Qlik Sense Topologies

Topology design principles

Dashboard (App) QIX Accociative Data Index Engine Overview Streams Central Governed Libraries Governed Data Extraction, Transformation and Load Process (ETL)
QVF A Qlik Sense app is a collection of reusable data items The QIX engine is the second generation of the A Qlik Sense Site consists of multiple servers. It is a 3. Qlik Central Node Windows Services The apps are organized in streams. A stream is Use of centrally governed libraries for dimensions, measures and charts ETL is a standard process which describes the moving and transformation of data from multiple data sources
(measures, dimensions and visualizations), sheets and stories. proven and patented engine technology that distributed architecture that consists of one or more Controls the whole Qlik Site. Qlik Senses services are the cornerstones of the Qlik a collection of apps that a specific group of A shared, centralized repository for data and visualization definition within a to more comprehensive views into a final sort of star schema model. By splitting the process into parts,
It is a self- has powered Qlik products for the last decade. nodes (that is, server machines) that together form a Sense architecture and is configured to support a variety users have access to. Qlik Sense app. Libraries provide a way for those responsible for managing the roles and responsibilties can be divided and each layer can have its own Sense Dashboard to monitor
Qlik Sense .QVF Application Structure Each role on the Central node is
contained entity site. By configuring which services to run on a node, it of enterprise-ready deployment scenarios, including The users of the stream can have different data to make it available to others in a way that makes it easier to use. But correctness of data and processes.
Using Qlik Associative Data Indexing and can be set up to perform a specific role within a site. considered to be the master and access rights. libraries include more than just data. Measures can be added to the library to key features
that includes the those that are geographically disbursed and with
Visual interface dynamic calculation that empowers people to contains information about the entire ensure that everyone has a common definition for calculated results. And, even IT maintains and monitors the data model
script, visualization multiple data centers. A node in a Qlik Sense site runs a Users might have read only access for the apps Dashboard design is splitted from data
A server can contain all roles or configured for a site. All nodes connect with this node.
and data to analyze Sheet with Bookmarks Stories with naturally explore using their intuition. The QIX set of Qlik Sense services. published to a stream, referred to as Consumer, visualizations can be added to the library for easy reuse by others. Master data stewards ensure correctness of master modeling
v isualizations specific Qlik Sense role. The following four roles are The QMC always uses the Qlik Sense
in a structured data engine exposes relationships in complex, multi- By configuring which windows services to run on a or they may also have publish rights, allowing data End users have self service BI without changing
source data sets that would otherwise be available. Proxy Service (QPS) on the central Without compromising the organisations IT security regulations, users can build Business key users can maintain business KPIs per
model. node, it can be set up to perform a specific role (for them to publish their apps to the stream, KPIs or measures
node their own visualizations from a centralized library of pre-built data sets,
QVD a Qlikview hidden in hierarchical or query based Qlik Sense Roles example, as a proxy node or a reload node) within a site. referred to as Contributor. department Data Loading Data Governance
approaches 4. Qlik Scheduler role (QSS) expressions, and visualizations to ensure consistent use of data and values.
Data file(QVD) can Deployment of these components requires planning.
Foundation 1. Proxy / Hub server Manage the scheduled tasks By default, Qlik Sense includes a stream called
store a single table Entities
Contains the user interface called Qlik Sense Hub. It Qlik Sense Node Everyone, in which all users have both read and Extraction Without modifications, make copy of data Verify data quality
of data for re-use Data model Measures Handles application reloads
handles requests through SSL/TLS and WebSockets. Servers in a site are referred as RIM nodesor slave publish rights to. QVF from source systems per table.
purposes.
Script Fx Dimensions
Handles task chaining nodes. A node in a Qlik Sense site runs a set of Qlik
(load * from table)
Store unedited separate tables in Qlik
Customers without
Data integration Acts as a webserver To which slave the QSS distributes Sense services. A multi-node environment name
Entry point into Qlik Sense for SALES compressed QVD files
the ability for the task ID to is determined by a load incorporates a minimum of one server. One node
cleansing, users and Administrators. balancing operation performed by assumes the role of central node, which is used as the Transformation Fast load of data from QVDs Verify relationships
transforming, and Entry point for external system the master QSS on the master QSS (Master)central point of control. User Consumer Align primary and foreign keys
Users benefit from QIX QVF
between table/
integration by using the mashup, Cast data to correct data format (date/
unifying multiple, disparate data sources for analysis, without Associative engine every time Node synchronization is based on a peer to peer entities. Invoices
authentication, session and load Qlik Repository Service & Database number/text/)
requiring external tools or data repositories. This includes they make a selection or concept. The P2P system connects all nodes in a Qlik without customer
balancing APIs FINANCE
Create entities like customer, materials,
databases, web content, and big data sources. (Similar to the search, allowing them to gain understanding Repository Services Manage multi-node database Sense Site. The configured nodes dynamically sales facts
activities which are normally done in a datawarehouse) and take next steps. communications. Each node requires a QRS and participate in traffic routing of apps in small Store in to QVD for re-use.
2. Consume Server Roles QIX / QES keeps a complete repository and transaction log. The segments to the designated nodes. When a change is Consumer
Direct Discovery Provides a "hybrid mode by loading the Define business
Provides the Engine service features of Qlik core repository contains information such as configuration, made on a node, the resulting transactions are Enrich master Ensure 1 version of truth of meta data
dimensions in the associative in-memory model. Combine this rules for process
technologies. Use the Engine on Qlik Management usage and rules. In a Qlik Sense environment, all Meta recorded in a transaction log. During the data QVF Enrich data with business KPIs
model with a real time SQL query connection to the Big Data analysis. (e.g. 3
Console to assign Engine services to specific QIX data is stored in a PostGreSQL Database. The synchronization, the latest set of transactions from Add extra master data (segments/clusters/
fact table. Provinding an associative experience combined with EVERYONE hierarchies) needed by business which are way match:
engine application servers. database can be deployed to multiple servers. the log is sent to the other nodes and replayed.
real time facts. Select chart type, dimension and the measure. (Definitions "can not" be changed) not in source Invoice without
In a multi-node environment, group databases Applications synchronization to the RIM Nodes does
In-Memory, Associative Data Indexing goods receipt)
by roles and deploy these to multiple database not include QVDs. The central node requests updates
Engine Create
servers. from the other nodes with an interval of 15 seconds. Create model per business area
Important The QIX Associative Data Indexing All engine services will utilize three business model Governed Self Service Analytics
Each of the other nodes initiates a synchronization Create star or snowflake schema data model to be used
Engine processing component in Qlik utilizes Services MY WORK QVF
Data Warehouse www
the CPU and memory from the sErver. The Qlik session with the central node every 15 seconds. by presentation apps.
Qlik Engine, Qlik Repository Services and Between nodes asynchronous communictation is
Custom Sense architecture provides application Qlik Repository Database Extend
Legend for database icons being used. Users consu me visualization apps to quickly answer the
orchestration, allows utilization of specific apps Identically configured application servers Dashboard next question
onto dedicated consume node. A combination can be used for application orchestrion. Entity (metadata) QVF Users create new sheets using the pre-built read only Libary Users
Binary Sync (application)
CRM ERP
Cloud Of Qlik Sense roles is possible but only if there Application nodes, look for services that The repository database contains the system The app data files contain th e data models and app Contributor Library with Dimensions, Measures and visualizations for
Big Data
are enough system resources available. Running configuration and all meta data about apps. The definitions. The app data files are controlled by the Qlik Developer own use.
consume a disproportionate amount of repository database is controlled by the Qlik Sense Sense Engine Service (QES). Publish
Consume
both of these roles on a single virtual machine Repository Service (QRS) and stored in the Repository Key users publish content into community.
resources and consider placing these Default Path =%ProgramData%\Qlik\Sense\Apps Users
Database (QRD) Power users can load extra data and create new measures
requires a 6-8-core VM. services on dedicated hardware. Default Path =%ProgramData%\Qlik\Sense\Repository\ and charts Collaborative environment

Enterprise Class Architecture

PostgreSQL\9.3

Qlik Sense Single Server (Node) Multi-Layer Security Qlik Sense Multi-Node Scalability Qlik Sense Communication Flow Installation concept Hybrid Solution - Qlikview and Qlik Sense
Qlik Senses singular architecture leverages the latest web specifications, Qlik Sense has multiple layers of security (both A common best practice for scalability, performance or security Virtual Topologies This scenario describes the traffic flow between multiple nodes. When you install multiple engines and proxies, the Hybrid Solution
data interchange formats, and protocols such as HTML5, CSS3, JSON and out of the box and custom) that all work reasons is to create a three-tier model; web, application and data Virtualized topologies are supported. Application behaviour depends on the capacity of Windows Server 2012 R2 proxies load balance users randomly across all This hybrid solution allows customers to reuse information generated by QlikView guided
Host A
WebSockets. together to conform to the organizations access. You can start with one node and increase nodes based on physical hosts, desired ratio or virtual machines to hosts, and the underlying virtualization engines by default. This can be modified so that a analytics. Qlik Sense can read Qlikview QVDs and QVWs (binary load) into Qlik Sense to
Proxy node Contains
security needs, resulting in A solid security requirements to create a high available and redundant clustered technology. This design model illustrates a fully fault-tolerant, virtual environment (VMs) Th e proxy server facilitate the proxy only talks to its local engine or to a sub-set of be used for self-service visualization users.
WebSockets protocol provides full-duplex communication between the Proxy / User traffic No Apps, No Engine
model. Qlik Sense utilizes the following environment. and a plan for scaling out different Qlik Sense roles. user request and lo adbalancing the engines, which caters for a number of
client and server over a single TCP connection.
infrastructure components: Each node operates independently, which increases the system Synchronization traffic Function deployment options to support various scenarios.
Users of all types can easily create and analyze in Qlik Sense using a Proxy to engine
Network security resilience, reduces maintenance, and increases the deployment
unified HTML5 web based client, with no necessity for browser plugins, Hardware Load Balancer Windows Server 2012 R2
nodes
In this deployment example, the Qlik Sense site
flexibility.
apps or fat desktop clients. All communications that build trust between Qlik consists of the following nodes:
Host B Host C Host D
IT can utilize a simple and powerful, web based management console Sense services and clients are based on web Web Host A Host B Engine node
Qlik Proxy Server can be load balanced via configured for
(QMC) for administration developed with the same technologies as the protocols such as Secure Socket Layer (SSL) and Delivers the end user interface Hub Proxy nod e Proxy node a hardware load balan cer. One or more Engine node 1 Engine node 2 Engine node 3
load balancing NODE NAME / REPOSITORY QIX ENGINE SCHEDULER PROXY
front-end client. Transport Layer Security (TLS). These protocols https://servername/hub proxy nodes can be included. over 2 nodes and SERVER ROLE SERVICE SERVICE SERVICE SERVICE
node for
Developers can integrate data and build analytics using the web client, handle encryption and The exchange of Authenticate user against external providers Contains Contains Co ntains development Central Node
and can extend and customize Qlik Sense through standard and open information, keys and authentication certificates. Published apps Published apps Published apps The Qlik
APIs DMZ Host C Host D dashboards will Scheduler Node
Web Portal Server security Qlik delivers all of this data and Scheduled reloads: No Scheduled reloads: No Scheduled reloads: No be consumed
HTML5 Web Client Firewall Consume Node 1 Consume Node 2 analysis through state-of-the-art Serves users: Yes Serves users: Yes Serves users: Yes
from the QIX
Consume Node Server A QlikView Server B Qlik Sense
Qlik Sense uses the servers operating system visualizations and interactive reports engine
Authentication Application that paint a clear picture of how the
Allow develop ment:
https:/ /portal.com security layer to control and protect Qlik Sense Allow development: No Allow development: No Yes Web Server Proxy
2 Providers Delivers only the dashboard app relevant for Contains Co ntains business is performin g. Proxy Node
1 resources (files, memory, processes, and Function Func tion Func tion IIS
https:/ /server/hub certificates) on the server. users to consume Published apps Published apps
Consume node Consume node Consume node
User types The data that is processed by Qlik server,
Consumer In-Memory, Associative Data Indexing Engine QIX Application QIX Application
QlikView Server Qlik Central Node
Processing Processing a highly scalable, secure and In very large environments roles can be directed to
Contributor Sense Server Application security manageable IT-friendly environment. Qlik associative
Oth er
dedicated server nodes. Qlik Management Console QIX Engine
Developer Combined with the security that Qlik Sense Host F
in-memory
computer Host E
technology
Qlik Admin provides authentication, rules based content Firewall The table above lists the Qlik Sense services that are QlikView Distribution Services
system Host E Scheduler node Contains Central node Co ntains Qlik Scheduler
Qlik Manage ment Qlik Proxy security, and dynamic data reduction, the result This multi server setup is referred to as All apps
deployed on each node in this site. Each node QlikView Publisher
Console
is an integrated, flexible and robust security Data Access Qlik site. The roles running on the
Published apps
requires a QRS and QRD
Dashboard Central node
model we call Qlik Sense Security This layer is Central node are considered to be the Function Scheduled reloads: Yes Function Scheduled reloads: No
Create compressed copy of source master node in the Qlik Sense
the core of the platforms protection, comprised
Reload node Serves users: No Proxy to engine Serves users: No A Qlik (application) environment separates the query
tables environment. nodes
of authentication, authorization, auditing, Data Model Allow development: No processing component and indexing component to Single point of loadin g data in both
Allow development: No
QlikView and Qlik Sense
dedicated servers. The remaining roles can reside on
QIX Engine Interact using APIs confidentiality and availability. ETL process
and Extentions 1:1 Data Load two all-purpose application servers. on two all-
After successful reload only sync per Table www
Continuous security test and enhancement Third Party Integration purpose application servers. Guided Analytics Applications Self Service Visualization &
Qlik creates an associative data dashboard(QVF) to application tier MetaData
Entity Sync
Mobility Applications
model, whic h acts as a full outer This layer of the security model focuses on
join between all sources based on a
matching key field which occurs in ensuring that the Qlik software is thoroughly Firewall License If the transaction log only contains entity data (that is, changes in the repository database), every 15 seconds an
Scheduler Repository Data Access Custom Con nectors Security Inte gration Qlik combines all of your data, whether
both or multiple sources: No data is Manage Persistance entity data synchronization is performed. The changes are applied immediately in the repository database on the
lost in this process. analyzed from a security perspective by using Dell ODBC LDAP its located in ERP , CRM, data
Source Boomi QVX Tivoli Software
warehouses, SQL databases, or even Synchronization receiving node. If a conflict occurs, the latest transaction is used.
rugged development practices as well as threat Systems Sybase XML IBM XLSX CSV CRM ERP SQL < Data Sources >
And many more.. Microsoft Active Excel. Security
analysis and exploratory security and Dire ctory
Service Configuration Data
penetration testing. - Being Rugged is
Legend for app icons Qlik has revolutionized the delivery of insights
about staying ahead of a threat. Using and value to every business stakeholder for App synch ron ization make use of peer-to-peer replication to speed up the synchronization of large apps and prevent As QVDs are Qlik proprietary way of staging and storing data it can provide a logical
experimental approach to learn and improve Node contains apps, Node contains apps but small data, to something more powerful in
Apps Binary Sync network bottlenecks. If the transaction log contains binary data (that is, changes to app data files), a binary data
serves dashboards serves no dashboards data tier as a solution. In certain cases it could remove the need for a data warehouse.
while actively seeking out threats and creating the Big Data world. Qlik enables customers to Stores App s structure synchronization, during which the receiving node obtains the updated data, is initiated. Only the components that have
to the users to the users combine Big Data and small data to yield changed will be copied. Using QVDs you can share data between multiple applications without having to keep

Qlik Sense Governance

defenses. actionable business insights. going back to the source data. In depicted scenario the Qlikview QVD shares the
source data with Qlik Sense.

Risk and Controls Governed Self Service BI Authorization Flow Data Reduction Authentication & Authorization overview Integration Overview (Embedded Analytics)
In order to provide a reliable and secure information management In the past, IT would create a reporting environment and the business users would just be able Stream and resource access control Row-level data security is accomplished by means of the Qlik Authentication: How do we provide Single Sign On using SAML, Context sharing between host website and Qlik Sense (e.g. share a session, share variables, transfer selections made) by making use
process the following key risks need to be mitigated: to read what IT delivered. But by definition, people learn and always want something else Which streams are you allowed to view or modify? Sense feature of the data model called Section Access tickets or header authentication using the Proxy API? of the "Session API" and "Mashup API. Integrate security to provide single sign on using web tickets, header authentication or SAML.
when they see the result. Qlik Sense has been developed with this is mind. Sense enables the Integrate the management/maintenance processes by using the API's
Unrealiable reporting Which dashboards are you allowed to view or
IT / Reporting team to create a starting template by filling "a library of master items" with 80% Authorization: What are you allowed to see/do on a specific User logs in into the generic portal.
Multiple versions of the truth modify? Iframe show a complete dashboard inside your website.
of the functionality and let the business create the remaining 20% in a controlled way. resource?
Users requirements are not met, causing additional decentralized and Which sheets, buttons or other resources are Single integrate a single chart from an app in your site with a iFrame
Content integration using Div-tag or Iframe Workbench DIV tag integration, create mashups of individual charts and sheets.
ungoverned tools (like Excel) to be used, e,g, not sufficient Self Se lf Service BI Process you allowed to view or modify. (Edit script, edit
Extensions exten ts Sense with custom visualizations or objects
Service capabilities IT / Reporting team Build template s heets with dimension, measures and charts dashboard, story telling)?
This type of authorizaton is managed by Web Portal
Performance issues Users Developer
Database specialist Reporting team Manager security rules in the management console. Two basic concepts in
Information is disclosed to unauthorized persons Using an authentication API / URL
Create Apps,Load and Model data Des ign Charts Application Validation security are Who are you: Authentication method. The user and
groups (like customer Bi-directional communication of
An integrated (business and IT) governance framework allows you authentication and Active Directory selections, content and user credentials.
Data reduction name) are passed onto
Know the risks that can cause unreliable reporting authorization. Proxy SAML Qlik
Validate correctness Answers the question: What data are you
Define controls to prevent or limit the impact of those risks allowed to see given your userId or group? Authentication HTTP Headers
and publish app into answers the question Ticket / Session API
Define a process with clear roles and responsiblities to ensure everyone stream to make it Rules engine Security Integration (SSO) Self Servic e BI
only performs the activities for which they are trained/authorized. The resource access control system in Qlik who is the user and 1. The user accesses Qlik Sense. CEO on mobile devices
available for the public Enforces Can see the
Sense is based on attributes. This means that how can the user prove 2. Qlik Sense redirects the user to the authentication whole company
resource access module of the portal. The authentication module verifies
1 Why is margin low?
the access is based on rules that refer to it?
control Repository the users identity and credentials with an identity provider. 2 Is it a specific , manager, region or
attributes connected to resources and users Usually the already existing authentication mechanism of product group?
Administrative Roles Application will only be visible in personal App now available for the the portal which stores users/groups in a SQL table.
in Qlik Sense. Regional Manager 3 Ah, product Y is not performing
Own work stream community in a specific Stream Authorization answers the question what does this specific user 3. Once the credentials have been verified, a ticket is See his or her departments in G ermany.
Sheets are marked as Base have access to, and what are they allowed to do. QIX Engine requested from Sense (QPS). Additional attributes like Because all data is included y ou can always
in different groups may be supplied in the request. find the next ans we r to your question, this
Administration roles sheets Authentication Authorization in contrast to query based tools, which
of the security We re-use the group membership of the Data reduction 4. The authentication module receives a ticket.
Sales.qvf Users provide you only a limited subsets of the
system Business users View or dup lica te the sheets or charts to build new visualizations source system (e.g. Active Directory) per QVF 5. The user is redirected back to the QPS with the ticket. The Only see his department, data.
QPS checks that the ticket is valid and has not timed out. company or cost center 4 Hey, is it only in Germany or do other
Consumer Contributor Developer Each department gets its own stream (Hierarchy)
6. A session is created for the user. countries also have issues?
(Finance, HR) Invoi ce
AD MIN 7. The user is now authenticated. Cu stomer Questions to answer
Complete Displa y Only Limited Self Service Full Self Service We map the stream name to the Active What are you allowed to do: Authorization
All Data User Order
Administrative Directory groups Role Re-use existing authorization or group membership
Client
Access Display only Create new sheet which is Create new Apps and sheets Server
No Sheet creation only visible for me which are only visible for me Authorization concept definition from the following sources:
User C Your company exists of processes in which entities are
Use a limited set of Use all dimensions If you see the stream you may see all dashboards in it. Security is managed using Lightweight Directory Access Protocol (LDAP) Authentication Module 2 Department
User B doing activities. These entities have relationship with each
Can audit who dimensions Use predefined measures one rule: Stream.name = user.group NTNAME or Microsoft Active Directory
5
sees what but User A other in a "Business way".
not change
Use predefined measures Create new measures Username & USERS Open DataBase Connectivity (ODBC)
4
Manages the 3
Load extra data Password Sales.qvf Sales.qvf Sales.qvf Examples are
anything content Qlik Sense groups Proxy
A Sales order has sales order lines.
Present a group via ticket (OEM) Session Module 6 7 An order line consists of products which you sold.
User can view the base These users can create sheets only visible in the users
sheets and the community private My sheets Authenticated So why not model this relationship one time in a datamodel in stead of
sheets created by the They can publish the sheet into the community sheets Reduced Reduced Reduced
2015 Qlik. All rights reserved. Data A Data B Data C
creating separate (MDX) queries each time you want to create a report?
Contributor and Developer section for collaboration with other application users