Inside OBIEE 10g Server

Inside the Oracle BI Server (10g)
Mark Rittman, Director, Rittman Mead ODTUG Kaleidoscope 2010, Washington D.C.
T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
Introductions
Oracle BI&W Architecture and Development Specialist Co-Founder of Rittman Mead Consulting Oracle BI&W Project Delivery Specialists 10+ years with Discoverer, OWB etc Oracle ACE Director, ACE of the Year 2005 Writer for OTN and Oracle Magazine Longest-running Oracle blog http://www.rittmanmead.com/blog Ex-Chair of UKOUG BIRT SIG ODTUG Director Editor of Oracle Scene Co-Chair of ODTUG BI&DW SIG Speaker at IOUG and BIWA events Ex-member of the Blue Man Group *
* Not technically true T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
Inside the Oracle BI Server

In this session, we will look in-depth at the Oracle BI EE Oracle BI Server What components are in the BI Server What is the Navigator, and how does it turn incoming logical requests into optimized physical SQL and MDX What happens when it processes a query What happens when the BI Server joins resultsets in memory And what influences SQL generation, join strategy and BI Server resource usage
Why Is This Topic Important?

Now this could be a pretty boring topic But its one of the most important parts of the BI EE technology stack It provides all of these key features Query Federation Heterogeneous Connectivity The Semantic Model Aggregate Navigation Time Travel (sort of) So what really happens when Answers fires off a query, and the BI Server goes and gets the data...?
The Oracle BI EE 10g Server Architecture
The Oracle BI Server 10g Architecture

Logical SQL interface via ODBC and JDBC Intelligent Request Generator, made up of Logical request generator Navigator Optimized Query Rewrites Execution Engine Cache Services Other services include Load balancing (in conjunction with cluster controller) Query governance Session Management System / Perf Monitoring (JMX etc)
Intelligent Request Generation and the BI Server Navigator

The Navigator is the key component that parses incoming requests and generates the relevant physical SQL and MDX queries Invoked if cache cannot be used Handles the following translations Function push-down Multi-pass logic (time series etc) Aggregate navigation Fragmentation navigation Responsible for generating optimized SQL and MDX for the mapped data sources Uses rules and definitions present in the RPD
The BI Server Functional Compensation Model

The BI Server tries to push functions and joins down to the underlying databases Where not possible, the BI Server will functionally compensate by calculation, joining data sources in memory
Parse Incoming Logical SQL
Generate Optimized Physical SQL(s)
Receiving Incoming Resultsets
Apply any required functions and joins
Return data to user
How Does The BI Server Process Queries?

So what happens when the BI Server processes an incoming query? What are the steps involved in translating logical requests to physical queries? What does the Navigator do? What rules and statistics does it use? How can we influence the physical queries that it generates? How can we tell what is going on?
Step 1 : Request Arrives at Intelligent Request Generator

Query arrives from Answers or other ODBC tool, against Semantic Model Passed to Intelligent Request Generation function
Step 2 : Request Passed to Logical Request Generation Engine

Request passed to Logical Request Generation engine, responsible for generating the one or more logical SQL required to satisfy the query In this case, request requires multiple columns and a derivation (calculation)
Step 3 : Check Cache if Enabled

If caching is enabled, Cache Services is checked for the result Avoids calls to physical databases for frequently-requested data Intelligent checking of cache including derivation, projection, nested formulas etc
Rules for a Cache Hit

Caching is enabled (CACHE=Y in the NQSConfig.INI file); WHERE clause is semantically the same or a logical subset of a cached statement; All of the columns have either to be present or derivable from cached copies It has equivalent join conditions If DISTINCT is used, the cached copy has to use this attribute as well Aggregation levels have to be compatible, being either the same or more aggregated than the cached query No further aggregation (for example, RANK) can be used in the incoming query Compatible ORDER BY clause
Step 4 : Request Passed to the Navigator

Navigator is responsible for navigating the semantic model and generating the execution plan, and most optimal physical SQLs, for the request Contains multiple functions that are used dependent on the nature of the request
Step 5 : Multi-Pass / Sub-Request Logic

Analyzes request and works out the complexity of the query Determines whether multi-pass logic is required (time-series etc) Works out whether one request depends on the results of another (sub-request) Navigator uses this function to determine the most efficient way to satisfy the request
Step 6 : Fragment Optimization

When a measure is fragmented, the Fragment Optimization function will transform base-level logical SQL into logical SQL fragments Also automatically resolves fan-trap issues
Step 7 : Aggregate Navigation

Aggregate Navigation checks aggregates mapped in RPD for relevance Execution Plan will be written to use aggregates if they return less rows than the default logical table sources
Step 8 : Optimized Query Rewrite, and pass to Execution Engine

Optimized Query Rewrite checks database capabilities for function shipping and ability to join subqueries Generates optimal physical SQL or MDX for relevant data sources Passed to Execution Engine for transfer to physical databases
Step 9 : Copy Back to Cache Services, if Enabled

Results are returned to calling application (Answers etc) If caching is enabled, results are copied back to Cache Services
Key Navigator Functionality

BI Server will pick one or more LTSs at run-time based on query context Generates an execution plan Will try and generate as few physical SQL as possible Will try to push down functions if possible Will try to push down joins if possible Will write the optimal SQL or MDX for the data source, based on DBFunctions.INI matrix
Level 5 Query Logging and Logical Execution Plans

The execution plan generated by the Navigator can be viewed when Level 5 logging is enabled Shows the execution plan, step-by-step row counts, and the physical SQL / MDX generated for the request
BI Server Joins
(Almost) Every request requires joins For simple queries, you dont notice this because they are pushed down to the DB For more complex queries, the BI Server may need to get involved Where more than one physical database provides data for the request Where multi-pass logic is used Where measures are fragmented Where more than one fact table is used in the request Basic principle is that the BI Server tries to push down all joins Depending on the database type and version and nature of request, the BI Server may need to perform the joins instead
The BI Server Source-to-Target Metadata Model

Facts and dimensions are mapped to physical sources Logical Table Sources LTS are chosen at run time based on RPD definitions (level mapping, fragmentation, federation etc)
So What Does The BI Server Do, When It Joins Data In Memory?

So just what goes on when the BI Server joins data in memory? How can we tell when the BI Server has performed the join? Why does the BI Server sometimes do the join, and sometimes get the database to do it instead?
BI Server Join Type 1 : Fact + Dimension from Single Database

Single logical fact and dimension table, joined by a logical join Both logical tables have single LTS, which both point to tables in the same physical database Physical tables are joined using a physical foreign key
Single Database LTS Mapping
Products Logical Dimension
Sales Logical Fact LTS Mapping
Products Physical Table
Sales Physical Table
Single Source Fact and Dimension Execution Plan & SQL

BI Server generates execution plan with single logical SQL including join clause
RqLis t <<2105>> [ fo r database 3023:2820:orc l3 ,44 ] PR O D U CTS.PR O D_SUB CATE G O R Y_DESC as c1 GB [ fo r da tabase 3023:2820,44 ] , sum(SALES.QUANTITY_S OLD by [ PRO D U CTS.PR O D_SUBCATE G O RY_DES C] ) as c2 GB [ fo r database 3023:2820,44] Chi ld Nodes (RqJoinSpec) : <<2136>> [ for database 3023:2820:orc l3 ,44] PR O D U CTS T2874 SALES T2911 Deta i lF i l te r : PRO D U C TS.PR O D_ID = SALES.PR O D_ID [ fo r da tabase 0 :0 ] GroupBy: [PR O D U CTS.PR O D_SUB CATE G O R Y_DESC] [ fo r da t abase 3023:2820,44 ] OrderBy: c1 asc [ fo r database 3023:2820,44]
Single physical SQL with join pushed down to database

se lec t T2874.PR O D_SUB CATE G O R Y as c1 , sum(T2911.QUANTITY_S OLD) as c2from PROD U CTS T2874, T2911 w here ( T2874.PR O D_ID = T2911.PR O D_ID ) group by T2874.PR O D_SUBCATE G O RYorder by c1 SALES
BI Server Join Type 2 : Fact + Dimension from Two Databases

Single logical fact and dimension table, joined by a logical join Both logical tables have single LTS, which point to different databases Physical tables are joined using a physical foreign key Database #1
LTS Mapping
Products Physical Table
Database #2
Multiple Source Fact and Dimension Execution Plan

Excerpt of plan (source rqList cropped) Two logical SQL blocks with InnerJoin between fact and dimension tables Can also be LeftOuterJoin, RightOuterJoin and FullOuterJoin
( RqLis t <<2495>> [ fo r database 3023:132:orc l ,44 ] SALES.PRO D_ID as c2 [ fo r da tabase 3023:132,44 ] , sum(SALES.QUANTITY_S OLD by [ SALES.PRO D_ID] ) as c3 [ fo r da tabase 3023:132,44] Chi ld Nodes (RqJo inSpec) : <<2504>> [ fo r database 3023:132:orc l ,44 ] SALES T211 GroupBy: [SALES.PRO D_ID] [ for da tabase 3023:132,44] OrderBy: c2 asc [ fo r database 3023:132,44 ] ) as D901 InnerJoin <<2492>> O n D901.c2 = D902.c2; actual j oin vectors: [ 0 ] = [ 1 ] ( RqLis t <<2517>> [ fo r database 3023:2500:orc l2 ,44 ] PR O D U CTS.PR O D_SUB CATE G O R Y_DESC as c1 GB [ fo r da tabase 3023:2500,44 ] , PR O D U CTS.PR O D_ID as c2 [ fo r da tabase 3023:2500,44] Chi ld Nodes (RqJo inSpec) : <<2523>> [ fo r database 3023:2500:orc l2 ,44 ] PR O D U CTS T2502 OrderBy: c2 asc [ fo r database 3023:2500,44] ) as D902 OrderBy: c2, c3 [ fo r database 0 : 0,0 ] ) as D1 OrderBy: c1 asc [ fo r database 0 : 0,0 ]
Multiple Source Fact and Dimension Physical SQL

Two physical SQL queries sent to the two databases BI Server joins the two sources in-memory using a sort-merge join algorithm
- Send ing query to database named orc l( i d : <<6057>>) :se lec t T211.PR O D ID as c2, su m(T211.Q U A N TIT Y S OL D) as c3 f ro m Send ing query to da tabase na med orc l2 ( id : <<6074>>) : se lec t T2502.PR O D_SUB CATE G O R Y as c1 , T2502.P R O D_ID as c2 f rom PRO D U C
BI Server Join Type 3 : Fact + Dimension Using Driving Table

Single logical fact and dimension table, joined by a logical join Two tables typically LTS mapped to different physical databases One of the logical tables defined as a driving table in the BMM logical join Driving table uses Parameterized Nested Loop Join (PNLJ) to probe other table Database #1
LTS Mapping
Database #2 Products Physical Table
PNLJ Probe Lookup
Driving Table Considerations

By default, federated queries will cause both tables to be loaded into the BI Server memory, and then joined (as per previous example) Can be expensive if one or more of the tables are large An alternative is to designate one of the tables as the driving table Needs to be small, typically <1000 rows This table is then queried and filtered first, then its remaining rows are then applied as a filter to the second table Driving table is defined in the Logical Join dialog
T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
Parameterized Nested Loop Join Algorithm & Physical SQL

1. 2. 3. 4. Start reading rows from the driving table Submit a parameterized query request to non-driving table Bind values from the drive-table rows into the parameterized query and fetch results Repeat until all rows from drive table are processed
PR O D U CTS.PRO D_SUBCATE G O RY as c1 GB,
RqLis t
QUANTITY_S OLD: [DAggr (SALES.QUANTITY S O LD by [ PRO D U CTS.PR O D SUB
se lec t T2502.PR O D_SUB CATE G O R Y as c1 ,
se lec t T211.PRO D_ID as c2 , sum(T211.QUA NTITY_S OLD) as c3f rom SALES T211where (T211.PR O D_ID in ( :PA R A M 1, :PA R A M 2, :PA R A M 3, :PARA M4, :PARA M5, :PARA M6, :PARA M7, :PARA T2502.PR O D_ID as c2 f rom PRO D U CTS T2502order by c2 M8, :PARA M9, :PARA M10, :PA R A M 11,:PARA M12, :PARA M 13, :PARAM14, :PA R A M 15, :PAR A M 16, :PA R A M 17,:PARA M18, :PARA M 19, :PARAM20) ) g roup by T211.P R O D_ID order by c2
Fact and Dimension Driving Table Execution Plan

Excerpt of plan Note InnerJoin (left drive) and parameterized DetailFilter in plan
( RqL is t <<13655>> [ for da tabase 3023:2500:orc l2 ,44 ] PR O D U CTS.PRO D_SUBCATE G O R Y as c1 GB [ fo r da tabase 3023:2500,44] , PROD U CTS.PRO D_ID as c2 [ fo r da tabase 3023:2500,44 ] Ch i ld Nodes (RqJo inSpec) : <<13658>> [ f or database 3023:2500:orc l 2,44 ] PR O D U CTS T2502 OrderBy: c2 asc [ fo r da tabase 3023:2500,44 ] ) as D901 InnerJoin ( l ef t dr ive) <<13689>> On D902.c2 = D901.c2 ; ac t ua ljo in vec t ors :[ 1 ]= [ 0] ( RqL is t <<13673>> [ for da tabase 3023:132:orc l ,46 ] SAL E S.P R O D _ID as c2 [ fo r da tabase 3023:132,46] , su m(S AL E S.Q U A N TITY_S O L D by [ SALES.PRO D_ID] ) as c3 [ fo r da tabase 3023:132,46 ] Ch i ld Nodes (RqJo inSpec) : <<13676>> [ f or da tabase 3023:132:orc l ,46 ] SALES T211 Deta i lF i l ter : SAL E S.P R O D _ID = ?1 or SAL E S.P R O D _ID = ?2 or SAL E S.P R O D _ID = ?3 or S A L E S.PRO D_ID = ?4 or SALE S.PR O D_ID = ?5 or SALE S .PR O D_ID = ?6 or SALE S.PR O D_ID = ?7 or SA L E S.PR O D_ID = ?8 or SALE S.PR O D_ID = ?9 or SALE S .PR O D_ID = ?10 or SALE S.PR O D_ID = ?11 or S A L E S.PRO D_ID = ?12 or SALES.PR O D_ID = ?13 or SALES.PR O D_ID = ?14 or SA L E S.P R O D _ID = ?15 or SAL E S.PR O D _ID = ?16 or SALES.PR O D_ID = ?17 or SALES.PR O D_ID = ?18 or SALES.PR O D_ID = ?19 or SALES.PR O D_ID = ?20 [ for database 0:0] GroupBy: [SALES.PRO D_ID] [ fo r database 3023:132,46 ] OrderBy : c2 asc [ fo r da t abase 3023:132,46 ] ) as D 902
BI Server Join Type 4 : Two Facts Using a Single Database

Query requires data from two fact tables, sharing a common dimension BI Server creates two logical queries and stitches results together, to avoid fan trap Join is pushed down to the database if possible
Single Database LTS Mapping Costs Logical Fact Products Physical Table Costs Physical Table
LTS Mapping
Sales Logical Fact
LTS Mapping
Facts with Conformed Dimensions Execution Plan

Two result sets are generated, with a FullOuterStitchJoin between them Capabilities of physical database determine whether join is pushed down to database
( RqL is t <<3474>> [ fo r da t abase 3023:2820:orc l 3,46 ] s um(COSTS.UNIT_C O ST by [ PRO D U CTS.PR O D_SUBCATE G O R Y_DESC] ) as c1 GB [ f or da tabase 3023:2820,46 ] , PR O D U CTS.PRO D_SUBCATE G O RY_DESC as c2 GB [ f or da tabase 3023:2820,46 ] Ch i l d Nodes (RqJo inSpec): <<3507>> [ fo r da tabase 3023:2820:orc l3 ,46] PR O D U C T S T2874 C O STS T2830 Deta i lF i l te r : CO STS.PRO D_ID = PR O D U CTS.PRO D_ID [ fo r da tabase 0 :0 ] GroupBy: [ PRO D U CTS.PR O D_SUBCATE G O R Y_DESC] [ fo r da tabase 3023:2820,46 ] ) as D1 Ful lOuterSt i tchJoin <<3565>> On D1.c2 = D2.c2 ( RqL is t <<3511>> [ fo r da t abase 3023:2820:orc l 3,46 ] s um(SALES.AM O U NT_S OLD by [PR O D U CTS.PR O D_SUB CATE G O R Y_DESC] ) as c1 G B [ fo r da t abase 3023:2820,46 ] , PR O D U CTS.PRO D_SUBCATE G O RY_DESC as c2 GB [ f or da tabase 3023:2820,46 ] Ch i l d Nodes (RqJo inSpec): <<3544>> [ fo r da tabase 3023:2820:orc l3 ,46] PR O D U C T S T2874 SALES T2911 Deta i lF i l te r : PRO D U CTS.PR O D_ID = SALES.PRO D_ID [ fo r database 0:0 ] GroupBy: [ PRO D U CTS.PR O D_SUBCATE G O R Y_DESC] [ fo r da tabase 3023:2820,46 ] ) as D2
Facts with Conformed Dimensions Single Physical SQL

Where database supports subquery factoring (Oracle 10gR2+) a single physical SQL will be generated with a full outer join between all shared dimension columns
WITH SA WITH0 AS (se lec t sum(T2830.UNIT_C O ST) as c1 , T2874.PR O D_SUBCATE G O RY_DESC as c2 f ro m PRO D U CTS T2874, COSTS T2830 where ( T2830.PR O D _ID = T2874.PR O D_ID ) group by T2874.PR O D_SUBCATE G O RY_DES C) , SA WITH1 AS (se lec t sum(T2911.AM O U NT_S OLD) as c1, T2874.PR O D_SUBCATE G O RY_DESC as c2 f ro m PRO D U CTS T2874, SALES T2911 where ( T2874.PR O D _ID = T2911.PR O D_ID ) group by T2874.PR O D_SUBCATE G O RY_DES C) se lec td is t i nct SA WITH0.c1 as c1, SA WITH 1.c1 as c2, case when SA WITH0.c2 is not nu l lthen S A WIT H0.c2 when SA WITH1.c2 is no t nu l lthen SA WITH1.c2 end as c3 f ro m SA WITH0 fu l l outer jo in SAWITH1 On SA WITH0.c2 = SA WITH1.c2 order by c3 T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com
Facts with Conformed Dimensions Multiple Physical SQL

Otherwise, multiple physical SQL queries will be generated BI Server will perform in-memory sort-merge stitch full outer join on shared dim columns
- Send ing query to database named orc l3 ( id : <<3018>>) : se lec t su m(T2830.UNIT_C O S T) as c1 , T2874.PR O D_SUBCATE G O RY_DESC as c2 f ro m PRO D U CTS T2874, COSTS T2830 where ( T2830.PR O D _ID = T2874.PR O D_ID ) group by T2874.PR O D_SUBCATE G O RY_DES C order by c2 +++Ad min i s t ra to r :2a0000:2a0005: -2010/02/28 15 :05:31 - Send ing query to database named orc l3 ( id : <<3055>>) : se lec t su m(T2911.A M O U N T_ S O L D) as c1 , T2874.PR O D_SUBCATE G O RY_DESC as c2 f ro m PRO D U CTS T2874, SALES T2911 where ( T2874.PR O D _ID = T2911.PR O D_ID ) group by T2874.PR O D_SUBCATE G O RY_DES C order by c2
Stitch Joins and Federated Queries

Stitch joins are used whenever multiple fact tables are used in queries Also used when federated fact table sources are used, and aggregate sources Basic principle is to join facts using FullOuterStitchJoin across all dimension attribute columns used in the query
. . . ) as D901 Ful lOuterSt i tchJoin <<7800>> On D901.ProductCategory =Nul lsEqual D902.ProductCategory and D901.ProductCategory =Nul lsEqual D902.ProductCategory; actual j oin vectors: [ 0 1 ] = [ 0 1 ] ( . . .
Rules for BI Server Join Strategy

Try to generate as few physical SQL / MDX queries as possible Push as many joins (and functions) down to the database as possible Where required, join fact and dimensions in memory using inner and outer sort-merge joins If driving table is specified, iterate through join using nested loop algorithm Generate one logical SQL query per fact table access, push down stitch join to database using full outer join if possible, if not have the BI Server do it Logical execution plan and resulting physical SQLs visible in the Level 5 log file
BI Server Join Variations 1 : Fragmentation (Partitioned Measures)

RPD has a measure mapped through two or more LTS to different physical tables One physical source for historic data, one for real-time One for Southern Region, one for Northern Region Known as fragmentation BI Server has to UNION the resultsets together
Fragmentation (Partitioned Measures) Logical Execution Plan

Execution Plan will contain two logical SQL blocks (excerpt shown below) RqUnion All unions the two resultsets, BI Server will attempt to push down to database if possible (if not, two physical SQLs will need to be retrieved and UNION done in memory)
( RqL is t <<7613>> [ f or da tabase 3023:6594:orc l7 ,44 ] PROD U CTS.PRO D_SUBC ATE G O R Y _DESC as c2 [ fo r da tabase 3023:6594,44 ] , SALES_UPT O_2003.AM O U NT_S OLD as c3 [ fo r database 3023:6594,44] Ch i ld Nodes (RqJoi nSpec) : <<7617>> [ fo r da tabase 3023:6594:orc l7 ,44 ] PROD U CTS T6596 SALES T6629 Deta i lF i l te r : PRO D U CTS.PR O D_ID = SALES_UPTO_2003.PR O D_ID [ for da tabase 0:0 ] RqUnion Al l <<7690>> [ for database 3023:6594:orcl7 ,44] RqL is t <<7668>> [ f or da tabase 3023:6594:orc l7 ,44 ] PROD U CTS.PRO D_SUBC ATE G O R Y _DESC as c2 [ fo r da tabase 3023:6594,44 ] , SALES_BEY O N D_2003.AM O U NT_SOLD as c3 [ fo r da tabase 3023:6594,44 ] Ch i ld Nodes (RqJoi nSpec) : <<7672>> [ fo r da tabase 3023:6594:orc l7 ,44 ] PROD U CTS T6596 SALES T6637 Deta i lF i l te r : PRO D U CTS.PR O D_ID = SALES_BEYO N D_2003.PROD_ID [ fo r da tabase 0 :0 ] ) as D3
BI Server Join Variations 2 : Time-Series Queries

Generated when time-series functions are used in a request Requires time dimension to be defined in RPD Currently limited to AGO and TODATE functions Generates two or more logical SQL queries which need to be joined together to return timeseries results
Time Series Logical Execution Plan

Execution plan will show multiple nested logical SQL blocks More complex set of nested blocks will be doing the time offset Time offset block will need to be FullOuterStitchJoined to the present-day resultset Excerpt shown below
. . . Deta i lF i l te r :I T E M S.O R DID = O R D E R S .OR DID and O R D E R S.O R D E R D A T E = D 4.c2 and D3.c1 = D 4.c1 and D3. c2 = 'May-2007 '[ fo r da tabase 0 :0 ] GroupBy: [ D3.c2 , D3.c3 ][ fo r da tabase 3023:4210,44] ) as D1 OrderBy: c1 asc [ fo r database 3023:4210,44 ] D901 Ful lOuterSt i tchJoin <<4892>> On D901.c1 =Nul lsEqual D902.c1; actual j oin vectors: [ 0 ] = [ 0 ] ( RqLis t <<4963>> [ fo r da tabase 3023:4210:orc l4 ,44 ] D2. c2 as c1 [ fo r database 3023:4210,44 ] , D2.c1 as c2 G B [ fo r da tabase 3023:4210,44 ] , D2.c3 as c3 [ fo r da tabase 3023:4210,44 ] . . .
) as
Depending on DB capabilities, will either be shipped as one SQL (using subqueries) or as two or more SQLs (with results full outer stitch joined in memory)
Memory and Disk Usage During BI Server Joins

So what happens when the BI Server performs an in-memory join? The BI Server requires memory to join the data sets using a sort-merge algorithm Initial memory usage is determed by size/complexity of the RPD Memory usage then rises with # of concurrent sessions, in-memory joins, aggregations Typically paged to file at {ORACLEBIDATA}/ tmp Because of paging, use of BI Server functional compensation is largely I/O intensive rather than memory intensive Extent of paging driven by VIRTUAL_TABLE_PAGE_SIZE parameter May benefit from increase on Unix systems Docs do not imply much benefit though
So What Influences Execution Plans and Physical Query(ies)?

Logical Execution Plan largely determined by the LTS mappings in the RPD BI Server Navigator generates execution plan based on context of the request and the LTS configuration in the BMM layer All can be determined by looking at the RPD No equivalent to the Oracle database CBO Physical SQL then determined by execution plan and DBFeatures.INI
Are There The Equivalent of Object Statistics in the RPD?

There are some limited statistics in the RPD that influence the query plan Row counts on physical tables are ignored Number of Elements at this Level value can be used by the Navigator to determine which of several aggregates to use for a request Needs to be manually populated (rough figure is OK)
Dimension Level Element Counts In Use

Consider the following dimension Time Dim -> Year (2), Month (24), Days (730) Products Dim-> Type (10), Sub-Type (20), SKUs (1000) Customers Dim-> Regions(20), Districts(500), Customers (100000) Sales Fact joins to these dimensions at the lowest level. Also assume I have a couple of aggregate tables : Sales Fact Agg1 -> (Time Dim)Year, (Customer Dim) Districts Sales Fact Agg2 -> (Time Dim)Year, (Product Dim) Sub-Type Consider this Query Example : Select Year, Sum(Revenue) from Sales ; This can be satisfied by either Agg1 or Agg2, but BI Server has no way of knowing which is the best option unless the number of elements are entered.
What Does the Logical Execution Plan Tell Us?

It tells us how the BI Server Navigator turned the logical request into physical SQL and MDX queries But theres nothing that you couldnt infer from the RPD There are no parameters etc that you can use to alter join strategies etc This is driven by the repository design only Extra information provided by Level 7 logging is interesting Times logged by each entry/step Tells us how long the BI Server takes to do each join Can be used to fine-tune the VIRTUAL_TABLE_POOL_SIZE parameter
So What Does This All Tell Us?

Co-locate data as much as possible, minimize in-memory joins Make sure the database version and capabilities is set correctly for each source Maximize function push-down, mimimize functional compensation Understand what happens with BI Server joins, and how to read the execution plan Understand what a FullOuterStitchJoin is, and how to avoid unneccesary ones Understand the role of LTSs and when to add or remove them There is some diagnostic data available for the BI Server, but most of what you need is contained in the RPD design
Thank You for Listening

More details on this topic, and on OBIEE in general, can be found on our blog
ht tp : / /ww w.r i t tmanmead. com/2010/02/25/ ins ide-the-oracl e -b i -server -par t 1the-b i server arch i tec t ure / ht tp : / /ww w.r i t tmanmead. com/2010/03/01/ ins ide-the-oracl e -b i -server -par t 2-howis-a -queryprocessed/ ht tp : / /ww w.r i t tmanmead. com/2010/03/03/ ins ide-the-oracl e -b i -server -par t 3-b i -server in -memoryj o ins /
Thank you for your time Any questions : Web : http://www.rittmanmead.com Email : mark.rittman@rittmanmead.com Twitter : @markrittman
Inside the Oracle BI Server (10g)

Mark Rittman, Director, Rittman Mead ODTUG Kaleidoscope 2010, Washington D.C.

Inside OBIEE 10g Server

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Inside OBIEE 10g Server

Uploaded by

Copyright:

Available Formats

Inside the Oracle BI Server (10g)

Inside the Oracle BI Server

Why Is This Topic Important?

The Oracle BI EE 10g Server Architecture

The Oracle BI Server 10g Architecture

Intelligent Request Generation and the BI Server Navigator

The BI Server Functional Compensation Model

Generate Optimized Physical SQL(s)

Receiving Incoming Resultsets

Apply any required functions and joins

Return data to user

How Does The BI Server Process Queries?

Step 1 : Request Arrives at Intelligent Request Generator

Step 2 : Request Passed to Logical Request Generation Engine

Step 3 : Check Cache if Enabled

Rules for a Cache Hit

Step 4 : Request Passed to the Navigator

Step 5 : Multi-Pass / Sub-Request Logic

Step 6 : Fragment Optimization

Step 7 : Aggregate Navigation

Step 8 : Optimized Query Rewrite, and pass to Execution Engine

Step 9 : Copy Back to Cache Services, if Enabled

Key Navigator Functionality

Level 5 Query Logging and Logical Execution Plans

The BI Server Source-to-Target Metadata Model

So What Does The BI Server Do, When It Joins Data In Memory?

BI Server Join Type 1 : Fact + Dimension from Single Database

Products Logical Dimension

Sales Logical Fact LTS Mapping

Products Physical Table

Sales Physical Table

Single Source Fact and Dimension Execution Plan & SQL

Single physical SQL with join pushed down to database

BI Server Join Type 2 : Fact + Dimension from Two Databases

Products Physical Table

Products Logical Dimension

Sales Logical Fact LTS Mapping

Sales Physical Table

Multiple Source Fact and Dimension Execution Plan

Multiple Source Fact and Dimension Physical SQL

BI Server Join Type 3 : Fact + Dimension Using Driving Table

Products Logical Dimension

Database #2 Products Physical Table

PNLJ Probe Lookup

Sales Logical Fact LTS Mapping

Sales Physical Table

Driving Table Considerations

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com

Parameterized Nested Loop Join Algorithm & Physical SQL

QUANTITY_S OLD: [DAggr (SALES.QUANTITY S O LD by [ PRO D U CTS.PR O D SUB

se lec t T2502.PR O D_SUB CATE G O R Y as c1 ,

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com

Fact and Dimension Driving Table Execution Plan

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com

BI Server Join Type 4 : Two Facts Using a Single Database

Products Logical Dimension

Sales Logical Fact

Sales Physical Table

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com

Facts with Conformed Dimensions Execution Plan

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com

Facts with Conformed Dimensions Single Physical SQL

Facts with Conformed Dimensions Multiple Physical SQL

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com