Advanced Database Concepts

4/29/12
605.786 Enterprise System Design and Implementation http://aplcenmp.apl.jhu.edu/Notes/PriyaCherry/605.786/
Topics
Enterprise Design and Implementation
!! Performance Tuning !! Security Features !! Database Utilities !! Other Database Concepts !! Non-Relational Approaches
Advanced Database Concepts
Database
!! Critical layer of the overall system !! Large percentage of response time is spent
collecting data !! Needs to be designed to fit the needs of the calling applications
!! Online !! Data
Performance Tuning
Transaction Processing (OLTP) Warehouse
!! Drive data model based on application use cases
Performance Tuning
!! Query Optimizations !! Explain Plans
!! Hints
Explain Plan
!! Oracle feature which provides the 'route' the query
optimizer is planning on taking to retrieve the data

!! Which !! What
!! Materialized Views !! Data Marts !! Partitioning
tables get accessed, in which order indexes are being used, if any
!! Best Guess does not actually execute the
query
!! In
rare cases, plan may not be the actual one used at execution time
!! May need to work with DBAs to setup and obtain
access to underlying tables to use this feature

5 6
4/29/12
Explain Plan SQLPlus

SQL> explain plan for (<sql statement>); SQL> select plan_table_output from table(dbms_xplan.display());
Explain Plan SQL Developer

Explain plan button
Explain Plan Using Indexes
Query Timing
!! Oracle utility that provides the total time it takes to
execute a query and return results
Single lookup
Full index scan

9 10
AUTOTRACE
!! Oracle utility that provides the same information as
AUTOTRACE SQLPlus
With timing option
Explain Plan as well as execution statistics

!! Unlike
Explain Plan, Auto Trace executes the query in question and displays the plan the optimizer actually chose !! Additional statistics can be used to determine in-memory reads vs. reads from disk
!! Can be used in conjunction with 'timing' !! May need to work with DBAs to setup and obtain
access to underlying tables to use this feature

11 12
Query statistics
4/29/12
AUTOTRACE SQL Developer

AUTOTRACE button
Hints
!! Ways to influence the optimizer to use or not use
specified routes for query execution

!! Included as part of a SQL Statement !! Optimizer may choose to ignore hint !! Commonly used hints
!! INDEX !! NO_INDEX !! FIRST_ROWS !! ALL_ROWS
13
14
Hints
Data Mart
!! A copy of data taken from one or more source tables and
stored in a single table structure

!!
Used for fast retrieval of data
!! Used for commonly joined data between two or more

By default, the optimizer uses the index
tables
!! Can store subset of activity
!!
Last 90 days of bank transactions A copy of a single table Made up of columns from various tables A materialized view
!! Can be
!!
Adding a hint tells the optimizer to ignore the index and perform a full table scan
!! !!
15
16
Materialized Views
!! Backed by a concrete table, a data structure that
Materialized View
contains its own copy of qualifying data resulting from a query !! Used to organize data that normally takes a long time to query
!! May !! Can
contain data from multiple tables be indexed
!! Can be setup to refresh at defined intervals !! Data may become stale if source tables are modified between refresh intervals
17 18
4/29/12
Data Warehousing
!! Designed for fewer users running long, process-
Partitioning
!! Allows tables and indexes to be subdivided into
intensive queries with little to no updates !! Contains archived, less volatile data !! Separate database instance
!! Optimized !! More
for long running queries memory allocation
smaller pieces enabling these database objects to be managed and accessed at a finer level of granularity !! Used for very large databases (VLDBs) !! Enables 'partition pruning'
!! Queries
!! Enables high volume OLTP systems to continue to
will only run against applicable partitions
perform without degrading performance due to excessive warehouse-type processing

19 20
!! Can partition by: !! Ranges (dates, alphabetic)

!! Lists !! Custom
(states, countries) Hash Algorithm
Security Features
Security Features
!! Users and Roles !! Setting the context !! Views !! Fine Grained Access Control !! Virtual Private Databases (VPD)
!! Oracle
Label Security (OLS)
21
22
Users
!! Created in the database with CREATE USER
Privileges
!! Granted to users using GRANT statement !! Revoked from users using REVOKE statement !! Two types
!! System !! Object
statement !! Automatically creates a schema

!! Container
for all objects created by the user (tables, views, indexes, etc)
!! Newly created user has no privileges, including
Privileges Privileges
being able to connect to the database

CREATE USER PRIYA IDENTIFIED BY ORACLE;
username
23
password
24
4/29/12
System Privileges
!! Allow the user to perform system level activities !! Connect to the database (create session)
!! Create
System Privileges
!! By default, scoped to user's schema can be
granted across schemas using ANY keyword

GRANT SELECT ANY TABLE TO PRIYA;
objects (create tables, alter tables, create views,
etc.)
!! Perform
DBA activities (backup the database) !! Alter configuration parameters

GRANT CREATE SESSION, CREATE TABLE, CREATE VIEW, CREATE INDEX, CREATE VIEW, CREATE SEQUENCE username TO PRIYA;
25
!! Can also allow the grantee to grant the same
privileges to other users

GRANT SELECT ANY TABLE TO PRIYA WITH ADMIN OPTION;
system privileges
26
Object Privileges
!! Allow the user to perform specific activities on an
Object Privileges
!! Grant query privileges on table
GRANT SELECT ON ACCOUNT TO PRIYA, MATT;
object
!! Insert
data into a table !! Update data in a table or view !! Delete data from a table or view !! Select from a table, view, or sequence !! Alter a table or view !! Execute a procedure
!! Owners have all privileges on their objects
!! Grant privileges to update specific columns

GRANT UPDATE(ADDRESS, PHONE_NUMBER) ON ACCOUNT_OWNERS TO PRIYA, MATT;
!! Grant user authority to pass along privileges

GRANT UPDATE(ADDRESS, PHONE_NUMBER) ON ACCOUNT_OWNERS TO PRIYA WITH GRANT OPTION;
28
27
Roles
Collection of system and object privileges Can encapsulate other roles as well !! User can be assigned several roles as well as a combination of roles and direct privileges !! Simplifies access control administration
!! !!
Roles
1.! Create a role
CREATE ROLE LIBRARIAN_ROLE;
2.! Grant privileges to a role

GRANT CONNECT, SELECT ANY TABLE TO LIBRARIAN_ROLE; GRANT INSERT, UPDATE ON PATRON_ACCOUNTS TO LIBRARIAN ROLE;
3.! Grant role to users

GRANT LIBRARIAN_ROLE TO PRIYA;
29 30
4/29/12
ORA-00942
!! "Table or View does not exist" !! Can occur for multiple reasons !! Table or view actually does not exist
!! User
Connection Pool Configuration Strategy

accounts
!! Application end-users do not have database !! Connection Pool uses single account to connect to
attempting to access table or view does not have privileges on the table or view !! Can also be thrown if schema name is not prefixed to table or view name
!!
the database
!! Associate end-users to roles !! LDAP !! Database Table
By default, Oracle only looks in the current user's (connecting user) schema
31
32
Interacting with Middle-Tier

!! Connection pools are configured with proxy user
Users and Roles

!! Create a user to 'own' the application database
accounts !! Database needs to know who the end-user is and what their access permissions are in order to audit and perform access control !! Call stored procedure to initialize database context before returning connection to the application
objects [Application Owner] !! Create a user to 'connect' on behalf of the application [Proxy User]
!! Should !! Has
not have any direct privileges to owner-objects synonyms for visibility into owner-objects
!! Create roles as appropriate for application
functionality
!! Grant
necessary privileges on owner-objects to each respective role
33
34
Connecting End-Users and Roles

!! Application end-users do not have database
Interacting with Middle-Tier

!! Database access use connection pools that are
accounts !! Associate end-users to roles

!! LDAP !! Database
configured with proxy user accounts

!! Database needs to know who the end-user is and
Table
what their access permissions are in order to audit and perform access control !! Call stored procedure to initialize database context before returning connection to the application
35
36
4/29/12
Setting the Database Context

CREATE OR REPLACE PROCEDURE initialize_connection( p_username varchar2) IS v_role_count NUMBER := 0; BEGIN Initialize DB Session with DBMS_SESSION.SET_IDENTIFIER(p_username); User Identifier FOR role_record IN (SELECT role FROM application_users WHERE username = p_username) Set the LOOP appropriate roles for user DBMS_SESSION.SET_ROLE(role_record.role); v_role_count := v_role_count + 1; END LOOP; IF v_role_count = 0 THEN RAISE_APPLICATION_ERROR(-20101, 'ERROR: Invalid User'); END IF; END; Throw error if user doesn't have any assigned roles /
37 38
Obtaining a Connection
public Connection getConnection(String username) { Connection connection = myDataSource.getConnection(); initializeConnection(connection, userName); return connection; } private void initializeConnection (Connection connection, String username) { // call the pl/sql package which sets the users db context CallableStatement procedure = connection.prepareCall( "{CALL INITIALIZE_CONNECTION(?)}"); // set the db context with username procedure.setString(1, username); procedure.executeQuery(); }
Cleaning up the Database Context

CREATE OR REPLACE PROCEDURE cleanup_connection( p_username varchar2) IS BEGIN DBMS_SESSION.SET_IDENTIFIER(''); --Removes all roles from session DBMS_SESSION.SET_ROLE('NONE'); END; /
Cleanup Connection
public Connection cleanupDBResources(Connection connection, Statement statement, ResultSet rs) throws SQLException { if(rs != null){ rs.close(); } if(statement != null){ statement.close(); } if(connection != null){ //call the package which cleans the users db context try{ CallableStatement procedure = connection.prepareCall( "{CALL CLEANUP_CONNECTION()}"); procedure.executeQuery(); }finally{ procedure.close(); } connection.close(); }
40
39
Views
!! Stored query accessible as a virtual table
Views
composed of the result set of that query

!! Different from materialized views !! Results generated dynamically every time it is accessed !! Does NOT provide any performance gains
!! May
decrease performance!
!! Can be used to limit access to data in underlying
tables
!! Can also be used to simplify repeated queries that
require complicated joins

41 42
4/29/12
Fine Grain Access Control - VPD

!! Create a policy which is fired when a specified
Virtual Private Database
operation (Select, Insert, Update, Delete) is performed by the user on the specified object !! Any user statement accessing a table that has a security policy associated with it causes the server to modify the SQL statement by appending a where clause predicate !! The appended where clause is dynamically generated by the function associated with the security policy
43 44
VPD Add Row-Level Policy

DBMS_RLS.ADD_POLICY( POLICY_NAME => 'PROJECT_MANAGER_POLICY', STATEMENT_TYPES => 'SELECT,UPDATE' OBJECT_SCHEMA => 'MYAPP', OBJECT_NAME => 'PROJECTS', FUNCTION_SCHEMA => 'MYAPP', POLICY_FUNCTION => 'GET_PROJECTS_PREDICATE', );
VPD Policy Function

CREATE OR REPLACE FUNCTION GET_PROJECTS_PREDICATE( p_schema IN VARCHAR2, p_object IN VARCHAR2) RETURN VARCHAR2 IS v_dept_id NUMBER := 0; BEGIN SELECT dept_id INTO v_dept_id FROM DEPARTMENTS WHERE PROJECT_MANAGER = SYS_CONTEXT('userenv', 'client_identifier'); RETURN 'DEPT_ID = ' || v_dept_id; END;
45
46
VPD Row Level Policy Example

SELECT * FROM PROJECTS;
VPD Add Column Level Policy

DBMS_RLS.ADD_POLICY( POLICY_NAME => 'PROJECT_MANAGER_POLICY', STATEMENT_TYPES => 'SELECT, UPDATE', OBJECT_SCHEMA => 'MYAPP', OBJECT_NAME => 'PROJECTS', FUNCTION_SCHEMA => 'MYAPP', POLICY_FUNCTION => 'GET_PROJECTS_PREDICATE', SEC_RELEVANT_COLS => 'FUNDING', SEC_RELEVANT_COL_OPT => DBMS_RLS.ALL_ROWS );
Predicate is only applied when operations include the FUNDING column. For SELECTs including FUNDING column where user is not allowed access, column value is set to NULL in the results
SELECT * FROM PROJECTS WHERE DEPT_ID = 29;
47
48
4/29/12
Oracle Label Security (OLS)

!! Framework built upon VPD to enforce complex
OLS Components
!! Security labels are defined to identify the access
security rules !! Add-on option to database !! Does not require coding PL/SQL functions !! Used mainly by military and government !! Performance trade-off
control specifications for the data

!! User labels are established that define what
security labels the user has access to

!! For each table that needs to enforce row-level
security, a label column is built and populated with the appropriate security label !! At data access time, the database performs access mediation which determines the set of action the user can perform on the data
50
49
Oracle Label Security (OLS)

row-label is a single column in the table
OLS Labels
!!
A label is made up of 3 dimensions:

1.! Level [REQUIRED]
!! !! !! Defines increasing data sensitivity Unclassified, Classified, Secret, Top Secret Public, Company Confidential, Restricted, Company Highly Restricted Defines the areas to which data access is restricted Sales, Finance, HR, IT Defines the owner of the data. Can be used to restrict data based on an organization s hierarchical structure A sales force regional manager can only access data related to sales generated within his region s districts
2.!
Compartment [OPTIONAL, MULTIPLE]

!! !!
3.!
Group [OPTIONAL, MULTIPLE]

!! !!
!!
51 52
Stored as a NUMBER
OLS Tutorial: http://www.oracle.com/technology/obe/obe10gdb/security/ls/ls.htm
Exporting/Importing Data
Database Utilities
!! Oracle Data Pump !! Formerly known as Export and Import utilities

!! Utility
!! !!
to export/import database objects into another schema/environment

Full database Schema !! Tables
!! With
or without data
!! SQL*Loader !! Oracle tool for parsing and loading bulk text-based external files into the database
53 54
4/29/12
Scheduled Jobs
!! Oracle provides a scheduler engine for executing
jobs at specified intervals

!! DBMS Scheduler !! Can be either simple SQL or can call PL/SQL procedure for more complicated logic
!! Jobs
Other Database Concepts
will execute even if connectivity to the database is not available application servers, 1 database
!! Good option in clustered middle tier environments

!! Multiple
55
56
Other Important Concepts

!! Backup and Recovery !! Have a plan!
!! Incrementals/Full !! Include !! RMAN
Database Design Best Practices

!! !! !! !! !! !! !! !! !!
58
Backups off site storage
Tool used for backing up and recovering Oracle databases !! If used properly, guarantees 100% data recovery up to time of system crash
!!
!! High Availability/Failover (RAC) !! Disaster Recovery (Data Guard)

57
Try to avoid data duplication Try to avoid nullable columns Use surrogate keys Generally, fewer calls to the database are better but not always! Don't violate the design use columns appropriately!! !! Don't insert data that doesn't belong just because it's easy Avoid views unless necessary Don't go index-crazy !! Only keep the ones that get used Use naming conventions Limit wildcard/function searches
What is LDAP?
!! What is LDAP?
LDAP
59
60
Lightweight Directory Access Protocol Application protocol for querying and modifying directory services running over TCP/IP !! Directories vs. Databases: !! Organized hierarchically !! Organized for lookups (searching and reading information) !! Used for higher read-to-write ratio !! Do not support secure transaction mechanisms !! More easily extended !! Support for standards is more important !! Examples: !! Phone Book, Library Card Catalog, Identity Management
!! !!
10
4/29/12
LDAP Architecture
!! LDAP Information Model !! Directory Information Base !! All information held in the directory !! Consists of hierarchically related entries !! Entries !! Attributes !! LDAP Schema Definitions !! Collection of Object class Definitions !! Typically found in the LDAP Server's configuration file
61 62
LDAP Object Classes

!! Used to group information !! Provides the following rules: !! Required attributes !! Optional attributes !! Entry can have multiple object classes !! Attributes for the entry are the union of all the attributes of the associated classes !! Object class inheritance !! Extend attributes of other objectclasses !! Can't override any of the rules !! No multiple inheritance !! All classes extend special class top
!!
Only required attribute is objectclass
LDAP Specification (RFC 2256)
LDAP Client/Server Interaction

!! Client establishes session with server (BIND) !! Hostname/IP and Port Number
!! Security
User ID/Password Encryption/Kerberos !! Anonymous Connection

!! !!
!! Client performs insert, update, search operation

!! Base
DN where to start search or perform insert/update
!! Client ends session (UNBIND) or kills session
(ABANDON)
63 64
References
!! !! !! !! !! !! !! !! !! !! !! !! !! !! !! !!
65
http://myweb.brooklyn.liu.edu/gnarra/database/downloads/Normalization.pdf http://en.wikipedia.org/wiki/Database_normalization http://en.wikipedia.org/wiki/ACID http://articles.techrepublic.com.com/5100-10878_11-1045461.html http://www.oracle.com/technology/obe/obe10gdb/security/vpd/vpd.htm http://science.kennesaw.edu/~mguimara/8080/dbsecurity.ppt#1 http://download.oracle.com/docs/cd/B19306_01/network.102/b14266/toc.htm http://www.oracle.com/global/yu/dogadjaji/otd_bgd/Oracle%20Database %20Security.pdf http://www.oracle.com/technology/obe/obe10gdb/security/ls/ls.htm http://download.oracle.com/owsf_2003/Leftik_40119.ppt http://wendtstud1.hpi.uni-potsdam.de/sysmod-seminar/SS2003/presentations/ gruppe-1/LDAP.pdf http://www.openloop.com/education/classes/sjsu_engr/engr_ms_network/ notes/ldap_preso.ppt#1 http://quark.humbug.org.au/publications/ldap/ldap_tut.html http://www.ieor.berkeley.edu/~goldberg/courses/F04/215/215-OODB.ppt#1 http://www.cse.iitb.ac.in/~laxman/cs617(oops)/project/lec6-OODBMS.pdf http://en.wikipedia.org/wiki/Object_database
11

Advanced Database Concepts

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Advanced Database Concepts

Uploaded by

Copyright:

Available Formats

4/29/12

605.786 Enterprise System Design and Implementation http://aplcenmp.apl.jhu.edu/Notes/PriyaCherry/605.786/

Enterprise Design and Implementation