Informix Architecture

(Informix on Campus Lecture Series)

September 13, 2010

2010 IBM Corporation

Information Management Informix

Agenda

Architecture overview Process model Memory model Storage model

2010 IBM Corporation

Information Management Informix

Architecture Overview

2010 IBM Corporation

Information Management Informix

IDS Server components

Process component One or more processes that do the tasks requested by the database server instance Shared memory component Shared memory segments used to cache table data from disk and maintain and control resources needed by processes Disk component Disk is a storage facility to hold table data and system information necessary to maintain the server
4 2010 IBM Corporation

Information Management Informix

PROCESS MODEL

2010 IBM Corporation

Information Management Informix

Process based database server

Each process receives a time-slice of CPU time, represented by CPU Usage Over Time
Notice that the oninit processes fall in line between other processes (p1, p2, p4, p5). As more users connect, the number of processes increase and the demand for resources does as well.

* Informix database server processes are called oninit

6 2010 IBM Corporation

Information Management Informix

Process based database server cont

Process-based database servers are highly inefficient!
100

CPU Usage
CPU being used CPU idle, not working on process

Can waste 50% or more of CPU resources!!

2010 IBM Corporation

Information Management Informix

Dynamic Scalable Architecture (DSA)

1993 Informix creates Dynamic Scalable Architecture and IDS is born!

Multi-threaded from the core out
Custom libraries, not Posix
Resource efficient Not O/S dependent
Easily ported

Integrated parallelism
Dynamic Scalable

2010 IBM Corporation

Information Management Informix

Dynamic Scalable Architecture (DSA) cont

Processes
Each database server process (oninit) is known as a virtual processor because it schedules and runs its own threads Virtual Processors (VPs) are divided into classes based on functionality
E.g. Writing to logical or physical log, reading data from disk, performing administrative tasks

Threads
Services requests from client application (user threads)
E.g. Select query issued by a client is serviced by an user thread

Accomplish internal tasks like database I/O, logging I/O, page cleaning, administrative functions (internal threads) A thread can run on any VP in its class To run a thread, VP retrieves threads data and environment stack from ready queue and executes request
9 2010 IBM Corporation

Information Management Informix

Dynamic Scalable Architecture (DSA) cont

Provides fan-in and fan-out parallelism
Fan-in
Physical Processor Physical Processor 1

Fan-out
Physical Processor 2 Physical Processor 3

Virtual Processor

Virtual Processor 1

Virtual Processor 2

Virtual Processor 3

10

2010 IBM Corporation

Information Management Informix

DSA - Thread based database server

Each thread receives a chance to run as a part of the oninit process on a physical processor.

As more users connect, the number of processes remains static as each connection represents a request to run a thread, not a process

Notice the bottom axis shows Virtual Processor Usage Over Time. This represents time the thread is running on a pre-existing oninit process

11

2010 IBM Corporation

Information Management Informix

DSA - Thread based database server

Result: almost complete usage of system resources and significantly better scalability!
100

CPU Usage

CPU being used CPU idle, not working on process

12

2010 IBM Corporation

Information Management Informix

DSA Putting it all together

(Two table join)

Write DSA processes tasks concurrently

Time to Process

Sort Join Scan

DSA breaks tasks into subtasks

Serial aka Single threaded

13

Parallel

Parallelized
2010 IBM Corporation

Information Management Informix

DSA - Putting it all together

A single query example: select * from flights order by flightno;
Scan Thread Sort Thread

Disk Scan

14

Introduction to Informix Dynamic Server

2010 IBM Corporation 14

Information Management Informix

DSA Putting it all together

A more complex example using a parallel tree
Sort Threads

Exchange Threads

select count(*) from tickets a, flightleg b where a.flno = b.flno and orig=DFW order by a.class
Exchange Threads

Hash Join Threads

Exchange Threads

Scan Threads (ticket)

Scan Threads (flightleg)

2010 IBM Corporation

15

Information Management Informix

MEMORY MODEL

16

2010 IBM Corporation

Information Management Informix

Shared Memory
O/S feature that allows database server processes to share data by sharing access to pools of memory Reduces disk I/O by caching data from disk Provides the fastest method of inter-process communication Provides communication channels for local client applications that use IPC communication

Virtual processor A memory space

Virtual processor B memory space

Unallocated space

Unallocated space

Private data

Private data

Program text

Shared Memory Segments

Program text

Client Client Data Client

17

2010 IBM Corporation

Information Management Informix

Shared Memory Segments

Resident segment
Contains the buffer pool and other system information
Buffer pool caches data from table

Can be configured to remain resident in main memory Fixed size

Virtual segment
Contains information about the threads and sessions, and the data used by them Could be paged out to disk by the operating system Expandable

Message segment
Holds the message buffers used in client-server communication if shared memory communication is configured
18 2010 IBM Corporation

Information Management Informix

Shared Memory Structure

19

2010 IBM Corporation

Information Management Informix

STORAGE MODEL

20

2010 IBM Corporation

Information Management Informix

Disk Page
The basic unit of storage in a server All database and system information is stored on pages The minimum unit of I/O in a server is a page The size of a page is by default 2KB on most UNIX systems and 4KB on AIX and Windows Page size can also be configured (max of 16KB)

21

2010 IBM Corporation

Information Management Informix

Extents and Tablespaces

Extent
Collection of contiguous pages Space for table is allocated in units of extents

Tablespace
Logical collection of extents Storage for table A table can have multiple tablespaces
22

Tablespace

Extent

2010 IBM Corporation

Information Management Informix

Chunks
Largest unit of contiguous disk dedicated to database server data storage Chunk can be a raw device (characterspecial device), a piece of a raw device, or a UNIX file (cooked file) Maximum size of chunk is 4TB Maximum allowable chunks is 32767

23

2010 IBM Corporation

Information Management Informix

Dbspaces
Logical collection of one or more chunks Can have between 1 and 32767 chunks Maximum allowable dbspaces is 2047

24

2010 IBM Corporation

Information Management Informix

Storage Model
Dbspace

Chunk

Chunk Chunk

Page

Extents

Tablespace

25

2010 IBM Corporation

Information Management Informix

Blobspaces
Special dbspace that stores simple large objects (TEXT and BYTE data) Basic unit of storage in blobspace is a blobpage Blobpage size can be configured as a multiple of the database server page size Database server writes data stored in a blobspace directly to disk (not buffered)

26

2010 IBM Corporation

Information Management Informix

Sbspaces
Special dbspace that stores smart large objects (BLOB and CLOB data) and user defined data Pages in sbspace are called sbpages Sbpage size is same as the database server page size and is not configurable Basic unit of allocation in sbspace is an extent Writes to sbspace are buffered by default
Buffering can be turned off at sbspace creation time

27

2010 IBM Corporation

Information Management Informix

Mirroring
Process of automatically writing same data to two disks Eliminates data loss due to disk failure If one disk fails, the data is still available on the other disk Costs Additional disk space Performance cost of having to write to two locations
Primary Chunk Mirrored Chunk

Writes

28

2010 IBM Corporation

Information Management Informix

Reading and Caching data

When a client issues a query, the table data is read from disk into buffers in shared memory I/O is performed in page units Subsequent access to the same data is from buffer pool Changes made to the data are visible to all database server process

29

2010 IBM Corporation

Information Management Informix

Physical and Logical Logging

Physical logging
Process of storing before-images of pages that are being modified Before-images are stored in physical log Physical log is a collection of contiguous pages on disk Required for recovery purpose in event of a system failure

Logical logging
Process of recording transaction details Transaction records are stored in logical log Logical log is comprised of logical log files, which are collections of contiguous pages on disk Required for transaction rollback and recovery purpose in event of a system failure

30

2010 IBM Corporation

Information Management Informix

Checkpoints and Recovery

Checkpoints
Periodic system event during which all modified buffers are written to disk Establishes a consistent state for database server Information on checkpoint event is recorded in the system pages and logical log (needed for recovery purposes)

Recovery
If a system failure occurs, the database server restarts at that established point Before-images of pages modified since checkpoint are restored from physical log Transactions since checkpoint are then replayed from the logical log to get the server to the consistent state just before the time of failure

31

2010 IBM Corporation

Information Management Informix

Resources
The Online IDS Information Center
http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp

IBM Informix DeveloperWorks Technical Articles

http://www.ibm.com/developerworks/db2/products/informix/index.html

IBM DeveloperWorks IDS Blogs

http://www-128.ibm.com/developerworks/blogs/page/roundrep (IDS

Replication)
http://www-128.ibm.com/developerworks/blogs/page/gbowerman (IDS Application Development) http://www-128.ibm.com/developerworks/blogs/page/idsteam (IDS Experts Blog)

32

2010 IBM Corporation