Teradata Unity Powers The UDA - Andy Sanderson

TERADATA UNITY
Teradata User Group

St. Louis
Andrew Sanderson
Product Marketing Manager
AGENDA
Unity Suite Overview

Unity Product Portfolio
> Director
> Loader
> Data Mover
> Ecosystem Manager
Demo
Questions and Answers
Copyright 2013-2014 by Teradata. All rights reserved.
Multi-System Environment
Requirements
INTEGRATED DATA
WAREHOUSE
DUAL SYSTEM
DATA
LAB
High Availability
INDEPENDENT
DATA MART
Disaster Recovery
DEPENDENT
DATA MART
Workload Isolation
Workload Balancing
Workload Shifting
Service Level Agreement
Management
Teradata UDA
TEST / DEV
DEEP HISTORY
ANALYTICAL
ARCHIVE
Teradata Unity
DIRECTOR
Query routing and SQL-based
database synchronization
DATA MOVER
Intelligent data movement
between UDA systems
LOADER
Scalable and intelligent high
volume data loading
ECOSYSTEM MANAGER
Monitoring data integration
processes and systems
Integrated Portfolio of Products That Turns a Multi-system

Environment into an Orchestrated Analytical Ecosystem
4
Teradata Multi-System Solution Architecture

Teradata Unity Product Portfolio
Monitoring/Control
UNITY ECOSYSTEM MANAGER
Monitoring
Synchronization
SQL Multicast
Data Loads
Administration
Read
Apply
Operational Control
UNITY LOADER
UNITY DIRECTOR
TPT Utilities
Stand-alone Utilities
Queries
DDL changes
SQL data updates
Parser
Table Copy *
Data Dictionary
Users/
Applications/
SQL Utilities
Users/
Applications/
SQL Utilities
Sequencer
Status
Users/
Applications/
SQL Utilities
UNITY
DATA MOVER
E
Schema/Data
Synchronization
System A
C
System B
* This method of data sync can coexist with Unity, but needs to be managed and coordinated outside Unity
5
D
Query Routing
UNITY DIRECTOR
Unity Director
Query Routing
> Controlled query routing for high availability and failover
> Automatic query routing for workload optimization
Data Synchronization
> Enables database (object) and transactional updates across participating systems to
maintain automatic consistency
Auto Resubmit
> Aborted in-flight queries are automatically re-submitted to an alternate system that
can satisfy the query
Integrated with Unity Ecosystem Manager

INTEGRATED DATA
WAREHOUSE
Source
Systems
Streaming
INTEGRATED DATA
WAREHOUSE
Analytics
Tools and
Users
Unity Director Provides Query Management

Reads
Two methods for Read queries
Controlled Routing
Automatic Routing
Great for session and workload

control across systems, especially
when high availability failover is
required
Great when systems contain

different data and/or for fast setup
where session control isnt required
(Named, Preferred)
(Automatic)
Unity Director Provides Query Management

Writes
Three methods for Write operations

> Default, Create Preferred, Create Balanced
> Can be used in combination
Default
(writes)
Writes to all systems

where objects reside
Great for keeping systems
in sync
Create Preferred
(creates)
(creates)
Creates to only one system
Creates to only one system
Subsequent writes to created

objects on just one system, such
as temporary tables
Designated system for

creates alternates among
systems for different
sessions based on a roundrobin algorithm
Prevents wasted resources on

other systems
9
Create Balanced
Multi-System Synchronization
Database Synchronization
Architectural Flexibility
All Platform Family supported

Any number of systems can participate
Databases objects dont have to be
identical (tables, users, view, macros,)
10
Data Synchronization
John 123
82.50
Bill
858
12.50
Jill
545
72.12
Tom 684
64.29
Joe
367
97.24
Bob
826
55.19
John 123
82.50
Bill
858
12.50
Jill
545
72.12
Data Flexibility
Tom 684
64.29
Joe
367
97.24
Bob
826
55.19
Tom 684
64.29
Joe
367
97.24
Bob
826
55.19
Not all tables have to be on all systems

Updates sent to all participating systems
Data updates applied in the order
received
Auto Resubmit
Unity Director will resubmit in-flight queries following a
database restart to an alternate Teradata system which can
satisfy the query
> Automatic feature
No changes required for applications or queries
Removes the burden from application developers
Improves user experience
> Auto Resubmit only available through Unity Director
UNITY
DIRECTOR
11
UNITY LOADER
Unity Loader
High Volume Loading
> Scalable to meet requirements of
data storage volume, load

window, and throughput
requirements
> Intelligently loads data to the
appropriate system(s)
> Validates successful loading to
eliminate DBA interventions
> Supports all load utilities (except
mainframe; on the roadmap)
INTEGRATED DATA
WAREHOUSE
Source
Systems
Bulk
Load
INTEGRATED DATA
WAREHOUSE
> Tightly integrated with Unity
Director to sequence data

updates in the order received
Integrated with Unity

Ecosystem Manager
13
INTEGRATED DATA
WAREHOUSE
Unity Loader
Bulk loads sent to all participating systems; updates are asynchronous
More than two sites are supported with additional Unity Loader and
Director servers
Site 1
UNITY
LOADER
Unity
Loader
Unity
Loader
Site 2
UNITY
LOADER
Unity
Loader
Unity Loader
14
Intelligent Sequencing
Unity Director and Unity Loader are tightly integrated to ensure
ordered writes and consistent read results across systems
Asynchronous
processing
W1
W2
W3
R1
W4
W5
W1
W2
W3
W4
W5
UNITY DIRECTOR
UNITY LOADER
Queries
15
SQL updates
Bulk data loads
Database updates
Trickle loads
Latest Data When

Submitted
Consistent Query
Results
Different Systems
Will Process At
Different Rates
UNITY DATA MOVER
Unity Data Mover

Powerful Data Management
Within The Teradata UDA
Bi-directional data movement
> Teradata-Teradata, Teradata-Aster,
Teradata-Hadoop
Continuous, periodic, or ad-hoc
Viewpoint portlet, command line, or
event driven
> No need to generate utility load scripts
Intelligently selects the most efficient
and appropriate utility
Full or Partial Table copies

Uses existing Teradata utilities
> Supports load operators, ARC, and JDBC
17
Unity Data Mover When To Use

Does not replace Unity Director or Unity Loader;
but works in conjunction with them
> Select the right tools for the right job
Use cases:
> Continuous data synchronization (low data lag required
between systems)
> Periodic / scheduled data movement (higher data lag
acceptable)
Note: Data Mover does not provide a scheduler
> Ad-hoc data movement via command line or GUI (copy

data to another system on demand)
18
Unity Data Mover

Data Mover User Portlet
Graphical point-and-click
data copy
> Select source and target
from dropdown lists

> Select one or more
database objects to copy
> No need to write data
movement scripts
For DBAs and Sys Admins

> Forced direction data
movement
> Complements the
command-line and event
driven data copy options
19
UNITY ECOSYSTEM MANAGER
Unity Ecosystem Manager

It provides an end-to-end approach to meeting application SLAs
through monitoring, administration, and control of single and
multi-system environments
Application 1
Application 2
SOURCE SYSTEMS
USER EXPERIENCE
Application 3
ETL
Servers
Processes
Unity Director
ETL
Servers
Processes
21
Teradata
System
Database
Currency of data
Unity Loader
Teradata
System
Database
Currency of data
BI
Servers
Processes
Data Mover
BI
Servers
Processes
Unity Ecosystem Manager

End-to-End Monitoring and
Control for the Teradata UDA
Relationship and dependency
monitoring across the Teradata UDA
> Teradata, Aster, Hadoop, supporting
servers, processes, tables, applications
Monitors thresholds and discrepancies
Automatic alerts and actions based on

thresholds and events
Enables graceful failover during
planned/unplanned Outages
Integrated with Unity Director, Unity

Loader, Unity Data Mover
22
Unity Ecosystem Manager - UDA

Monitoring
Hardware
Software
Jobs
Tables
Applications
BI Server
Tableau
T1
Load
Server
Raw Data
Informatica
23
Hadoop
Refined Data
Sensor Data Load
Teradata
Aster
UNITY DEMO
Order tracking demo shows how Unity can

provide high availability
Ecosystem Manager
MIRA
SAN DIEGO
UNITY
DIRECTOR
(daunity)
25
ZETAR
ATLANTA
Thank You!
Questions
and Answers
26
Teradata Unity
Enabling Capabilities Aligned To Products
Teradata Unity
Unity
Director
Unity
Loader
Unity Data
Mover
Use of Bulk Load Utilities
Large batch, low latency
Unity
Ecosystem
Manager
Query Routing
Session-based routing
Query-based routing
Data & Database Synchronization

Data model/schema updates
SQL-based transactional updates
Large batch, mid/high latency
Mini-batch, low latency
Mini-batch, mid/high latency
Monitoring/Control
27
Component / Process health
Application readiness
SLA management
Professional Services
Customer Services
SQL Multicast
SQL Multicast used in two ways
> Synchronization (write to all)
> Queries (read from one)
Queries
Synchronization
(reads)
(writes)
Systems kept in sync
Data and database updates sent to

all participating systems
Not all systems have to be the same
Updates can come from users,
applications, or SQL-based utilities
(TPump or TPT Streams)
28
Intelligent query routing
Director determines which systems can

satisfy the query and sends to the one
with the shortest update queue
Read-only queries sent to only one
system (not query federation)
Teradata Unity Integration Points

Sequencing of Events
Unity Director / Unity Loader

Data and Database updates are ordered to ensure that changes occur in the order received; maintaining
integrity across systems
Teradata Unity
User
Unity Director
Unity Loader
Customer
Admin
Unity Data
Mover
Future
Unity Ecosystem
Manager
USE CASE Data changes originating from multiple sources

7:45pm customer calls into call-center
7:46pm customer places new order through web
7:47pm batch load updates order status
7:48pm batch update changes last purchase date
Sequence of transactions maintained across systems
29

Coordination and Message Sharing
Unity Ecosystem Manager / Unity Data Mover

Ecosystem Manager records and compares the results of load jobs from TTU, including Data Mover,
to ensure:
Successful job
completion
Teradata Unity
Compare results
across tables on
different systems
Unity Director
Alert on discrepancies
Unity Loader
Take pre-defined
automated action
1,000 rows
Unity Data
Mover
Admin
Unity Ecosystem
Manager
Discrepancy
Reported
998 rows
USE CASE Automated action on data movement failures

Administrator establishes data validation rules within Ecosystem Manager for select load processes
A Data Mover job is run that extracts 1,000 rows from System A, but only 998 rows successfully
loaded to System B
Ecosystem Manager recognizes the mismatch and alerts on discrepancy
Predefined actions may be taken
Ensures data integrity across systems via Admin defined actions
30

Unity Director / Unity Loader / Unity Ecosystem Manager

These tools are integrated to automatically and intelligently coordinate tasks
Tables under Director
control are known by
Ecosystem Manager
Table validation
requests by Ecosystem
Manager coordinate
with Director to control
table sync status
before issuing
validation directly
w/systems
System and Table
state changes by
either product are
reflected in the other
Teradata Unity
Unity Director
Request
Sync Point
Unity Loader
Sync Point
Unity Data
Mover
Unity Ecosystem
Manager
Sync Point
Admin
USE CASE Table/data validation across systems

DBA initiates data validation of the Inventory table on System A and B through Ecosystem Manager
Ecosystem Manager knows that updates to this table occur through Director and requests a sync point
Director holds new change requests for the table and flushes in transit activity to achieve a sync point
Ecosystem Manager then queries the table on each system to calculate data content metrics
Ecosystem Manager informs Unity Director that is can release the hold on change requests for the table
Seamless, automated coordination of products
31

Unity Director / Unity Loader / Unity Data Mover / Unity Ecosystem Manager
Ecosystem Manager receives, records, and displays all Teradata Unity alerts, providing centralized
ecosystem monitoring and alerting
Teradata Unity
Unity Director
Unity Loader
Unity Data
Mover
Admin
Unity Ecosystem
Manager
USE CASE Viewing Teradata Unity Alerts

Administrator wants to view any issues across the Teradata Unity portfolio of products
Uses the Unity Ecosystem Manager portlets as the central point for all Unity related alerts
Reduces administrator workload and simplifies ecosystem management
32

Shared Infrastructure
Unity Director / Unity Loader / Unity Ecosystem Manager

Shared database repository for events and metadata
Reduced installation
footprint
Teradata Unity
One repository to
maintain
Unity Director
SQL-accessible
ecosystem information
Unity Loader
Unity Data
Mover
Future
Admin
Unity Ecosystem
Manager
USE CASE Customer purchases full Unity portfolio

One database repository is created for all Teradata Unity products to access
This reduces solution complexity and speeds time to value
Reduces solution complexity and speeds time to value
33
Unity Director / Loader Roadmap

Unity Director / Loader 14.10
January 2014
Unity Director /
Loader Features

June 2014
Feature Enhancements
Teradata DB 14.10 Support
Viewpoint 15.00 support
Teradata DB 15.00 certification
Data Center Affinity
UDT Support
LOB Support
Support for Table Depth (by view)
Infrastructure
Recursive SQL support
TMS server with SLES11 only
Viewpoint Enhancements
Unity Loader
(separately
priced)

Target Q4 2014
No JSON support
Workload Management (TASM
integration)
Session routing option
Operational Enhancements
Infrastructure
Swing version supporting the TMS
server on both SLES10 & SLES11
not a committed release
TPT Export
TPT Update
TBD
Legacy Utility Support:
Legacy Utility Support:
o Fastload
o Multiload
o FastExport
Localization
Support Versions
Japanese & Simplified Chinese
TD 13.00 - 14.10
TD 13.00 - 14.10
TD 13.00 - 15.00
The products, features, & functionality described in this document represent Teradatas plans as of the date issued. Those plans may change at any time, without notice, at
Teradatas sole discretion. Teradata is making no commitment to develop or release any of these products, features, or functionality.
34
Unity Data Mover Roadmap

Unity Data Mover
Features
Unity Data Mover 14.11
June 2014
Target June 2014
Considerations - TBD
Product Integration
Product Integration
Product Integration
Aster (TD/Aster and Aster/TD)
Certify with Teradata DB 15.00
Aster to Hadoop
Hadoop (TD/Hadoop and Hadoop/TD)
JSON Support
Hadoop to Aster
Viewpoint 15.00 support
Ease of Use
Ease of Use
Co-residence with Ecosystem

Manager for new installs.
Editing a job via the Command Line

I/F and change priority of a job while
in the queue.
Performance
Support additional protocols
Other Features
Infrastructure
Swing version supporting both

SLES10 & SLES11
Co-residence with Ecosystem

Manager as in-field upgrade
Infrastructure
SLES11 only
Dynamic default stream and session

settings and improved Job start
time.
Availability
Streamline failover scripts and
improved DM failure recovery, and
remote reporting via TVI.
Localization
Support Versions
TD 13.00, 13.10, 14.00, 14.10
TD 13.00, 13.10, 14.00, 14.10, 15.00
TD 13.0, 13.10, 14.0, 14.10, 15.0
Aster 5.00, 5.10, 6.00;
Aster 5.00, 5.10, 6.00;
Aster & HDP TBD
HDP 1.3
HDP 1.3
35
Unity Ecosystem Manager - Roadmap
Feature
Enhancements
Unity Ecosystem
Manager 14.10
Unity Ecosystem
Manager 15.00
Unity Ecosystem
Manager 15.10
Jan 2014
Q2 2014
TBD
Product Integration
Product Integration
Themes
Ecosystem Repository (Phase 1)
Viewpoint 15.0
Ecosystem Repository (Phase 2)
Data Collection Service

Integration
HA Monitoring/Control
Foundation APIs (Phase 2)
HA Monitoring/Control
Foundation APIs (Phase 1)
Table Validation Improvements
Enhanced TPT/OMD
Data Lineage as data moves

through Ecosystem
Aster/Hadoop Monitoring
Integration
Workflow Engine Enhancements

Scheduling
Ease of use
Workflow Configuration Portlets
Ecosystem Explorer Dashboard

o Show Component, Job,
Table and application
metrics
o Filter views based on
relationships
o Full Rewind/Trend Analysis
New Client OS:
Solaris 11
New Client OS:

Z/OS 1.13
Mac OS X (Post GCA)
Windows 8 (Post GCA)
Localization
Database Version
TD 13.00, 13.10. 14.00,

14.10
TD 13.00, 13.10. 14.00,

14.10, 15.00
13.10. 14.00, 14.10, 15.00,

15.xx
36
Unity Data Mover

Data Mover Setup Portlet
Configuration of the Data

Mover portlet
> Set defaults for copying
data, transfer encryption,

and other performance and
security settings
> Set defaults for the number
of load slots, excluded
target systems, and force
direction pairs
37
38

Teradata Unity Powers The UDA - Andy Sanderson

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Teradata Unity Powers The UDA - Andy Sanderson

Uploaded by

Copyright:

Available Formats

TERADATA UNITY

Teradata User Group

Unity Suite Overview

> Ecosystem Manager

Copyright 2013-2014 by Teradata. All rights reserved.

Copyright 2013-2014 by Teradata. All rights reserved.

Integrated Portfolio of Products That Turns a Multi-system

Copyright 2013-2014 by Teradata. All rights reserved.

Teradata Multi-System Solution Architecture

Copyright 2013-2014 by Teradata. All rights reserved.

maintain automatic consistency

can satisfy the query

Integrated with Unity Ecosystem Manager

Copyright 2013-2014 by Teradata. All rights reserved.

Unity Director Provides Query Management

Two methods for Read queries

Great for session and workload

Great when systems contain

Copyright 2013-2014 by Teradata. All rights reserved.

Unity Director Provides Query Management

Three methods for Write operations

Writes to all systems

Creates to only one system

Creates to only one system

Subsequent writes to created

Designated system for

Prevents wasted resources on

Copyright 2013-2014 by Teradata. All rights reserved.

All Platform Family supported

Not all tables have to be on all systems

Copyright 2013-2014 by Teradata. All rights reserved.

> Auto Resubmit only available through Unity Director

Copyright 2013-2014 by Teradata. All rights reserved.

data storage volume, load

> Tightly integrated with Unity

Director to sequence data

Integrated with Unity

Copyright 2013-2014 by Teradata. All rights reserved.

Copyright 2013-2014 by Teradata. All rights reserved.

Bulk data loads

Latest Data When

UNITY DATA MOVER

Unity Data Mover

> No need to generate utility load scripts

Intelligently selects the most efficient

and appropriate utility

Full or Partial Table copies

Copyright 2013-2014 by Teradata. All rights reserved.

Unity Data Mover When To Use

> Ad-hoc data movement via command line or GUI (copy

Copyright 2013-2014 by Teradata. All rights reserved.

Unity Data Mover

from dropdown lists

For DBAs and Sys Admins

Copyright 2013-2014 by Teradata. All rights reserved.

UNITY ECOSYSTEM MANAGER

Unity Ecosystem Manager

Copyright 2013-2014 by Teradata. All rights reserved.

Unity Ecosystem Manager

servers, processes, tables, applications

Monitors thresholds and discrepancies

Automatic alerts and actions based on

Integrated with Unity Director, Unity