BCO Best Practice Webinars - Designing The BCO Data Integration Architecture

BMC Capacity Optimization - Best Practice Webinar
Designing the BCO data integration architecture

INTERNATIONAL TOLL FREE
Int'l Toll: 913-312-0828
US/CAN Toll free: 866-816-1982
Greece: 00 800 161 2205 6440

Hong Kong: 800 968 066
Hungary: 06 800 112 82
Argentina: 0800 444 6440

Australia: 1 800 612 415
Austria: 0800 295 780
Bahamas: 1 800 389 0491
Belgium: 0 800 75 636
Brazil: 0800 891 0266
Bulgaria: 00 800 115 1141
Chile: 123 0020 6707
China, Northern Region: 10 800 714 1509
China, Southern Region: 10 800 140 1376
Colombia: 01 800 518 1171
Czech Republic: 800 700 715
Denmark: 80 883 277
Dominican Republic: 1 888 752 0002
France: 0 800 914 176
Germany: 0 800 183 0299
Copyright 12/4/2012 BMC Software, Inc
India: 000 800 1007 613

Indonesia: 001 803 017 6440
Ireland: 1 800 947 415
Israel: 1 80 925 6440
Italy: 800 789 377
Japan: 00348 0040 1009

Latvia: 8000 3523
Lithuania: 8 800 3 09 64
Luxembourg: 800 2 3214
Malaysia: 1 800 814 723
Mexico: 001 800 514 6440
Monaco: 800 39 593
Netherlands: 0 800 022 1465
New Zealand: 0 800 451 520
Norway: 800 138 41

Panama: 00 800 226 6440
Peru: 0800 54 129
Philippines: 1 800 111 010 55
Poland: 00 800 112 41 42
Portugal: 800 827 538
Russia: 810 800 2915 1012
Singapore: 800 101 2320
Slovenia: 0 800 80439
South Africa: 0 800 982 304
South Korea: 003 0813 2344
Spain: 900 937 665
Sweden: 02 079 3266
Switzerland: 0 800 894 821
Taiwan: 00 801 127 186
Thailand: 001 800 156 205 2068
Trinidad and Tobago: 1 800 205 6440
United Kingdom: 0 808 101 7156
Uruguay: 0004 019 0348
Venezuela: 0 800 100 8540
1
Designing the BCO data integration architecture

Speaker:
Giuseppe Nardiello
Sr Mgr BCO Product Management
Panelists:
Sudheer Apte
Marco Colombo
Charlie Geisler
Giorgio Gasparini
Mike Paska
Mike West
Evan Wetstone
December 4th, 2012

Legal Notice
The information contained in this presentation is the confidential

information of BMC, Inc. and is being provided to you with the
express understanding that without the prior written consent of
BMC, customers and partners may not discuss or otherwise disclose
this information to any third party or otherwise make use of this
information for any purpose other than for which BMC intended.
All of the recommendations and information described herein are at
the sole discretion of BMC and are subject to change and/or
cancellation, and in no way should this content be viewed as
guarantees or warrantees on BMCs part.

Topic: Designing the BCO data integration architecture
We will discuss topics such as:
How do I select the right data source for managing a specific
environment? Do I need to deploy new data collection
capabilities or can I rely on already deployed management
tools? How do I design the data integration architecture from
a logical perspective?
We will NOT discuss topics such as:
How to integrate BCO to extract data from external reporting
tools or applications
How do I size the BCO infrastructure based on the number of
systems to manage? How many connectors do I need? How do
I make it scalable? How do I design the BCO infrastructure
from a physical perspective?
How do I configure a connector to a specific data source? How
do I manage historical recovery? How to ensure BCO
resilience?

Agenda
FIRST PART
A (short) introduction to BMC

Capacity Optimization (BCO)
architecture
data integration
Designing data integration in BCO
general criteria
some real-world use cases
Conclusions
SECOND PART
Some of your use cases will be

selected to be discussed please
have them at hand !
BMC Capacity Optimization

High-level architecture
Cloud Admin
Capacity Manager
Storage Admin
IT Operation Manager
Service Manager
IT Financial Manager
technical and business-oriented dashboards & scheduled, on-demand and by-exception reports
Web Interface (OOTB and customizable views)

Analytical and Modeling Engines
BMC Capacity
Optimization
Capacity Database
(CDB)
Reporting Engine
Data Integration Engine
Performance Metrics
Physical/Mainframe/Virtual/Clouds
Business KPIs
Configuration Metrics
Databases & Middleware
Applications
Financial Metrics
Events
Storage
Networks
Data Center Facilities
Facility Metrics
Business Drivers

Connectors
MarketZone Direct Connectors (**)
OOTB Connectors (*)

Specific Connectors
AIX Topas / CEC

BMC Atrium CMDB
BMC Atrium Discovery and Dependency Mapping
(ADDM)
BMC Patrol
BMC Performance Assurance
BMC ProactiveNet Performance Mgmt
BMC Cloud Lifecycle Management
BMC Capacity Management for Mainframe (CMM)
Citrix Xen server
EMC Storage Scope Control Center
nMon for AIX
Microsoft Performance Monitor (perfmon)
Microsoft SCOM 2007
NetApp Data Fabric Manager (DFM)
Oracle Enterprise Manager (OEM)
Solaris Prstat
Solaris Extended Accounting
UNIX OS configuration (AIX, Solaris, )
UNIX CLI tools (DiskFree, IOstat, )
UNIX System Activity Report (SAR)
VMware vCenter
General-purpose Connectors
CSV file
NCSA and custom text log files
SQL queries to RDBMS (e.g. Oracle, IBM DB2
and MS SQL Server)
SNMP interfaces (standard/proprietary MIBs)
HTTP and JMX interfaces
Template Connectors (***)
Custom connectors (SDK)

Custom connectors can be developed from an
Eclipse-based Integrated Development
Environment (IDE) provided as part of BCO SDK
IBM Tivoli Monitoring

HP Reporter
HP uCMDB
MS SCOM DW
Entuity
EMC SMI-S
Hitachi Device Manager
more to come...
CA Unicenter TNG
HP SiteScope
HP Business Availability Center (BAC)
HP Load Runner
HP OVO EPC Agents (Coda Agents)
HP OpenView Performance Agents
HP OpenView Performance Insight
HP OpenView Performance Manager
NetIQ AppManager
Oracle AWR / STATSPACK
SolarWind Orion NPM
TeamQuest Enterprise Database
Zenoss
(*) OOTB connectors are supported by BMC - please verify the availability for the specific platform, version and metrics with BMC representatives
(**) Market Zone Direct (MZD) connectors are supported connectors that require a separate license
(***)Template
are Software,
providedInc
as-is (at no additional cost) and are nor supported by BMC - please read BMC support policies for template connectors
Copyright connectors
12/4/2012 BMC
Movri
Integration for BMC Capacity Optimization
Moviri Integration for BMC Capacity Optimization v3.0

provides supported connectors to:
HP Reporter
ENHANCED
HP Reporter 3.8, 3.9
HP uCMDB
HP uCMDB 9 and 10
HP Reporter

ITM 6.2.1, 6.2.2, 6.2.3
ENHANCED
MS SCOM Data Warehouse
uCMDB
NEW
MS SCOM DW 2007, 2012
BMC
Capacity
Optimization
Entuity
Entuity EYE 2012 and 12.5
Entuity
ENHANCED
MS SCOM DW
Sentry Software
Integration for BMC Capacity Optimization
Sentry Software Integration for BMC Capacity Optimization v2.0

provides supported connectors to
EMC SMI-S provider
EMC Symmetrix DMX Series (DMX-4,
DMX-3, DMX-2)
EMC Symmetrix V-Max Series
EMC CX4 Series, CX3 Series, CX Series
EMC VNX series
EMC
NEW
Hitachi Device Manager

Hitachi Adaptable Modular Storage
2000 family
Hitachi Universal Storage Platform
(USP, USP-V/USP-VM)
Hitachi Virtual Storage Platform (VSP)
NEW
BMC
Capacity
Optimization
Hitachi

Visibility, Planning and Optimization
Technical and business oriented views
Capacity Manager
Storage Admin
BMC
Capacity
Optimization
Database Admin
Service Manager
IT Financial Manager
Scheduled, on-demand and by-exception reports

Operation Manager
10

OOTB Views
DataCenter Overview
AIX PowerVM frame

vSphere infrastructure
vSphere datastore allocation

11
BMC Capacity Optimization for Mainframes

OOTB Views
Mid Short term mainframe capacity summary
z/OS Image Summary View 4 Hour Rolling Average
4 Hour Rolling Average MAX value KPIs
z/OS Image MIPs consumed
12

Agenda
FIRST PART

architecture
data integration
general criteria
Conclusions
SECOND PART

have them at hand !
13

General criteria
When designing the BCO data integration archicturecture, you are required
to evaluate several factors, such as:
- data availability (e.g. integration of already adopted management tool vs
new data collection mechanism to be deployed)
- data quality (e.g. data collection mechanisms guaranteed to return CPU
utilization values <=100% for all managed platforms)
- data richness (e.g. more comprehensive set of metrics or at higher
granularity that can satisfy already known or potential requirements)
- easiness of integration (e.g. access to repository with public data schema vs
parsing of exported reports in complex formats)
- robustness of integration (e.g. parsing of files located on a remote file
system or pushed to a local file system)
- cost of integration (e.g. parsing exported CSV files versus creating a custom
connector to leverage complex APIs)
In general, there is not a right solution as these and other factors need to
be balanced carefully. Also, a one fits all approach may not be suitable and
different design may apply to different environments to be managed.
In the following, we will discuss some general criteria that can be used as a
guidance to design the BCO data integration archicturecture.
14

General Criteria
#1 It is not about data integration it is about Capacity Mgmt
BCO has the ability to virtually integrate any existing data source thanks to
BCO flexible and scalable data integration layer
Therefore, when discovering the existing management environment and
identifying potential data sources to be integrated, you might be tempted to
consider any availble data source
However, only data sources that are would provide useful data to support
Capacity Management use cases should be considered
In general, make sure that you understand how existing or new data sources
are supporting your Capacity Management use cases of interest
Also take into account that your environment can change over time in terms
of available data sources
Best Practices:
Dont integrate data sources that you are not sure how to leverage
Be driven by your Capacity Management use cases
Adopt a stepwise approach perfect knowledge is not always available
15

General Criteria - Real-World Use Case
Case 1: Can we integrate WebSphere Application Server (WAS)?
First of all, integrating WAS data in BCO may mean different use cases:
perform a seasonal analysis on WAS instances (e.g. heap / thread pool size)
execute extrapolation models on applications running on WAS containers
based on technical proxy (e.g. number of servlet calls) of Business KPIs
Depending on which use case is of interest, the available integration options
can be very different (e.g. a custom connector to IBM Tivoli Monitoring for
WAS or a custom connector to JMX provider or more simply parse an
application log file, respectively)
While these use case are valid, do not forget that Capacity Management is
done OS level (e.g. to right size CPU or memory resources) not to identify AS
level configuration
Case 2: Can we integrate BMC EUEM?
Measured response times provided by BMC End-User Experience
Management (EUEM) or similar tools can be collected to provide trending
analysis or to calibrate what-if scenarios but the latter constitutes quite an
advanced use case
16

General Criteria
#2 Horses for courses
BCO can provide support for easily geneating seasonal and trending analyses
and reports on any metric collected from the integrated data sources
However, not all metrics that might be potentially available from a data source
need to be collected unless useful from a Capacity Management perspective
BCO supported connectors usually collect only a subset of all potentially
available metrics, that correspond to typically useful deliverables (e.g. views)
BCO supported connectors usually also provide a way of filtering specific
Datasets of interest (e.g. filtering out NIC-related metrics)
While BCO allows additional custom metrics to be added, existing BCO Data
Sets usually provide a quite comprehensive set of metrics can be expected to
be collected for each managed resource type
Best Practices:
Dont integrate metrics that you are not sure how to leverage
Perform a match & gap analysis before adding custom metrics
17

Case 1: Can we integrate NetBackup metrics?
Again it is important to identify the intended use cases in order to evaluate
whether it might be useful to integrate specific metrics, for example:
job execution times: to execute trend analyses
volumes of backup data: to be correlate to number of clients
While some other metrics may not be integrated as being more performance
oriented or more useful for an administration purpose, as for example:
disk performance, drive performance, ...
real-time library drive status
Case 2: Should I create a new custom metrics for these NetBackup metrics (and
in case for each different backup technology of interest)?
BCO Datasets in #13 WKLGEN (Business Driver Generic Metrics) contains
many (56) predefined metrics that can used to map the previously listed
metrics when used as Business KPIs:
ELAPSED_TIME or BYSET_ELAPSED_TIME
TOTAL_EVENTS or BYSET_EVENTS
18

General Criteria
#3 Not all data sources are created equal
Data sources providing the same metric can still differ in terms of:
Support only for specific platforms (e.g. not for HP IVM)
Support for different time granularities (e.g. 5m or 1h, possibly depending
on how original data collection has been configured)
Support also for by-metrics (e.g. storage utilization by tablespace, or
process-level CPU utilization, by custom workloads or transaction classes)
Availability of historical data (based on configured retention policies, that
usually cannot be changed easily)
Since BCO provides decoupling between data collection and data model for
managed systems, it is possible to have different data sources providing
different set of metrics (provided corresponding connectors are shared)
Best Practices:
Evaluate your specific needs in terms of metrics
Consider using different data sources for different platforms
19

Case 1: Can we leverage BPPM data for AIX workload characterization?
BCO 9.0 can leverage BPPM (9.0) data collection for performing capacity
management activities on AIX platforms (both standalone and partitioned)
However BPPM does not provide support workload characterization (that is
resource utilization breakdown by custom group of processes running on an
LPAR) that is supported by BPA (a BPA agents needs to run on each LPAR)
Case 2: Do I need to use BPA for collecting vSphere data or can I use BCO?
BCO 9.0 collects an extensive set of vSphere metrics, but there are cases
where an higher level of detail is required, for example for accounting or
chargeback purposes
BPA the ability to collect (via agent-less or agent-based data collection
depending on the specific platform) data from inside the VMs (at the virtual
node level) related to transaction classes
Example: banking applications accessed via Citrix terminal servers with SAP
(Linux) servers virtualized with VMware - each Citrix server runs multiple
applications that are accounted individually and also Linux guests run multiple
services (e.g. WAS JVMs) that need to be accounted individually
20

General Criteria
#4 Data sources should not be multiplied beyond necessity
BCO may collect metrics related to a managed system (aka BCO entity) from
diverse data sources - and this is what usually is used to combine configuration
data (e.g. from a CMDB) with system metrics (e.g. from BPPM)
However, notice that BCO does not support multiple datasources for the same
metric for the same systems (e.g. to compare their data quality) BCO has
been designed to supports Capacity Management use cases, including ability
to generate automated forecasting and automated by-exception reporting, to
provide resource utilization visibility also from a service/business perspective
and to provide IT cost visibility (showback/chargeback) that require
Therefore, it is highly discouraged to configure BCO to collect the same set of
metrics from different datasources unless this is done in a test environment,
for a limited time period and only to evaluate the best data sources to
integrate in the production environment
Best Practices:
Evaluate in advance the quality of data sources (in case, in your test env)
Dont collect twice the same metric in your production env
21

Case 1: Can we collect the same VM metrics both from the hypervisor level (e.g.
vSphere or Hyper-V) and from inside the VM?
In general, metrics measured from inside the VM and at the hypervisor level
may not provide the same values - as matter of fact they do not represent
exactly the same metrics
In order to make this distinction clear, metrics collected at the VM level are
referred to a different entity (Virtual Node) that is distinct from the VM entity
associated to vSphere or Hyper-V hosts and clusters
Case 2: Can we collect the same metrics from two data sources (in our test
environment) whitout creating custom shadow metrics?
The simplest approach consists in not sharing the connectors extracting data
from the different data sources so that managed entities are populated as
completely separated entities
Notice that also with this appoach, the complexity of the BCO environment
(and the size of the database) increases for example, entities of the same
type can be counted twice and be visible under different names
22

General criteria
#5 Keep it simple
When multiple equivalent BCO data integration options are available, always
balance the advantage of the most complex and complete possible integration
with the benefits of a simpler while still robust integration approach
BCO provides several mechanisms that can ensure an automatic and reliable
data integration:
last counter mechanism: retrieve only new data or data that were missed
in a previous run (also used to retrieve historical data)
done mechanism: parse only new files not already parsed in previous runs
Better control of connector execution, support for migration from test to
production environments and for database cleanup is being introduced in the
upcoming Service Pack
Best Practices:
Perform a cost / benefit analysis
Consider also how your environment may change over time
23

General criteria
#5 Keep it simple
Case 1: How can we integrate network metrics from a Network Management solution
that does not provide any interface that can be leveraged?
This is a real case that was solved by having reports scheduled on a regular base from
the Network Management solution, so as to allow BCO to automatically parse all
reports files via a custom connector (few lines of code)
In some cases it is required to keep the integration simple as there are no other way
of integrating closed solutions
Case 2: How can we integrate storage management metrics from a Storage
Management solution that had a complex API (WBEM) for extracting data?
This is a real case that was solved by having reports scheduled on a regular base from
the Storage Management solution, so as to allow BCO to automatically parse all
reports files via a custom connector (few lines of code)
Even when some different options are available, do not understimate how effective a
simple integration approach can be at least as a first integration step that allows to
quickly demonstrate BCO value
As a matter of fact, as high as 80% of data integrations can be easily done via general
purpose connectors or via simple parsing
24

General criteria - summary
#5 Keep it simple
but most and foremost
25

From criterias to a decision tree
PHASE 1:
use cases
Identify the use cases of interest
Identify the required metrics
PHASE 2:
metrics and
data sources
Identify the potential datasources

Identify potential integration
mechanisms and interfaces
Is there an OOTB or a MZD connector?
Is a general-purpose connector viable?
PHASE 3:
connectors
Is there a template/community connector?

/
create a custom connector
26

Agenda
FIRST PART

architecture
data integration
general criteria
Conclusions
SECOND PART

have them at hand !
27

Your cases
Question
Answer
28

Upcoming Webinars
Thanks
for attending
and
... completing the survey!
January 8th
Managing BPA to BCO data
integration
February 5th
Deploying and configuring
BCO-CLM integration
March 5th
BCO for Storage Capacity
Management
29
Learn more at www.bmc.com

30

BCO Best Practice Webinars - Designing The BCO Data Integration Architecture

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BCO Best Practice Webinars - Designing The BCO Data Integration Architecture

Uploaded by

Copyright:

Available Formats

BMC Capacity Optimization - Best Practice Webinar

Designing the BCO data integration architecture

Greece: 00 800 161 2205 6440

Argentina: 0800 444 6440

Copyright 12/4/2012 BMC Software, Inc

India: 000 800 1007 613

Japan: 00348 0040 1009

Norway: 800 138 41

BMC Capacity Optimization - Best Practice Webinar

Designing the BCO data integration architecture

December 4th, 2012

BMC Capacity Optimization - Best Practice Webinar

The information contained in this presentation is the confidential

Copyright 12/4/2012 BMC Software, Inc

BMC Capacity Optimization - Best Practice Webinar

BMC Capacity Optimization - Best Practice Webinar

A (short) introduction to BMC

Designing data integration in BCO

Some of your use cases will be

Copyright 12/4/2012 BMC Software, Inc

BMC Capacity Optimization

Web Interface (OOTB and customizable views)

Data Integration Engine

Copyright 12/4/2012 BMC Software, Inc

Databases & Middleware

Data Center Facilities

BMC Capacity Optimization

OOTB Connectors (*)

AIX Topas / CEC

Template Connectors (***)

Custom connectors (SDK)

IBM Tivoli Monitoring

Moviri Integration for BMC Capacity Optimization v3.0

HP Reporter 3.8, 3.9

IBM Tivoli Monitoring

MS SCOM Data Warehouse

MS SCOM DW 2007, 2012

Copyright 12/4/2012 BMC Software, Inc

Sentry Software Integration for BMC Capacity Optimization v2.0

Hitachi Device Manager

Copyright 12/4/2012 BMC Software, Inc

BMC Capacity Optimization

Scheduled, on-demand and by-exception reports

BMC Capacity Optimization

AIX PowerVM frame

vSphere datastore allocation

BMC Capacity Optimization for Mainframes

Mid Short term mainframe capacity summary

z/OS Image Summary View 4 Hour Rolling Average

4 Hour Rolling Average MAX value KPIs

z/OS Image MIPs consumed

Copyright 12/4/2012 BMC Software, Inc

BMC Capacity Optimization - Best Practice Webinar

A (short) introduction to BMC

Designing data integration in BCO

Some of your use cases will be

Copyright 12/4/2012 BMC Software, Inc

BMC Capacity Optimization

BMC Capacity Optimization

BMC Capacity Optimization

BMC Capacity Optimization

BMC Capacity Optimization

Copyright 12/4/2012 BMC Software, Inc