The General Electric Power Journey:

Oracle E-Business Suite 12.2 on Oracle

Exadata and Exalogic
Gary Gordhamer GE Power
2016 Exadata Virtual Conference

Imagination at work
Agenda

1.  Introduction 4.  What’s not "in the box”

2.  Oracle Design a.  security
a.  Hardware, VM, Storage b.  operations
b.  Network 5.  Working with Oracle
c.  OS + Software a.  EBS 12.2 - cutting edge
3.  The implementation b.  Exalogic team - fast and spunky
a.  Our team c.  Exadata team - tried and true
b.  the timeline / the goal
6.  results / questions
c.  the process (DevOps / agile / etc...)

2
Introduction
Speaker background
Gary Gordhamer
Principal Technologist ERP Infrastructure
Power & Water Technology
GE Power & Water
25 years of IT experience
24 years with Oracle (6.x up to 12c)
Worked in many different industries including healthcare,
manufacturing, utilities, banking, and printing
@ggordham

linkedin.com/in/ggordhamer/

oraontap.blogspot.com
Power EBS journey - 2014 plan
2014 2015 2016
Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Power Generation 400GB 5TB
Parts 250 2,500
Users Users
FS
Distributed Power
Exadata
Dresser 2.5TB 8TB +
1,000 3,000 Exalogic
Aero Users Users
Jenbacher
Renewables 2.5TB 1,000 Users

Goals: ERP reductions 83%

reduction
•  Reduce number of ERP’s in ERPs
•  Move to R12.2
•  Modern technology stack 40 37
•  Highly available, reliable, with performance 28
20
7
*Not shown, VCP, FMW SOA, etc… ‘12 ‘13 ’14E ’15F ’16F
6
Options considered
Option
Internal Cloud
Build development instances on
cloud VMs with shared SAN
storage
Commodity physical hardware
Traditionally built physical hardware
with shared SAN storage
Exa*
Commission new Exalogic and
Exadata racks for Prod / QA ERP
instances
Fewer larger ERPs requires rethinking existing architecture standards
Pros
•  Existing support teams
•  Well known architecture
•  Existing PWT tech stack for ERP
•  Existing support teams
•  Well known architecture
•  Existing PWT tech stack for ERP
•  Engineered for high
performance
•  Speed of implementation
•  Simplified single vendor
support
Cons
•  Can not scale vertically to
support extremely large ERPs
•  Concerns about long term
performance with extremely
large ERPs
•  Complex build, long lead times
•  More complex for DBA team to
support
•  Need to build new
infrastructure support team
7
 Our 2014 project
•  Acquire Exadata + Exalogic system and install hardware by Feb 28th
•  Convert to virtual ExaLogic and install ZFS storage for backup by April
•  Train team, and build out all needed infrastructure support for EBS
production go live by August (Latter moved to October)
•  EBS SIT / UAT testing to be completed at least one month prior to go live

Feb Mar Apr May Jun Jul Aug Sep Oct

Hardware Team Build out all
Install Finish build
Training “services” for
out of all
production
“services” for
ExaLogic LDAP / IDM non-
Virtual integration Deliver QA
production system for Deliver PROD
Reconfiguration hardware SIT / UAT for go live
Collaborate '15 - IOUG3236 8
Power EBS journey - 2016 status
2014 2015 2016
Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Power Generation 4TB 250 Users

Global GL 3TB 100 Users 200 interfaces

Supply Chain 4TB 100 Users Single

Global SOA FMW
Exadata
+
Renewables 2.5TB 1,000 Users Exalogic
Nuclear 4TB 250 Users

Global VCP

Consolidation of our Oracle ERP systems on a single platform, some EBS 12.1, mostly 12.2.
Some new installs, and some platform migrations from Solaris to Linux and Linux to Linux.
Completed In Process / production go live date
9
Oracle Design

Collaborate '15 - IOUG3236 10

GE Experiences with EBS on Exadata + Exalogic
Lots of groups to work with inside of Oracle / some conflicting information
EBS team, Exadata hardware, Exalogic hardware, Database team, OVM team,
EMCC team, etc…
Good engagement from sales and product teams to help with these interactions
Single support tickets, but lots of hand offs in the support process
Some growing pains as we were an early adopter of EBS R12.2
There are
“updates” since
Design bible: 2014 that are in
MOS notes
“Oracle E-Business Suite and Oracle Maximum Availability Architecture Best
Practices, Including a Case Study on Oracle Exalogic and Oracle Exadata
Database Machine” – An Oracle White Paper, April 2014.
System performance has been very good to date (if we set things up right)
11
Our Exa* Architecture
Exadata Exalogic
ZFS Storage •  Virtualized server
Appliance •  Database server
•  Database storage infrastructure
•  NFS based •  Internal ZFS
storage •  Hosts the
database storage for
•  Used for Application
Database, server, Mounts
and infrastructure •  Internal backup
backups for infrastructure

All racks are physically interconnected by InfiniBand – recommended by MAA and product teams!
All systems also have redundant 10Gig Ethernet interfaces to our internal GE Network

Collaborate '15 - IOUG3236 12

Oracle Engineered System for e-Business (EBS)
EBS instance detailed infrastructure components

ExaLogic ExaData ZFS

Each instance will require the integration of components from each of the three
Oracle engineered components.
Each engineered component can host multiple instances with simple tweaks to
Oracle’s design
13
EBS on Exadata + Exalogic MAA design
vServer 1 OEL 5.8 4CPU x 64GB vServer N OEL 5.8 4CPU x 64GB

WebLogic (30GB) WebLogic (30GB)

Admin (2GB) Admin (2GB)
Forms C4ws (4GB) Forms C4ws (4GB)
oacore (8GB) oacore (8GB)
GSM Svc oafm (4GB) GSM Svc oafm (4GB)
(3GB) Forms (2GB) x 4 (3GB) Forms (2GB) x 4
oaea (4GB) oaea (4GB)
WLS WLS
Listener Listener
(1GB) CM + Other 5GB (1GB) CM + Other 5GB

OS + Misc 25GB OS + Misc 25GB

Exalogic ZFS storage Head (NFSv4 shared over IPoIB Default)

ZFS Project ZFS backup / DR

/p01/<inst>/appl - hourly snapshots, keeping 24

- daily snapshots, keeping 15
/p01/<inst>/csf
Replicated off-site every 30 minutes
/p01/<inst>/temp

Production ZFS Storage •  HA built into the

Projects
Exadata + Exalogic
DB Backup Exadata

•  DR provided through
OS Backup

Replicated Dataguard and

off-site
vServer OS
Backup
every hour storage replication

14
Networking, networking, networking
Admin Network 192.168.2.x/22
Very difficult
Client Network 10.200.192.x/22 •  Exalogic has 8 networks
EoIB external-mgmt
IPoIB OVM Mgmt 10.2.10.0/24 •  Exadata has 3 networks
ZFS Backup Exadata
Appliance scamd01 Exalogic OVM control, heartbeat, movement
aiezd01 scand01

scand01cn01 IPoIB virt-admin 172.16.0.0/16

Each Network has it’s own
scand01cn02
scand01cn03
Exalogic Internal interconnect purpose (short list)
scamd01db01
scand01cn04
•  IPoB Default
scand01cn05 IPoIB Admin 10.2.5.0/24
scamd01db02 scand01cn06
Interconnects of Exalogic Control
•  Admin
scamd01db03
scamd01cel04
scand01cn07
scand01cn08
•  Client (Public)
IPoIB Storage 10.2.1.0/24
scamd01cel01
scamd01cel02 OVS Repo mount network

aiezd01sn01
scamd01cel03 Need to make sure traffic is
aiezd01sn02
scamd01cel04
scamd01cel05
always routed over the right
scamd01cel06
scamd01cel07
scand01sn01
IPoIB vserver-shared-storage
network
scand01sn02 172.17.0.0/16
vServer access to ZFS

IPoIB Default 10.1.1.0/22

GRIDLINK, DB Backup, shared EBS mounts
10g Ethernet
InfiniBand 15
The Implementation
The build / design team (four people)

Lead / Architect EBS Expert DMA (Database Infrastructure Build

Machine Administrator) Lead
•  20+ years •  20+ years
•  5+ years •  12+ years
•  Servers •  EBS architecture
•  Exadata install •  Servers
•  Storage •  EBS administration
•  Exadata patching •  Storage
•  Network •  EBS patching
•  Mostly DW •  Network
•  Oracle DB •  EBS custom code
background
•  Oracle EBS (Some) •  EBS upgrades •  GE processes
•  Oracle DB
•  Java •  Oracle DB •  Data Center team
•  Linux OS processes
•  Oracle Tech stack

Lots of help and expertise from our central network, and facilities teams

17
Infrastructure Impact of Exa

ExaData – two ½ racks ZFS Backup Appliance

•  190 passwords •  Four head units
•  5 disk trays
•  72 IP Addresses
•  4 passwords
•  106 OEM targets (excludes databases)
•  16 IP Address
ExaLogic – two ¼ racks Three physical racks at each location lots
•  112 passwords We upgraded of cabling work
1 Exalogic to
•  158 222 IP Addresses ½ rack
280 IP’s
•  6 VLAN’s 306 passwords
•  74 OEM targets (excludes vServers) 184 OEM targets

Lot’s of effort from Corporate IT, Power IT, and Oracle to design and install all this equipment

18
Things we did not do
•  Oracle Traffic Director (OTD) – We continue to use external
hardware load balancers
•  Oracle VM Templates for Exalogic – we made our own
templates (needed to GE’ize them either way)
•  Automated EBS instance cloning (yet)
•  Enable app to DB tier traffic over InfiniBand – still no clear
instructions for this (note AutoConfig leanings)
Note: We also had a lot of back and forth on NFS v4 mounting
options, and managing the number of files in CM log & out
directory. (NFS4 rw,bg,hard,nointr,rsize=131072,wsize=131072)
19
What’s not in the box?
What “services” did we have to build out
•  Setting up admin users
(EMOC, ZFS, local OS)
•  Configure EMOC
•  vServer template with “GE”
customizations
•  Domain Name Services (DNS)
•  LDAP / IDM integration (vs. NIS)
•  SNMP / Mail relay
•  Linux OS RPM updates
•  EBS minimum RPM / OS requirements
•  ExaLogic vServer build process
•  Internal YUM REPO
•  Configuration / usage of ExaBR (scripts)
•  OS Backup scripts for Exadata
•  Exadata GridLink Listener
•  OEM Cloud Control agent install
•  Monitoring template setup in OEMCC
•  ZFS Backup Storage setup
•  ZFS Replication configuration
•  QFSDP management process (patching)
•  New DR processes / instructions
•  Change management process
•  Incident management process
21
Some more on LDAP vs. NIS (is there out of the box security?)
Network Information System (NIS)
•  Pre-installed by oracle on the ADM node of
Exalogic
•  Stores users and passwords in the native files on
the ADM node (E.G. /etc/passwd)
•  All servers have access to view all data (all users,
passwords, etc..)
•  If a file gets messed up (passwd), creates lots of
issues
•  ZFS is pre-configured for this, but vServer
templates still need customization
•  Exadata needs to be updated with this setup
Lightweight Directory Access Protocol (LDAP)
•  You need to provide, not part of delivered system
•  Data is stored in a “database”
•  Access to individual data is controlled by LDAP
server
•  Managed through LDAP tools (less likely to mess
up)
•  ZFS needs to be re-configured to use LDAP
•  Exadata, and vServer templates need to be
updated to add LDAP support / configuration

NFS v4 (part of ZFS) requires a central user directory NIS, AD or LDAP in order to work

22
More Security + DR + DB Connectivity + Platinum Support
Getting people used to security model on Exadata
•  Shared ASM, GRID user vs. dedicated servers per database

New procedures for DR

•  Exa* MAA designs are different than our traditional DR

Cleaning up a lot of MAA setup items in our ERP eco-system

•  SCAN IP vs. Virtual IP (VIP) vs. InfiniBand
•  GRC product using SID vs. SERVICE_NAME
•  FMW SOA using SID vs. GridLink connection
We expect to use Platinum Support / ASR gateway to enhance our uptime, speed response
to issues, and stay updated on patches
(still working networking issues getting everything setup between GE Corporate and Oracle)

2
3
Working with Oracle
Exa* marketing vs. reality
Lots of Exa* features are not fully supported / working
with e-Business Suite (yet)
•  EBS R12.2 IB app to DB connection
•  ExaLogic enhancements for WebLogic
•  Exadata – must use PFILES + Autconfig, can not use SPFILES
This system is
now the •  Exadata Hostname vs. Public network
ultimate power
in the universe! Exa* specific MOS notes for EBS
•  Needed to start on 11g version of Exadata (2014)
We are now on 12c / OEL 6

•  ExaLogic virtual did not have OEL 6 till late 2014

•  Had to push Oracle for up to date notes on patching DB on
Exadata for EBS in 2014.
2015 and 2016 have been much better

Oracle continues suggesting going to newest EBS delta

releases as they come out 25
Challenges
•  Lots of infrastructure knowledge needed
Network, storage, OS, DNS, LDAP, etc…
•  Networking is very complex (lots of help from our corporate network team on
this)
•  Still a pretty limited customer base of Exadata + Exalogic + EBS 12.2
•  Exalogic is still pretty new, few resources available (continue to train internal)
•  Engineered systems are all based on OEL 5 (not 6) – no longer true
•  EBS add-on / bolt-on compatibility with Exa* (GRC + SID + ExaData = bad)
•  Still no published benchmarks / sizing information for EBS 12.2 on Exa*
•  Working across multiple Oracle teams (support, product, sales) that don’t
normally work together

26
Challenges Cont.
•  Oracle EBS teams do not seem to be doing testing / development on Exa*
platform (yet)
•  Platinum support = Look at patching Exadata + Exalogic at least twice a year
(we patch 4 times a year today with N-1)
•  No Single documentation to follow
(lots of papers for each technology / application)
•  Need to verify post Oracle build (ACS) – missing settings or setups on Exadata +
Exaloigc
•  Password Management – you will need a good post-it note, or SAPM solution
(Shared Password Management Utility)
•  EBS 12.2 DR process is longer than EBS 12.1 due to Weblogic layer configuration
•  No updated MAA document for EBS 12.2 (yet)

Lots of MOS notes that contradict each other (or at least collide with each other)
27
Partial references list (60 and counting)
•  Deploying Oracle Traffic Director as a Load Balancer for Oracle E-Business Suite R12 (White paper)
•  Exalogic – Walgreens – Customize EBS vServer Template, Shanna Chan 8/25/2013
•  MOS Doc ID 1616827.1, Managing Concurrent Manager Log and Out Directories
•  Oracle E-Business Suite and Oracle Maximum Availability Architecture Best Practices Including a Case Study on Oracle Exalogic and Oracle
Exadata Database Machine (Oracle White paper Feb 2014)
•  Oracle Data Guard: Disaster Recovery for Oracle Exadata Database Machine (Oracle Maximum Availability Architecture White Paper April
2012)
•  Consolidating Oracle Applications on Exalogic (An Oracle White Paper March, 2012)
•  MOS Doc ID 1133355.1 Migrating an Oracle E­Business Suite Database to Oracle Exadata Database Machine
•  E-Business Suite on Exadata (Oracle Maximum Availability Architecture White Paper October 2012)
•  MOS Doc ID 1590941.1 Oracle VM Templates for Oracle E­Business Suite Deployment Guide, Release 12.2.2
•  Oracle VM Templates for E-Business Suite: Deployment Guide (December 2009)
•  MOS Doc ID 741818.1 Export/import process for 12.0 or 12.1 using 11gR1 or 11gR2
•  MOS Doc ID 438086.1 Application Tier Platform Migration with Oracle E­Business Suite Release 12
•  Deploying Oracle E-Business Suite on Oracle Exalogic Elastic Cloud and Oracle Exadata Database Machine (An Oracle White Paper
September 2012 (Updated June 2013))
•  MOS Doc ID 1546861.1 [Linux OS] System Hung with Large Numbers of Page Allocation Failures with "order:5" on Exadata Environments
•  MOS Scenarios Doc ID 1363188.1 Configuring Exadata I/O Resource Manager for Common
•  MOS Doc ID 762374.1 Step by Step - Configure Direct NFS Client (DNFS) on Linux (11g)
•  MOS Doc ID 560853.1 HTTP Server Is Either Slow Or Stops Responding When Installed On A NFS Mounted Drive
•  Exalogic Backup and Recovery Best Practices (An Oracle White Paper 16 August 2012)
•  MOS Doc ID 1516025.1 How To Configure NIS Master, Slave And Client Configuration In Exalogic Virtual Environment
•  MOS Doc ID 1274318.1 Oracle Sun Database Machine Setup/Configuration Best Practices
•  MOS Doc ID 396009.1 Database Initialization Parameters for Oracle E-Business Suite Release 12

28
Tools, Tools, Tools, Scripting, Scripting, Scripting
•  Enterprise Manager Operations Center (EMOC)
•  Oracle Virtual Machine Manager (OVMM)
•  ZFS Storage appliance BUI / CLI
•  Integrated Lights Out Management (ILOM)
•  Shell scripting
•  ZFS scripting
•  Python (ExaTools)
•  Oracle Enterprise Manager Cloud Control (OEM CC)
•  Exacheck (Both Exadata and Exalogic)
•  ExaTools (Exalogic)
•  ExaBR (Exalogic)

Collaborate '15 - IOUG3236 29

Specific tools issues
•  ExaBR – backup of VM server fails if login banner exists
•  OACore server “warning state” – EBS 12.2.3 AD5 + TXK – due to “Exalogic enhancements
for WebLogic”
•  ExaBR – backup of “Control Stack” causes OEMCC to show “Control Stack” as down
•  ExaBR – (No) swap space issue on vServer causes backup to fail
•  OEMCC – Exadata alerting daily on TNS Listener errors (false errors / ExaData plug-in) –
(There is a patch for this now)
•  EBS server crash "WARNING: COULD NOT OBTAIN LOCK ON POLLTRACKING.LCK” –
required OS update, which required QFSDP patch for both Exadata + Exalogic
•  IaaS CLI for Exalogic – filled up root volume on Exalogic compute node
•  ExaCheck – plugin for OEM lacks documentation, ORAchk Collection Manager is nice!
•  TCP/IP – vServer to vServer connectivity issue
•  QFSDP patching – continue to plan on outages for EBS

30
 Patching ExaData + ExaLogic together (QFSDP)
Interactions on IB Switch firmware
Exalogic)Patching)task(s) On3line Off3line Exadata)Patching)task(s) On3line Off3line
•  QFSDP in sync across both racks Pro$active+Reboot+Infiniband+switches 0.5 Pro$active+Reboot+Infiniband+switches 0.5
Upgrade+Exalogic+Control+Services 0.5
Upgrade+the+Sun+Network+QDR+InfiniBand+
Patches are “Rolling” Gateway+Switches. 0.75
Upgrade+the+Sun+Datacenter+InfiniBand+ 0.75
•  EBS does not support rolling Upgrade+the+ZFS+Storage+Appliance+(ILOM+
and+Software) 2.5
•  Some infrastructure items done on-line Shut+down+application+services+(EBS)
Upgrade+Compute+Node+#1+and+verify+
0.5
Upgrade+the+DB+node+#1+operating+
0.5

successful 0.3 systems+and+verify+successful 2.0

•  Optimized outage to 12 hours Upgrade+remaining+Compute+Nodes+in+ 1.5 Upgrade+remaining+DB+Nodes+in+Parallel 1.5
quarterly Refresh+the+Oracle+VM+Manager+data+if+
Patch+the+OVMM,+PC1,+and+PC2+vServers
0.2 Upgrade+the+Grid+Infrastructure
0.5 Upgrade+the+databases+in+parallel
3.0
0.0
Run+Exachk 0.5
•  Outage is mainly for OS restarts Patch+VM+Clients+(Security+fixes+and+Bug+ 4.0
Sanity+Checks 0.5 Sanity+Checks 1.0
Becomes a go-forward or go back Startup+Applications+(after+all++Exadata+and+
Exalogic+Off$line+steps+are+complete)
Startup+Applications+(after+all++Exadata+and+
1.0 Exalogic+Off$line+steps+are+complete) 1.0
TOTAL+APPLICATION+OUTAGE+(Largest+ 9.0 TOTAL+APPLICATION+OUTAGE+(Largest+ 9.0
•  apply all patches or roll back all Upgrade+Storage+Cells 18
patches Upgrade+Infiniband+Switches
Run+Exachk
1
1

Very complex overall

31
Patching – option B
“Site A / Site B” model – Logical split up of the Exa* rack hardware

1.  deploy application so it is only on ½ of the logical split

Exadata Exalogic 2.  During patching, shutdown application
3.  Migration application items from “Site A” to “Site B”
4.  Startup application on “Site B”
Compute Node 8
5.  Patch infrastructure items on “Site A”

Site B
DB Node 4 Compute Node 7 6.  Shutdown application again (same time or in the future)
DB Node 3 Compute Node 6 7.  Migrate application back to “Site A” from “Site B”
Compute Node 5 8.  Startup application on “Site A”
9.  Patch “Site B”

Compute Node 4
Works well if applications are “regional” in scope
DB Node 2 Compute Node 3
Site A
(you can affect one install at a time)
DB Node 1 Compute Node 2
There multiple hardware layouts to support this
Compute Node 1
(this is an example)
You will need excess capacity of some level
32
More networking headaches

Which Network would you want your

✔ application to connect to the Exadata over?
•  “Public” = 10 Gig Ethernet (redundant)
•  “Admin” – 1 Gig Ethernet (not redundant)
•  “IB” – 40 Gig (redundant)
We had issues with both Database and
application configuration for proper
✗ connectivity from Exalogic to Exadata.
•  EBS AutoConfig wants REMOTE_LISTENER /
LOCAL_LISTENER set based on DB “HOSTNAME”
✔ •  ExaData default “HOSTNAME” = admin network

✔ •  ExaData best practices is to set

LISTENER_NETWORKS and not use
* Screen shot from my SR to explain to Oracle LOCAL_LISTENER + REMOTE_LISTENER at all
Collaborate '15 - IOUG3236 33
Results / Questions
Where are we at today?
New Oracle PCA rack for development EBS
instances
•  We have 2 EBS instances in build now
Non-productions Exadata / Exalogic
•  Hosts 150 virtual servers on Exalogic
•  About 35 databases on Exadata
•  DR site for production
Production Exadata / Exalogic
•  36 virtual machines on Exalogic
•  10 databases on Exadata
Single support team for Exa* equipment
•  7 person team for OS / hardware
•  2 person team
“Database Machine Administrator” (DMA)
•  4 person team for build of new ERP systems
(we have been doing 1 – 2 builds a week for
a year now)
35
Exa* Benefits
•  Relatively quick to bring basic hardware up (3-ish months)
•  Full Database RAC system + 60TB of storage
•  Fully Virtualized server environment
•  Very high speed private network
•  200TB+ dedicated backup storage with off-site replication
• Everything is certified with EBS 12.2 out of the box
• No custom design of cluster, fail over, no 3rd party vendor interactions
• Infrastructure patch interactions are all pre-accounted for
• Fast, Fast, Fast
(except when it’s not, or we are not using it right)
We have a small, dedicated, and talented team for all this, so that resulted in
fewer people to interact with, and very quick turn around.
This was somewhat forced due to the engineered system design.

36
Some initial performance data
Item Exa* Time VMWare servers
Autoconfig < 2 Minutes
DB Restore (500GB) < 10 Minutes
DB Backup (500GB) < 8 Minutes
ADOP FSClone 1 hour 2 ½ - 3 hours
ADOP Prepare 1 hour 2 hours
ADOP Cutover 1 hour 2 ½ - 3 hours
APP Tier backup (APPL_TOP+) Instantaneous 1+ hours
(ZFS Snapshot) (tar + compress)

37
 EXA* Benefits for Global GL
• Performance improvements in multiple areas
!  Physical deployment
Real savings for
"  Delivery time of new environments cut in half
DBA & project teams
"  Cloning time reduced from 72 hours to 36 hours
!  Application performance
"  Throughput of feeder file processing has increased by 40-50%.
"  Throughput of GL interface extraction has increased by 30-40%
"  Performance of WEB ADI upload has increased by more than 50%
"  Outbound file creation time (such as Taxpro) has reduced by more than 50%
"  Performance of ledger run and closing (month end) programs has increased by 60-70%

•  Some examples:
•  Ledger run reduced from 2 hours 32 minutes to 50 minutes Increased end user
•  Month end closing process reduced from 36 min to 9 minutes analysis windows
•  Opening new period reduced from 2.5 hours to 30 minutes between runs

38
QA Load Test July non Exa vs Oct Exa (% faster shown)

GE Power & Water | 4 December 2013 39

Added Nov 2015

Supply Chain batch processes performance
Prior Performance
Concurrent Request Exa* Production
Production Improvement
Planning Manager 23:57:51 04:45:53 80.12%
PO To IBS Extract Program 22:22:03 00:00:19 99.98%
WIP Job Status Change Program 16:59:15 00:00:20 99.97%
VE WIP Job Status Change Program 15:57:48 00:00:50 99.91%
Generic File Attachment Mailing Program 15:40:26 00:00:33 99.94%
MMF PO to IBS Extract Program 11:13:20 00:00:59 99.85%
Gather Schema Statistics 06:03:58 01:36:56 73.37%
BMA TRS Batch Program 04:51:54 00:00:13 99.93%
BMA TRS FTP Program 04:50:01 00:00:03 99.98%
Purge Concurrent Request and/or Manager Data 02:51:39 00:00:11 99.89%
Request Set Stage 00:01:00 00:00:11 81.67%
VN WIP Move Transaction Error Report 00:00:18 00:00:02 88.89%
Sales Pick Slip Wrapper Program 00:00:07 00:00:00 100.00%
Report Set 00:03:52 00:00:30 87.07%
ES PO Print Wrapper Program 00:03:08 00:01:11 62.23%
ES PO Print Report for Communication 00:00:59 00:00:09 84.75%
ES PO Print Extension Program 00:00:08 00:00:00 100.00%
PO Output for Communication - File Move 00:00:18 00:00:00 100.00%
Generate Invoices after Holds release Collaborate '15 - IOUG3236 00:00:08 00:00:00 100.00% 40

Overall Average 92.88%

Summary
Overall we have had a good experience
"  Since the Exa* stack facilitates a collapsed support team, the design and support is
somewhat simplified.
"  Our team is still getting up to speed on managing hardware end to end
(storage, OS, Virtualization, network, cluster, etc…)
"  The Oracle MAA paper and design is a great starting place
"  We have been able to automate much more of our build since it’s a single group
"  Performance has been good (no performance issues identified)
  The Oracle MAA solution lacks sufficient security and identity controls built in,
much of our time has been integrating these items into the design
  We are not getting full realization of all Exa* benefits yet (EBS 12.2 limitations)
  Lots of mis-understanding / miss-communication on “Full Stack” patching,
“on-line” patching in EBS 12.2
Exa* technology is moving fast.
Is the Oracle EBS team ready for the new technology in Engineered systems?
4
1
Questions
References
•  Oracle E-Business Suite and Oracle Maximum Availability Architecture Best Practices
http://www.oracle.com/technetwork/database/availability/ebs-maa-2158952.pdf

•  Oracle Data Guard: Disaster Recovery for Oracle Exadata Database Machine
http://www.oracle.com/technetwork/database/features/availability/maa-wp-dr-dbm-130065.pdf

•  Backup and Recovery Performance and Best Practices using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine
http://www.oracle.com/technetwork/database/features/availability/maa-wp-dbm-zfs-backup-1593252.pdf

•  Disaster Recovery for Oracle Exalogic Elastic Cloud

http://www.oracle.com/technetwork/database/focus-areas/availability/maa-exalogic-dr-401789.pdf

•  Exalogic Backup and Recovery Best Practices

http://www.oracle.com/technetwork/database/features/availability/maa-exalogic-br-1529241.pdf

•  Architecture Principles and Implementation Practices for Remote Replication Using Oracle ZFS Storage Appliance
http://www.oracle.com/technetwork/server-storage/sun-unified-storage/documentation/zfssa-replication-2014-1-2120969.pdf

Oracle MAA for Exalogic site: http://www.oracle.com/technetwork/database/features/availability/exalogic-maa-1529215.html

Oracle MAA for Exadata site: http://www.oracle.com/technetwork/database/features/availability/exadata-maa-best-practices-155385.html

Oracle MAA for Applications site: http://www.oracle.com/au/products/database/oracle-applications-maa-155388.html

Collaborate '15 - IOUG3236 44

My Oracle Support Notes (MOS)
Note ID! Note Name!
438086.1! Application Tier Platform Migration with Oracle E-Business Suite Release 12!
741818.1! Export_import process for 12.0 or 12.1 using 11gR1 or 11gR2!
1087430.1! Are NFS v4 and automount supported with dNFS (Direct NFS) in 11g!
1244344.1! Exadata Starter Kit!
1269706.1! Best Practices for OLTP on the Sun Oracle Database Machine!
1274318.1! Oracle Sun Database Machine Setup:Configuration Best Practices!
1274322.1! Oracle Sun Database Machine X2-2 X2-8 X3-2 X3-8 and X4-2 High Availability Best Practices!
1314535.1! Exalogic Patch Set Updates (PSU) Master Note!
1330701.1! Oracle E-Business Suite Installation and Upgrade Notes Release 12_12.2_for Linux x86-64!
1330701.1! Oracle E­Business Suite Installation and Upgrade Notes Release 12 (12.2) for Linux x86­64!
1337265.1! Tool for Gathering IO Resource Manager Metrics- metric_iorm.pl!
1357457.1! How to Expand Exadata Compute Node File Systems!
1375670.1! Oracle E-Business Suite Release 12.2 Configuration in a DMZ!
1375769.1! Sharing The Application Tier File System in Oracle E-Business Suite Release 12.2!
1392527.1! Database!Patches!Required!by!Oracle!E5Business!Suite!on!Oracle!Engineered!Systems:!Exadata!Database!Machines!and!SuperClusters!!
1460921.1! Error incorrect password reports when run command su after inputting correct password on exadata server!
1481713.1! NFSv4 mount directories hang on Exalogic Machine!
1481713.1! NFSv4 mount directories hang on Exalogic Machine!
1491855.1! Business Continuity for Oracle E-Business Release 12.2 Using Oracle 11g Physical Standby Database!
1525155.1! 12c or 11g Alerts on ILOM Target With Message Error- Unable to Establish IPMI v2 RMCP session!
1533415.1! How To Configure Oracle Virtual Machine Manager (OVMM) for read-only access by Enterprise Manager Cloud Control!
1580584.1! Setup Listener on Infiniband Network using both SDP and TCP Protocol!
1582091.1 Exalogic Virtual dom0 Memory Recommendations
1968231.1 Configuring Oracle E­Business Suite Release 12.x Using Logical Host Names

Collaborate '15 - IOUG3236 45

My Oracle Support Notes (MOS) Continued
Note ID! Note Name!
1590941.1! Oracle VM Templates for Oracle E-Business Suite Deployment Guide, Release 12.2.2!
1591073.1! Enhanced Jar Signing for Oracle E­Business Suite!
1594223.1! How To Stop and Start the Entire Exalogic Control Stack In An Exalogic EECS v2.0.6.0.0 and later Virtual releases!
1595549.1! Sun Storage 7000 Unified Storage System- How to set up Replication!
1617454.1! ExaWatcher utility on Exadata database servers and storage cells!
1679481.1! Cannot Start:stop Guest Because Guest Status Is Stuck On Stopped On Oracle VM Manager!
17165204.8! Bug 17165204 - Self deadlock while updating HCC compressed tables!
1923481.1! How to Configure PCG On RAC With Scan HOST!
1901242.1! Script to Diagnose adop and Other AD-TXK Issues in Oracle E-Business Suite Release 12.2!
181244.1! Configuring VNC Or XVFB As The X Server For Applications 11i!
352363.1! LTOM - The On-Board Monitor User Guide!
384248.1! Sharing The Application Tier File System in Oracle E-Business Suite Release 12!
396009.1! Database Initialization Parameters for Oracle E-Business Suite Release 12!
415244.1! Error on libdb Library Installing, Patching or Starting OHS 10g on Linux Platforms!
422073.1! How to Configure the Enterprise Management Agent Host Credentials for PAM and LDAP!
560992.1! RedHat and Oracle Linux kernel versions and release strings!
808017.1! Files created using utl_file do not have world read permission!
560853.1! HTTP Server Is Either Slow Or Stops Responding When Installed On A NFS Mounted Drive!
762374.1! Step by Step Configure Direct NFS Client (DNFS) on Linux_11g!
1516025.1! How To Configure NIS Master Slave And Client Configuration In Exalogic Virtual Environment!
1546861.1! System Hung with Large Numbers of Page Allocation Failures on Exadata Environments!
1363188.1! Configuring Exadata IO Resource Manager for Common Scenarios!
1616827.1! Managing Concurrent Manager Log and Out Directories !
1133355.1! Migrating an Oracle E-Business Suite Database to Oracle Exadata Database Machine!

Collaborate '15 - IOUG3236 46

My Oracle Support Notes (MOS) Continued
Note ID! Note Name!
2087231.1! Guidelines!When!Using!ZFS!Storage!in!an!Exadata!Environment!
1274475.1! Oracle!Sun!Database!Machine!X252/X258,!X352/X358!and!X452!Performance!Best!PracTces!
2102784.1! How!To!Free!Up!Disk!Space!on!Exalogic!Nodes!Prior!to!Patching!
1354980.1! Oracle!ZFS!Storage:!FAQ:!Exadata!RMAN!Backup!with!The!Oracle!ZFS!Storage!Appliance!
1996349.1! Oracle!ZFS!Storage!Appliance:!Using!ZFS!DeduplicaTon!!

Collaborate '15 - IOUG3236 47