IBM Server Hardware Training

Compiled By : Ravinder Singh 5/3/12

What Is Required to Build a Server?

Hardware components
Processing capability

CPU Memory Disk Physical devices and media LAN cards and cables or modems and telephone lines

Storage system

Communication method

Software components
Operating system Applications

5/3/12

Industry Standard 1U, 2way Server

Scalability, Performance and Availability Features Limited by Thermal Envelope

2 PCI slots Optional hot swap power

6 DIMM slots 2 hot swap hard drives

5/3/12

Xtended Design Architecture

2 PCI slots Increase performance with Optional hot swap power Maximize uptime with rear access Optional hot swap power Hot swap fans Maximize uptime with easy
access redundant fans
SA R SL II

optional PCI-Express

hot swap power for easy upgrades

2 PCI slots
re ss IE xp

Optional RSA II SlimLine Preserve full PCI

expandability and improve mgmt with optional integrated systems mgmt controller IBM Director ships standard with every system

Up to 4 hot swap HDDs 2 hot swap hard drives Higher availability with

RAID 5 plus hot spare Additional flexibility with support for SCSI or SATA Better performance with more spindles

Drop down light path Improves in-rack

manageability Improve uptime

PC

8 6 DIMM slots Maximize performance with max memory Maximum investment protection by

upgrading without parts on the floor Higher availability: protect more memory with mirroring and hot-spare Save money by using less expensive 2GB DIMMs to get to maximum memory

5/3/12

IBM eServer xSeries Servers with Simple Swap SATA Drives

Remove front panel and security bar and slide out drive Increased uptime Easy, tool-free upgrades and serviceability

5/3/12

X-Architecture
IA-64 Advanced Clustering
Power

n-way servers Light path Diagnostics xSeries

Scalability Service

Remote Connect
Control

Enterprise Storage

ChipKill Memory PCI-X

Hot Plug I/O

5/3/12

Enterprise X-Architecture
199 7 199 9 200 3

4 way SMP ECC Memory Redundant Power Redundant Cooling Service Processor

8 way SMP Chipkill Memory Active PCI Redundant Power Redundant Cooling Service Processor Systems Management

Xpand on Demand 4-32 way SMP Partitioning XceL4 Server Accelerator Cache Active Memory Remote I/O Chipkill Memory Active PCI-X Redundant Power 5/3/12 Redundant Cooling

X-architecture (continued)
Industry standard
~ xSeries

Enterprise server computing ~ iSeries ~ pSeries ~ zSeries

~ xSeries x440 with RXE100

Industry standard Price/performance Windows NT SCO UNIX Novell

Integrated solutions Reduced complexity Cost of ownership Performance tuning

POWER Scalability focus UNIX High-speed interconnect Cluster management

Availability focus Highly manageable Concurrent diagnostics

Solutions

Scalability

Manageability

5/3/12

X-Architecture Memory Subsystem Increased memory reliability Technology capacity (up to 64GB), the density of memory The increase of memory

on a single DIMM (up to 2GB) and the increase in speed of the memory subsystem has significantly increased the risks of multi-bit memory errors that cannot be corrected by standard ECC memory and results in systems hanging

X-Architecture Memory subsystem features includes:

256GB memory capacity Active Memory High speed (DDR) memory Memory ProteXion Chipkill memory Memory mirroring Hot-add/hot-swap memory Memory eXpansion Technology (MXT)
5/3/12

Types of Memory

Nonparity Parity ECC Double Data Rate memory (DDR) Chipkill Interleaved Memory*

* This is an implementation of the memory, not a type of memory

5/3/12

Types of Memory

Nonparity Parity ECC Double Data Rate memory (DDR) Chipkill Interleaved Memory*

* This is an implementation of the memory, not a type of memory

5/3/12

Standard ECC SDRAM: 64-bit Data Dispersal Example

0 -3 32 35 4 bits

X
4-7 36 39

8-11 40 43

12-15 44 47

16-19 48 51

20-23 52 55

24-27 56 59

28-31 60 63

C0 -C3 C4 C7

5/3/12

Standard ECC SDRAM: 64-bit Data Dispersal Example

0 -3 32 35 4 bits

X
4-7 36 39

8-11 40 43

12-15 44 47

16-19 48 51

20-23 52 55

24-27 56 59

28-31 60 63

C0 -C3 C4 C7

5/3/12

Chipkill Error Recovery

Worst recoverable case is a DRAM device failure.

Four data bits lost, each from different 64-bit segments Four separate checksums correct four single-bit errors

0 8,16, 24 , 32,40,48 ,56

1, 9,17, 1 9 17, 25 , , 33,41,49 ,57

1, 9,17, 25 2 10,18, 26 , 34,42,50,5 8 3, 11,19, 27 35,43,51, 59 4 12, 20, 28 , 36,44,52, 60 5 13, 21 29 , , 37,45,53,6 1 6 14, 22, 30 , 38,46,54,6 2 7 15, 23, 31 , 39,47,55,6 3 C0,C1,C2,C3 C4,C5,C6, C7

5/3/12

System Management Hardware (Service Processors)

5/3/12

Service Processor Introduction

The

service processor is:

A powerful hardware solution for proactive monitoring and management of xSeries servers.
This feature, integrated into the planar on many of the IBM servers, contains its own processor, memory and communications interface IBMs Service Processors include:

Integrated System Management Processor (ISMP), Remote Supervisor Adapter (RSA), Remote Supervisor Adapter II (RSA-II), and Baseboard Management Controller (BMC)

5/3/12

Naming Conventions
Integrated Baseboard Remote

System Management Processor (ISMP) Management Controller (BMC)

Integrated on the system board of the xSeries 255, 360 & 365 Standard on the xSeries 260, 366, 460 and MXE 460

Supervisor Adapter (RSA) Supervisor Adapter II (RSA-II)

Standard on xSeries 360, 440, 445, 450 and 455 Optional on xSeries 255
Remote

Standard on xSeries 365, 460 (Slimline) and MXE 460 (Slimline) Optional on xSeries 260 (Slimline) and 366 (Slimline)

5/3/12

Integrated Service Processors

Integrated

System Management Processor

Service processor for local system management

Stores event log and other system information Accessible via <F1> Setup or <F2> Diagnostics but requires the OS to be stopped

Secondary access via an SP Interconnect bus without stopping the OS but requires configuration and installation ahead of time

Baseboard

Management Controller

IPMI-compliant service processor

Stores event log and other system information Accessible via an ethernet connection to the (shared) eth0 port of the host (default values are known)

Accessible via <F1> Setup or <F2> Diagnostics but requires the OS to be stopped
5/3/12

Remote Supervisor Adapters

RSA,

RSA II and RSA II Slimline

Management independent of server or OS status

Full remote control of hardware and OS (only direct via LAN) PFA and Automatic Server Restart after fail Remote power control LAN, RS-232, and ASM Interconnect remote access Remote flash update User administration and security

Additional features of RSA II

Default states is fixed IP address (192.168.70.125) Dongle provides RS-485 and serial ports Host video is provided by the RSA II RSA II Slimline does not provide host video

Management tool access

IBM Director, Telnet, ANSI terminal, Web browser

5/3/12

RSA Serial Interface

Works

port

with all Service Processors that have a serial

Not available on RSA II Slimline Null modem and dial connections supported

RSA serial logon screen

5/3/12

RSA Web Interface

RSA Web interface

RSA II Web interface

5/3/12

Event Log
If

a failure occurs in the system:

A fault LED on the operator panel card will be illuminated Level 2 Light Path LEDs indicate which subsystem has failed Verify the fault by accessing the event log Take appropriate action based on errors in the event log

5/3/12

Flash Updates
Host,

and RSA/RSA-II have upgradeable BIOS and firmware

Diskette and remote images available via the IBM support web sites Diskettes require physical access to the host Remote flash images are also available by using remote CD or floppy control Remote flash does not require physical access to the host Updates are applied via the RSA/RSA II web interface

1.

2.

Browse for downloaded file Select to Update

5/3/12

SP Functions Comparison
Feature / Function
Monitoring Automatic Server Restart Capture Windows Blue Screens Environmental Monitors Interface with Light-Path Optional Power Source PFA on system components POST, Loader, O/S Timeouts Alerting Alert to pager SMTP Email SNMP Traps SNMP via PPP Management/configuration ANSI-based Management Director-based Management Telnet-based Management Web-based Management Remote BIOS Update Remote Control Remote POST / Diagnostics View Status Logs View Vital Product Data Connectivity 10/100 Ethernet DHCP support DNS support PPP Shared serial support No No No No No Yes (shared) No No No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No No No No No No No <F1> <F2> <F1> <F2> Yes (via SoL) Yes Yes No No No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes** Yes Yes Yes Yes No No No No Yes No No No Yes Yes Yes Yes Yes Yes Yes* No Yes No Yes Yes No Yes No Yes No Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

ISMP

BMC

RSA

RSAII

5/3/12

Accessing the Adaptec SAS/SATA Disk BIOS

5/3/12

SAS/SATA Disk Diagnostics

5/3/12

Diagnostics and Problem Isolation Aids

POST

error messages and error logs Problem isolation tables Light Path diagnostics Diagnostic programs error messages

5/3/12

Identifying Problems with Hardware

The

system error LED is viewable from the front of the server

This gives the first indication of a problem in the system
Activity LED

System Error Led

Release Lever

USB

Location LED Power on Led Power control switch Information LED

5/3/12

Light Path Problem Notification

Light

Path panel is visible from the front of the unit

First and second indication of where the problem is

18 LEDs for system components One remind button to acknowledge sight of fault indicator Additional LEDs are located close to failing components
Remind Button Error LED Scalability (link) LED

5/3/12

Processor Failure
If

a failure occurs, the following should happen:

The failure will be logged in the BMC Log Operator Panel Fault LED will light

Error LED

5/3/12

Scalability Port test

5/3/12

Scalability Port Test

5/3/12

Error logs

POST

The POST error log contains the three most recent error codes and messages that were generated during POST The BMC log and the system-error log contain messages that were generated during POST and all system status messages from the service processor When troubleshooting an error, make sure to clear the BMC log so that you can find current errors more easily

BMC

RSA

II (optional on x260/x366, standard on x460)

Will contain translated BMC errors messages Primary RSAII will contain all critical errors from remote nodes (except from environmental events) in scalability mode 5/3/12

Four methods to access the BMC

BIOS

Configuration/Setup utility System Analysis (DSA)

Dynamic

SMBridge Diagnostics

5/3/12

IBM POST/BIOS F1 Setup Utility BMC Settings

5/3/12

IBM POST/BIOS F1 Setup Utility BMC Settings (contd)

5/3/12

IBM POST/BIOS F1 Setup Utility BMC Settings (contd)

5/3/12

IBM POST/BIOS F1 Setup Utility BMC Log

5/3/12

SMbridge
C:\Program Files\OSA>smbridge -ip 192.168.70.183 -u USERID -p PASSW0RD sel get 1= 2= 3= 4= 5= 6= 7= 8= 9= 10= 11= 12= 13= 14= 15= 2005/04/25 15:12:09, 2005/04/25 15:26:26, 2005/04/25 15:26:26, 2005/04/25 15:26:26, 2005/04/25 15:26:26, 2005/04/25 15:26:27, 2005/04/25 15:26:27, 2005/04/25 15:26:28, 2005/04/25 15:26:28, 2005/04/25 15:26:28, 2005/04/25 15:26:28, 2005/04/25 15:26:29, 2005/04/25 15:26:29, 2005/04/25 15:26:29, 2005/04/25 15:26:30, #95 #80 #82 #84 #86 #193 #131 #113 #156 #158 #145 #44 #134 #161 #163 Drive Slot, Device Inserted/Device Present Fan, Device Inserted/Device Present Fan, Device Inserted/Device Present Fan, Device Inserted/Device Present Fan, Device Inserted/Device Present Temperature, State Deasserted Cable, Device Inserted/Device Present

Power Supply, Presence detected Processor, State Deasserted Processor, State Deasserted Processor, Processor Presence detected Chassis, State Deasserted Cable, Device Inserted/Device Present Add-in Card, Device Inserted/Device Present Add-in Card, Device Removed/Device 5/3/12 Absent

DSA
DSA

is supported by the xSeries 460

Use DSA to gather system information in a single application

5/3/12

BMC System Event log using Diagnostics (F2)

5/3/12

BMC Remote access

The

BMC also supports remote server management

Accessed through the OSA SMBridge management utility:

Command-line interface (IPMI Shell) Serial over LAN

Establish a Serial over LAN (SOL) connection to manage servers from a remote location

5/3/12

IBM x460 Enabling and Configuring Serial over LAN (SOL) and configure the server for Serial over To enable
LAN (SOL) by using the OSA SMBridge management utility program

Update and configure the BIOS code and enable the operating system for an SOL connection.

5/3/12

IBM x460 BMC logs

The

BMC log can be viewed from the Diagnostic programs (F2)

The Diagnostic menus Hardware information Tab select BMC Log Backing up the BMC log can be accomplished USB diskette drive or memory Key Viewing the BMC log can also be accomplished through the web interface of the optional Remote Supervisor Adapter II SlimLine allows all messages to be translated.

5/3/12

IBM x460 Diagnostic BMC Log

5/3/12

IBM x460 Diagnostic BMC Log

5/3/12

IBM x460 Diagnostic BMC Log continued

5/3/12

IBM x460 RSA II Slimline Adapter

5/3/12

IBM x460 RSA ll Slimline Event Log

5/3/12

xSeries Documentation
Problem

Determination Guide (PDG formerly Hardware Maintenance Manual HMM)

Available electronically (Adobe Acrobat PDF format) from the IBM support web site or on the Service Update CDROM Primary support document for diagnostics and troubleshooting

System

documentation (Users guide, installation guide, etc.)

Useful for confirming shipping group contents (missing parts, etc.) and initial customer setup

5/3/12

Server Support Web Site

New

central web site for all products

www.ibm.com/server/support

Navigation bar

Hyperlink to xSeries products

xSeries Device Driver Matrices

New

central web site for all products

http://www-3.ibm.com/pc/support/site.wss/document.do? lndocid=MIGR-4JTS2T

ServerProven Web Site

Reference

site for confirmation of compatibility between xSeries servers and third party devices and software

www.pc.ibm.com/us/compat/index.html

Hyperlink to xSeries products

5/3/12

xRef Web Site

xRef

refers to the xSeries reference sheets

Sheets show all the current marketed solutions and the features of each system The site is located at:

http://www1.ibm.com/servers/eserver/education/cust/xseries/xref.html

5/3/12

RAID 0

Data1 Data4 Data7

72GB

Data2 Data5 Data8

72GB

Data3 Data6 Data9

72GB

RAID 0 Striping without Parity Advantages: Large Drive Array with the most useable space Performance acceleration through data striping Disadvantages: No Redundancy

5/3/12

RAID 1
RAID 1 Disk Mirroring Advantages: Good Read Performance Excellent Data Redundancy Disadvantages: Large Overhead for Redundancy Two Physical Disks Limitation

Data1 Data2 Data3

72GB

Mirror1 Mirror2 Mirror3

72GB

5/3/12

RAID 5
RAID 5 Data Striping with Skewed Parity Advantages: Extremely efficient use of space for redundancy Disadvantages: Lower read and write performance compared to RAID 1

Data1 Data3 Parity

72GB

Data2 Parity Data5

72GB

Parity Data4 Data6

72GB

5/3/12

ServeRAID and Hot Spare Drives Array A

Data1 Data2 Data3 72GB Mirror1 Mirror2 Mirror3 Data1 Data2 Data3 72GB 72GB

RAID 1

Array B
Data1 Data5 Parity 72GB Data2 Parity Data9 Parity Data6 Data10 72GB

RAID 5
Data2 Parity Data9

72GB

Array C
Parity Data6 Data10 72GB Hot Spare

Data1 Data5 Parity 72GB

RAID 5

72GB

5/3/12

ServeRAID and Hot Spare Drives Array A (continued)

Data1 Data2 Data3 Mirror1 Mirror2 Mirror3 Data1 Data2 Data3 72GB Hot Spare

RAID 1

72GB

72GB

Array B
Parity Data6 Data10 72GB

Data2 Parity Data9 72GB

Data1 Data5 Parity 72GB

Data2 Parity Data9

RAID 5
Data2 Parity Data9

72GB

Array C
Parity Data6 Data10 72GB

Data1 Data5 Parity 72GB

RAID 5

72GB

5/3/12

THANK YOU

5/3/12