Professional Documents
Culture Documents
BOOKS
Installation Guide
Server Administration
Guide
Introduction
Topology Management
Configuration Management
Connection Management
Cross-Connect Management
NE Maintenance
NE Software Management
NETSMART 1500
VERSION 5
ISSUE 1, JANUARY 2007
NE Performance Management
NE User Management
TL1 Editor
External Interfaces
Reports
Master Index
All other products or services mentioned in this document are identified by the trademarks, service marks, or product names as
designated by the companies that market those products or services or own those marks. Inquiries concerning such products,
services, or marks should be made directly to those companies.
This document and its contents are provided by Fujitsu Network Communications Inc. for guidance purposes only. This
document is provided as is with no warranties or representations whatsoever, either express or implied, without notice, to make
including without limitation the implied warranties of merchantability and fitness for purpose. Fujitsu Network
Communications Inc. does not warrant or represent that the contents of this document are error free. Furthermore, the contents
of this document are subject to update and change at any time without notice by Fujitsu Network Communications Inc., since it
reserves the right, changes in equipment design or components as progress in engineering methods may warrant. No part of the
contents of this document may be copied, modified, or otherwise reproduced without the express written consent of Fujitsu
Network Communications Inc.
Unpublished work and only distributed under restriction.
Copyright Fujitsu Network Communications Inc. All Rights Reserved.
FNC-1500-0050-110
Issue 1, January 2007
Location of Change
Description of Change
Throughout
Section 2.2.2
Section 2.10.2
Added note that certain Oracle Database Maintenance tasks are no longer
supported. Although those tasks still display in the UNIX terminal view,
the descriptions of the tasks have been removed.
Section 3.23
Section 3.24
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
Also includes:
List of Figures
List of Tables
1
Introduction
1.1
Overview of Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-2
1.2
Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . .
1-3
1.3
Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-3
1.3.1
1.3.2
1-3
1-4
2.1
Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-2
2.2
2-5
2.2.1
2.2.2
2.2.3
2.2.4
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Changing Database Passwords . . . . . . . . . . . . . . .
Verifying Database Passwords . . . . . . . . . . . . . . . .
Resetting Database Passwords to Default . . . . . . . .
2-5
2-6
2-7
2-9
Status Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-10
2.3.1
2.3.2
2.3.3
2.3.4
2-11
2-13
2-14
2-15
2
Oracle Administration
2.3
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
2.4
Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-18
2.4.1
2.4.2
2.4.3
Startup Normal . . . . . . . . . . . . . . . . . . . . . . . . . . .
Startup Archive . . . . . . . . . . . . . . . . . . . . . . . . . . .
Startup by Ending Backup . . . . . . . . . . . . . . . . . . .
2-18
2-19
2-22
Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-23
2.5.1
2.5.2
Shutdown Normal . . . . . . . . . . . . . . . . . . . . . . . . .
Shutdown Abort (Abnormal) . . . . . . . . . . . . . . . . .
2-23
2-24
2.6
2-25
2.7
2-29
2.7.1
2.7.2
2.7.3
2.7.4
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recovering from a Backup . . . . . . . . . . . . . . . . . . .
Restoring Backup of HA on a non-HA System . . .
Restoring Backup of HA on an HA System . . . . . .
2-29
2-30
2-35
2-35
2.8
Oracle Purge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-36
2.9
Alert Scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-37
2.9.1
2.9.2
2.9.3
2.9.4
2-37
2-38
2-39
2.5
Alert Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Check Size of Alert File . . . . . . . . . . . . . . . . . . . . .
Check Size of File Systems . . . . . . . . . . . . . . . . . . .
Check Size of File System Containing Archived
Redo Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Check for Missing Archived Redo Log Files . . . . . .
2-40
2-41
Automated Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-43
2.10.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.10.2 Maintenance Scripts. . . . . . . . . . . . . . . . . . . . . . . .
2-43
2-46
2-59
2.11.1
2.11.2
2.11.3
2.11.4
2.11.5
2.11.6
2.11.7
2-60
2-63
2-63
2-65
2-65
2-66
2-66
2.9.5
2.10
2.11
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
2-66
2-66
Export Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-67
2-68
2-69
3.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-3
3.2
3-7
3.3
3-8
2.12
3
NETSMART 1500
Administration
3.3.1
3.3.2
3.3.3
3.3.4
3.4
Start Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Start Client on the Server . . . . . . . . . . . . . . . . . . . .
Shut Down Server . . . . . . . . . . . . . . . . . . . . . . . . .
Shut Down Client . . . . . . . . . . . . . . . . . . . . . . . . .
3-8
3-9
3-10
3-11
3-12
3.4.1
3.4.2
3-12
3-13
3-14
3-15
3-16
3.5.1
3.5.2
3.5.3
3.5.4
3.5.5
3.5.6
3.5.7
3.5.8
3.5.9
3.5.10
3-17
3-18
3-19
3-19
3-20
3-21
3-22
3-22
3-23
3-24
3.4.3
3.4.4
3.4.5
3.5
FNC-1500-0050-110
Issue 1, January 2007
3-16
iii
Table of Contents
3.5.11
3.5.12
3.5.13
3.5.14
3-25
3-26
3-26
3-27
3.6
3-28
3.7
3-28
3.7.1
3-29
3-34
3-36
3.8
3-37
3.9
3-39
3.10
3-40
3.11
Resync NE Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-41
3.12
3-42
3-42
3-44
3-44
3.13
3-45
3.14
Broadcast Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-46
3-46
3-49
3-50
3-51
3.7.2
3.7.3
3.15
3-48
3-48
3-51
3-52
3-53
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
3.16
3.17
3.18
3-55
3-55
3-56
3-58
3-58
3-59
3-69
3-73
3-73
3-74
3-76
3-77
3-77
3.20
LNE Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-80
3.21
Connection Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-82
3.22
Topology Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-84
3.23
3-88
3.24
3-90
3.25
3-90
4.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-2
4.2
4-3
4.3
Start Up Workstation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-4
4.4
4-4
4.5
4-5
4
UNIX System
Administration
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
4.6
4-5
4.6.1
4.6.2
4.6.3
4-6
4-6
4.7
4-11
4.8
4-13
4.9
4-13
4.10
4-14
4-14
4-16
4-17
4.11.1
4.11.2
4.11.3
4.11.4
4.11.5
4.11.6
4.11.7
System Processes . . . . . . . . . . . . . . . . . . . . . . . . . .
System Messages . . . . . . . . . . . . . . . . . . . . . . . . . .
Disk Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Swap Info. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
System Activities Menu . . . . . . . . . . . . . . . . . . . . .
Cronjobs Menu . . . . . . . . . . . . . . . . . . . . . . . . . . .
Defunct Processes. . . . . . . . . . . . . . . . . . . . . . . . . .
4-18
4-19
4-20
4-20
4-21
4-25
4-26
4-27
4-27
4-28
4-30
4.13
Admintool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-31
4.14
4-31
4.15
4-31
4.11
4.12
4-8
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
5
Network
Administration
5.1
5.2
5.3
5.4
5.5
5.6
FNC-1500-0050-110
Issue 1, January 2007
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-2
5.1.1
5.1.2
Start FTAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Stop FTAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-4
5-4
5-5
5.2.1
5.2.2
OSI Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . .
TCP/IP Connectivity . . . . . . . . . . . . . . . . . . . . . . .
5-5
5-6
5-7
5.3.1
5.3.2
5-8
5-9
OSI Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-11
5.4.1
5.4.2
5.4.3
5.4.4
5.4.5
5.4.6
5.4.7
5.4.8
5-12
5-12
5-13
5-15
5-16
5-16
5-17
5-17
TARP Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-18
5.5.1
5.5.2
5.5.3
5.5.4
5.5.5
5.5.6
5.5.7
5.5.8
5-19
5-19
5-19
5-20
5-20
5-21
5-21
5-21
IP Address Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-22
vii
Table of Contents
6
Service Support
6.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-2
6.2
6-2
6.3
6-3
6.4
Support Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-3
6.5
FTP Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-3
6.6
6-4
6.7
Archive System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-6
6.8
6-7
6.9
System Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-12
6.10
6-12
6.11
6-14
6.11.1
6.11.2
6.11.3
6.11.4
System Information . . . . . . . . . . . . . . . . . . . . . . . .
NETSMART Information . . . . . . . . . . . . . . . . . . .
ORACLE Information . . . . . . . . . . . . . . . . . . . . . .
All Information . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-15
6-16
6-17
6-17
7.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-2
7.2
7-2
7.3
7-5
7.3.1
7.3.2
7-5
7-8
7-13
7
High Availability
Administration
7.4
7.4.1
7.4.2
7-13
7-14
FNC-1500-0050-110
Issue 1, January 2007
Table of Contents
7.5
7-15
7.6
7-18
7.7
7-21
7.8
7-23
7.9
7-24
7.9.1
7.9.2
7-24
7-25
7.10
7-26
7.11
7-27
7.12
HA Quick Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-31
7.12.1
7.12.2
7.12.3
7.12.4
7.12.5
7.12.6
7.12.7
Cluster Startup. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Start in Single-Node Mode . . . . . . . . . . . . . . . . . .
Shutdown Cluster . . . . . . . . . . . . . . . . . . . . . . . . .
Enable Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Disable Failover . . . . . . . . . . . . . . . . . . . . . . . . . . .
Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recovering Crashed Node . . . . . . . . . . . . . . . . . . .
7-31
7-31
7-32
7-32
7-32
7-33
7-33
7-34
7-34
7-35
A.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A-2
A.2
A-2
7.13
A
Server and Client Port
Usage
FNC-1500-0050-110
Issue 1, January 2007
ix
Table of Contents
A.3
A-3
Index
FNC-1500-0050-110
Issue 1, January 2007
List of Figures
LIST OF FIGURES
FIGURE
TITLE
Figure 3-1
Figure 3-2
Figure 3-3
Figure 4-1
Figure 7-1
Figure 7-2
Figure 7-3
Figure 7-4
FNC-1500-0050-110
Issue 1, January 2007
xi
List of Tables
LIST OF TABLES
TABLE
TITLE
Table 1-1
Table 2-1
Table 2-2
Table 2-3
Table 2-4
Table 2-5
Table 2-6
Table 3-1
Table 3-2
Table 3-3
Table 3-4
Table 3-5
Table 4-1
Table 4-2
Table 4-3
Table 4-4
Table 5-1
Table 6-1
Table 7-1
Table 7-2
Table 7-3
Table 7-4
Table 7-5
Table A-1
FNC-1500-0050-110
Issue 1, January 2007
1
CHAPTERS/APPENDIXES
INTRODUCTION
1 Introduction
2 Oracle Administration
3 NETSMART 1500
Administration
NO.
SECTION
4 UNIX System
Administration
1.1
Overview of Practice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-2
5 Network Administration
1.2
Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-3
6 Service Support
1.3
Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-3
PAGE
7 High Availability
Administration
A Server and Client Port Usage
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 1
Introduction
Overview of Practice
1.1
Overview of Practice
This practice provides descriptions and procedures related to server administration.
Information in this practice is organized as follows:
Chapter 1, IntroductionProvides a brief introduction to this book and information
about contacting and using Fujitsu Network Communications Inc. (FNC) technical
support
Chapter 2, Oracle AdministrationDescribes procedures for managing the Oracle
database, including startup, shutdown, recovery, and backup
Chapter 3, NETSMART 1500 AdministrationDescribes procedures for
NETSMART 1500 software, including optional features such as creating schedules
for NE remote backup
Chapter 4, UNIX System AdministrationDescribes the tasks usually performed by a
UNIX system administrator; assumes the user has root privileges
Chapter 5, Network AdministrationDescribes procedures for protocol and interface
settings
Chapter 6, Service SupportDescribes the Service Support menu options available in
in Heartbeat
Chapter 7, High Availability AdministrationDescribes procedures related to systems
configured with high availability (HA)
Appendix A, Server and Client Port UsageLists the ports used in NETSMART 1500
client and server systems
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 1
Introduction
Typographical Conventions
1.2
Typographical Conventions
Visual cues throughout the NETSMART 1500 Element Management System documentation
are used to help you easily locate and interpret information. Typographical conventions
(particular fonts used to indicate a type of word) are listed and explained in Table 1-1 [p. 1-3].
Table 1-1: Typographic Conventions
1.3
Type Style
Used for:
Initial Caps
Bold
Italics
Menu, menu item, button, icon, field, region, or other GUI object
Courier
Text that the user types, text that is displayed as a message on the screen, or a file name
ALL CAPS
< variable>
Technical Support
This section includes the following subsections:
NO.
SUBSECTION
PAGE
1.3.1
1-3
1.3.2
1-4
1.3.1
FNC-1500-0050-110
Issue 1, January 2007
1-3
CHAPTER 1
Introduction
Technical Support
1.3.2
Information Bulletins
Manuals
Training information
Keyword searching
E-mail to FNC
Document downloads
FNC-1500-0050-110
Issue 1, January 2007
2
CHAPTERS/APPENDIXES
ORACLE ADMINISTRATION
1 Introduction
WARNING:
2 Oracle Administration
3 NETSMART 1500
Administration
4 UNIX System
Administration
5 Network Administration
6 Service Support
7 High Availability
Administration
SECTION
PAGE
2.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-2
2.2
2-5
2.3
Status Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-10
2.4
Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-18
2.5
Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-23
2.6
2-25
2.7
2-29
2.8
Oracle Purge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-36
2.9
Alert Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-37
2.10
Automated Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-43
2.11
2-59
2.12
Export Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-67
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Overview
2.1
Overview
The Oracle database is used to store configuration and state information for NEs and for
internal NETSMART 1500 objects.
Database administration uses the Oracle Administration option of the Heartbeat tool. This
option provides the following menu, which is the starting point for most of the procedures
described in this chapter:
ORACLE DATABASE ADMINISTRATION
1. Status Scripts
2. Startup/Shutdown Procedures
3. Backup/Recovery Procedures
4. Alert Scripts
5. Maintenance Scripts
6. Export Procedures
d. Disable Root Access (return to netsmart)
m. Go back to the Main Menu
q. Quit
Table 2-1 [p. 2-2] lists the menu and submenu items available from this menu and shows
where to go for additional information.
Table 2-1: Oracle Database Administration Menu Items
Menu Item
Reference
1. Status Scripts
3. OS Monitor
4. Database Monitor
2. Startup/Shutdown Procedures
1. Startup Normal
2. Startup Archive
3. Shutdown Normal
4. Shutdown Abort
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Overview
Reference
3. Backup/Recovery Procedures
1. Backup
4. Alert Scripts
5. Maintenance Scripts
FNC-1500-0050-110
Issue 1, January 2007
2-3
CHAPTER 2
Oracle Administration
Overview
Reference
6. Export Procedures
2. Export of PM LOGS
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Oracle User Account
2.2
2.2.1
NO.
SUBSECTION
PAGE
2.2.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-5
2.2.2
2-6
2.2.3
2-7
2.2.4
2-9
Overview
An Oracle UNIX user account is required to:
The Oracle UNIX user account information is listed in Table 2-2 [p. 2-5].
Table 2-2: Oracle UNIX User Account Information
User
Shell
Home
Group
oracle
/bin/csh
/opt/NetSmart/oracle
dba
Note: Typical installations use local files rather than Network Information Services (NIS) or NIS+.
This information is needed for configuring the UNIX user account in NIS or NIS+. Provide
this information to the network System Administrator to set up the NIS or NIS+ UNIX user
account.
If the Oracle UNIX user account is set up using local files on the NETSMART 1500 server
instead of NIS or NIS+, the installation sets up the account automatically. No additional
configuration is required.
FNC-1500-0050-110
Issue 1, January 2007
2-5
CHAPTER 2
Oracle Administration
Oracle User Account
2.2.2
Task
cd $MAINT
$>
orapass_chng
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Oracle User Account
The system prompts you to reenter the new password for confirmation and then displays that
the password was changed successfully.
5
2.2.3
Enter q to quit.
Task
cd $MAINT
$>
orapass_chng
FNC-1500-0050-110
Issue 1, January 2007
2-7
CHAPTER 2
Oracle Administration
Oracle User Account
Enter 2.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Oracle User Account
Enter 1.
The system prompts you to enter the password you want to verify and then displays that the
password was verified.
2.2.4
Enter q to quit.
Task
cd $MAINT
$>
orapass_chng
FNC-1500-0050-110
Issue 1, January 2007
2-9
CHAPTER 2
Oracle Administration
Status Scripts
2.3
Status Scripts
This section describes the Oracle database status scripts. It includes the following subsections:
NO.
SUBSECTION
PAGE
2.3.1
2-11
2.3.2
2-13
2.3.3
OS Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-14
2.3.4
Database Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-15
The database status scripts monitor availability and collect statistics on the Oracle database.
Many of these scripts are run daily from the Oracle users crontab. The scripts that run daily
automatically notify you of potential database availability problems though e-mail to the
Oracle user account.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Status Scripts
If needed, the database status scripts can be run manually through Heartbeat.
2.3.1
Step
1
Task
From the common desktop environment (CDE) toolbar, open the Heartbeat menu
OR
From the Command Line enter netsmart heartbeat
HINT:
To determine if any failures occurred, search for IR.Failure in the output file. Report any
failures to NETSMART product support.
FNC-1500-0050-110
Issue 1, January 2007
2-11
CHAPTER 2
Oracle Administration
Status Scripts
auditdetail_tasklog_check
clienthashmap_proplist_check
clienthashmap_check
binfileobject_check
CTPUtilization_NE
CTPUtilization_PNEComponent
ne_customerinfo_check
ne_neuser_check
ne_servicemanager_1_check
ne_servicemanager_2_check
ne_servicemanager_check
NeDatabase_swfile
nesync_ne_check
neuser_actgrp_check
neuser_ne_check
neusr_actgrp_phase2
actgrp_actgrpdetails_check
actgrp_neact_actgrp_check
actgrp_neact_neact_check
pconnection_customerattr_check
pconnection_endpoints_check
pcourseleg_ne_check
pcourseleg_pconnection_check
pcourselegpcrossconnect
PEC_NE_Check
peqptconn_attributes_check
peqptconn_tne_check
playoutleg_ne_check
playoutleg_pconnection_check
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
13:47:31
13:47:31
13:47:31
13:47:32
13:47:32
13:47:32
13:47:33
13:47:33
13:47:33
13:47:34
13:47:34
13:47:34
13:47:34
13:47:35
13:47:35
13:47:35
13:47:36
13:47:36
13:47:36
13:47:36
13:47:37
13:47:37
13:47:37
13:47:38
13:47:38
13:47:38
13:47:39
13:47:39
13:47:39
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Status Scripts
checking
checking
checking
checking
checking
checking
checking
checking
checking
checking
checking
pnec_ne
pnecomponent_do_storage_check
pnecomponent_tne_check
psnc_snccustomerattr_check
resyncdetail_tasklog_check
Swgen_swfile
xcnecheck
topo_check.run_consistency_checks(&m)
pm_15min_check
pm_1day_check
pm_polled_check
:
:
:
:
:
:
:
:
:
:
:
13:47:39
13:47:40
13:47:40
13:47:40
13:47:41
13:47:41
13:47:41
13:47:42
13:47:42
13:47:42
13:47:43
2.3.2
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
2-13
CHAPTER 2
Oracle Administration
Status Scripts
2.3.3
If any error is reported, contact FNC technical support for further assistance.
OS Monitor
This script displays a report on operating system (OS) performance monitoring.
Note: This item is used only by FNC personnel for troubleshooting.
Task
Enter 3 OS Monitor.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Status Scripts
The performance monitoring report displays. The following example shows a system with no
reported problems:
Note: This item is used only by FNC personnel for troubleshooting.
Sun Microsystems Inc.
SunOS 5.10
Generic June 2006
You have new mail.
Performance Monitoring Report for node nvt1 at 06/20/06 11:37:18
Collecting Unix system performance statistics - Please wait 5 seconds...
Free swap space 4878 Mbytes
Free memory 120 Mbytes
CPU user mode 0
CPU system mode 10
CPU idle mode 98
Analyzing disk system performance statistics - Please wait 10 seconds...
The disk subsystem is performing normally...)
Processing Unix cache performance statistics - Please wait 5 seconds...
(The Cache statistics is performing normally...)
Hit any key to Continue.
2.3.4
Database Monitor
This script collects the Oracle database statistics listed in Table 2-3 [p. 2-15]. These statistics
are used by a database administrator to tune database performance.
Table 2-3: Database Monitor Statistics
Name
Description
Ideal Value
90% or higher
2% or lower
85% or higher
Monitors statistics for the three types of redo log latches: redo
allocation, redo copy, redo writing
Not applicable
Extents report
Not applicable
FNC-1500-0050-110
Issue 1, January 2007
2-15
CHAPTER 2
Oracle Administration
Status Scripts
Description
Ideal Value
Tablespace summary
report
System input/output on
physical disks
Lists the number of physical reads and writes for each data file
Not applicable
Rollback segment
details
Not applicable
Rollback shrink/extend
report
Not applicable
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Status Scripts
The script executes. The values of the statistics listed in Table 2-3 [p. 2-15] display on the
screen as shown in the following example:
DBA Report for nms on Tue Jun 11 11:48:04 CDT 2006
****************************************
Buffer Cache Hit Ratio is : 99.6970796
SQL Cache Hit Ratio is : .096648812
Data Dictionary Hit Ratio is : 99.385048
Sortarea Info
=============
Sort (Memory) : 15119 Sort (Disk) : 19 Sort (Rows) : 3559268
Tunebuffer Info
===============
>
SGA CACHE STATISTICS
>
********************
>
SQL Cache Hit rate = 99.9
>
Dict Cache Hit rate = 99.38
>
Buffer Cache Hit rate = 99.7
>
Redo Log space requests = 86
>
>
INIT.ORA SETTING
>
****************
>
Shared Pool Size = 47000000 Bytes
>
DB Block Buffer = 30000 Blocks
>
Log Buffer = 25600000 Bytes
:
:
Note: For a complete listing, see Section 2.3.4, Database Monitor [p. 2-15].
Review the statistics reported. Refer to Table 2-3 [p. 2-15] for detailed information about the
individual statistics. If any of the values in the report are not in the standard operating range,
contact FNC technical support for further assistance.
FNC-1500-0050-110
Issue 1, January 2007
2-17
CHAPTER 2
Oracle Administration
Startup
2.4
Startup
Note: The procedures in this section should be used only if NETSMART 1500 is not running.
This section describes the three ways to start the Oracle database with NETSMART 1500
software not running. It includes the following subsections:
NO.
SUBSECTION
PAGE
2.4.1
Startup Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-18
2.4.2
Startup Archive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-19
2.4.3
2-22
Under normal operation, NETSMART 1500 starts the Oracle database. The procedures in
this section should be used only if NETSMART 1500 is not running. Refer to Section 3.4,
Troubleshooting Server and Clients [p. 3-12], for instructions on NETSMART 1500
shutdown.
2.4.1
Startup Normal
The startup-normal method sets the database to run in archive mode so that archive redo files
are created. Archive redo files allow the database to recover if a problem occurs. When
NETSMART 1500 starts up, it starts up the Oracle database in this way.
Start Oracle using the startup-normal method as follows:
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Startup
Otherwise, the system display for a normal startup is similar to the one shown below:
Sun Microsystems Inc. SunOS 5.10 Generic Patch October 2006
You have new mail.
Database is started.
Hit any key to Continue.
2.4.2
Startup Archive
WARNING:
Do not start the Oracle database using the startup-archive method without the direction of
Fujitsu Network Communications Inc. (FNC) technical support.
The startup-archive method forces the database to start in archive mode after it has been
running in a nonarchive mode. The startup-archive method normally should not be used. It is
used only after the database has been run in nonarchive mode for database recovery.
Start Oracle using the startup-archive method as follows:
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
2-19
CHAPTER 2
Oracle Administration
Startup
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Startup
The database starts in archive mode. The system display is similar to the one shown below:
Sun Microsystems Inc.
You have new mail.
SunOS 5.10
Generic Patch
October 2006
FNC-1500-0050-110
Issue 1, January 2007
2-21
CHAPTER 2
Oracle Administration
Startup
332108028
454908
184549376
134217728
12886016
bytes
bytes
bytes
bytes
bytes
Database altered.
Statement processed.Database altered.
2.4.3
This method is used only after a backup of the Oracle database is aborted abruptly; for
example, if the NETSMART 1500 system is rebooted while the NETSMART 1500 software
and Oracle database are running. The startup-by-ending-backup method starts the Oracle
database by ending the previously aborted backup.
Start Oracle using the startup-by-ending-backup method as follows:
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Shutdown
2.5
Shutdown
Note: The Oracle shutdown procedures described in this section should be used only if NETSMART
1500 is not running or if NETSMART 1500 is running in an error condition.
This section describes two ways to shut down the Oracle database. It includes the following
subsections:
NO.
SUBSECTION
PAGE
2.5.1
Shutdown Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-23
2.5.2
2-24
Under normal operation, the Oracle database is shut down by the NETSMART 1500
shutdown procedures described in Chapter 3, NETSMART 1500 Administration,
Section 3.3.3, Shut Down Server [p. 3-10].
2.5.1
Shutdown Normal
The shutdown-normal method produces a graceful shutdown. All Oracle database users (such
as the NETSMART 1500 software) must be disconnected from the database when the
shutdown-normal method is invoked. Shutdown-normal is the method NETSMART 1500
uses to shut down the Oracle database.
Shut down Oracle using the shutdown-normal method as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
2-23
CHAPTER 2
Oracle Administration
Shutdown
SunOS 5.10
Generic Patch
October 2006
2.5.2
The shutdown-abort (abnormal) method is not used in normal situations. This method shuts
the database down without warning users (such as the NETSMART 1500 software). This
shutdown is used for database recovery (refer to Section 2.7, Recovery from a Backup
[p. 2-29]) or for emergencies, such as when normal shutdown fails or when the database
hangs up.
Shut down Oracle using the shutdown-abort (abnormal) method as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Standard Database Backup
2.6
HINT:
Check that the backup device type is set correctly before following this procedure. If the
backup device type is set to Standalone or Autoloader, the restore tries to restore from
tape. Restore from tape will not be tried if the backup device is set to None or Custom.
Refer to Section 2.10.2, subsection Select Backup Device and Path [p. 2-57], for more
information.
A backup of the Oracle database is done while the Oracle database is running. The backup
stores Oracle tablespaces, archived redo log files, and control files on the tape.
Note: Backups should be run during off-peak hours.
FNC-1500-0050-110
Issue 1, January 2007
2-25
CHAPTER 2
Oracle Administration
Standard Database Backup
Task
Enter 1 Backup.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Standard Database Backup
FNC-1500-0050-110
Issue 1, January 2007
2-27
CHAPTER 2
Oracle Administration
Standard Database Backup
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Recovery from a Backup
2.7
2.7.1
NO.
SUBSECTION
PAGE
2.7.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-29
2.7.2
2-30
2.7.3
2-35
2.7.4
2-35
Overview
This section describes how to restore the contents of Oracle database and how to restore the
backup from an high availability (HA) system on a non-HA system.
FNC-1500-0050-110
Issue 1, January 2007
2-29
CHAPTER 2
Oracle Administration
Recovery from a Backup
2.7.2
This section describes how to restore the contents of the Oracle database.
CAUTION:
After the Oracle database recovery is completed, making a backup of the newly restored
Oracle database is important. The following procedure includes steps for backing up the
Oracle database after recovery is complete.
Task
Shut down each NETSMART 1500 client by selecting Exit from the File menu of each
NETSMART 1500 client graphical user interface (GUI).
Stop the NETSMART 1500 server. (Refer to Section 3.3.3, Shut Down Server [p. 3-10].)
Enter 3 Backup/Recovery.
Insert the backup tape from the previous day into the tape drive on the NETSMART 1500
server.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Recovery from a Backup
Data recovery begins. A screen similar to the following displays. A completion message
displays when recovery is complete.
--------------------------------------For Logs, Please see
/opt/NetSmart/oracle/scripts/operation/out/nsdbbackup.25Jun132023
--------------------------------------Starting Restoration from TAPE to DISK
Sun Jun 25 13:20:26 PDT 2006
Restoration of Files from HOT Backup
Restoration will be done from /dev/rmt/0n
Make sure that /dev/rmt/0n is ONLINE
Please insert database BACKUP DAT in /dev/rmt/0n
Restoration will start in 1 Minute.
Restoration can proceed
-----------------------------------------------------------------Date
: Sun Jun 25 12:15:48 PDT 2006
Type of Backup
: On Demand HOT/OnLine Backup of Database
System
: swagat.nms.fnc.fujitsu.com
NetSmart Version
: 5.0
Database Name
: nms
Size of Backup
: 109459456 Bytes
MaxNeCount
: 1000
App Servers
: 2
CheckSum
: 60111496853
-----------------------------------------------------------------Are you restoring correct backup ?
Do you want to continue ? :y/n: y
STARTING RESTORATION Of Database Files AT Sun Jun 25 13:20:46 PDT 2006
Restoring all the files
It will take time, please wait .....
screen display continues on next page
FNC-1500-0050-110
Issue 1, January 2007
2-31
CHAPTER 2
Oracle Administration
Recovery from a Backup
fufullDb_0528031208/rbs_01.dbf.Z
fullDb_0528031208/tools_01.dbf.Z
fullDb_0528031208/users_01.dbf.Z
fullDb_0528031208/d_meta_01.dbf.Z
fullDb_0528031208/d_docmgt_01.dbf.Z
fullDb_0528031208/replic_ts_01.dbf.Z
fullDb_0528031208/d_activityevent_01.dbf.ZfullDb_0625061208/d_emlnml_01.db
f.Z
fullDb_0528031208/d_faulthistory_01.dbf.Z
fullDb_0528031208/d_netsmart_01.dbf.Z
fullDb_0528031208/d_of_01.dbf.Z
fullDb_0528031208/d_pm_01.dbf.Z
fullDb_0528031208/d_pmcore_01.dbf.Z
fullDb_0528031208/d_systemevent_01.dbf.Z
fullDb_0528031208/d_tl1event_01.dbf.Z
fullDb_0528031208/d_tl1uplinkevent_01.dbf.Z
fullDb_0528031208/i_emlnml_01.dbf.Z
fullDb_0528031208/i_netsmart_01.dbf.Z
fullDb_0528031208/i_pm_01.dbf.Z
fullDb_0528031208/i_pmcore_01.dbf.Z
fullDb_0528031208/temp_01.dbf.Z
fullDb_0528031208/28_1.arc.Z
fullDb_0528031208/27_1.arc.Z
fullDb_0528031208/bkp_control.ctl
fullDb_0528031208/dbfile_bkp.ctl
fullDb_0528031208/new_control_resetlog.sql
fullDb_0528031208/fullDb_headerRestoration of Database files is over Sun
Jun 25 13:22:58 PDT 2006
Restoration Can Proceed after controlfile for backup check
Restoration From Tape to Backup Disks is Over
Program Successfully Completed ..
Restoring from DISK for fullDb onLine lastCommit
Starting Recovery lastCommit Based
Checking for Validation of database installation
Checking for Host names
Host names mismatch
database is going down.. In abort mode
Database shutdown complete...
database is going down.. In abort mode
Database shutdown complete...
Database is up and running !!!
Connectivity test is successful !!!
screen display continues on next page
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Recovery from a Backup
auditdetail_tasklog_check : 13:54:21
clienthashmap_proplist_check : 13:54:22
CTPUtilization_NE : 13:54:22
CTPUtilization_PNEComponent : 13:54:23
inactive_nemgrarray_check : 13:54:23
ne_customerinfo_check : 13:54:23
ne_neuser_check : 13:54:24
ne_servicemanager_1_check : 13:54:24
ne_servicemanager_2_check : 13:54:25
ne_servicemanager_check : 13:54:25
ne_sm_coe_check : 13:54:25
NeDatabase_swfile : 13:54:26
nesync_ne_check : 13:54:26
neuser_actgrp_check : 13:54:27
neuser_ne_check : 13:54:27
neusr_actgrp_phase2 : 13:54:27
neusr_neact_phase1 : 13:54:28
pconnection_customerattr_check : 13:54:28
pconnection_endpoints_check : 13:54:29
pconnpcourseleg2 : 13:54:29
pconnpcourseleg3 : 13:54:30
pcourseleg_ne_check : 13:54:30
pcourseleg_pconnection_check : 13:54:30
pcourselegpcrossconnect : 13:54:31
PEC_NE_Check : 13:54:31
PECATTR_PEC_Check : 13:54:31
peqptconn_attributes_check : 13:54:32
peqptconn_tne_check : 13:54:32
playoutleg_ne_check : 13:54:33
playoutleg_pconnection_check : 13:54:33
screen display continues on next page
FNC-1500-0050-110
Issue 1, January 2007
2-33
CHAPTER 2
Oracle Administration
Recovery from a Backup
Remove the backup tape from the tape drive to prevent accidental erasure.
10
Do a normal shutdown of the Oracle database. (Refer to Section 2.5.1, Shutdown Normal
[p. 2-23].)
11
Start the NETSMART 1500 server. (Refer to Section 3.3.1, Start Server [p. 3-8].)
12
Do a backup of the system. (Refer to Section 2.6, Standard Database Backup [p. 2-25].)
13
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Recovery from a Backup
2.7.3
Step
Task
Use Heartbeat menu to restore backup. Refer to Section 2.7.2, Recovering from a Backup
[p. 2-30], for more information.
Use the Heartbeat menu to Unconfigure HA. Refer to Section 7.11, Unconfiguring
NETSMART 1500 HA Service [p. 7-27], for more information.
2.7.4
Step
Task
Bring down the cluster. Refer to Section 7.12.3, Shutdown Cluster [p. 7-32], for more
information.
Use Heartbeat menu to restore backup. Refer to Section 2.7.2, Recovering from a Backup
[p. 2-30], for more information.
Bring up the active system in single node. This puts the active in active suspended (AS) state.
Do a netsmart startup on single node on the active system by choosing cluster startup. Refer
to Section 7.12.1, Cluster Startup [p. 7-31], for more information. This triggers a active
suspended to active fail (AS_AF) and put the passive also in passive fail (PF) state.
Do a startup on the active system choosing cluster startup. Refer to Section 7.12.1, Cluster
Startup [p. 7-31], for more information. This internally triggers an active failed to active up
(AF_AU) state, internally adding the second node.
FNC-1500-0050-110
Issue 1, January 2007
2-35
CHAPTER 2
Oracle Administration
Oracle Purge
2.8
Oracle Purge
NETSMART 1500 stores TL1 events, fault history, 15-minute performance monitoring data,
and 24-hour performance monitoring data (all referred to as log data) in log tables. This
historical data is available from the log tables until it is deleted by purging. Purging is required
to make space for new data. There are two purging mechanisms:
Daily rolloverDoes a backup of the log data that is older than 365 days and then
deletes that data from the system at the end of the day.
Threshold purgeDeletes log data from the log tablespaces when the disk is at or
above 80 percent of maximum capacity. The purge deletes historical log data until the
disk falls below 60 percent capacity. This task runs every hour.
If your system has a very large storage capacity, threshold purge does not occur, but the log
table can contain no more than 365 days of historical data due to daily rollover.
When disk usage for the log data tablespaces reaches 70 percent, the purge monitoring utility
sends an e-mail to the Oracle Administrator. Upon receiving this e-mail, the Oracle
Administrator should back up all of the data using the backup procedure described in
Section 2.6, Standard Database Backup [p. 2-25].
Various scheduled reports are generated by NETSMART 1500. All of these reports are stored
in the appropriate directories on the disk and are backed up daily by the Oracle nightly
backups.
Depending on the size of the reports and the configuration of the database, the disk space is
appropriately allocated. For reports reaching the thresholds size limit on the disk, the reports
are deleted starting with the oldest one. The corresponding database entry for the report is
also dropped. The delete threshold limit for the reports is 90 percent of the allocated directory
space in the disk. On HA systems, periodically the reports are the snapshot reports copied to
the backup system from the active system. When user-initiated, the reports are recreated on
the backup system through the help of scripts.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Alert Scripts
2.9
Alert Scripts
This section describes the database alert scripts. It includes the following subsections:
NO.
SUBSECTION
PAGE
2.9.1
Alert Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-37
2.9.2
2-38
2.9.3
2-39
2.9.4
2-40
2.9.5
2-41
Database alerts notify the Oracle user if there may be a problem with the Oracle database.
The database alert scripts are run daily from the Oracle users crontab. The Oracle user is
automatically notified of potential database problems through e-mail to the Oracle user
account.
If required, database alert scripts can be run manually through Heartbeat.
2.9.1
Alert Monitor
The alert monitor script is responsible for logging errors reported by the Oracle database.
Execute the alert monitor script as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
2-37
CHAPTER 2
Oracle Administration
Alert Scripts
The alert monitor script runs and checks for warnings or errors in the alert_nms.log file
as shown in the following output sample:
Sun Microsystems Inc.
You have new mail.
SunOS 5.10
2.9.2
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Alert Scripts
The script runs and displays the size of alert_nms.log file. If the original size of the alert
file is 5 MB or greater, it is compressed and renamed as shown in the following output sample:
Sun Microsystems Inc.
You have new mail.
SunOS 5.10
1 oracle
dba
2.9.3
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
2-39
CHAPTER 2
Oracle Administration
Alert Scripts
The script runs and displays available free space in the UNIX partitions. A screen similar to
the following displays:
Sun Microsystems Inc.
SunOS 5.10
Generic Jun 2000
You have new mail.
Sun Jun 25 10:22:25 CDT 2006
Filesystem / has 14016 MB of free space left
Filesystem /nsdata has 26182 MB free space left
Hit any key to Continue.
2.9.4
If there is less than 300 MB free in one of the partitions, contact FNC technical support for
further assistance.
WARNING:
If 90 percent of the available space is used and the backup of the archived redo logs fails
due to unavailability of a tape, the archived redo log files are deleted.
Task
Enter 4 Check size of File System containing Archived Redo Log Files.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Alert Scripts
The script runs and displays the available disk space in the archived redo log file partition. If
70 percent of the space is used, a message indicates that a backup of the archived log files has
been initiated. A screen similar to the following displays:
Sun Microsystems Inc.
You have new mail.
SunOS 5.10
2.9.5
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
2-41
CHAPTER 2
Oracle Administration
Alert Scripts
The script runs and verifies that all redo log files are available. If any are missing from
sequence, they are displayed. A screen similar to the following displays:
Sun Microsystems Inc.
SunOS 5.8
Generic June 2006
You have new mail.
Last Archived Redo Log File Copied to tape was 10_1.arc
Beginning to search for missing Archived redo Log files...
Search Completed...
All Archived Redo Log Files are present.
Hit any key to Continue.
If any redo log files are reported missing, you should do a backup of the database immediately
so that a valid and current backup of the database is available. (Refer to Section 2.6, Standard
Database Backup [p. 2-25].)
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
2.10
Automated Maintenance
This section includes the following subsections:
NO.
SUBSECTION
PAGE
2.10.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-43
2.10.2
Maintenance Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-46
Oracle maintenance jobs are set up during NETSMART 1500 installation. These jobs run
from the Oracle users crontab.
Note: Modifying Oracle maintenance jobs installed in the Oracle users crontab is not
recommended.
2.10.1
Overview
One of the maintenance tasks that is automatically configured to run daily at midnight is a
backup of the database (part of the eod.sh script). For this job, the tape used to store the
backup must be changed daily. The tapes storing the backup information should be stored in
off-site archives.
The Oracle Administrator must check e-mail regularly, because various cronjobs can generate
warning and error e-mail messages. E-mails are sent to the Oracle user for database-related
script status. Section 2.11.9, Database E-Mail Notification [p. 2-66], explains how the Oracle
Administrator e-mail address is specified.
FNC-1500-0050-110
Issue 1, January 2007
2-43
CHAPTER 2
Oracle Administration
Automated Maintenance
The maintenance jobs listed in Table 2-4 [p. 2-44] are executed at specific intervals during the
day to check for various system problems.
Note: These jobs can be executed manually, if required, from the Oracle user. All the scripts
listed in Table 2-4 [p. 2-44] are on the NETSMART 1500 server in the following directory:
/opt/NetSmart/oracle/scripts/maintenance.
Table 2-4: Automated Oracle Maintenance Jobs
Script Name
Description
alert_mon
Every 15 minutes
arch_miss
Every 3 hours
This job monitors the Oracle archived redo log files and verifies
that all the redo logs exist. If any are missing, an e-mail is sent to
the Oracle user, informing the administrator of the problem and
suggesting that an immediate backup of the system be
performed.
audit_purge.sh
chk_alert
This job checks the size of the alert log. If the size exceeds a
predefined threshold (typically 5 MB), the file is compressed
and renamed with a timestamp. A new alert log is created.
chk_arch
Every 3 hours
This job monitors available space in the file system where the
Oracle archived redo log files are stored. If the archived redo logs
are occupying more than 70 percent of the partition capacity, a
backup of the archived log files is initiated. If the disk space use
reaches 90 percent, the archived redo log files are deleted.
chk_fs
Every 3 hours
This job checks for disk space problems in the following disk
partitions: /, /var, /usr/, and several of the /opt partitions. E-mail
is sent to the Oracle user, identifying the disk space use status.
chk_iostat
Every hour
chk_rpt_space
Every 3 hours
This job monitors the report file system. If the files have
occupied more than 90 percent of the allotted space it deletes
the files starting with the oldest one and drops the
corresponding database table till the space reaches less than 75
percent of the allotted space
clean_alert
This job removes compressed Oracle alert log files that are more
than 30 days old.
clean_arch
This job removes archived log files that are more than 7 days
old.
clean_log
This job removes various log and output files that are more than
45 days old.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
Description
clean_trc
This job removes any Oracle trace files that are more than 7 days
old.
conn_db
Every 15 minutes
dbarpt
Every hour
gather_stats
Every 3 hours
icheck
monitor_log_space
Every hour
This job monitors the log tablespaces to see whether capacity has
reached the 70 or 90 percent thresholds. The tables in the log
tablespaces include fault history, TL1 event, 15-minute
performance monitoring, and 24-hour performance
monitoring. If the capacity has reached 70 percent, an e-mail is
sent to the Oracle user. If the 90 percent threshold is reached, a
purge is initiated.
nesecurity_purge
os_monitor
Every 20 minutes
prebkup
This job creates a list of control files, redo log files, and database
files that are to be backed up. The output of this script is used by
the eod.sh script, which performs a backup of the database.
The location of this script is
/opt/NetSmart/oracle/scripts/operation/bkup.
roll_ns
Every hour
This jobs removes the oldest partition from the log tables (front
end).
report_mail
rmbu_purge.sh
FNC-1500-0050-110
Issue 1, January 2007
2-45
CHAPTER 2
Oracle Administration
Automated Maintenance
Description
stat_reco
This job creates a list of all of the control files, redo log files, and
database files. This list is used by database recovery scripts. The
location of this script is
/opt/NetSmart/oracle/scripts/operation/reco.
tl1batch_purge
Every hour
validate.ksh
This job checks for invalid database objects and then validates if
it finds any.
pmregister_purge.sh
2.10.2
Maintenance Scripts
This subsection includes the following items:
ITEM
PAGE
2-48
2-48
2-49
2-50
2-50
2-51
2-52
2-52
2-54
2-54
2-55
2-55
2-56
2-57
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
The ORACLE DATABASE MAINTENANCE menu options shown below are run through
cronjobs. However, they also can be run from Heartbeat.
Note: Menu options 8, 10, 14, and 15 involving disk arrays and RAID are not supported in this
release. Menu option 18 is partially supported. Autoloader and RAID are not supported.
O R A C L E
D A T A
B A S E
M A I N T E N A N C E
FNC-1500-0050-110
Issue 1, January 2007
2-47
CHAPTER 2
Oracle Administration
Automated Maintenance
./nms_ora_11935.trc
-rw-r----- 1 oracle dba 2504 Jun 25 20:00 ./nms_ora_11935.trc
./nms_ora_28102.trc
-rw-r----- 1 oracle dba 2505 Jun 25 1 20:00 ./nms_ora_28102.trc
clean_trc over Sun Jun 25 11:02:37 CDT 2006
./nms_ora_11935.trc
-rw-r----- 1 oracle dba 2504 Jun 25 20:00 ./nms_ora_11935.trc
./nms_ora_28102.trc
-rw-r----- 1 oracle dba 2505 Jun 25 20:00 ./nms_ora_28102.trc
./nms_ora_17859.trc
-rw-r----- 1 oracle dba 2503 Jun 25 20:00 ./nms_ora_17859.trc
./nms_ora_13277.trc
-rw-r----- 1 oracle dba 2504 Jun 25 20:00 ./nms_ora_13277.trc
./nms_ora_25411.trc
-rw-r----- 1 oracle dba 2505 Jun 25 20:00 ./nms_ora_25411.trc
./nms_ora_17859.trc
-rw-r----- 1 oracle dba 2503 Jun 25 20:00 ./nms_ora_17859.trc
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
SunOS 5.10
FNC-1500-0050-110
Issue 1, January 2007
2-49
CHAPTER 2
Oracle Administration
Automated Maintenance
SunOS 5.10
1661160
1996752
511992
Owner Object_name
Datafile
Name Max_next(KB)
------ --------------- -------------------- -----------Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
SunOS 5.10
FNC-1500-0050-110
Issue 1, January 2007
2-51
CHAPTER 2
Oracle Administration
Automated Maintenance
CAUTION:
Contact FNC technical support at 1-800-USE-FTAC before using this option.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
This option opens the crontab file in your default editor. A screen similar to the following
displays:
This file contains cron entries which are to be run through "oracle" id.
DAILY SCRIPTS
Every 15 mins.
5,20,35,50 * * * * cd /opt/NetSmart/oracle/scripts/maintenance ;
nohup /opt/NetSmart/oracle/scripts/maintenance/alert_mon &
5,20,35,50 * * * * cd /opt/NetSmart/oracle/scripts/maintenance ;
nohup /opt/NetSmart/oracle/scripts/maintenance/conn_db &
5,20,35,50 * * * * cd /opt/NetSmart/oracle/scripts/maintenance ; nohup
/opt/NetSmart/oracle/scripts/maintenance/stat_db &
Every 20 mins.
.....
/bkup/backup &
WEEKLY SCRIPTS
Every Sunday at 7.00 hours
0 7 * * 0 cd /opt/NetSmart/oracle/scripts/maintenance ; nohup /opt
NetSmart/oracle/scripts/maintenance/clean_alert &
0 7 * * 0 cd /opt/NetSmart/oracle/scripts/maintenance ; nohup /opt
NetSmart/oracle/scripts/maintenance/clean_trc &
0 1 * * 6 cd /opt/NetSmart/oracle/scripts/maintenance ; nohup /opt
NetSmart/oracle/scripts/maintenance/analyze_tab &
FNC-1500-0050-110
Issue 1, January 2007
2-53
CHAPTER 2
Oracle Administration
Automated Maintenance
SunOS 5.10
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
FNC-1500-0050-110
Issue 1, January 2007
2-55
CHAPTER 2
Oracle Administration
Automated Maintenance
SunOS 5.10
Enter the no of Hours you want the Oracle Scheduled jobs to be offset (for
HELP enter '?') [?,q] ?
The Input entered will be used to Offset the Scheduled Oracle jobs
by
that many Hours (valid range is +-1 to +-12)
Enter the no of Hours you want the Oracle Scheduled jobs to be offset (for
HELP enter '?') [?,q] +1
Changed the timings of Scheduled Oracle cron jobs by 1 Hours
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Automated Maintenance
SunOS 5.10
Standalone(default) Backup uses both the hard-drive and a tape device. The tape
device holds only one tape. This backup method is used for configurations of fewer
than 4000 NE.
Note: Autoloader Backup is not supported in this release.
FNC-1500-0050-110
Issue 1, January 2007
CustomBackup uses both the hard-drive and a tape device, however, the backup tape
device is an external entity and user is responsible for moving the backup from the
hard-drive to the tape device. Used for 8000 NE configurations. The tape device
requires additional software not included with the NETSMART 1500 NMS
application.
2-57
CHAPTER 2
Oracle Administration
Automated Maintenance
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
System and Configuration Information
2.11
SUBSECTION
PAGE
2.11.1
2-60
2.11.2
Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-63
2.11.3
Tablespaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-63
2.11.4
2-65
2.11.5
Redo Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-65
2.11.6
2-66
2.11.7
Core Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-66
2.11.8
Trace Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-66
2.11.9
2-66
FNC-1500-0050-110
Issue 1, January 2007
2-59
CHAPTER 2
Oracle Administration
System and Configuration Information
2.11.1
Task
Log on the NETSMART 1500 server as user oracle (refer to Section 2.2, Oracle User
Account [p. 2-5]) or netsmart (refer to Section 3.2, Creating UNIX Accounts [p. 3-7]).
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
System and Configuration Information
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
Enter the number of the attribute you want the search criteria to apply to.
In the following example, number 3 was entered to select the TYPE attribute:
3
Enter the Search value for TYPE ( Press Enter for selecting all values )
Enter a search string for the selected attribute (or press ENTER to get a full report).
Note: The search string can include the * (asterisk) wildcard character. For the example shown
below, the search string DS* was entered.
FNC-1500-0050-110
Issue 1, January 2007
2-61
CHAPTER 2
Oracle Administration
System and Configuration Information
The report is generated, compressed, and stored as shown in the following example:
DS*
*********************************************
Start time is Sun Jun 25 10:47:49 CDT 2006
*******************************************************
Creating Query Files Sun Jun 25 10:47:49 CDT 2006
Processing for AID Sun Jun 25 10:48:43 CDT 2006
Processing for TYPE Sun Jun 25 10:48:46 CDT 2006
Processing for CLEI Sun Jun 25 10:49:49 CDT 2006
Processing for DOM Sun Jun 25 10:49:55 CDT 2006
Processing for SERIALNO Sun Jun 25 10:50:15 CDT 2006
Processing for UNITNAME Sun Jun 25 10:50:20 CDT 2006
Processing for VENDOR_ID Sun Jun 25 10:50:49 CDT 2006
Processing for VENDOR_REV Sun Jun 25 10:51:20 CDT 2006
*******************************************************
Creating NETSMART1500 System Equipment Inventory Information File :
/opt/NetSmart/oracle/out/eqpt_inventory_report/sys_info_TYPE.26Jun104749.Z
Sun Jun 25 10:51:20 CDT 2006
*******************************************************
Deleting Temporary Files
*******************************************************
Process Finished Sun Jun 25 10:51:20 CDT 2006
*******************************************************
The display shows the name of the compressed inventory report file. Its file name is:
sys_info_<query attribute>.<time stamp>.Z
Enter the following commands to change directories and uncompress the report file:
cd /opt/NetSmart/oracle/out/eqpt_inventory_report
uncompress sys_info_<query attribute>.<time stamp>.Z
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
System and Configuration Information
2.11.2
Log Files
Log files keep records of various activities, events, and reports:
2.11.3
The log files generated from various Oracle maintenance scripts are stored in
/opt/NetSmart/oracle/scripts/maintenance/out.
Tablespaces
Oracle stores data logically in tablespaces and physically in data files associated with the
corresponding tablespace. Each tablespace consists of one or several data files. NETSMART
1500 and Oracle-specific data are collectively stored in many tablespaces.
FNC-1500-0050-110
Issue 1, January 2007
2-63
CHAPTER 2
Oracle Administration
System and Configuration Information
Table 2-5 [p. 2-64] defines tablespaces for Oracle data, and Table 2-6 [p. 2-64] defines
tablespaces for NETSMART 1500.
Table 2-5: Tablespaces Used for Oracle Data
Tablespace
Description
SYSTEM
ROLLBACK
TEMP
TOOLS
USERS
Used to store data for users other than NETSMART 1500, such as other
Oracle users created by the System Administrator
SYSAUX
Description
D_ACTIVITYEVENT
D_DOCMGT
D_EMLNML
I_EMLNML
D_FAULTHISTORY
D_META
Contains data and indexes for the Metamodel data (Metamodel data consists
of static tables that describe the characteristics of the NEs managed by
NETSMART 1500.)
D_NETSMART
I_NETSMART
D_OF
D_PM
D_PMCORE
I_PMCORE
REPLIC_TS
I_PM
D_RNAUDITEVENT
D_SYSTEMEVENT
D_TL1EVENT
D_TL1UPLINKEVENT
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
System and Configuration Information
2.11.4
Tablespace
Description
I_SYSTEMEVENT
I_TL1EVENT
I_TL1UPLINKEVENT
I_FAULTHISTORY
I_ACTIVITYEVENT
I_RNAUDITEVENT
MIG
DW
CDC
Used for storing change records from transaction tables that are involved in
data warehouse schema
2.11.5
/opt/NetSmart/oracle/nms/oradata/control01.ctl
/nsdata/nms/oradata/control02.ctl
/nsdata/nms/oradata/control03.ctl
Redo Files
Redo log files collect the transaction history of the NETSMART 1500 Oracle database. The
log files are mirrored over two different locations. If one of the mirrored files is damaged, the
database continues to run using the second mirror.
The following are the primary redo log files for a 4x4x8 system:
/nsdata/nms/oradata/redonms01a.log
/nsdata/nms/oradata/redonms02a.log
/nsdata/nms/oradata/redonms03a.log
FNC-1500-0050-110
Issue 1, January 2007
2-65
CHAPTER 2
Oracle Administration
System and Configuration Information
Archived redo log files are created when the NETSMART 1500 database is running in archive
mode, which is the default mode for NETSMART 1500. The archived log files are backed up
nightly by the NETSMART 1500 backup script eod_backup executed as root as a nightly
cronjob.
2.11.6
2.11.7
Core Files
Oracle core dumps are stored in /opt/NetSmart/oracle/admin/nms/cdump. This
directory is cleaned on a weekly basis by an Oracle maintenance job.
2.11.8
Trace Files
Trace files can provide information related to an error identified by the Oracle database. Trace
files are generated by various background processes and by Oracle. These trace files are stored
in the following locations:
/opt/NetSmart/oracle/admin/nms/bdump
/opt/NetSmart/oracle/admin/nms/udump
2.11.9
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Export Procedures
2.12
Export Procedures
This section includes the following subsections:
NO.
SUBSECTION
PAGE
2.12.1
2-68
2.12.2
Export of PM LOGS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-69
The processes in the Oracle Database Import/Export menu shown below are on-demand
maintenance scripts. They are not monitoring scripts and should be run only by designated
DBA users when necessary.
1.) Export
2.) Export
d. Disable
m. Go back
p. Go back
q. Quit
FNC-1500-0050-110
Issue 1, January 2007
2-67
CHAPTER 2
Oracle Administration
Export Procedures
2.12.1
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 2
Oracle Administration
Export Procedures
2.12.2
Export of PM LOGS
This option exports PM log tables in compressed mode and stores them on tape using cpio.
Connected to: Oracle8i Enterprise Edition Release 10.2.0.2.0 - Production
With the Partitioning option
JServer Release 10.2.0.2.0 - Production
Export done in US7ASCII character set and US7ASCII NCHAR character set
About to export specified tables via Conventional Path ...
. . exporting table
PMMONVAL15MIN_0
0 rows exported
. . exporting partition
PMMONVAL15MIN_0
0 rows exported
. . exporting partition
PMMONVAL15MIN_1
0 rows exported
. . exporting partition
PMMONVAL15MIN_2
0 rows exported
. . exporting partition
PMMONVAL15MIN_3
0 rows exported
. . exporting partition
PMMONVAL15MIN_4
0 rows exported
. . exporting partition
PMMONVAL15MIN_5
0 rows exported
. . exporting partition
PMMONVAL15MIN_30
0 rows exported
.................................................................
. . exporting table
PMMONVAL24HOUR
. . exporting partition
PMMONVAL24HOUR_0
0 rows exported
. . exporting partition
PMMONVAL24HOUR_1
0 rows exported
. . exporting partition
PMMONVAL24HOUR_6
0 rows exported
................................................................
. . exporting partition
PMMONVAL24HOUR_30
0 rows exported
Export terminated successfully without warnings.
Backup will be done on /dev/rmt/0n
Make sure that /dev/rmt/0n is ONLINE
and containing Write Enabled DAT for Backup
STARTING BACKUP Of export File/s AT Sun Jun 25 13:10:09 CST
2006
Backup will Start after 1 seconds.
Starting backup to Tape...
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
2-69
3
CHAPTERS/APPENDIXES
1 Introduction
2 Oracle Administration
3 NETSMART 1500
Administration
NO.
SECTION
4 UNIX System
Administration
3.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-3
5 Network Administration
3.2
3-7
6 Service Support
3.3
3-8
7 High Availability
Administration
3.4
3-12
3.5
3-16
3.6
3-28
3.7
3-28
3.8
3-37
3.9
3-39
3.10
3-40
3.11
Resync NE Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-41
3.12
3-42
3.13
3-45
3.14
Broadcast Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-46
3.15
3-49
3.16
3-55
3.17
3-58
3.18
3-73
FNC-1500-0050-110
Issue 1, January 2007
PAGE
CHAPTER 3
NETSMART 1500 Administration
NO.
SECTION
PAGE
3.19
3-76
3.20
LNE Discovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-80
3.21
Connection Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-82
3.22
Topology Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-84
3.23
3-88
3.24
3-90
3.25
3-90
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Overview
3.1
Overview
This chapter provides administrative information and describes administrative tasks for the
NETSMART 1500 server and client software.
WARNING:
Many of the operations described in this chapter use the Heartbeat tool. When using
Heartbeat with root enabled, do not press CTRL+C. Doing so may cause Heartbeat to
terminate abnormally. CTRL+C may be used with Heartbeat when root is not enabled.
FNC-1500-0050-110
Issue 1, January 2007
3-3
CHAPTER 3
NETSMART 1500 Administration
Overview
NETSMART 1500 is administered using the NETSMART 1500 Administration option of the
Heartbeat tool. This option provides the following menu, which is the starting point for most
of the procedures described in this chapter.
N E T S M A R T 1500 A D M I N I S T R A T I O N
1. Start NETSMART 1500 Server Processes +
2. Stop NETSMART 1500 Server +
3. Check NETSMART 1500 Server Status +
4. Start NETSMART 1500 Client +
5. Enable Program Groups +
6. Disable Program Groups +
7. NETSMART 1500 Trace Menu
8. Load NE Software Generic Files *
9. Start ObjectStream License Manager *
10. Export Security Data +
11. Import Security Data +
12. Resync NE Data
13. NETSMART 1500 HA Configuration
14. NETSMART 1500 HA Unconfiguration
15. Pre-login Warning Banner Operations
16. Enable/Disable RTRV-HDR Logging
17. Upgrade Netsmart 1500 License *
18. License Information
19. Transaction Debugging Client
20. LNE Discovery
21. Connection Discovery
22. Topology Discovery Administration
23. Set Branding Image to display in Topology GUI client
24. Remove Branding Image from Topology GUI client
25. PCN and PNE Update
26. Change PCN and PNE Update Schedule
27. Display PCN and PNE Update Schedule
d. Disable Root Access (return to netsmart)
m. Go back to the Main Menu
q. Quit
* may require root access
+ can only be executed by NETSMART 1500 administrator
Select a number/letter (case insensitive):
Note: Menu items denoted with an asterisk (*) require root access. Menu items denoted with a
plus sign (+) can be executed only by the NETSMART 1500 Administrator.
Table 3-1 [p. 3-5] lists the menu and submenu items for NETSMART 1500 administration
and where they are described in this practice.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Overview
Reference
1. Enable PM
2. Enable EMLNML
3. Enable TL1Uplink
4. Enable FW74NM
5. Enable Webservice
6. Disable SNMPUplink
1. Disable PM
2. Disable EMLNML
3. Disable TL1Uplink
4. Disable FW74NM
5. Disable Webservice
6. Disable SNMPUplink
FNC-1500-0050-110
Issue 1, January 2007
3-5
CHAPTER 3
NETSMART 1500 Administration
Overview
Reference
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Creating UNIX Accounts
Reference
3.2
The user name for this account is netsmart. Table 3-2 [p. 3-7] lists the user account
information.
Table 3-2: NETSMART 1500 Administrator UNIX User Account Information
User
Shell
Home
Group
netsmart
/bin/csh or
/bin/ksh
/opt/NetSmart
netsmart
This information is needed for Lightweight Directory Access Protocol (LDAP), NIS, or NIS+
setup of the UNIX user account. Provide this information to the Network System
Administrator to set up LDAP, NIS, or NIS+ user account verification.
If the NETSMART 1500 system is configured to use local files instead of LDAP, NIS, or
NIS+, the installation sets up the account automatically. No additional configuration is
required.
The NETSMART 1500 software requires a separate logon for the NETSMART 1500 system
(used when starting the NETSMART 1500 client graphical user interface [GUI]) in addition
to the UNIX logon. Upon installation, the NETSMART 1500 software is configured to use
the NETSMART 1500 UNIX user account and password for both logons.
Note: The NETSMART 1500 UNIX user account is not required on NETSMART 1500 clients, and
the install does not configure a NETSMART 1500 UNIX user account on client systems.
FNC-1500-0050-110
Issue 1, January 2007
3-7
CHAPTER 3
NETSMART 1500 Administration
Starting Up and Shutting Down
The NETSMART 1500 software on NETSMART 1500 clients is owned by the root UNIX
user account. Therefore, any UNIX user accounts that are created for NETSMART 1500
should not have a home directory under /opt/NetSmart. They should use a home
directory such as /home or /users, depending on the local site conventions for UNIX user
accounts.
3.3
SUBSECTION
PAGE
3.3.1
Start Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-8
3.3.2
3-9
3.3.3
3-10
3.3.4
3-11
The NETSMART 1500 server and client applications are started separately. The following
procedures describe how to start up and shut down the NETSMART 1500 server and clients.
Note: A NETSMART 1500 server system contains both server and client software; therefore, both
the NETSMART 1500 server and client can be started on a NETSMART 1500 server system. A
NETSMART 1500 client system contains only client software; therefore, only the NETSMART 1500
client can be started on a NETSMART 1500 client system.
3.3.1
Start Server
Start the NETSMART 1500 server application as follows:
Step
Task
At the common desktop environment (CDE) welcome screen, log on with the NETSMART
1500 Administrator user name (netsmart) and password.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Starting Up and Shutting Down
3.3.2
CAUTION:
In general, you should not run the NETSMART 1500 client on the server. Running the client
on the server consumes server memory and degrades performance. Some graphics cards
may also degrade performance because of Java compatibility issues. A list of acceptable
graphics cards is included in the technical information bulletin (TIB) titled NETSMART 1500
Version 5 Software Release Notes. This TIB is available from the FOCIS Web site at
https://partners.fnc.fujitsu.com.
Note: To start a NETSMART 1500 client on UNIX workstation or PC (not the server), refer to
FNC-1500-0050-200, Introduction, Section 4.7, Client Startup [p. 4-5].
FNC-1500-0050-110
Issue 1, January 2007
3-9
CHAPTER 3
NETSMART 1500 Administration
Starting Up and Shutting Down
Task
Enter a valid NETSMART 1500 user name and password, and then click Login.
The Topology Management window displays when the system has finished initializing.
3.3.3
WARNING:
When the NETSMART 1500 server is shut down, the managed NEs are not affected.
However, the NETSMART 1500 software no longer monitors all NEs.
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Starting Up and Shutting Down
NETSMART 1500 shuts down, and the server displays messages similar to those shown
below:
Checking for NETSMART 1500 services...
NETSMART 1500 Servers are going down. This will take a while....
Using OpenORB 1.3.1
Shutting down HA federation bubs (Tue Dec 19 10:41:39 PST 2006)
Checking current federation status (Tue Dec 19 10:41:39 PST 2006)
bubs.nms.fnc.fujitsu.com: Stopping NETSMART services (Tue Dec 19 10:41:39
PST 2006)
bubs.nms.fnc.fujitsu.com: Stopping NETSMART Database (Tue Dec 19 10:42:49
PST 2006)
Validating the cookie (Tue Dec 19 10:44:34 PST 2006)
Stopping HANS (Tue Dec 19 10:44:34 PST 2006)
null (Tue Dec 19 10:46:01 PST 2006)
HA federation is now down (Tue Dec 19 10:46:01 PST 2006)
======================================================================
NETSMART 1500 Federation bubs
Hostname: bubs.nms.fnc.fujitsu.com: Unknown
======================================================================
done
Hit any key to Continue.
If any clients are attached to the server, a message displays to each client that the connection
has been lost, and each client stops functioning.
3.3.4
Step
1
Task
In the NETSMART 1500 client GUI, select Exit from the File menu.
A confirmation dialog box opens stating that this action will halt the NETSMART 1500
client.
Click OK to continue.
The NETSMART 1500 client shuts down.
FNC-1500-0050-110
Issue 1, January 2007
3-11
CHAPTER 3
NETSMART 1500 Administration
Troubleshooting Server and Clients
3.4
3.4.1
NO.
SUBSECTION
PAGE
3.4.1
3-12
3.4.2
3-13
3.4.3
3-14
3.4.4
Restore Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-15
3.4.5
3-16
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Troubleshooting Server and Clients
status"
status ha"
status list"
status all"
3.4.2
Step
1
Task
Shut down the client. (Refer to Section 3.3.4, Shut Down Client [p. 3-11].)
FNC-1500-0050-110
Issue 1, January 2007
3-13
CHAPTER 3
NETSMART 1500 Administration
Troubleshooting Server and Clients
Reboot the NETSMART 1500 client using the following command as the root user:
/usr/sbin/shutdown -i6 -y -g30
Note: The -g30 option specifies a 30-second delay period, which may or may not be needed.
After the reboot is complete, restart the NETSMART 1500 client. (Refer to
FNC-1500-0050-200, Introduction, Section 4.7.1, Starting a Client on Solaris [p. 4-6].)
If the NETSMART 1500 client still fails, contact Fujitsu Network Communications Inc.
(FNC) technical support.
3.4.3
Step
Task
Shut down all NETSMART 1500 clients. (Refer to Section 3.3.4, Shut Down Client
[p. 3-11].)
Shut down the NETSMART 1500 server. (Refer to Section 3.3.3, Shut Down Server
[p. 3-10].)
Restart the NETSMART 1500 server. (Refer to Section 3.3.1, Start Server [p. 3-8].)
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Troubleshooting Server and Clients
Reboot the NETSMART 1500 server using the following command as the root user:
/usr/sbin/shutdown -i6 -y -g30
After the reboot is completed, restart the NETSMART 1500 server. (Refer to Section 3.3.1,
Start Server [p. 3-8].)
Restart the NETSMART 1500 clients. (Refer to Section 3.3.2, Start Client on the Server
[p. 3-9].)
3.4.4
Restore Server
Use the Solaris ufsrestore command to restore files from tape. Refer to the
ufsrestore man page for additional information. There are no Heartbeat menu items
available for restoring UNIX files backed up through the NETSMART 1500 back-up
procedure.
You can also use the Admintool described in Section 4.13, Admintool [p. 4-31].
FNC-1500-0050-110
Issue 1, January 2007
3-15
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.4.5
3.5
Service packsare released periodically to provide support for new releases and NEs,
feature enhancements, and problem corrections. They are cumulative.
SUBSECTION
PAGE
3.5.1
3-17
3.5.2
3-18
3.5.3
Enable EMLNML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-19
3.5.4
Disable EMLNML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-19
3.5.5
3-20
3.5.6
3-21
3.5.7
3-22
3.5.8
3-22
3.5.9
Enable Webservice. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-23
3.5.10
Disable Webservice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-24
3.5.11
3-25
3.5.12
3-26
3.5.13
3-26
3.5.14
3-27
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
NETSMART 1500 contains optional features. These features are disabled by default during
the NETSMART 1500 installation. The optional NETSMART 1500 features are as follows:
PMThis feature collects performance monitoring (PM) statistics from NEs.
WebserviceThis feature allows users limited access to NETSMART 1500 through a
Web browser.
EMLNMLThis feature supports the Element Management Layer Network
Management Layer (EMLNML) interface for communicating with an
EMLNML-compliant NMS.
TL1 uplinkThis feature provides uplink to TL1OSS communications.
SNMP uplinkThis feature acts as an SNMP agent for SNMP-based fault
management.
FW74NMThis feature enables and disables the node managers for FLASHWAVE
7410 and FLASHWAVE 7420 NEs.
Note: You can check the status (active/inactive) of these features using Heartbeat as described
in Section 3.4.1, Check the Status of Server Processes [p. 3-12].
The NETSMART 1500 processes associated with each of the above features are referred to as
a program group. Optional program groups can be started and stopped independently from
the set of core (nonoptional) NETSMART 1500 processes.
The NETSMART 1500 software remembers which optional program groups have been
enabled/disabled even after NETSMART 1500 is stopped and subsequently restarted.
3.5.1
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-17
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
The PM optional program group is enabled. Messages similar to the following display:
Running /opt/NetSmart/bin/netsmart startgroup pm
Group pm starting...
nvt1, PM-ES, Server started, Sun Jun 25 14:53:55 PDT 2006
nvt1, PerformanceManagement, Server started, Sun Jun 25 14:53:55 PDT 2006
pm, Server group started, Sun Jun 25 14:53:55 PDT 2006
Group pm started.
done
Hit any key to Continue.
3.5.2
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.3
Enable EMLNML
This procedure enables the EMLNML optional program group.
Enable EMLNML as follows:
Step
1
Task
3.5.4
Disable EMLNML
Disable EMLNML as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-19
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.5
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
The TL1 uplink optional program group is enabled. Messages similar to the following display:
Running /opt/NetSmart/bin/netsmart startgroup tl1uplink
Group tl1uplink starting...
nvt1, TL1UplinkServer, Server started, Sun Jun 25 14:53:55 PDT 2006
tl1uplink, Server group started, Sun Jun 25 14:53:55 PDT 2006
Group tl1uplink started.
done
Hit any key to Continue.
3.5.6
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-21
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.7
Step
1
Task
3.5.8
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.9
Enable Webservice
Enabling the Webservice option allows access to limited NETSMART 1500 features through
a Web browser.
Enable Webservice as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-23
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.10
Disable Webservice
Disable Webservice as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.11
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-25
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.12
Step
1
Task
3.5.13
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling Program Groups
3.5.14
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-27
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 Trace Menu
3.6
3.7
SUBSECTION
PAGE
3.7.1
3-29
3.7.2
3-34
3.7.3
3-36
An NE software generic is the software component for a Fujitsu NE. As new features are
added to the NEs, new software generics are distributed. The software generics are distributed
on CDs.
This section describes how to install the NE software generics into NETSMART 1500.
Procedures for this activity differ based on the configuration of the machine used. The
following subsections describe procedures for installing NE software generics using a machine
with a CD drive and a machine without a CD drive.
Note: For more information about administration of NE software, refer to FNC-1500-0050-260,
NE Software Management.
Note: For an alternative method of loading NE software generic files using a server script, refer
to Section 3.7, Installing NE Software Generics [p. 3-28].
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
3.7.1
Task
N E
S O F T W A R E
G E N E R I C
F I L E S
This option will get the NE generic software files from the CD
and load them into a user specified directory.
Enter the media in which you have files to be copied
1 floppy
2 cdrom
3 harddisk
Enter selection (default: harddisk) [?,??]:
FNC-1500-0050-110
Issue 1, January 2007
3-29
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
Enter the desired destination directory, or press RETURN/ENTER to accept the default.
Enter a destination directory for the download files
[default=/opt/NetSmart/download_files]:
Creating subdirectory /opt/NetSmart/download_files/F600201V ..
Copying files from CDROM #1 ..
1) a600201v.pgm to
/opt/NetSmart/download_files/F600201V/A600201V.PGM
2) d600201v.pgm to
/opt/NetSmart/download_files/F600201V/D600201V.PGM
3) f600201v.con to
/opt/NetSmart/download_files/F600201V/F600201V.CON
4) f600201v.sig to
/opt/NetSmart/download_files/F600201V/F600201V.SIG
5) h600201v.pgm to
/opt/NetSmart/download_files/F600201V/H600201V.PGM
6) ipl_25~1.hex to
/opt/NetSmart/download_files/F600201V/IPL_25~1.HEX
7) ipl_25~2.hex to
/opt/NetSmart/download_files/F600201V/IPL_25~2.HEX
8) m600201v.pgm to
/opt/NetSmart/download_files/F600201V/M600201V.PGM
9) s600201v.pgm to
/opt/NetSmart/download_files/F600201V/S600201V.PGM
Copy of files from CDROM #1 completed
Ejecting CDROM ...
Is there a CDROM #2 in this set (y/n)? [y,n,?]
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
b.
c.
Enter y.
d.
If NO:
Enter n.
Is there a CDROM #2 in this set (y/n)? [y,n,?] n
Copy completed successfully for NE F600201V
Hit any key to Continue.
Start the NETSMART 1500 client. (Refer to Section 3.3.2, Start Client on the Server
[p. 3-9].)
The NETSMART 1500 client starts up and displays the Topology Management window after
logon and initialization.
10
From the Applications menu, select Maintenance > Software Repository Management.
FNC-1500-0050-110
Issue 1, January 2007
3-31
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
The Software Repository Management window opens (Figure 3-1 [p. 3-32]).
11
Select the Software Generic tab by clicking the Software Generic tab in the lower-left corner of
the screen.
12
Click the Install toolbar icon in the upper-left corner of the screen.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
The Install Software Generic dialog box opens (Figure 3-2 [p. 3-33]).
13
In the Source Hostname field, enter the name of the machine where the temporary directory in
Step 7 [p. 3-30] was created. The name of the machine may already be entered correctly by
default.
OR
In the Source IP Address field, enter the IP address of the machine where the temporary
directory in Step 7 [p. 3-30] was created. The IP address may already be entered correctly by
default.
14
In the User Name field, enter a valid UNIX user name. The user name is used by the File
Transfer Protocol (FTP) process to log on the machine with the IP address specified in
Step 13 [p. 3-33].
15
In the Password field, enter the password associated with the UNIX user name specified in
Step 14 [p. 3-33].
16
In the Source Directory field, enter the name of the directory where the software generic files
are located. If the default directory suggested in Step 7 [p. 3-30] was used, then enter
/opt/NetSmart/download_files.
17
In the Select FGA Date fields, select the date that should be associated with the software
generic. By default, the current date and time are selected.
FNC-1500-0050-110
Issue 1, January 2007
3-33
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
18
Click Ok.
The files associated with the software generic are now installed in NETSMART 1500.
19
3.7.2
Task
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
Enter the following command to create a temporary directory to store the files that make up
the software generic:
mkdir /tmp/software_generic
Note: You must have a directory for each NE type and load level.
Insert the first (or next) CD into the CD drive of the machine.
The CD is automatically mounted.
Copy the files on the CD to the temporary directory created in Step 2 [p. 3-34] by entering
the following command:
cp /cdrom/cdom0/* /tmp/software_generic
If there are more CDs that are part of the software generic, remove the last CD from the drive,
and repeat Steps 3 [p. 3-34] through 5 [p. 3-34] for the next CD.
Start the NETSMART 1500 client. (Refer to Section 3.3.2, Start Client on the Server
[p. 3-9].)
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
The NETSMART 1500 client starts up and opens the Topology Management window after
logon and initialization.
8
From the Administration menu, select Maintenance > Software Repository Management.
The Software Repository Management window opens (Figure 3-1 [p. 3-32]).
Click the Software Generic tab in the lower-left corner of the screen.
10
Click the Install toolbar icon in the upper-left corner of the screen.
The Install Software Generic dialog box opens (Figure 3-2 [p. 3-33]).
11
In the Source Hostname field, enter the name of the machine where the temporary directory in
Step 2 [p. 3-34] was created. The name of the machine may already be entered correctly by
default.
OR
In the Source IP Address field, enter the IP address of the machine where the temporary
directory in Step 2 [p. 3-34] was created. The IP address may already be entered correctly by
default.
12
In the User Name field, enter a valid UNIX user name. The user name is used by the FTP
process to log on the machine with the IP address specified in Step 11 [p. 3-35].
13
In the Password field, enter the password associated with the UNIX user name specified in
Step 12 [p. 3-35].
14
In the Source Directory field, enter the name of the directory where the software generic files
are located. If the directory suggested in Step 4 [p. 3-34] was used, then enter
/tmp/software_generic.
15
In the Select FGA Date fields, select the date that should be associated with the software
generic. By default, the current date and time are selected.
16
Click Ok.
The files associated with the software generic are now installed in NETSMART 1500.
17
At the UNIX prompt in the terminal window, enter the following command to delete the
temporary directory that was created and all files in that directory:
FNC-1500-0050-110
Issue 1, January 2007
3-35
CHAPTER 3
NETSMART 1500 Administration
Installing NE Software Generics
rm -fr /tmp/software_generic
18
3.7.3
Step
Task
Start the NETSMART client. (Refer to Section 3.3.2, Start Client on the Server [p. 3-9].)
Click the Software Generic tab in the lower-left corner of the screen.
Click the Install toolbar icon in the upper-left corner of the screen.
The Install Software Generic window opens (Figure 3-2 [p. 3-33]).
In the Source Hostname field, enter the name of the remote machine where the software
generic files are loaded. If the remote machine does not have a name, leave this field blank.
In the Source IP Address field, enter the IP address of the remote machine where the software
generic files are loaded.
In the User Name field, enter a valid user name for the remote machine. The user name is used
by the FTP process to log on the remote machine.
In the Password field, enter the password associated with the user name specified in Step 7
[p. 3-36].
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Configuring the ObjectStream License
In the field under Directory Info, enter the fully qualified directory path where the software
generic files are located.
Note: You cannot use the CD-ROM or browse buttons; you must type the path.
10
In the Select FGA Date fields, select the date that should be associated with the software
generic. By default, the current date and time are selected.
11
Click Ok.
The files associated with the software generic are now installed in NETSMART 1500.
3.8
Step
Task
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
Stop the ObjectStream License server. At the prompt, enter the following command:
/etc/init.d/OBJStream stop
To confirm that the ObjectStream License server has stopped, enter the following command:
ps -ef | grep -i objstream | grep -v grep
FNC-1500-0050-110
Issue 1, January 2007
3-37
CHAPTER 3
NETSMART 1500 Administration
Configuring the ObjectStream License
If the process is still running, a line displays showing details of the ObjectStream process;
otherwise, no line displays.
8
Start the ObjectStream License server. At the prompt, enter the following command:
/etc/init.d/OBJStream start
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Export Security Data
3.9
Running ExportSecurity
Enter a directory for exporting files or use default
[/opt/NetSmart/Properties/nif]:
/opt/NetSmart/Properties/nif
Found addAccessGroup.cfg
Found addSecurityUser.cfg
ContainerInitComponent.initializeComponents(): total no. of application
components loaded = 1
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
3-39
CHAPTER 3
NETSMART 1500 Administration
Import Security Data
3.10
Running ImportSecurity
This utility expects data files addSecurityUser.cfg and/or
addAccessGroup.cfg
Enter a directory for importing files or use default:
[/opt/NetSmart/Properties/nif] [?,q]
/opt/NetSmart/Properties/nif
Importing Roles and Users
ContainerInitComponent.initializeComponents(): creating component "FwJVM"
ContainerInitComponent.initializeComponents(): component "FwJVM" was
created successfully
ContainerInitComponent.initializeComponents(): total no. of application
components loaded = 1
Role ATMManagementUser already exists.
Role NetSmartUserAdmin already exists.
Role EmlNmlUser already exists.
Role CrossConnectsUser already exists.
Role NetSmartServerAdmin already exists.
Role ProfileManagementUser already exists.
Role BLSRUser already exists.
Role PerformanceManagementUser already exists.
Role ConfigurationUser already exists.
Role NECutThroughUser already exists.
Role ConnectionManagementUser already exists.
Role AlarmsUser already exists.
Role ReadOnlyUser already exists.
User netsmart already exists
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Resync NE Data
3.11
Resync NE Data
This option should be executed during a migration process with a release upgrade or a
hardware upgrade of NETSMART 1500. Normally, the option is invoked automatically by
the logical restore function. However, you can choose to invoke it after the logical restore. The
resync function, which performs a resync and update of the NETSMART 1500 database,
performs a first time logon of the NEs so that NETSMART 1500 has the updated NE
information.
In such scenarios, you can execute it from this menu option. This action forces logons to all
the NEs that existed and were autodiscovered before migration. It also resynchronizes all
equipment, cross-connects, shelves, and alarms for those NEs.
Resync Network Initiated. Please wait..
Done with initialization
Logged into MAXFLM1502402 successfully
Logged into MAXFLM1502405 successfully
Logged into MAXFLM1502437 successfully
Resync Network Done.
For all NEs that failed, manually login from GUI
FNC-1500-0050-110
Issue 1, January 2007
3-41
CHAPTER 3
NETSMART 1500 Administration
Configuring the Communication Server/NE Logon
3.12
SUBSECTION
PAGE
3.12.1
3-42
3.12.2
3-44
3.12.3
3-44
This section describes the parameters that control NE logon retries and NE logon delay. An
NE logon retry is initiated in NETSMART 1500 when the NE logon fails on initial
NETSMART 1500 startup. The NE logon delay is the time that the system waits after TCP is
established with an NE before sending an ACT-USER command to the NE. This section
provides an overview of the algorithms and describes how to modify configurable parameters.
Note: UID should be a Level 4 UID user to avoid password-aging problems.
3.12.1
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Configuring the Communication Server/NE Logon
Table 3-3 [p. 3-43] lists the configurable communication server parameters associated with
the NE logon reconnect algorithm, a description of each, and their default values.
Table 3-3: NE Logon Reconnect Algorithm Parameters
Parameter Name
Description
Default Value
comms.connection.loginInitialRetryTime
45 seconds
comms.connection.loginInitialRetries
comms.connection.loginInitialRetryTime
300 seconds
Upon initial logon failure, the NETSMART 1500 communication servers use the reconnect
algorithm to control logon attempts. There are two parts to the algorithm: the initial retry and
the back-off retry. The initial retry is used after the first failed attempt.
During the initial retry, the communication servers wait 45 seconds (default value for the
comms.connection.loginInitialRetryTime parameter) between each retry and
attempt to connect to the NE five times (default value for the
comms.connection.loginInitialRetries parameter). If the NE is not
successfully logged on during the initial retry period, then the back-off retry time period is
used.
During this period, the communication servers try to log on an NE every 300 seconds
(default value of the comms.connection.loginBackOffRetryTime). The number
of tries during this period is infinite.
A NETSMART 1500 user can force an NE to the initial retry algorithm by logging off and
logging back on an NE through the NETSMART 1500 client GUI. If a NETSMART 1500
end user tries to initiate a logon by clicking login on the NETSMART 1500 client GUI, the
following message may be displayed: NETSMART is already attempting to
re-establish communications with this NE. In this scenario, the user must
select logout and then login to force the NE back into the initial retry algorithm.
FNC-1500-0050-110
Issue 1, January 2007
3-43
CHAPTER 3
NETSMART 1500 Administration
Configuring the Communication Server/NE Logon
3.12.2
3.12.3
Parameter Name
Description
Default Value
comms.connection.ACT-USER.waitTime
1000
milliseconds
Step
Task
Identify the parameters that must be changed. (Refer to Table 3-3 [p. 3-43].)
Edit the following communication server properties file using the preferred UNIX editor:
/opt/NetSmart/Properties/DefaultProperties/CommsServer/
CommunicationServer.properties
Stop the NETSMART 1500 servers. (Refer to Section 3.3.3, Shut Down Server [p. 3-10].)
Start the NETSMART 1500 servers. (Refer to Section 3.3.1, Start Server [p. 3-8].)
The new values of the communication server configuration parameters are in effect.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Security E-Mail Notification
3.13
FNC-1500-0050-110
Issue 1, January 2007
3-45
CHAPTER 3
NETSMART 1500 Administration
Broadcast Messages
3.14
Broadcast Messages
This procedure includes the following subprocedures:
NO.
SUBPROCEDURE
PAGE
3.14.1
3-46
3.14.2
3-48
3.14.3
Client Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-48
At times, NETSMART 1500 Administrators must inform users (NETSMART 1500 Clients)
about any administrative tasks to be done on the server, such as maintenance work, server
shutdown, restart, or failover. Using the Broadcast feature, these messages can be broadcast to
all clients connected to the federation. These broadcast messages give users the opportunity to
save their work and exit their respective client before systems are turned off.
The NETSMART 1500 administrator can send broadcast messages in the two ways:
Messages contain a priority, time tamp, host name from which they are sent, and the content
itself.
3.14.1
Use the following procedure to send a broadcast message from a NETSMART 1500 server
using a command line:
Step
1
Task
command is a NETSMART 1500specific function and does not use the UNIX rwall
command.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Broadcast Messages
Assign a priority to the message by typing 1 for High priority or 2 for Normal
priority.
Example: Choose a number from the above: 2
Press RETURN.
The command tries to send messages to the connected NETSMART 1500 clients. After the
command completes sending the message, a confirmation message is displayed at the prompt.
This confirmation message displays the delivery status of the message, including information
about any delivery successes, failures, and time-outs that occurred while the message was
being sent.
FNC-1500-0050-110
Issue 1, January 2007
3-47
CHAPTER 3
NETSMART 1500 Administration
Broadcast Messages
3.14.2
3.14.3
Client Notification
No action is required on the part of a NETSMART 1500 client user to view a Broadcast
message. When a NETSMART 1500 client receives a message, the Admin Broadcast Message
dialog box pops up, displaying the details associated with the message. Refer to
FNC-1500-0050-200, Introduction, Section 8.3.7, Sending a Broadcast Message [p. 8-21], for
a detailed description of the Client Notification feature.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Pre-Logon Warning Banner Administration
3.15
SUBPROCEDURE
PAGE
3.15.1
3-50
3.15.2
3-51
3.15.3
3-51
3.15.4
3-52
3.15.5
3-53
The pre-logon warning banner displays legal or business policies that you want users to
acknowledge before they log on to your NETSMART 1500 system. By default, the feature is
disabled. If enabled, the Warning dialog box (Figure 3-3 [p. 3-49]) opens, and the user must
click I Agree before they can log on.
FNC-1500-0050-110
Issue 1, January 2007
3-49
CHAPTER 3
NETSMART 1500 Administration
Pre-Logon Warning Banner Administration
3.15.1
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Pre-Logon Warning Banner Administration
3.15.2
Step
1
Task
3.15.3
FNC-1500-0050-110
Issue 1, January 2007
3-51
CHAPTER 3
NETSMART 1500 Administration
Pre-Logon Warning Banner Administration
Task
Enter the absolute path and name of the file that you want to import.
Example: /opt/NetSmart/tmp/newBanner
The file is imported. A screen similar to the following displays:
Enter absolute path and name for the warning banner file : [?,q]
/opt/NetSmart/tmp/newBanner
Please wait while we import pre-login warning banner file :
/opt/NetSmart/tmp/newBanner
PreLogin Warning Banner: Imported warning banner text
Hit any key to Continue.
3.15.4
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Pre-Logon Warning Banner Administration
Task
3.15.5
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-53
CHAPTER 3
NETSMART 1500 Administration
Pre-Logon Warning Banner Administration
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling RTRV-HDR Logging
3.16
SUBSECTION
PAGE
3.16.1
3-55
3.16.2
3-56
These subsections describe how to enable or disable TL1 Heartbeat event logging. When TL1
Heartbeat event logging is enabled, commands sent from NETSMART 1500 and responses
received are logged in the TL1 Heartbeat event log, which can be viewed using TL1 Heartbeat
Log Management (refer to FNC-1500-0050-420, Reports, Section 3.6, TL1 Heartbeat Log
Management [p. 3-12]).
CAUTION:
These procedures are not permanent across server restarts. After a restart, TL1 Heartbeat
event logging reverts to the default as described below.
The default state of TL1 Heartbeat event logging is controlled by the following
communication server property, which has a value of false (disabled) at installation.
comms.mediator.enableRTRV-HDRLogging=false
However, you can change the default to enabled by changing the value of
comms.mediator.enableRTRV-HDRLogging to true using the process described in
Section 3.12.3, Modifying Communication Server Properties [p. 3-44].
3.16.1
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-55
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling RTRV-HDR Logging
The TL1 Heartbeat event logging is enabled. A screen similar to the following displays:
Note: This update is lost when servers restart unless steps are taken to save it. Refer to
Section 3.16, Enabling/Disabling RTRV-HDR Logging [p. 3-55], for information on making
changes permanent across restarts.
Running tl1heartbeat enable
VBBindInterceptor() instantiated
POA created: /
POA created: fwjvm-zappa:1024699431705ContainerPOA
POA created: ComponentPOAPOA created:
ContainerInitComponentContainerInitComponent.initializeComponents():
creating component
"FwJVM"
POA created: FwJVM
ContainerInitComponent.initializeComponents(): component "FwJVM" was
created successfully
ContainerInitComponent.initializeComponents(): total no. of
applicationcomponents loaded = 1
============================================================
Found 2 Communication Servers
Updating CommunicationServer-2
Updated CommunicationServer-2 Successfully
Updating CommunicationServer-1
Updated CommunicationServer-1 Successfully
============================================================
Updated 2/2 Servers Successfully
This update will be lost when Servers Restart. Please consult the
documentation on how to make these changes permanent
============================================================
Hit any key to Continue.
3.16.2
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Enabling/Disabling RTRV-HDR Logging
FNC-1500-0050-110
Issue 1, January 2007
3-57
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
3.17
3.17.1
NO.
SUBSECTION
PAGE
3.17.1
3-58
3.17.2
3-59
3.17.3
3-69
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
-> M A I N
M E N U
*****
Userid: netsmart
Federation: henley
L I C E N S E
I N F O R M A T I O N
: 50
: 10
: BRONZE
Maintenance Duration
: 0
: PM EMLNML TL1UPLINK
3.17.2
PAGE
3-62
Upgrading a Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-65
3-67
FNC-1500-0050-110
Issue 1, January 2007
3-59
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
This subsection describes how to upgrade your NETSMART 1500 license. You can upgrade
your NETSMART 1500 license to:
You can do a license upgrade when servers are running (online upgrade) or when servers are
not running (offline upgrade). All upgrades require a new license key. Refer to
FNC-1500-0050-100, Installation Guide, Section 2.4.2, Obtain NETSMART 1500 License
Key [p. 2-10], for more information. The new license key can include any combination of the
possible upgrades.
You can add the number of NEs and the concurrent user sessions by obtaining a license key
for the additional number of NEs or user sessions that you want. For example, if you have a
50-NE license and want to upgrade to a system with 200 licensed NEs, obtain a license key
for the 150 additional NEs, and then do the NETSMART 1500 license upgrade process.
CAUTION:
You cannot upgrade the number of NEs beyond the hardware capacity of your system. The
NE Utilization report (FNC-1500-0050-300, Datacomm and Security Management,
Section 2.3, NEU Management [p. 2-5]) displays the maximum capacity of your hardware.
The licensing factor for FLASHWAVE 4010 NEs is 0.2 for 10 percent of licensed network
element units (NEUs) and 1 for the remaining FLASHWAVE 4010 NEs. For example, if you
have purchased a license key for 50 NEUs but you want to add only FLASHWAVE 4010
NEs, you can calculate how many FLASHWAVE 4010 NEs you can add based on the
following:
The weight for FLASHWAVE 4010 NEs is 0.2; therefore, if you add five
FLASHWAVE 4010 NEs, it would be equal to one regular NE.
The total FLASHWAVE 4010 NEs that can be added is 45 (90 percent of 50 NEUs at
regular factor 1) plus 25 (10 percent of 50 NEUs at FLASHWAVE 4010 factor 0.2.).
Thus, you can add 70 FLASHWAVE 4010 NEs for a 50 NE license key.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
For changing a maintenance contract, upgrade the license policy (GOLD, SILVER, or
BRONZE) or the maintenance duration (specified number of days). As an example of adding
to the duration of the contract, if the initial maintenance duration was for 200 days from the
installation date, upgrading the maintenance duration by 100 days sets the maintenance
duration to 300, starting from the installation date.
Upgrade your NETSMART 1500 license as follows:
Step
Task
Obtain a license key for the upgrade. Refer to FNC-1500-0050-100, Installation Guide,
Section 2.4.2, Obtain NETSMART 1500 License Key [p. 2-10].
FNC-1500-0050-110
Issue 1, January 2007
3-61
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
-> M A I N
M E N U
Federation: henley
L I C E N S E
U P G R A D E
: 50
: 10
: BRONZE
Maintenance Duration
: 0
: EMLNML TL1UPLINK
/opt/NetSmart/install/licenseupgrade
Please enter the License Key for NETSMART 1500 License Upgrade [?,q]
47beqed2531jn3f6547b9x8ytknafy85sohb3f0rscilu
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
After entering the license key for the upgrade, the system completes the upgrade and displays
a summary of how the upgrade impacts your system. This example shows information from
the license key that indicates an upgrade in the licensed number of NEs and user sessions:
*************** Information from License key ***************
: 1
: 1
: none
Maintenance Duration
: 0
************************************************************
If the attempted upgrade is beyond the hardware capacity of your system, the license upgrade
aborts, and a message displays similar to the one below. For example, if you try to update
1,000 NEs to 3,000 NEs, the procedure will fail if the hardware supports only 2,000 NEs. For
hardware upgrade information, refer to FNC-1500-0050-100, Installation Guide, Chapter 3,
Software and Hardware Upgrades.
The NE capacity does not allow update of NEs to 3000. The System allows
upgrade up to 2000 NEs. You are required to do an Hardware Upgrade first,
if you want to update the installation to 3000 NEs
License upgrade was not successful
License upgrade was unsuccessful for the following reason:
Check failed on the license key. Upgrade attempted with an invalid license
key
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
3-63
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
The summary information for upgrading the NEs and user sessions displays:
*****
Heartbeat
-> M A I N M E N U
-> 1. NETSMART 1500 Administration
-> 18. License Information
*****
Userid: netsmart
Federation: henley
L I C E N S E
I N F O R M A T I O N
:
:
:
:
:
51
11
BRONZE
0
EMLNML TL1UPLINK
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
Upgrading a Feature
After entering the license key for the upgrade, the system completes the upgrade and displays
the optional feature licensed (in this example, PM) as shown in the following example.
Additionally, for feature upgrades, online upgrades prompt you for automatic enabling of the
feature.
*************** Information from License key ***************
: 0
: 0
: none
Maintenance Duration
: 0
: PM
************************************************************
If you enter Y, the feature is enabled. Otherwise, it is disabled by default. Certain features may
require additional configuration and cannot be enabled automatically. During online feature
upgrade, the system displays:
Feature X could not be enabled because it requires configuration.
During feature upgrades, certain features, such as EMLNML and HA, cannot be upgraded
because they need to be installed. If an upgrade is attempted for a feature that requires
installation, the system displays:
Feature EMLNML could not be licensed because it requires installation.
Please re-install by obtaining a new license key.
FNC-1500-0050-110
Issue 1, January 2007
3-65
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
-> M A I N
M E N U
*****
Userid: netsmart
Federation: henley
L I C E N S E
I N F O R M A T I O N
: 51
: 11
: BRONZE
Maintenance Duration
: 0
: PM EMLNML TL1UPLINK
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
: 0
: 0
: GOLD
Maintenance Duration
: 200
: none
************************************************************
FNC-1500-0050-110
Issue 1, January 2007
3-67
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
-> M A I N
M E N U
*****
Userid: netsmart
Federation: henley
L I C E N S E
I N F O R M A T I O N
: 51
: 11
: GOLD
Maintenance Duration
: 200
: PM EMLNML TL1UPLINK
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
3.17.3
If an upgrade is tried with either an unreachable peer node or the role of the peer node is not
in the UP state, the following message displays:
Upgrade failed due to
1. Unreachable peer node
2. Required servers not running on peer node. Please check connectivity to
the peer node and try again.
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-69
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
Federation: clapton
L I C E N S E
I N F O R M A T I O N
: 1000
: 50
: SILVER
Maintenance Duration
: 0
: PM TL1UPLINK HA
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
Federation: clapton
L I C E N S E
U P G R A D E
: 1000
: 50
: SILVER
Maintenance Duration
: 0
***
*** Finding Current Federation
***
Federation: clapton
Please enter the License Key for NETSMART 1500 License Upgrade [?,q]
FNC-1500-0050-110
Issue 1, January 2007
3-71
CHAPTER 3
NETSMART 1500 Administration
NETSMART 1500 License Administration
: 0
: 1
: none
Maintenance Duration
: 0
: none
************************************************************
-> M A I N
M E N U
*****
Userid: netsmart
Federation: clapton
L I C E N S E
I N F O R M A T I O N
: 1000
: 51
: SILVER
Maintenance Duration
: 0
: PM TL1UPLINK HA
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Service Pack Installation on HA-Enabled Federations
3.18
SUBSECTION
PAGE
3.18.1
3-73
3.18.2
3-74
Service pack installation on HA-enabled federations is similar to that for non-HA enabled
systems. However, the service pack installation process must be done on both server machines
of the HA pair.
Depending on the fixes in the service pack, there are two ways of installation on HA
platforms:
3.18.1
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
3-73
CHAPTER 3
NETSMART 1500 Administration
Service Pack Installation on HA-Enabled Federations
Follow the procedure specified in Section 2.8, Service Pack Installation [p. 2-42] to install the
service pack on the passive server. During this process, the active server continues to be
available for NETSMART 1500 clients.
Note: The active host should be used mostly for monitoring and minimal configuration changes to
prevent the replication queues from getting full before the passive is brought back. Do not do a bulk
resynchronization of NEs during the service pack installation processthis is to avoid full database
resynchronization after the passive node is brought up.
After the passive server installation of the service pack is completed, the passive node must be
reactivated. To do this, log on to the active node as netsmart and execute netsmart
start to start the HA cluster.
If the queues are not full, this step does not cause a full database resynchronization and
finishes in 5 to 10 minutes.
CAUTION:
Do not start in single node modethis results in two active systems updating the same
database.
4
After the completion of Step 3 [p. 3-74], and the passive node is UP, a manual HA failover
can be initiated.
This process switches the roles of active and passive servers and is documented in
FNC-1500-0050-110, Server Administration Guide, Chapter 7. During this time, the clients
cannot communicate with the servers and the downtime is less than 20 minutes.
The clients are automatically prompted to auto-upgrade, after the failover is completed and
passive server with the service pack installed becomes active.
3.18.2
Install the service pack on the now passive server using the procedure described in Step 1
[p. 3-73] above.
Step
Task
Login on to the active node with your netsmart user login and password.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Service Pack Installation on HA-Enabled Federations
Install the service pack following the procedure specified in FNC-1500-0050-100, Installation
Guide, Chapter 3.
From the active server, login as netsmart user and initiate netsmart start to start the HA
cluster. The netsmart services will start on the active and passive servers.
Note: After Step 3 [p. 3-75], the clients will lose communication with the server cluster. Once the HA
cluster is brought back up in Step 6 [p. 3-75], the clients will auto-detect the upgraded server software
and prompt for an auto-upgrade of client software.
FNC-1500-0050-110
Issue 1, January 2007
3-75
CHAPTER 3
NETSMART 1500 Administration
Transaction Debugging Client
3.19
SUBSECTION
PAGE
3.19.1
3-77
3.19.2
3-77
This section describes the Transaction Debugging Client and explains how to start and use the
utility.
At times, the incorrect behavior of the system might leave a transaction incomplete. Because
the transaction service acquires write locks on the participating persistent objects, an
incomplete transaction makes these objects unusable.
The transaction debugging client provides a way to find out about incomplete transactions,
their duration, the list of persistent objects the transactions are holding, and a way to roll back
the transactions forcefully. The client also identifies transactions that have gone out of context
and that have left some persistent objects locked. For these transactions, the utility provides
details about where and when the locks were acquired on them and releases them if possible.
The following are the most common causes of incomplete transactions:
A transaction was started inside a try catch block and, in an exception, was not rolled
back.
A transaction was started inside a try catch block, and the exception was never caught.
Attempt to acquire a lock outside the transaction. This lock is not released unless it is
specifically released by the developer.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Transaction Debugging Client
3.19.1
Step
Task
Start Heartbeat.
3.19.2
PAGE
3-77
3-78
3-78
3-78
3-78
3-78
3-78
3-78
3-79
Get the Dump of Information about the Reader/Writer Queues for a Specific
Transaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3-79
3-79
3-79
FNC-1500-0050-110
Issue 1, January 2007
3-77
CHAPTER 3
NETSMART 1500 Administration
Transaction Debugging Client
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Transaction Debugging Client
Get the Dump of Information about the Reader/Writer Queues for a Specific
Transaction
The option retrieves the information about the reader/writer queues of the specific lockset for
a specific transaction.
FNC-1500-0050-110
Issue 1, January 2007
3-79
CHAPTER 3
NETSMART 1500 Administration
LNE Discovery
3.20
LNE Discovery
This section describes using the LNE Discovery feature in Heartbeat. With this feature, you
can discover, view, and delete subnetworks. This procedure uses the option LNE Discovery as
an example.
Follow these steps for LNE Discovery:
Step
Task
Start Heartbeat.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
LNE Discovery
/*****************************************/
/*****************************************/
Enter choice :
Option 2 discovers subnetworks for links that do not belong to any subnetwork and
preserves existing subnetworks.
FNC-1500-0050-110
Issue 1, January 2007
3-81
CHAPTER 3
NETSMART 1500 Administration
Connection Discovery
3.21
Connection Discovery
This section describes using the Connection Discovery feature in Heartbeat. With this menu,
you can do a variety of discovery functions. This procedure uses the option Discover TDM
(SONET/SDH) Connections as an example.
Note: The Connection Discovery script does not delete existing connections.
Task
Start Heartbeat.
kzhu
Federation:
frakenberry
C O N N E C T I O N
D I S C O V E R Y
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Connection Discovery
kzhu
Federation:
frakenberry
FNC-1500-0050-110
Issue 1, January 2007
3-83
CHAPTER 3
NETSMART 1500 Administration
Topology Discovery
3.22
Topology Discovery
If the Network Topology Discovery component is installed, you may access the Topology
Discovery Administration client by selecting the 22 Topology Discovery Administration option
from the NETSMART 1500 Administration menu. From the Topology Discovery
Administration menu you can select the options described in this section.
The state of the Topology Discovery server displays along with the available administrative
options for the server. The Topology Discovery server can be in any one of the following four
states:
DOWNState that indicates that the Topology Discovery server is not active; that is, no
NE polling activity is scheduled.
UPState that indicates that the Topology Discovery server is active; that is, NE
polling is scheduled for execution to discover a network's NEs and links.
The Topology Discovery server is not enabled as part of the initial NETSMART 1500 server
startup. To enable the server, please refer to the Enable server menu option description. The
enabling and disabling of the Topology Discovery server is logged in the Activity Console and
Log of NETSMART 1500.
Menu options for the Topology Discovery Administration Client menu include:
0) Quit
Selecting this option exits the client and returns the user to the Heartbeat administration
menu.
1) Disable server
Selecting this option disables the Topology Discovery server if it is in the UP state, and stops
all NE polling activities. If the Topology Discovery server is in the DOWN, STARTING, or
STOPPING state, this action has no effect.
Note: When the Topology Discovery server has been disabled, all discovered data is flushed
from the server. The Refresh button flashes in the NETSMART 1500 Topology window and the
disable operation is logged in the NETSMART 1500 Activity Console and Log.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Topology Discovery
This indicates that the server is terminating polling activities and releasing resources. When
the cleanup is complete, the following message displays:
StateChange from STOPPING to DOWN
Once this message is received, disabling of the Topology Discovery server is complete. Note
that it may take several minutes for this last message to appear due to active polling activities
that are allowed to complete before cleanup can be initiated.
CAUTION:
If you disable and then restart the NETSMART 1500 topology server, the disable operation is
remembered and the topology discovery server is not enabled when the server comes back
into service.
2) Enable server
Selecting this option enables the Topology Discovery server if it is in the DOWN state, to
start NE polling. If the Topology Discovery server is in the UP, STARTING, or STOPPING
state, this action has no effect.
Note: The TOPODISCOVERY security area is automatically created by the NETSMART 1500
Version 5 application if Topology Discovery is enabled during the NETSMART 1500 Version 5
installation. You must add the user ID and password ID (UID/PID) for this security areaif no
UID/PIDs are found for the TOPODISCOVERY security area, the Topology Discovery server will not
start.
The TOPODISCOVERY account group should not be shared with any other security
areas.
The TOPODISCOVERY security area should not be associated with any NEs.
The TOPODISCOVERY security area and account group should not be deleted.
You may modify the accounts in the TOPODISCOVERY account group to further
customize the NE logon accounts that Topology Discovery will use for its NE polls.
If the enable request is accepted, the following message displays:
StateChange from DOWN to STARTING
FNC-1500-0050-110
Issue 1, January 2007
3-85
CHAPTER 3
NETSMART 1500 Administration
Topology Discovery
This indicates that the server is initializing for NE polling. When initialization is complete,
the following message displays:
StateChange from STARTING to UP
When this message is received, enabling of the Topology Discovery server is complete. The
Topology Discovery server continuously polls NEs to discover new NEs and links until it is
disabled.
If the NETSMART 1500 Topology Server is restarted, the enable operation is remembered
such that the Topology Discovery server is enabled when the Topology Server comes back into
service.
3) Re-load filters
Selecting this option reloads the filters specified in the property file:
${NETSMART_HOME}/Properties/TopologyServer/discovery/
Filter.properties
Filters allow you to fine-tune the set of discovered NEs that will be reported by Topology
Discovery. Currently, filters can be defined based on TID, OUI (Organizationally Unique
Identifier), or system ID (from the OSI network address for the NE).
For each filter, you can specify one of the two available policies: ACCEPT or REJECT.
ACCEPT indicates that any NEs matching the filter criteria will be reported. REJECT
indicates that any NEs matching the filter criteria will not be reported.
There is a set of preloaded filters that is stated in the comments of the property file. By
default, discovered NEs are reported if no matching filters are found.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Topology Discovery
Modify the property file if you wish to specify filters. Each filter is in the following format:
filter.<type>.<target> = <policy>
where:
Examples:
Define a REJECT TID filter for the NE FLASHWAVE4100-B:
filter.tid.FLASHWAVE4100-B = REJECT
filter.oui.00107B = REJECT
filter.sysid.00000E4622A4 = ACCEPT
After the changes are complete, select this option to reload the filters. The following message
displays:
* All previously loaded filters will be removed and replaced with the
filters defined in the property file <full path of the property file>
Continue (y/n) [n]?
Regardless of which option you select, the Topology Discovery Administration menu is
displayed afterwards.
FNC-1500-0050-110
Issue 1, January 2007
3-87
CHAPTER 3
NETSMART 1500 Administration
Configure SONET-SDH Terminology Mapping
Task
Start Heartbeat.
D i s c o v e r y
A d m i n i s t r a t i o n
=================================================================
* Server state is DOWN
0) Quit
1) Disable server
2) Enable server
3) Re-load filters
Enter choice:
3.23
If you make the SONET-SDH terminology feature available, you must choose between three
modes of availability (values 1, 2, or 3 as described in Table 3-5 [p. 3-89]).
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Configure SONET-SDH Terminology Mapping
CAUTION:
If the SONET-SDH terminology mapping feature is available (configuration value set to 1, 2,
or 3 as described in Table 3-5 [p. 3-89]), you cannot switch directly to another mode of
availability (value 1, 2, or 3). You must first make the feature unavailable (set configuration
value to 0, and restart the server). For example, to change from 2 to 3, apply the procedure
below to change from 2 to 0, and then apply it again to change from 0 to 3.
Task
Open the following server properties file in your preferred UNIX editor.
/opt/NetSmart/Properties/TopologyServer/meta/
UserPrefDefault.properties
Edit the file to change the value of the following property to 0, 1, 2, or 3 (refer to Table 3-5
[p. 3-89]):
fenics.gui.crossconnect.isMappedDisplay
Note: You cannot switch between values 1, 2, and 3. Refer to the earlier caution.
Table 3-5: Property Values to Configure SONET-SDH Terminology Mapping
Value
Description
If SONET-SDH terminology mapping is available, users can enable, disable, or configure the feature (for their
client only) as described in FNC-1500-0050-235, Cross-Connect Management, Section 6.2.1, Set SONET-SDH
Terminology Mapping Preferences [p. 6-4].
FNC-1500-0050-110
Issue 1, January 2007
3-89
CHAPTER 3
NETSMART 1500 Administration
Set Branding Image in Topology View
Stop the NETSMART 1500 servers. (Refer to Section 3.3.3, Shut Down Server [p. 3-10].)
Start the NETSMART 1500 servers. (Refer to Section 3.3.1, Start Server [p. 3-8].)
3.24
Step
Task
Start Heartbeat.
From the menu that displays, enter 23 Set Branding Image to display in Topology GUI client.
The following dialog displays:
Please select the image file that needs to be displayed in Topology GUI
clients
Note: The image has to be of 48 or less pixels wide by 24 or less
pixels high and should be in jpg format
Enter absolute path and name for the branding image file:
[?,q]
After typing the absolute path and name for the branding image file, press Enter.
The image selected will now display on the Topology View GUI.
3.25
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 3
NETSMART 1500 Administration
Remove Branding Image from Topology View
Task
Start Heartbeat.
From the menu that displays, enter 24 Remove Branding image from Topology GUI client.
The following dialog displays:
Running removeCustomerBrandingImage
Do you want to remove customer brand image from Topology GUI clients? [y/n]
(default=n)
FNC-1500-0050-110
Issue 1, January 2007
3-91
4
CHAPTERS/APPENDIXES
1 Introduction
2 Oracle Administration
3 NETSMART 1500
Administration
NO.
PROCEDURE
4 UNIX System
Administration
4.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-2
5 Network Administration
4.2
4-3
6 Service Support
4.3
Start Up Workstation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-4
7 High Availability
Administration
4.4
4-4
4.5
4-5
4.6
4-5
4.7
4-11
4.8
4-13
4.9
4-13
4.10
4-14
4.11
4-17
4.12
4-27
4.13
Admintool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-31
4.14
4-31
4.15
4-31
FNC-1500-0050-110
Issue 1, January 2007
PAGE
CHAPTER 4
UNIX System Administration
Overview
4.1
Overview
WARNING:
Some operations described in this chapter use the Heartbeat tool. When using Heartbeat
with root enabled, do not press CTRL+C. Doing so may cause Heartbeat to terminate
abnormally. CTRL+C may be used with Heartbeat when root is not enabled.
This chapter provides UNIX system administration procedures for NETSMART 1500
client workstations and servers.
UNIX administration uses the UNIX System Administration option of the Heartbeat tool.
This option provides the following menu, which is the starting point for most of the
procedures described in this chapter:
U N I X A D M I N I S T R A T I O N
1. System Processes
2. System Backup *
3. NETSMART 1500 Backup *
4. Backup Log
5. System Messages
6. Disk Usage
7. Swap Info *
8. System Activities Menu
9. Cronjobs Menu
10. Defunct Processes
11. Admintool *
e. Enable Root Access
m. Go back to the Main Menu
q. Quit
Table 4-1 [p. 4-2] lists the menu and submenu items for UNIX administration, as well as
where they are described in this practice.
Table 4-1: UNIX Administration Menu Items
Menu Item
Reference
1. System Processes
2. System Backup
4. Backup Log
5. System Messages
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Power Down UNIX Workstation
Reference
6. Disk Usage
7. Swap Info
1. sar
2. vmstat
3. iostat
4. uptime
5. netstat
9. Cronjobs Menu
11. Admintool
4.2
CAUTION:
The NETSMART 1500 client or server application must be properly shut down before
turning off power to the UNIX workstation or server. Turning off the NETSMART 1500 client
workstation or server without using the following procedures could result in corrupted files
or a loss of data. Refer to Chapter 3, NETSMART 1500 Administration, Section 3.3.3, Shut
Down Server [p. 3-10], for NETSMART 1500 server shut-down procedures. Refer to
Chapter 3, NETSMART 1500 Administration, Section 3.3.4, Shut Down Client [p. 3-11], for
NETSMART 1500 client shut-down procedures.
FNC-1500-0050-110
Issue 1, January 2007
4-3
CHAPTER 4
UNIX System Administration
Start Up Workstation
Task
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
At the UNIX prompt, enter init 5 to immediately shut down the system.
OR
At the UNIX prompt, enter shutdown -i0 -g60 -y to warn any users logged on the
system before shutdown. Shutdown occurs 60 seconds after users are warned. The -g60
option specifies a 60-second period where any users on the system are informed that the
system is about to be shut down.
4.3
When the OK prompt is displayed, turn off the computer, monitor, and any peripheral
equipment, if necessary.
Start Up Workstation
The operating system automatically starts when the UNIX workstation or server is turned on.
There are no additional startup procedures required.
4.4
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Shut Down Workstation
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
4.5
4.6
SUBSECTION
PAGE
4.6.1
4-6
4.6.2
4-6
4.6.3
4-8
This section describes the NETSMART 1500 server and client accounts needed to run the
NETSMART 1500 software.
Note: In this section, adding, changing, and deleting UNIX accounts is done through
commands. However, doing UNIX administrative functions through AdminTool, described in
Section 4.13, Admintool [p. 4-31], is also possible.
To log on as the root user, you can enter su - or su -u; however, by doing so, you no
longer have access to NETSMART 1500 user commands.
FNC-1500-0050-110
Issue 1, January 2007
4-5
CHAPTER 4
UNIX System Administration
Configure UNIX User Accounts
4.6.1
Shell
Home
Group
swdl1
/bin/csh
/opt1/swdl1
other
NE software download to an NE
This information is needed for Network Information Services (NIS) or NIS+ setup of the user
account. Provide the information to the Network System Administrator to set up NIS or
NIS+ user account verification.
Note: The NETSMART 1500 software allows any valid group for the software download user.
If the UNIX system is configured to use local files instead of NIS or NIS+, the NETSMART
1500 installation creates the account automatically. No additional configuration is required.
4.6.2
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Configure UNIX User Accounts
The following configuration tasks must be done for a UNIX user to have the correct
configuration to execute the NETSMART 1500 client application.
Configure a NETSMART 1500 client account as follows:
Step
Task
Determine whether the users UNIX shell is ksh or csh. The shell was specified when the
UNIX user account was initially added to the system.
Note: To determine their UNIX shell, a user can enter echo $shell. The system displays either
/bin/csh or /bin/ksh.
Use a text editor, such as vi, to edit the appropriate file in the users home directory:
File for csh
.cshrc
.profile
Note: To determine their home directory, a user can enter cd (to make their home directory the
current directory) and then enter pwd. The system displays the current (home) directory.
Note: In the second ksh line, the second character is a space. The first three characters are
period (.), space ( ), and then dollar sign ($).
FNC-1500-0050-110
Issue 1, January 2007
4-7
CHAPTER 4
UNIX System Administration
Configure UNIX User Accounts
4.6.3
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Configure UNIX User Accounts
Task
FNC-1500-0050-110
Issue 1, January 2007
4-9
CHAPTER 4
UNIX System Administration
Configure UNIX User Accounts
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Add UNIX User Accounts Using Local Files
CONTAINER_NAME NETSMARTSubpanel
CONTAINER_TYPE SUBPANEL
ICON Dtsiflexrclient
LABEL NETSMART Client
PUSH_ACTION NetSmartClientPrompt
HELP_TOPIC FPOnItemTerm
HELP_VOLUME FPanel
}
Restart the users Workspace Manager. To do this, right-click on the desktop and click Restart
Workspace Manager.
4.7
Task
Log on the UNIX host as the root user (or enter su -u).
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
FNC-1500-0050-110
Issue 1, January 2007
4-11
CHAPTER 4
UNIX System Administration
Add UNIX User Accounts Using Local Files
Field
Description
comment
homedir
groupname
shell
username
User ID
To configure the new user to be able to run the NETSMART 1500 client software, follow the
instructions in Section 4.6.2, Configure NETSMART 1500 Client Accounts [p. 4-6].
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Delete UNIX User Accounts Using Local Files
4.8
CAUTION:
Because you may uninstall NETSMART 1500 if the users home directory is
/opt/NetSmart, check with your system administrator before using this procedure.
Step
4.9
Task
Log on the UNIX host as the root user (or enter su -u).
Open a terminal window. (Right-click in the desktop background to bring up the workspace
menu, and then select Tools and then Terminal.)
Any errors that may have occurred during the last reboot
FNC-1500-0050-110
Issue 1, January 2007
4-13
CHAPTER 4
UNIX System Administration
Set the Date and Time on the UNIX System
The /var/adm directory contains several message files. The most recent messages are in
/var/adm/messages (and in messages.0); the oldest are in messages.3. After a
certain time (usually 10 days), a new messages file is created. The messages.0 file is
renamed messages.1, messages.1 is renamed messages.2, and messages.2 is
renamed messages.3. The old /var/adm/messages.3 is deleted.
Because /var/adm stores large files containing messages, crash dumps, and other data, this
directory can consume a lot of disk space. To keep the /var/adm directory from growing
too large, unneeded files should be removed periodically.
4.10
4.10.1
NO.
SUBPROCEDURE
PAGE
4.10.1
4-14
4.10.2
4-16
CAUTION:
This procedure must not be used to change the date and time forward. Refer to
Section 4.10.2, Change the Date and Time Forward [p. 4-16], for the procedure on changing
the date and time forward.
Note: The ObjectStream license key associated with the NETSMART 1500 server is required for
this procedure. Refer to Section 3.8, Configuring the ObjectStream License [p. 3-37], for more
information on configuring the ObjectStream license.
Task
Shut down all NETSMART 1500 clients. On the client GUIs, select Exit from the File menu.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Set the Date and Time on the UNIX System
Shut down the NETSMART 1500 server and all Oracle processes. Type netsmart
stopall at the UNIX command prompt, and then press RETURN.
Description
mm
dd
HH
2-digit value for the hour of the day, using a 24-hour clock (for example, 00 is 12:00 a.m., and 23 is
11:00 p.m.)
MM
yy
Last two digits of the year (for example, 06 for year 2006)
Example: To set the date to November 20, 2006 at 1:19 p.m., the following command
is entered: /usr/bin/date 1120131906.
6
10
11
12
13
FNC-1500-0050-110
Issue 1, January 2007
4-15
CHAPTER 4
UNIX System Administration
Set the Date and Time on the UNIX System
14
Enter the ObjectStream license associated with the server when requested.
15
16
After the server has rebooted, restart the NETSMART 1500 server. (Refer to Section 3.3,
Starting Up and Shutting Down [p. 3-8].)
4.10.2
Step
Task
Verify that NETSMART 1500 is up and that all NEs are logged on.
As user netsmart, shut down the NETSMART 1500 server. (Refer to Section 3.3.3, Shut
Down Server [p. 3-10].)
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
UNIX System Information
4.11
SUBPROCEDURE
PAGE
4.11.1
System Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-18
4.11.2
System Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-19
4.11.3
Disk Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-20
4.11.4
Swap Info. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-20
4.11.5
4-21
4.11.6
Cronjobs Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-25
4.11.7
Defunct Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-26
FNC-1500-0050-110
Issue 1, January 2007
4-17
CHAPTER 4
UNIX System Administration
UNIX System Information
4.11.1
System Processes
The System Processes command displays a list of all system processes. A screen similar to the
following displays:
S Y S T E M
P R O C E S S E S
You can either list a particular application process name, or list all
system processes.
Enter process name (or leave blank for all):
UID
PID PPID C
STIME TTY
TIME CMD
root
0
0 0 Sep 21 ?
0:00 sched
root
1
0 0 Sep 21 ?
0:25 /etc/init root
2
0 0 Sep 21 ?
0:00 pageout
root
3
0 0 Sep 21 ?
6:36 fsflush
root
234
1 0 Sep 21 ?
0:00 /opt/NetSmart/OBJStream
License/ObjectStream_lmd
-r /opt/NetSmart/OBJ
root
160
1 0 Sep 21 ?
0:00 /usr/lib/nfs/lockd
root
309
1 0 Sep 21 ?
0:00 /usr/lib/saf/sac -t 300
root
179
1 0 Sep 21 ?
0:00 /usr/lib/autofs/automountd
root
318
291 0 Sep 21 ?
0:00 mibiisa -p 32823
root
123
1 0 Sep 21 ?
0:00 /usr/sbin/rpcbind...
root
194
1 0 Sep 21 ?
0:00 /usr/sbin/cron
root
231
1 0 Sep 21 ?
0:00 /usr/lib/power/powerd
root
247
1 0 Sep 21 ?
0:02 /usr/lib/utmpd
root
315
309 0 Sep 21 ?
0:00 /usr/lib/saf/ttymon
netsmart
316
313 1 Sep 21 ?
0:53 /usr/openwin/bin/Xsun :0
-nobanner -auth /var/dt/A:0Ew6gR_
root
310
1 0 Sep 21 console 0:00 /usr/lib/saf/ttymon -g -h -p
pslab6 console login:
-T sun -d /dev/console -l c
netsmart
422 379
0 Sep 21 pts/2
0:00 [ sdt_shel ]
root
291
1 0 Sep 21 ?
0:00 /usr/lib/snmp/snmpdx -y -c
/etc/snmp/conf
root
301
1 0 Sep 21 ?
0:00 /usr/lib/dmi/dmispd
root
302
1 0 Sep 21 ?
0:00 /usr/lib/dmi/snmpXdmid -s
pslab6
--More--
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
UNIX System Information
4.11.2
System Messages
Executing this command displays the latest UNIX system messages. A screen similar to the
following displays:
S Y S T E M
B A C K U P
SunOS pslab6 5.10 Generic_105181-14 sun4u sparc SUNW,Ultra-60
Filesystem
kbytes
/dev/dsk/c0t0d0s0
/dev/dsk/c0t0d0s4
/dev/dsk/c0t0d0s3
/dev/dsk/c0t0d0s5
/dev/dsk/c0t1d0s1
/dev/dsk/c1t2d0s1
/dev/dsk/c1t3d0s0
120739
578351
290065
5601004
6718025
6718025
8782097
used
avail
21053
87613
393818 126698
18577 242482
1528175 4016819
6459706 191139
6560447
90398
5020224 3674053
capacity
20%
76%
8%
28%
98%
99%
58%
Mounted on
/
/usr
/var
/opt
/opt2
/opt3
/opt4
To restore filesystems: Select the required volume from the System Messages
File
Sep 21 07:27:53 pslab6 unix: pci@1f,4000/scsi@3/disk@0,0:a fstype ufs
Sep 21 07:27:53 pslab6 unix: PCI-device: ebus@1, ebus #0
Sep 21 07:27:53 pslab6 unix: su0 at ebus0: offset 14,3083f8
Sep 21 07:27:53 pslab6 unix: su0 is /pci@1f,4000/ebus@1/su@14,3083f8
Sep 21 07:27:53 pslab6 unix: su1 at ebus0: offset 14,3062f8
Sep 21 07:27:53 pslab6 unix: su1 is /pci@1f,4000/ebus@1/su@14,3062f8
Sep 21 07:27:53 pslab6 unix: keyboard is </pci@1f,4000/ebus@1
su@14,3083f8> major <37> minor <0>
Sep 21 07:27:53 pslab6 unix: mouse is </pci@1f,4000/ebus@1/su@14,3062f8>
major <37> minor <1>
Sep 21 07:27:53 pslab6 unix: stdin is </pci@1f,4000/ebus@1/su@14,3083f8>
major <37> minor <0>
Sep 21 07:27:53 pslab6 unix: SUNW,ffb0 at root: UPA 0x1e 0x0
Sep 21 07:27:53 pslab6 unix: SUNW,ffb0 is /SUNW,ffb@1e,0
Sep 21 07:27:53 pslab6 unix: stdout is </SUNW,ffb@1e,0> major <51> minor
- more --<56%>
FNC-1500-0050-110
Issue 1, January 2007
4-19
CHAPTER 4
UNIX System Administration
UNIX System Information
4.11.3
Disk Usage
Executing this command lists the current usage of the disk partitions. A screen similar to the
following displays:
Disk Utilization
/dev/dsk/c0t0d0s0
/dev/dsk/c0t0d0s4
/dev/dsk/c0t0d0s3
/dev/dsk/c0t0d0s5
/dev/dsk/c0t1d0s1
/dev/dsk/c1t2d0s1
/dev/dsk/c1t3d0s0
4.11.4
Swap Info
Executing this command displays the current status of the swap configuration on the server. A
screen similar to the following displays:
swap space information for pslab6
# swap -s
total: 28808k bytes allocated + 6176k reserved = 34984k used, 8011256k
available
# swap -l
swapfile dev swaplo blocks free
/dev/dsk/c1t2d0s0 32,256 16 4194272 4194272
/dev/dsk/c0t0d0s1 32,121 16 4194272 4194272
/dev/dsk/c0t1d0s0 32,128 16 4194272 4194272
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
UNIX System Information
4.11.5
PAGE
1. sar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-22
2. vmstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-22
3. iostat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-23
4. uptime. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-23
5. netstat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4-24
The SYSTEM ACTIVITY menu provides submenu options that aid in checking for
bottlenecks, disks, CPU, or I/O. This menu appears as follows:
S Y S T E M A C T I V I T Y
1. sar
2. vmstat
3. iostat
4. uptime
5. netstat
e. Enable Root Access
m. Go back to the Main Menu
q. Quit
FNC-1500-0050-110
Issue 1, January 2007
4-21
CHAPTER 4
UNIX System Administration
UNIX System Information
1. sar
System activity report (sar) outputs CPU usage for usr, sys, wio, and idle. A screen
similar to the following displays:
S A R
SunOS
07:51:14
07:51:19
07:51:24
07:51:29
07:51:34
07:51:39
Average
%usr
9
16
19
20
23
17
06/24/02
%sys
8
8
11
10
11
%wio
0
0
0
0
0
%idle
84
75
70
70
66
10
73
2. vmstat
Executing this command displays CPU and disk statistics. A screen similar to the following
displays:
V M S T A T
procs
r b w
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
swap
202568
6785008
6785008
6784992
6784944
memory
free
159896
718528
718512
718496
718448
re mf
0 102
0 254
0 106
0
56
0
0
page
disk
pi po fr de sr f0 s1 s1 s2 in
0 0 0 0 0 0 2 0 0 136
0 0 0 0 0 0 1 0 0 197
0 0 0 0 0 0 1 0 0 281
0 1 1 0 0 0 8 1 0 252
0 0 0 0 0 0 1 0 0 266
faults
cpu
sy cs us sy id
2517 91 2 1 97
1711 144 7 2 91
1691 202 1 1 98
1415 113 0 1 99
1142 148 0 0 99
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
UNIX System Information
3. iostat
Executing this command displays the output from the Solaris iostat utility. A screen similar to
the following displays:
I O S T A T
tin
0
0
0
0
0
tty
fd0
sd15
sd16
sd21
tout kps tps serv kps tps serv kps tps serv kps tps serv us sy
4
0
0 468 15
2
92
3
0
3
0
0
0 2 1
47
0
0
0
6
2
10
2
0
1
0
0
0 6 4
16
0
0
0
3
0
11
3
0
1
0
0
0 0 0
16
0
0
0
3
0
7
3
0
1
0
0
0 0 0
16
0
0
0
5
1
17
2
0
1
0
0
0 0 0
cpu
wt id
1 96
1 89
0 99
0 100
0 100
4. uptime
Executing this command displays uptime statistics. A screen similar to the following displays:
U P T I M E
1:43pm up 1 day(s), 6:16, 2 users, load average: 0.08, 0.11, 0.11
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
4-23
CHAPTER 4
UNIX System Administration
UNIX System Information
5. netstat
Executing this command displays the output from the Solaris netstat utility. A screen similar
to the following displays:
N E T S T A T
TCP: IPv4
Local Address
Remote Address
Swind Send-Q Rwind
-------------------- -------------------- ----- ------ ----localhost.32850
localhost.32772
32768
0 32768
localhost.32772
localhost.32850
32768
0 32768
localhost.32853
localhost.32772
32768
0 32768
localhost.32772
localhost.32853
32768
0 32768
localhost.32856
localhost.32848
32768
0 32768
:
:
:
nvt1.57171
nvt1.listener
32768
0 32768
nvt1.57172
nvt1.12350
32768
0 32768
nvt1.57173
nvt1.listener
32768
0 32768
nvt1.57174
nvt1.12350
32768
0 32768
nvt1.57175
nvt1.listener 32768 0 32768 0 TIME_WAIT
Recv-Q State
------ ------0 ESTABLISHED
0 ESTABLISHED
0 ESTABLISHED
0 ESTABLISHED
0 ESTABLISHED
0
0
0
0
TIME_WAIT
TIME_WAIT
TIME_WAIT
TIME_WAIT
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
UNIX System Information
4.11.6
Cronjobs Menu
This subsection includes the following items:
ITEM
PAGE
4-25
4-26
This submenu helps administer crontab files. This submenu appears as follows:
C R O N T A B
1.
2.
e.
m.
p.
q.
A D M I N I S T R A T I O N
FNC-1500-0050-110
Issue 1, January 2007
4-25
CHAPTER 4
UNIX System Administration
UNIX System Information
Users can use this option to edit their own crontab. This example displays the user root
crontab:
#ident "@(#)root 1.14 06/07/31 SMI" /* SVr5.0 1.1.4.1 */
#
# The root crontab should be used to perform accounting data collection.
#
# The rtc command is run to adjust the real time clock if and when
# daylight savings time changes.
#
10 3 * * 0,4 /etc/cron.d/logchecker
10 3 * * 0 /usr/lib/newsyslog
15 3 * * 0 /usr/lib/fs/nfs/nfsfind
1 2 * * * [ -x /usr/sbin/rtc ] && /usr/sbin/rtc -c > /dev/null 2>&1
~
~
....
~
~
"/tmp/crontab0M1P3j" 21 lines, 742 characters
4.11.7
Defunct Processes
This command checks for defunct processes. A screen similar to the following displays:
netsmart 1170
481
0:00 <defunct>
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Back Up the UNIX System
4.12
SUBPROCEDURE
PAGE
4.12.1
4-27
4.12.2
4-28
4.12.3
4-30
This section describes the tasks used for a full (that is, Level 0) backup of all the files on the
NETSMART 1500 system as well as a NETSMART 1500 backup.
4.12.1
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
4-27
CHAPTER 4
UNIX System Administration
Back Up the UNIX System
B A C K U P
kbytes
120739
578351
290065
5601004
6718025
6718025
8782097
used
21053
393818
18577
1528175
6459706
6560447
5020224
avail
z87613
126698
242482
4016819
191139
90398
3674053
capacity
20%
76%
8%
28%
98%
99%
58%
Mounted on
/
/usr
/var
/opt
/opt2
/opt3
/opt4
4.12.2
WARNING:
This section does not contain procedures for backing up the Oracle database. Refer to
Section 2.6, Standard Database Backup [p. 2-25], for more information on backing up the
Oracle database.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Back Up the UNIX System
The following procedure backs up all files and directories listed in the file
/opt/NetSmart/bin/admin/backup_list.txt. By default, only the following files
and directories are backed up:
/opt/NetSmart/bin
/opt/NetSmart/client_env.csh
/opt/NetSmart/client_env.sh
/opt/NetSmart/Properties
/opt/NetSmart/install
The user can edit the backup_list.txt file to include any combination of files and/or
directories for backup.
Back up the NETSMART 1500 server application software as follows:
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
4-29
CHAPTER 4
UNIX System Administration
Back Up the UNIX System
4.12.3
Step
Task
B A C K U P
L O G S
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 4
UNIX System Administration
Admintool
4.13
Admintool
This option displays the Solaris Admintool: Users dialog box. For further information, refer to
the Sun system administration documentation. An example of the Admintool: Users dialog is
shown in Figure 4-1 [p. 4-31].
4.14
4.15
FNC-1500-0050-110
Issue 1, January 2007
4-31
5
CHAPTERS/APPENDIXES
NETWORK ADMINISTRATION
1 Introduction
WARNING:
2 Oracle Administration
3 NETSMART 1500
Administration
4 UNIX System
Administration
5 Network Administration
6 Service Support
7 High Availability
Administration
SECTION
PAGE
5.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-2
5.2
5-5
5.3
5-7
5.4
OSI Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-11
5.5
TARP Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-18
5.6
IP Address Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-22
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
Overview
5.1
Overview
This section includes the following subsections:
NO.
SUBSECTION
PAGE
5.1.1
Start FTAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-4
5.1.2
Stop FTAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-4
Network administration uses the Network Administration option of the Heartbeat tool. This
option provides the following menu, which is the starting point for most of the procedures
described in this chapter:
N E T W O R K
1.
2.
3.
4.
5.
6.
e.
m.
q.
A D M I N I S T R A T I O N
Table 5-1 [p. 5-2] lists the menu and submenu items for network administration as well as
where they are described in this practice.
Table 5-1: Network Administration Menu Items
Menu Item
Reference
1. Start FTAM
2. Stop FTAM
3. Start OSI
4. Stop OSI
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
Overview
Reference
5. OSI Menu
5. Add Route
6. Delete Route
7. Display Route
6. TARP Menu
FNC-1500-0050-110
Issue 1, January 2007
5-3
CHAPTER 5
Network Administration
Overview
5.1.1
Start FTAM
This option starts the File Transfer, Access, and Management (FTAM) protocol process. A
screen similar to the following displays:
O S I
1.
2.
d.
m.
p.
q.
S T A C K
S E L E C T I O N
Ftl2k.1
Ftl2k.2
Disable Root Access (return to netsmart)
Go back to the Main Menu
Go back to the previous menu
Quit
1
Starting (possibly additional) FTAM Responder...(results described in
/opt/NetSmart/Log/osi.log)
Hit any key to Continue.
5.1.2
Stop FTAM
This option stops the FTAM process. A screen similar to the following displays:
Note: The FTAM process restarts if the NETSMART server is running.
O S I
1.
2.
d.
m.
p.
q.
S T A C K
S E L E C T I O N
Ftl2k.1
Ftl2k.2
Disable Root Access (return to netsmart)
Go back to the Main Menu
Go back to the previous menu
Quit
1
Stopping FTAM Responder...
(results described in /opt/NetSmart/Log/osi.log)
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
TCP/IP and OSI Networks with NETSMART 1500
5.2
SUBSECTION
PAGE
5.2.1
OSI Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-5
5.2.2
TCP/IP Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-6
The NETSMART 1500 server can support the following protocols when communicating
with the NEs:
TCP/IP only
Before installing NETSMART 1500, you should know what types of communication
protocols the NETSMART 1500 server will be using. Adding OSI support to NETSMART
1500 after the installation of a TCP/IP-only configuration is more difficult than adding it at
time of installation. If a network uses purely TCP/IP communication with the NEs but may
use OSI support later, installing both TCP/IP and OSI support during initial installation is
recommended rather than TCP/IP-only support.
5.2.1
OSI Connectivity
NETSMART 1500 requires a minimum of two network interface cards (NICs) when running
with OSI connectivity. Some server configurations require more. Refer to
FNC-1500-0050-100, Installation Guide, Section 1.4, Hardware Platforms [p. 1-4].
For NETSMART 1500 to function properly, the selected NICs must be within the first 10
devices for a particular driver. If the machine is set up using hme or qfe devices, only hme0
hme9 or qfe0qfe9 are allowed.
The NICs chosen for OSI communication can be used for both TCP/IP and OSI
communication.
Each OSI interface requires a unique Ethernet address. The Ethernet address is also referred to
as the media access control (MAC) address. By default, all Ethernet cards on a PCI-based Sun
Solaris machine are configured with the same Ethernet address. By setting the EEPROM
variable local-mac-address? to true and rebooting the system, each Ethernet card is
configured with a unique address. If the EEPROM setting is not available on the system,
unique addresses must be manually assigned to each of the cards. These addresses can be
calculated by the installation or can be specified for each Ethernet card during the installation.
FNC-1500-0050-110
Issue 1, January 2007
5-5
CHAPTER 5
Network Administration
TCP/IP and OSI Networks with NETSMART 1500
To determine whether the system supports the EEPROM variable, enter the command
eeprom | grep mac. If the response is local-mac-address?=false, the variable
is supported, and the command EEPROM local-mac-address?=true can be used to
tell the system to use unique addresses. This command takes effect upon rebooting the
system.
Each interface used to connect with OSI NEs is required to have a unique network service
access point (NSAP) address. The NSAP address is similar to an IP address in TCP/IP
connectivity. Refer to Section 5.4.3, Set NSAP Address [p. 5-13], for more information on
the NSAP address.
5.2.2
TCP/IP Connectivity
This subsection includes the following items:
ITEM
PAGE
General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-6
5-7
General
The NETSMART 1500 server software handles TCP/IP connectivity using standard routing
tables set up on the host machine. These routes are set up from within the Solaris operating
system, not from within the NETSMART 1500 server application. (An exception is using a
single IP address for all communication to the NEs, which is discussed in Section 5.2.2,
subsection Single IP Address for All TCP/IP Communication [p. 5-7].) The main pages for
ifconfig, route, and routed contain more information about routing. The assumption is that
the user has a basic understanding of IP routing.
Each NIC used to communicate to IP-based NEs must be configured with an IP address and
a host name. The IP address of each card should be on the subnet of the destination IP NEs.
If the NIC cards are on a different subnet, a static route should be added to the system.
These IP addresses and static routes need to be visible even after a NETSMART 1500 server
host reboot. Therefore, the IP addresses and static routes should be configured in the
appropriate /etc files on the host machine.
Setting up IP Multipathing is recommended if multiple NICs are used to connect to NEs.
Multipathing provides fault tolerance to NIC failures and load balancing of outbound traffic.
Multipathing avoids the need for interface load balancing, as was required in previous versions
of Solaris.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
OSI Startup and Shutdown
to
comms.networkaddress.localaddress = xxx.xxx.xxx.xxx
5.3
SUBPROCEDURE
PAGE
5.3.1
5-8
5.3.2
5-9
Under normal operating circumstances, OSI is started when the NETSMART 1500 server
application is started using Heartbeat (refer to Section 3.3.1, Start Server [p. 3-8]). The OSI is
shut down when the NETSMART 1500 server application is stopped using Heartbeat (refer
to Section 3.3.3, Shut Down Server [p. 3-10]).
There may be instances, such as NETSMART 1500 troubleshooting, when starting and
stopping the OSI processes independent of the main NETSMART 1500 server application
are necessary.
FNC-1500-0050-110
Issue 1, January 2007
5-7
CHAPTER 5
Network Administration
OSI Startup and Shutdown
Refer to Section 5.3, OSI Startup and Shutdown [p. 5-7], for more information on OSI
startup and shutdown menu options.
Note: Stopping the OSI processes works only when the NETSMART 1500 server application is not
running. If the NETSMART 1500 server application is running and the OSI processes are shut
down, the shutdown appears as a communications failure to the NETSMART 1500 server
application, and the OSI processes are automatically restarted.
Refer to Section 3.3.3, Shut Down Server [p. 3-10], for procedures on shutting down the
NETSMART 1500 server.
5.3.1
Step
Task
S T A C K
S E L E C T I O N
tl2k.1
tl2k.2
Disable Root Access (return to netsmart)
Go back to the Main Menu
Go back to the previous menu
Quit
Enter the number corresponding to the OSI stack that needs to be started. A screen similar to
the following displays:
Do you want to [c]ontinue or [q]uit (default = c)?
Enter c to continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
OSI Startup and Shutdown
The OSI processes associated with the selected OSI stack start. A screen similar to the
following displays:
Starting (possibly additional) OSI Stack...(results described in
/opt/NetSmart/gordon/Log/osi.log)
Hit any key to Continue.
5.3.2
Task
S T A C K
S E L E C T I O N
tl2k.1
tl2k.2
Disable Root Access (return to netsmart)
Go back to the Main Menu
Go back to the previous menu
Quit
Enter the number corresponding to the OSI stack that must be stopped. A screen similar to
the following displays:
Do you want to [c]ontinue or [q]uit (default = c)?
FNC-1500-0050-110
Issue 1, January 2007
5-9
CHAPTER 5
Network Administration
OSI Startup and Shutdown
Enter c to continue.
The OSI processes associated with the selected OSI stack stop. A screen similar to the
following displays:
Stopping OSI Stack... (results described in
/opt/NetSmart/gordon/Log/osi.log)
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
OSI Menu
5.4
OSI Menu
This section includes the following subsections:
NO.
SUBSECTION
PAGE
5.4.1
5-12
5.4.2
5-12
5.4.3
5-13
5.4.4
5-15
5.4.5
Add Route . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-16
5.4.6
Delete Route . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-16
5.4.7
Display Route. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5-17
5.4.8
5-17
FNC-1500-0050-110
Issue 1, January 2007
M E N U
5-11
CHAPTER 5
Network Administration
OSI Menu
5.4.1
S T A C K
S E L E C T I O N
gotl2k.2
gotl2k.1
Enable Root Access
Go back to the Main Menu
Go back to the Previous Menu
Quit
P R O C E S S E S
This option will tell you if the FTAMResponder and OSIstack are running.If
they are not, you can go back to the previous menu and start them ifneeded.
Ftl2k.2
FTAMResponder Ftl2k.2 is running
ptl2k.2
OSI stack ptl2k.2 is running
Hit any key to Continue.
5.4.2
O S I
A N D
F T A M
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
OSI Menu
5.4.3
CAUTION:
Using an incorrect OSI address causes permanent loss of service to NEs until a valid address
is configured.
This section describes the format of an OSI address and how to change it on the system.
The OSI addresses required by NETSMART 1500 are normally configured during the initial
NETSMART 1500 server installation. However, there may be times when modifying the OSI
address (or addresses) for a NETSMART 1500 server is necessary. Each NETSMART 1500
server has a minimum of two OSI interfaces, each with a unique OSI address (also known as
an NSAP address).
An OSI address consists of 40 hexadecimal characters divided into several sections. The
user-modifiable portions of the OSI address are:
Organization6 characters
Domain4 characters
Area4 characters
Task
FNC-1500-0050-110
Issue 1, January 2007
5-13
CHAPTER 5
Network Administration
OSI Menu
S T A C K
S E L E C T I O N
gotl2k.1
gotl2k.2
all
Disable Root Access (return to netsmart)
Go back to the Main Menu
Go back to the Previous Menu
Quit
Enter the new values for the Organization, Domain, and Area sections of the OSI address, and
then press ENTER.
OR
Accept the default value, if it is correct, by pressing ENTER.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
OSI Menu
After the address modification is completed, Heartbeat automatically restarts the OSI
processes to ensure that the new addresses take effect. A screen similar to the following
displays:
Using Ethernet hme1
Editing file /opt/NetSmart/marben/bin/gotl2k.1
Device=hme1
Stack =Setting the following variables
LOCNET =H39840f80000000000000000000080020ab294500
LOCNSAP =H39840f80000000000000000000080020ab29451d
TARPNSAP=H39840f80000000000000000000080020ab2945af
def suf 0 Chme
def pri 0 01000000
$load ositl2k ptl2k.1 Ftl2k.1
$def ptl2k.1
calling setnsap with parameter: -dgotl2k.1
FTAM Responder and OSI Stack has been restarted
(results described in /opt/NetSmart/Log/osi.log)
Hit any key to Continue.
5.4.4
S T A C K
S E L E C T I O N
gotl2k.1
gotl2k.2
all
Disable Root Access (return to netsmart)
Go back to the Main Menu
Go back to the Previous Menu
Quit Select a number/letter (case insensitive):
D I S P L A Y
N S A P
A D D R E S S
NSAP Address:
H39840f800000000000000001090800209ac7080020a4c4330800209ac3a71d
Hit any key to Continue
FNC-1500-0050-110
Issue 1, January 2007
5-15
CHAPTER 5
Network Administration
OSI Menu
5.4.5
Add Route
This option adds an OSI route for an OSI stack. The first screen lists the OSI stacks. The
following sample screen then displays:
A D D
R O U T E
0. Complete
1. Relayed
Select a route type (default=Complete) [?] 0
Enter NSAP to route to: ........................................
[?] 123987ADBH234234098SD<N098345MNBSDF12346
Enter SNPA to route from: ............
[?] 987654321abc
5.4.6
Delete Route
This option deletes an OSI route. The first screen lists the OSI stacks. The following sample
screen then displays:
Select the number of the route to delete
1) R(Route)_234444444444444444444444444444444444(To)_5555(From)_0(Entry)
2) C(Route)_39840f80001090800208096e91d(To)_none(From)_0(Entry)
3) 1(Route)_none(To)_none(From)_0(Entry)#? 1
Deleting Route (234444444444444444444444444444444444)
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
OSI Menu
5.4.7
Display Route
This option displays the static OSI routes. The first screen lists the OSI stacks. The following
sample screen then displays:
D I S P L A Y
R O U T E S
Rte: C=Complete
R=Relayed
Rte
===
R
C
1
To
======================================
39840f8000000000000000000000000060ac
1235987987345asdf12312987asdf3245988
none
From
===========================
000000000000000
987654987456abc
none
5.4.8
Snpa
=============
00000e4610af
00000e6402ab
00000e6402ac
00000e3a0fee
00000e4625f7
FNC-1500-0050-110
Issue 1, January 2007
5-17
CHAPTER 5
Network Administration
TARP Menu
5.5
TARP Menu
This section includes the following subsections:
NO.
SUBSECTION
PAGE
5.5.1
5-19
5.5.2
5-19
5.5.3
5-19
5.5.4
5-20
5.5.5
5-20
5.5.6
5-21
5.5.7
5-21
5.5.8
5-21
This option displays the target identifier (TID) Address Resolution Protocol (TARP)
submenu:
T A R P
1.
2.
4.
5.
6.
7.
8.
9.
d.
m.
p.
q.
M E N U
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
TARP Menu
5.5.1
T A R P
T I D
5.5.2
T A R P
T I D
5.5.3
FNC-1500-0050-110
Issue 1, January 2007
5-19
CHAPTER 5
Network Administration
TARP Menu
5.5.4
S T A C K
S E L E C T I O N
1. gotl2k.
2. gotl2k.2
3. s/gotl2k/gotl2k/
Enter stack? 1
Enter the number of the manual adjacency to delete
1) 1(Level)_123456789a123456789b123456789c123456789d(NET)
#? 1
Deleting Manual Adjacency (1 H123456789a123456789b123456789c123456789d)
Hit any key to Continue.
5.5.5
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 5
Network Administration
TARP Menu
5.5.6
5.5.7
5.5.8
FNC-1500-0050-110
Issue 1, January 2007
5-21
CHAPTER 5
Network Administration
IP Address Change
5.6
IP Address Change
CAUTION:
Changing the IP address is not recommended. If the IP address or host name of the server
must be changed, contact Fujitsu Network Communications Inc. (FNC) technical support at
1-800-USE-FTAC.
CAUTION:
The host name must stay the same, or licensing will fail.
CAUTION:
Changing IP addresses on a NETSMART 1500 server disrupts service to TCP/IP NEs
temporarily while the address change takes effect.
CAUTION:
Using an incorrect IP address causes permanent loss of service to TCP/IP NEs until a valid
address is configured. In addition, an invalid IP address may make the NETSMART 1500
server inaccessible on the TCP/IP network, causing you to be unable to Telnet or log on the
NETSMART 1500 server. If you change IP addresses, you should change them only while
logged on the console of the NETSMART 1500 server, not over the network.
FNC-1500-0050-110
Issue 1, January 2007
6
CHAPTERS/APPENDIXES
SERVICE SUPPORT
1 Introduction
2 Oracle Administration
3 NETSMART 1500
Administration
NO.
SECTION
4 UNIX System
Administration
6.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-2
5 Network Administration
6.2
6-2
6 Service Support
6.3
6-3
7 High Availability
Administration
6.4
Support Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-3
6.5
FTP Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-3
6.6
6-4
6.7
Archive System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-6
6.8
6-7
6.9
System Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-12
6.10
6-12
6.11
6-14
FNC-1500-0050-110
Issue 1, January 2007
PAGE
CHAPTER 6
Service Support
Overview
6.1
Overview
The Service Support menu provides access to the service support logs. The menu options
shown below are described in the following subsections:
S E R V I C E
S U P P O R T
Hotline: 1-800-USE-FTAC
1. Edit Support Log
2. List Support Log
3. Support Information
4. FTP Information
5. Archive Log Files
6. Archive System *
7. Edit Property Files +
8. System Information
9. Check Solaris Patches
10. Collect Debug Information *
e. Enable Root Access
m. Go back to the Main Menu
q. Quit
* may require root access
+ can only be executed by NETSMART 1500 administrator
Select a number/letter (case insensitive):
6.2
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
List Support Log
6.3
S U P P O R T
L O G
6.4
Support Information
This option displays the address and phone number for Fujitsu Technical Assistance Center
(FTAC) support. A screen similar to the following displays:
G E N E R A L
S U P P O R T
I N F O R M A T I O N
6.5
FTP Information
FTP Information displays information for FTP file transfer with FNC. A screen similar to the
following displays:
F T P
I N F O R M A T I O N
Site : fncftp.fnc.fujitsu.com
Login : nms
Password : nms
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
6-3
CHAPTER 6
Service Support
Archive Log Files
6.6
L O G
F I L E S
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Archive Log Files
06/25/06
%idle
13
11
15
8
61
FNC-1500-0050-110
Issue 1, January 2007
6-5
CHAPTER 6
Service Support
Archive System
6.7
Archive System
The Archive System option creates an archive file. An example of the system display is shown
below:
A R C H I V E S Y S T E M
Creating archive in /opt/netsmart5/archives/files/062402.143750
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Edit Property Files
6.8
F I L E
D I R E C T O R I E S
FNC-1500-0050-110
Issue 1, January 2007
21)
22)
23)
24)
25)
26)
27)
28)
29)
30)
31)
32)
33)
34)
35)
36)
37)
38)
39)
40)
neTimeSyncScheduler
nif
NotifService
Nsh
NSWebService
PerformanceManagement
PM-ES
PresentationServer-1
rmbuscheduler
Scheduler
Security
SessionManager
SNMPServer
SwmgrServer
TARPServer
TL1UplinkServer
TopologyServer
UPSManager
Return to previous menu
Stop listing directories
6-7
CHAPTER 6
Service Support
Edit Property Files
As an example, if you select option 1 (Current directory), the list of property files to edit is
listed (as shown in the following example). When you select a file, the file is opened in your
default editor.
E D I T
P R O P E R T Y
F I L E S
Use this option to edit one of the many property files that wereconfigured
during installation.
Please select one of the following
1) Component.properties
2) Container.properties
3) CORBAFactory.properties
4) DB.properties
5) FenicsNodes.properties
6) JavaGroups.properties
7) NetsmartServer.properties
8) NetsmartServices.properties
9) NSContexts.properties
10) PersistentFactory.properties
#?
11)
12)
13)
14)
15)
16)
17)
18)
19)
Process.properties
Server.properties
ServerAssignment.properties
ServerGroups.properties
System.properties
Trace.properties
vbj.properties
Return to previous menu
Stop editing files
Table 6-1 [p. 6-9] lists some of the more important property files and their key values. All
property files are located in the following directory:
/opt/NetSmart/Properties
CAUTION:
When editing property files, ensure there are no extra spaces at the end of each line.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Edit Property Files
Function(s)
Key Value(s)
ApplicationServer-<n>/
ApplicationServer-<n>
.properties
FNC-1500-0050-110
Issue 1, January 2007
emlcrossconnectmanager.
validateFacility=false to true.
6-9
CHAPTER 6
Service Support
Edit Property Files
Function(s)
Key Value(s)
Properties/
TimeZone.properties
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Edit Property Files
Function(s)
Key Value(s)
GUIClient/
ApplicationGraphics/
ApplicationGraphics_en
.properties
FNC-1500-0050-110
Issue 1, January 2007
6-11
CHAPTER 6
Service Support
System Information
6.9
System Information
System Information displays details about the host system, including host name, IP address,
host ID, software level, and patches that have been applied. A screen similar to the following
displays:
S Y S T E M
I N F O R M A T I O N
Hostname: nvt1
Hostid: 80a76a66
Release: 5.10
Kernel architecture: sun4u
Application architecture: sparc
Hardware provider: Sun_Microsystems
Domain: nvt.domain
Kernel version: SunOS 5.10 Generic 108528-14 June 2006
IP Address: 167.254.235.1 n
6.10
S O L A R I S
P A T C H E S
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Check Solaris Patches
Refer to FNC-1500-0050-100, Installation Guide, Section B, Solaris Patches [p. B-1], for
more information.
FNC-1500-0050-110
Issue 1, January 2007
6-13
CHAPTER 6
Service Support
Collect Debug Information
6.11
SUBSECTION
PAGE
6.11.1
System Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-15
6.11.2
NETSMART Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-16
6.11.3
ORACLE Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-17
6.11.4
All Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-17
The Collect Debug Information menu shown below provides menu options to archive debug
data to tar files as described below.
D E B U G I N F O R M A T I O N
1. System Information
2. NETSMART Information
3. ORACLE Information
4. All Information
d. Disable Root Access (return to netsmart)
m. Go back to the Main Menu
q. Quit
Select a number/letter (case insensitive):
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Collect Debug Information
6.11.1
System Information
The System Information option archives system debug data to
/tmp/NETSMARTDataLog_system.tar.Z. A screen similar to the following displays:
Getting System information
Generating file :/tmp/NETSMARTDataLog_system.tar
a /tmp/NETSMARTDataLog_system/ 0K
a /tmp/NETSMARTDataLog_system/hostname.xxx 1K
a /tmp/NETSMARTDataLog_system/hosts 1K
a /tmp/NETSMARTDataLog_system/defaultdomain 1K
a /tmp/NETSMARTDataLog_system/defaultrouter 1K
a /tmp/NETSMARTDataLog_system/nodename 1K
a /tmp/NETSMARTDataLog_system/nsswitch.conf 1K
a /tmp/NETSMARTDataLog_system/inetd.conf 7K
a /tmp/NETSMARTDataLog_system/uname-a 1K
a /tmp/NETSMARTDataLog_system/ifconfig-a 1K
a /tmp/NETSMARTDataLog_system/netstat-rn 1K
a /tmp/NETSMARTDataLog_system/sysdef 17K
a /tmp/NETSMARTDataLog_system/env.lst 3K
a /tmp/NETSMARTDataLog_system/crontab.lst 1K
a /tmp/NETSMARTDataLog_system/messages 0K
a /tmp/NETSMARTDataLog_system/varadm_nsevents.tar 1K
a /tmp/NETSMARTDataLog_system/pkgparam-v 1K
a /tmp/NETSMARTDataLog_system/pkginfo-i 1K
a /tmp/NETSMARTDataLog_system/showrev-p 47K
End Time : Sun Jun 25 14:55:44 CDT 2006
Archive of log Data created : /tmp/NETSMARTDataLog_system.tar.Z
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
6-15
CHAPTER 6
Service Support
Collect Debug Information
6.11.2
NETSMART Information
The NETSMART Information option archives NETSMART 1500 debug data to
/tmp/NETSMARTDataLog_netsmart.tar.Z. A screen similar to the following
displays:
Getting NETSMART information
/opt/NetSmart/Properties/ApplicationServer-1/ApplicationServer-1.propertie
s 1K
a /opt/NetSmart/Properties/ApplicationServer-1/DB.properties 1K
a /opt/NetSmart/Properties/BankClient/BankClient.properties 1K
a /opt/NetSmart/Properties/BankClient/Container.properties 1K
a /opt/NetSmart/Properties/BankServer-1/BankServer-1.properties 1K
a /opt/NetSmart/Properties/BankServer-1/Container.properties 1K
:
:
:
a /opt/netsmart5/install/log/dbConfig.log 1K
a /opt/netsmart5/install/log/dbConfig.trace 18K
a /opt/netsmart5/install/log/DBInstall.log 45K
a /opt/netsmart5/install/log/applyOraclePatches.tmp.2531 5K
a /opt/netsmart5/install/log/DBInstall.sh.tmp.685 1K
a /opt/netsmart5/install/log/start_schema.tmp.3041 1K
a /opt/netsmart5/install/log/start_schema.tmp.14790 1K
Generating file : /tmp/NETSMARTDataLog_netsmart.tar
a /tmp/NETSMARTDataLog_netsmart/ 0K
a /tmp/NETSMARTDataLog_netsmart/NETSMART_PROPERTIES.tar 898K
a /tmp/NETSMARTDataLog_netsmart/NETSMART_Trace.tar 15K
a /tmp/NETSMARTDataLog_netsmart/NETSMART_Log.tar 334K
a /tmp/NETSMARTDataLog_netsmart/NSINSTALL_LOG.tar 166K
End Time : Sun Jun 25 14:56:17 CDT 2006
Archive of log Data created : /tmp/NETSMARTDataLog_netsmart.tar.Z
Hit any key to Continue.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 6
Service Support
Collect Debug Information
6.11.3
ORACLE Information
The ORACLE Information option archives database debug data to /tmp/NSDBlog.tar.Z.
A screen similar to the following displays:
Sun Microsystems Inc.
SunOS 5.10
You have new mail.
Generating file : /tmp/NSDBlog.tar
6.11.4
All Information
The All Information option performs all of the previous menu options. Debug data is archived
to the following files:
/tmp/NETSMARTDataLog_system.tar.Z
/tmp/NETSMARTDataLog_netsmart.tar.Z
/tmp/NSDBlog.tar.Z
FNC-1500-0050-110
Issue 1, January 2007
6-17
7
CHAPTERS/APPENDIXES
1 Introduction
2 Oracle Administration
3 NETSMART 1500
Administration
NO.
SECTION
4 UNIX System
Administration
7.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-2
5 Network Administration
7.2
7-2
6 Service Support
7.3
7-5
7 High Availability
Administration
7.4
7-13
7.5
7-15
7.6
7-18
7.7
7-21
7.8
7-23
7.9
7-24
7.10
7-26
7.11
7-27
7.12
HA Quick Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-31
7.13
7-34
FNC-1500-0050-110
Issue 1, January 2007
PAGE
CHAPTER 7
High Availability Administration
Overview
7.1
Overview
This chapter provides procedures for systems configured with NETSMART 1500 high
availability (HA). This chapter assumes that all users are System Administrators experienced
with UNIX who have read and understand FNC-1500-0050-100, Installation Guide,
Section 4.1, Overview [p. 4-2].
The procedures in this chapter use the following host names in the examples:
7.2
Definition
Active
Intended to be used
Passive
Definition
Up
Resource is working.
InProgress
Failed
GracefulDown
Suspended
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Understanding HA States and Roles
Table 7-3 [p. 7-3] defines valid combinations of roles and states for an HA node.
Table 7-3: Possible Combinations of Roles and States
Role/State
Definition
Abbreviation
Active/Up
a/u
Active/Failed
a/f
Active/GracefulDown
a/g
Active/Suspended
a/s
Passive/Up
p/u
Passive/Failed
p/f
Passive/GracefulDown
p/g
Passive/Suspended
p/s
FNC-1500-0050-110
Issue 1, January 2007
7-3
CHAPTER 7
High Availability Administration
Understanding HA States and Roles
Figure 7-1 [p. 7-4] details all of the transitions that are possible for an HA system.
Installed & Configured state on Primary
Active/Failed
Failure
Active/Suspended (as)
suspend/resume
ActiveGracefulDown (ag)
shutdown
(re)start
error starting up
failure detected
Active/Up (au)
manual
failover
cleanup
Active/Failed (af)
repaired
manual/auto
failover
detects another
active resource
failure detected
repaired
Passive/Up (pu)
(re)start
cleanup
shutdown
Passive/Suspended (ps)
error starting up
Passive/GracefulDown (pg)
Failure
Passive/Failed
m1615ao_2
resume/suspend
Passive/Failed (pf)
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
7.3
SUBPROCEDURE
PAGE
7.3.1
7-5
7.3.2
7-8
In the examples used in this procedure, the intent is to start services on the primary machine
in the active role and on the secondary machine in the passive role. By default, primary is
configured to come up in the active role and secondary in the passive role. These roles can be
changed while starting the servers.
In the examples used in this procedure, host names haorbison and garcia were installed
with all optional packages, but none of the optional packages were enabled. Also, this example
federation supports both IP and OSI.
7.3.1
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
7-5
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
FNC-1500-0050-110
Issue 1, January 2007
7-7
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
Services may take about 1530 minutes to start before presenting the following status:
NETSMART 1500 Federation haorbison
Primary: haorbison.nms.fnc.fujitsu.com: Active
Backup: garcia.nms.fnc.fujitsu.com: Passive
======================================================================
haorbison.nms.fnc.fujitsu.com%
NETSMART 1500 HA services are now activated and are up and running on both
haorbison and garcia. A client can connect to the hafed federation and invoke any of
the NETSMART 1500 features. In the above example, node garcia would take over the
functionality of haorbison in case of failure. Note that the NETSMART 1500 Heartbeat
state shows as Passive/Up. HA software has successfully connected itself with the services on
haorbison.
7.3.2
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
Task
FNC-1500-0050-110
Issue 1, January 2007
7-9
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
From
To
Status
Throughput Latency
Queue
(Bytes/sec) (ms)
(bytes)
.......................................................................
Heartbeat
Primary
Backup
Up
47.25
Heartbeat
Primary
Backup
Up
47.25
Heartbeat
Backup
Primary
Up
48.0
Heartbeat
Backup
Primary
Up
69.625
Replication Primary
Backup
Up
-1.0
-1.0
null
Replication Backup
Primary
Up
-1.0
-1.0
null
FNC-1500-0050-110
Issue 1, January 2007
7-11
CHAPTER 7
High Availability Administration
Starting NETSMART 1500 HA Services
status"
status ha"
status list"
status all"
Down
Up
Up
Up
Up
Down
Up
Up
Up
Up
Up
Up
Down
Down
Up
Up
Up
Down
Up
Up
Up
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
Jun
10:29:16
10:38:50
10:38:50
10:38:51
10:38:50
10:29:16
10:38:54
10:38:49
10:38:49
10:38:50
10:38:49
10:38:51
10:29:16
10:29:16
10:38:49
10:38:51
10:38:50
10:29:16
10:38:49
10:38:51
10:38:51
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
PDT
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
2006
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
25
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Stopping NETSMART 1500 HA Services
7.4
SUBPROCEDURE
PAGE
7.4.1
7-13
7.4.2
7-14
NETSMART 1500 HA services can be brought down as a cluster from both primary and
back-up systems. Optionally, the system can be brought down on passive-only systems.
7.4.1
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
7-13
CHAPTER 7
High Availability Administration
Stopping NETSMART 1500 HA Services
7.4.2
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Initiating Manual Failover
7.5
The manual failover request continues only if these conditions are met.
In the examples used in this procedure, haorbison is assumed to be in Active-Up state, and
garcia is assumed to be in Passive-Up state.
Initiate manual failover as follows:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
7-15
CHAPTER 7
High Availability Administration
Initiating Manual Failover
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Initiating Manual Failover
In the above example, haorbison reports Passive-Up state, and garcia reports Active-Up
state.
5
Optionally, check the status of NETSMART 1500 HA services on both nodes. Refer to
Section 7.3.2, Checking NETSMART 1500 HA Status [p. 7-8], for more information.
During a manual failover, clients are notified of the failover in progress. After the passive
system is in a state to allow clients to connect, it sends a message to the clients, and the clients
can then failover to the new active system. During failover, clients are not allowed to do any
operations other than getting progress information of the failover. Refer to Section 7.10,
Understanding Client Installation and Behavior [p. 7-26], for more information.
FNC-1500-0050-110
Issue 1, January 2007
7-17
CHAPTER 7
High Availability Administration
Understanding Automatic Failover
7.6
NETSMART 1500
Clients
NETSMART CORBA
Traffic
Primary
Backup
Active
Passive
DCN
DCN
NEs
m1615ar_1
In use
Standby
Failed
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Understanding Automatic Failover
NETSMART 1500
Clients
NETSMART CORBA
Traffic
Primary
Backup
Passive
Active
DCN
DCN
NEs
m1615as_1
In use
Standby
Failed
Failover time varies but is generally about 30 minutes. Failover time is defined as the interval
between the time the passive system detects the active system failure and the time the passive
system is connected and logged on (in a best-effort manner) to the same number of NEs that
the active system was logged on during the last snapshot before the active failure. The event
and best-effort cause of the failover is logged as a System Event. In addition, an e-mail is sent
to the administrator.
After the passive system is functional, connected clients are notified and start using the new
active system. The clients cannot autonomously decide to switch to another machine.
FNC-1500-0050-110
Issue 1, January 2007
7-19
CHAPTER 7
High Availability Administration
Understanding Automatic Failover
Failover is non-revertive. After a passive system has become the new active system, a user or
administrator must actively initiate the process of reverting back to the repaired active system.
After a failover, no monitoring of the failed active system is done. The system goes into
Single-Node NETSMART 1500 mode.
When the failed active system is repaired and NETSMART 1500 HA services are restarted,
the system comes up as a passive system. A manual failover can then be initiated for reverting
back.
During an automatic failover, clients are notified of the failover in progress. After the passive
system is in a state to allow clients to connect, it sends a message to the clients, and the clients
can then failover to the new active system. During failover, clients may not be allowed to do
any operations other than getting progress information of the failover. Refer to Section 7.10,
Understanding Client Installation and Behavior [p. 7-26], for more details.
During a passive system failure or crash, the active system detects the failure, logs a System
Event, and sends an e-mail to the administrator. All transactions that are executed on the
active system begin queueing up after the passive system is down. Every 10 percent increment
in queue size results in an e-mail being sent to the administrator. If the queue becomes full,
queueing is disabled, and the passive system must be re-added to the active system to be
functional again. The queue can usually accommodate 24 hours worth of NETSMART 1500
activity. The passive system, after being repaired, can be restarted by executing the
netsmart start command. Refer to Section 7.3, Starting NETSMART 1500 HA
Services [p. 7-5], for more information.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Recovering from a Crash
7.7
A system is determined to be in a failed state when an attempt to start the services is not
successful (for example, if the netsmart start command terminates without completing
properly). In such cases, when a reattempt is made to start services, NETSMART 1500 HA
service detects and does recovery before proceeding to the normal startup procedure.
The recovery process puts the node in gracefuldown (gd) state of the last known role
(passive or active). For example, a node that is in a passive-failed state would be put into
passive-gracefuldown state.
In the examples used in this procedure, the assumption is that haorbison was in active-up
state before it crashed. The system then repairs and restarts haorbison and tries to restart
NETSMART 1500 HA services. The peer system (garcia) was also down, and, after
recovery, a try is made to bring this system back up in the active-up state.
For using a database recovery on an HA system using the database backup, refer to
Section 2.7.4, Restoring Backup of HA on an HA System [p. 2-35].
Follow this procedure to recover from a crash:
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
7-21
CHAPTER 7
High Availability Administration
Recovering from a Crash
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Understanding Node Isolation Behavior
7.8
Behavior
Active
Passive
This parameter is defined in the HA properties file but is not visible as a configurable parameter in the
configuration process. The default value is 50 percent.
FNC-1500-0050-110
Issue 1, January 2007
7-23
CHAPTER 7
High Availability Administration
Understanding Data Loss Scenarios
7.9
SUBPROCEDURE
PAGE
7.9.1
7-24
7.9.2
7-25
NETSMART 1500 HA services make every attempt to minimize data loss, but there are some
scenarios where data loss can occur. This section documents data loss scenarios.
7.9.1
Heartbeat
Passive
m1615an_1
DRE
Active
After a certain time (for example, an hour), the active system crashes, resulting in an
automatic failover to the passive system. Because the data replication engine (DRE) link was
down, the passive side NETSMART 1500 HA services are activated in active mode with one
hour of old data. When the failed active system is repaired and recovered, the data in the
queues is dropped, and the node database is resynced with the surviving active system, thus
resulting in data loss.
Note: Before initiating recovery, NETSMART 1500 database backup can be initiated from the
Heartbeat tool. This backup ensures that any log information or faults are not lost.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Understanding Data Loss Scenarios
7.9.2
PAGE
7-25
7-25
Data loss can occur if the active system is isolated, failover occurs, and the old active node
continues to operate as active. When communication between the servers is reestablished, the
old active system goes into a passive-failed state, but the data that was accumulated on the
active server is not resynced during recovery. If the data must be saved, NETSMART 1500
database backup can be initiated from Heartbeat. This backup ensures that any log
information or faults are not lost.
FNC-1500-0050-110
Issue 1, January 2007
7-25
CHAPTER 7
High Availability Administration
Understanding Client Installation and Behavior
7.10
During a manual or automatic failover, a failover progress dialog opens, freezes all other
application windows, and displays the progress of the failover. When the failover is
complete, all open application windows are closed and reopened automatically. This
action ensures that new connections are established to the new active system.
The NETSMART 1500 HA client adds a new application called HA Status Monitor,
which can be invoked from the Applications > Administration menu of any application
window. The HA Status Monitor displays the current status of the HA subsystem and
has a console that displays real time streaming of HA messages.
Dashboard displays the federation name and role of both the primary and back-up node in
the federation status panel. The federation name displays above the icons for each node. The
larger icon represents the active node, and the smaller icon represents the passive node. Colors
of the Dashboard node monitor icon represent the state of the primary and backup node as
defined in Table 7-5 [p. 7-26].
Table 7-5: Color Conventions for Dashboard Node Icons
Color
State
Green
Up
Red
Down
Yellow
In progress
Gray
Unknown
Run your mouse pointer over the federation icon for a ToolTip that displays the role of the
node and the state of the node.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Unconfiguring NETSMART 1500 HA Service
7.11
Step
Task
Shut down the system by entering the following command at the command prompt:
netsmart stopall
M E N U
FNC-1500-0050-110
Issue 1, January 2007
7-27
CHAPTER 7
High Availability Administration
Unconfiguring NETSMART 1500 HA Service
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Unconfiguring NETSMART 1500 HA Service
A D M I N I S T R A T I O N
FNC-1500-0050-110
Issue 1, January 2007
7-29
CHAPTER 7
High Availability Administration
Unconfiguring NETSMART 1500 HA Service
Enter q Quit.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
HA Quick Commands
7.12
HA Quick Commands
This section describes some quick commands that can be used. It includes the following
subsections:
7.12.1
NO.
SUBSECTION
PAGE
7.12.1
Cluster Startup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-31
7.12.2
7-31
7.12.3
Shutdown Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-32
7.12.4
Enable Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-32
7.12.5
Disable Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-32
7.12.6
Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7-33
7.12.7
7-33
Cluster Startup
Start a cluster as follows:
Step
Task
7.12.2
Step
1
Task
FNC-1500-0050-110
Issue 1, January 2007
7-31
CHAPTER 7
High Availability Administration
HA Quick Commands
7.12.3
Shutdown Cluster
Shut down a cluster as follows:
Step
Task
7.12.4
Enable Failover
Go from single-node mode to cluster mode as follows:
Step
Task
7.12.5
Disable Failover
Shut down passive node only as follows:
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
HA Quick Commands
7.12.6
Failover
Use the command netsmart failover for this operation. No input is required.
7.12.7
Step
Task
FNC-1500-0050-110
Issue 1, January 2007
7-33
CHAPTER 7
High Availability Administration
Hints and FAQs
7.13
7.13.1
NO.
SUBSECTION
PAGE
7.13.1
7-34
7.13.2
7-35
Before configuring NETSMART 1500 HA for a federation that manages a large set of
NEs, populate the NETSMART 1500 database first with NEs, and then initiate HA
configuration. If you populate the data after HA is configured, the passive receives this
data through the relatively slower replication process. If you populate the data first, the
configuration does an export/import of the data to the backup, which is relatively faster
because it is a bulk operation.
Set up e-mail systems in such a way that e-mails are received by the administrator from
both the primary and back-up systems. Ideally, NETSMART 1500 and Oracle account
e-mails should be forwarded from both the primary and secondary systems to the
administrator.
Because NETSMART 1500 HA does not replicate the Solaris user IDs and passwords,
administrators are advised either to use a non-file-based authentication mechanism
(such as common LDAP, NIS, and NIS+) or to create user IDs on both the HA nodes
with the same password.
FNC-1500-0050-110
Issue 1, January 2007
CHAPTER 7
High Availability Administration
Hints and FAQs
7.13.2
FNC-1500-0050-110
Issue 1, January 2007
7-35
A
CHAPTERS/APPENDIXES
1 Introduction
2 Oracle Administration
3 NETSMART 1500
Administration
NO.
SECTION
4 UNIX System
Administration
A.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A-2
5 Network Administration
A.2
A-2
6 Service Support
A.3
A-3
PAGE
7 High Availability
Administration
FNC-1500-0050-110
Issue 1, January 2007
APPENDIX A
Server and Client Port Usage
Overview
A.1
Overview
This appendix lists the ports used in NETSMART 1500 client and server systems.
A.2
Ports
Containermanagement
120002049
fenicsd
1205012099
Oracle
1521
httpd
7600
COSServer
1210012199
PM-ES
1220012249
Framework
1225012299
Security
1230012349
TARPServer
1235012399
CommunicationServer-1
13000(13000+NumOfCommunicationServers*50)
SessionManager
1240012449
ApplicationServer
15000(15000+NumOfApplicationServers*50)
EmlNmlServer
1260012649
TL1UplinkServer
1265012699
PerformanceManagement
1270012749
FNC-1500-0050-110
Issue 1, January 2007
APPENDIX A
Server and Client Port Usage
Client Port Usage
Ports
DataServicesManager
1275012799
ConnectionManager
1280012849
SNMP Uplinka b
1600016050
SNMPServer
1290012949
ReportServer
1700017050
a SNMP
b
A.3
uplink listens on UDP port 162. If required, modifiy this with the following properties file:
$NETSMART_PROP_HOME_/Properties/SNMPUplinkServer
/SNMPUplinkServer.properties
If other SNMP agents, such as Sun, are running on this system, SNMPUplink can be reconfigured by editing the
same properties file in footnote a.
FNC-1500-0050-110
Issue 1, January 2007
A-3
Index
A
sending
from command line 3-46
from Web administration page 3-48
viewing 3-48
alert scripts
alert files
checking size 2-38
deleting 2-48
alert monitor, running 2-37
archived redo log files
missing, checking for 2-41
size, checking 2-40
file systems, checking size 2-39
notifying Oracle user 2-37
C
CDE (common desktop environment)
controls, adding to NETSMART 1500 client
accounts 4-8
client
recovering from errors 3-13
shutting down 3-11
starting
on server 3-9
troubleshooting 3-12
communication
server, NE logon. See NE log-on configuration.
configurations
SONET-SDH terminology mapping 3-88
backup
automated maintenance 2-43
full system 4-27
NETSMART 1500 application 4-28
performing 2-25
preventing erasure 2-34
recovering from 2-30
recovery from
HA on HA system 2-35
HA on non-HA system 2-35
Oracle database 2-29
startup by ending 2-22
UNIX system 4-27, 4-30
broadcast messages
FNC-1500-0050-110
Issue 1, January 2007
D
database
administration. See Heartbeat tool.
alert scripts. See alert scripts.
automated maintenance jobs, listed 2-44
backing up, online 2-25
backup files, location of 2-66
configuration information 2-59
connection status, checking 2-13
consistency, checking 2-11
Index
E
e-mail
NETSMART 1500 security 3-45
Oracle administrator 2-66
EML-NML
disabling 3-19
enabling 3-19
F
files
archive, creating 6-6
checking size, UNIX 2-39
database 2-59
properties
editing 6-7
key values table 6-8
restoring UNIX 4-31
FLASHWAVE 7400 node managers
disabling 3-22
enabling 3-22
fragmented tables, recreating indexes on 2-51
FTAM
restarting 5-12
starting 5-4
stopping 5-4
FTP (File Transfer Protocol)
information to FNC 6-3
G
generics. See software generics.
H
HA (high availability) 2-35
automatic failover, understanding 7-18
checking status 7-8
client installation behavior 7-26
crashes, recovering from 7-21
data loss scenarios 7-24
manual failover, initiating 7-15
procedures 7-1 7-33
quick commands 7-31 7-33
starting 7-5
stopping
on active and passive systems 7-13
on passive only systems 7-14
system isolation behavior 7-23
unconfiguring 7-27
Heartbeat tool
administration procedures 3-4 3-88
FNC-1500-0050-110
Issue 1, January 2007
Index
N
NE software generics. See software generics.
NEs
log-on configuration
communication server properties, modifying 3-44
log-on delay 3-44
log-on retries and delay parameters 3-42
reconnect algorithm 3-42
resynchronizing
data 3-41
software generics. See software generics.
NETSMART 1500
administration
optional features 3-16
See also Heartbeat tool.
license
upgrading 3-59
upgrading for HA 3-69
log tables, exporting 2-68
ObjectStream license, configuring 3-37
optional features
disabling 3-19 3-27
enabling 3-19 3-26
patches, managing 3-16
security e-mail 3-45
system information 6-12
user
accounts. See user accounts, NETSMART 1500.
warning banner, pre-logon 3-49
I
IP address
changing 5-22
IP multipathing
setting up 5-6
L
license
NETSMART 1500
displaying 3-58
upgrading 3-59
upgrading for HA 3-69
ObjectStream, configuring 3-37
log
files
archiving 6-4
database 2-63
for UNIX system backup 4-30
logical
restore, invoking resync function 3-41
M
maintenance services
database. See database maintenance.
FNC-1500-0050-110
Issue 1, January 2007
I-3
Index
starting 3-8
starting up 2-18
archive 2-19
by ending backup 2-22
normal 2-18
trace files 2-66
troubleshooting 2-59
user account 2-5
network
administration. See Heartbeat tool.
NSAP address
displaying 5-15
See also OSI address.
O
object tablespace, finding contiguous space 2-50
ObjectStream license, configuring 3-37
optional features
disable (all) 3-27
enable (all) 3-26
See also EML-NML, PM, TL1 uplink.
Oracle
alert scripts. See alert scripts.
automated maintenance jobs, listed 2-44
backup
backup 2-25
control files, location 2-65
core dumps, location 2-66
data, tablespaces for 2-64
database administration, Heartbeat menu items 2-2
database objects, validating 2-54
database scripts, status 2-10 2-15
editing crontab 2-52
e-mail to Oracle administrator 2-66
files. See database files.
import/export procedures 2-67
logs, generating ZIP files 2-55
maintenance scripts 2-47 2-57
maintenance, automated 2-43
passwords
changing 2-6, 2-9
verifying 2-7
PM log tables, exporting and storing 2-69
purging 2-36
queries, checking index usage 2-54
restoring database 2-29, 2-30
See also database.
shutting down 2-23
abnormal 2-24
normal 2-23
P
parameters
configuring communication servers 3-42
patches
checking Solaris 6-12 6-17
NETSMART 1500 3-16
performance management (PM)
disabling 3-18
enabling 3-17
port usage
client A-3
server A-2
pre-logon warning banner 3-49
checking status 3-53
disabling 3-51
enabling 3-50
importing text 3-51
retrieving and editing 3-52
properties
FNC-1500-0050-110
Issue 1, January 2007
Index
file
service
support, Heartbeat 6-2
shutting down
database. See Oracle.
NETSMART 1500 client 3-11
NETSMART 1500 server 3-10
SNMP uplink
disabling 3-26
enabling 3-25
reconnect algorithm
and NE logon 3-42
modifying properties 3-44
software
patches, managing 3-16
recovery
from client error 3-13
from server error 3-14
software generics
CD installation
on NETSMART 1500 server 3-29
on UNIX client 3-34
installing from
network drive 3-36
server 3-29
UNIX client 3-34
restore
files, UNIX 4-31
HA on HA system 2-35
HA on non-HA system 2-35
Oracle database 2-29
resync
NE data 3-41
SONET-SDH
terminology mapping
enabling and disabling feature 3-88
RTRV-HDR 3-55
support
log
editing 6-2
listing 6-3
scripts
automated Oracle maintenance 2-43
database status 2-10
Oracle database alerts 2-37
security
data
tablespaces
database 2-64
extending 2-52
exporting 3-39
importing 3-40
servers
checking processes status 3-12
communication, configuring log-on retries and delays
3-42
FNC-1500-0050-110
Issue 1, January 2007
TARP
manual adjacency
adding 5-19
deleting 5-20
displaying 5-20
manual entries
adding 5-21
deleting 5-21
displaying 5-21
menu 5-18
TID
displaying 5-19
setting 5-19
I-5
Index
TCP/IP
and OSI networks with NETSMART 1500 5-5
connectivity 5-6
time
and date, setting on UNIX system 4-14
TL1 Heartbeat log manager
and RTRV-HDR logging 3-55
disabling 3-56
enabling 3-55
TL1 uplink
disabling 3-21
enabling 3-20
trace
files, deleting 2-48
menu 3-28
transaction debugging client 3-76
options 3-77
starting 3-77
troubleshooting
database information 2-59
U
UNIX
accounts. See user accounts, UNIX.
adding CDE controls to NETSMART 1500 client
accounts 4-8
administration menu items 4-2
backing up system 4-27 4-30
backup log 4-30
changing date and time
backward 4-14
forward 4-16
cronjobs menu 4-25
date and time, setting 4-14
defunct processes, checking 4-26
disk usage 4-20
files
checking size 2-39
checking size of archived redo log files 2-40
installing NE software generics 3-34
log files, understanding 4-13
user accounts
Oracle 2-5
UNIX
adding using local files 4-11
configuring 4-5
creating NETSMART 1500 administrator 3-7
deleting using local files 4-13
W
warning banner, pre-logon 3-49
Web administration operations
sending and receiving broadcast messages 3-48
Web GUI
Web administration
disabling 3-24
enabling 3-23
workstation
UNIX, powering down 4-3
UNIX, restarting 4-4
FNC-1500-0050-110
Issue 1, January 2007