You are on page 1of 40

Tellabs 8000 Intelligent Network Manager SR2.

1
Maintenance Procedures
76.8021-70211A
14.02.2011

Document Information

2011 Tellabs. All rights reserved.


This Tellabs manual is owned by Tellabs or its licensors and protected by U.S. and international copyright laws, conventions and
treaties. Your right to use this manual is subject to limitations and restrictions imposed by applicable licenses and copyright laws.
Unauthorized reproduction, modification, distribution, display or other use of this manual may result in criminal and civil penalties.
The following trademarks and service marks are owned by Tellabs Operations, Inc. or its affiliates in the United States and/or
other countries: TELLABS , TELLABS logo, TELLABS and T symbol , and T symbol .
Any other company or product names may be trademarks of their respective companies.
The specifications and information regarding the products in this manual are subject to change without notice. All statements,
information, and recommendations in this manual are believed to be accurate but are presented without warranty of any kind,
express or implied. Users must take full responsibility for their application of any products.
Adobe Reader are registered trademarks of Adobe Systems Incorporated in the United States and/or other countries.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
2

76.8021-70211A
2011 Tellabs.

Document Information

Terms and Abbreviations

76.8021-70211A
2011 Tellabs.

Term

Explanation

AMPS

Automatic Maintenance Procedures Service

CST

Configuration and Service Tracker

DB

Database

LAN

Local Area Network

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
3

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
4

76.8021-70211A
2011 Tellabs.

Table of Contents

Table of Contents

About the Manual .............................................................................................................. 7


Objectives.......................................................................................................................................................................
Audience.........................................................................................................................................................................
Related Documentation ..................................................................................................................................................
Conventions....................................................................................................................................................................
Document Conventions ..................................................................................................................................................
Documentation Feedback...............................................................................................................................................

7
7
7
8
8
8

Introduction ................................................................................................................. 9

Automatic Maintenance Procedures ......................................................................... 10


2.1
2.2

Fault History Maintenance.......................................................................................... 14


3.1

Activating Automatic Maintenance Procedures .................................................................................................. 11


Configuring Automatic Maintenance Procedures on Database High Availability Setup .................................... 13

Configuring Automatic Deletion of Fault History............................................................................................... 14

Daily Maintenance ....................................................................................................... 15


4.1

4.2
4.3

4.4
4.5

76.8021-70211A
2011 Tellabs.

Backing Up Tellabs 8000 Intelligent Network Manager Database ..................................................................... 15


4.1.1
General................................................................................................................................................. 15
4.1.2
Verifying Scheduled Backups on Windows Database Server in Case Backup Scripts Are Used ....... 15
4.1.3
Verifying Scheduled Backups on Solaris Database Server in Case Backup Scripts Are Used ........... 16
4.1.4
Verifying Tape Backups....................................................................................................................... 17
Monitoring Size of Data Devices and Transaction Logs..................................................................................... 17
Checking Disk Space Usage and Logfiles on Database Server........................................................................... 19
4.3.1
Checking Disk Space........................................................................................................................... 19
4.3.2
Investigating and Removing Core Files from Solaris Database Server .............................................. 19
4.3.3
Truncating Log File on Solaris Database Server ................................................................................. 19
Checking Tellabs 8000 Intelligent Network Manager Servers............................................................................ 19
Checking Communication Links and Paths to All Network Nodes .................................................................... 20

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
5

Table of Contents

Weekly Maintenance .................................................................................................. 21


5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
5.10

Monthly Maintenance.................................................................................................. 31
6.1
6.2
6.3
6.4
6.5

6.6
6.7

Verifying Database Configuration Parameters .................................................................................................... 21


Verifying Status of Scheduled Procedures in Automatic Maintenance Procedures Tool.................................... 22
Reorganizing Indexes in Tellabs 8000 Intelligent Network Manager Database ................................................. 23
Updating Database Statistics after Reorganizing Indexes ................................................................................... 24
Checking Tellabs 8000 Intelligent Network Manager Database Consistency .................................................... 25
Monitoring Size of Performance Data Size......................................................................................................... 25
Testing Restoration of Backup ............................................................................................................................ 26
Checking Disk Space of Tellabs 8000 Intelligent Network Manager Computers (and Moving Log Files) ....... 28
Checking Disk Space on ThinClient Server ........................................................................................................ 29
Checking Disk Space on Route Master ............................................................................................................... 29

Deleting Audit Log.............................................................................................................................................. 31


Monitoring Size of Fault Tables .......................................................................................................................... 31
Accounting Management Events......................................................................................................................... 33
Private Subnetwork Trunk Accounting Events for Tellabs 8100 Nodes ............................................................. 33
Recovery Management System ........................................................................................................................... 34
6.5.1
Spontaneous Fault Reporting (GMH Units)........................................................................................ 34
6.5.2
Spontaneous Fault Reporting (GMU, GMU-A and GMU-M Units) .................................................. 34
6.5.3
Spontaneous Fault Reporting (Other Units) ........................................................................................ 34
Maintenance of Element Software in Database................................................................................................... 35
Node Configuration Backup ................................................................................................................................ 35

Additional Tools for Sun Solaris Server.................................................................... 37


7.1
7.2
7.3

Sun Explorer ........................................................................................................................................................ 37


Installation of Explorer........................................................................................................................................ 37
Configuration and Service Tracker (CST) Latest Version ................................................................................... 38

Suggested Weekly Schedule for Automatic Maintenance Procedures........................ 39

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
6

76.8021-70211A
2011 Tellabs.

About the Manual

About the Manual


This section discusses the objectives and the intended audience of this manual, Tellabs 8000
Intelligent Network Manager Maintenance Procedures, and consists of the following sections:
Objectives
Audience
Related Documentation
Conventions
Documentation Feedback

Objectives
This manual provides instructions on the procedures that have to be executed on a regular basis in
order to maintain the Tellabs 8000 intelligent network manager system.

Audience
This manual is designed for system administration personnel.

Related Documentation
The new document numbering scheme consists of the document ID, indicated by numbers, and the
document revision, indicated by a letter. The references in the Related Documentation table below
are generic and include only the document ID. To make sure the references point to the latest
available document versions, please refer to the relevant product document program on the Tellabs
Portal by navigating to www.portal.tellabs.com > Product Documentation > Data Networking >
Tellabs 8000 Intelligent Network Manager > Technical Documentation.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Software Installation Manual (76.8021-70201)

Provides instructions on how to install and


configure Tellabs 8000 intelligent network
manager in different Tellabs 8000 intelligent
network manager computer platforms
(workstation, servers).

Tellabs 8000 Intelligent Network Manager SR2.1


Third party Hardware and Software Requirements
(76.8021-70222)

Describes all the needed third party software


components and versions in the Tellabs 8000
intelligent network manager system (operating
systems, service packs, database server versions,
client versions, web browsers). Describes also
the recommended computer platforms, LAN
configurations and minimum performance
requirements for computers.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
7

About the Manual

Conventions
The following typographical conventions are used to help distinguish elements in the text in this
manual.
Text Element

Explanation

Bold

Indicates the name of an item (window/dialog/button, etc.) you need to select.


When more than one item needs to be selected in a step, the items to be selected
are separated by a dash (-).

Italics

Normally italics is used when referring to another document.

Courier

Indicates the contents of a file, a command or a part of a program.

Document Conventions
This is a note symbol. It emphasizes or supplements information in the document.

This is a caution symbol. It indicates that damage to equipment is possible if the instructions
are not followed.

This is a warning symbol. It indicates that bodily injury is possible if the instructions are not
followed.

Documentation Feedback
Please contact us to suggest improvements or to report errors in our documentation:
Email: fi-documentation@tellabs.com
Fax: +358.9.4131.2430

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
8

76.8021-70211A
2011 Tellabs.

1 Introduction

1 Introduction
This document is intended for system administrators of the Tellabs 8000 intelligent network
manager. It contains step lists for the procedures that must be executed on a regular basis to properly
maintain the Tellabs 8000 intelligent network manager system.

Please note that if these system maintenance procedures are not carried out, it may lead to
total management outage. If there isnt a valid database backup available, system recovery to
operational state can take several days or even weeks, and in most cases requires a massive
amount of manual work by Tellabs. This work is not covered by the Support Agreement
but is charged separately at an hourly rate.

The procedures are classified into four categories, as listed below:


1.

What can be done automatically.

2.

What should be done every day.

3.

What should be done every week.

4.

What should be done every month or whenever necessary.

These procedures are to be used with the Tellabs 8000 intelligent network manager. In addition to
these procedures, the normal maintenance tasks for the operating system should be performed. For
example, events in the Event Viewer such as Application Log, Security Log and System Log should
be monitored on a regular basis and truncated before they become too big.
Instructions in this manual are written for Classic Start menu of Windows Server 2003/2008.
You can turn on the Classic Start menu in Windows 2003/2008/XP by right clicking Start and
selecting Properties, clicking the Start Menu tab, selecting the Classic Start menu radio button
and clicking OK.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
9

2 Automatic Maintenance Procedures

2 Automatic Maintenance Procedures


Tellabs 8000 intelligent network manager provides a way to automate some database maintenance
and backup procedures that previously had to be done manually. The procedures that can be
automated are listed below:
1.

Rebuild Indexes (ReorgIndex.cmd) [ID 10001]


Schedule the rebuilding of indexes to be run before the reclaiming of unused space.

2.

Reclaim Unused Space (ReorgIndex.cmd) [ID 20004]

3.

Update Table Statistics (UpdStat.cmd) [ID 30002]


Schedule the updating of table statistics to be run before the updating of index statistics.

4.

Update Index Statistics (UpdStat.cmd) [ID 40003]

5.

Check/Fix Table Corruptions (CheckDb.cmd) [ID 60006]

6.

Check/Fix Index Corruptions (CheckDb.cmd) [ID 70007]

7.

Dump Database [ID 80000]

8.

Dump Transaction [ID 90000]

9.

Dump Master [ID 100000]

10.

Resourcelock Housekeeper [ID 110008]

11.

DB Server Housekeeper [ID 120009]

12.

Server Monitoring Housekeeper [ID 130010]

The database maintenance procedures (16) can be scheduled to run weekly during low Database
Server load.
For the database backup procedures (Dump Database, Dump Transaction and Dump Master),
there are two alternatives: using the Automatic Maintenance Procedures Tool or using scheduled
scripts. The procedure for using scheduled scripts is presented in Tellabs 8000 Intelligent Network
Manager Software Installation Manual in chapters Database Backup Procedures for Windows
2003/2008 Database Server and Database Backup Procedures for Solaris.

Note that both alternatives of taking database backup should not be used simultaneously.
When configuring the backups to be taken with Automatic Maintenance Procedures Tool,
make sure to delete the related AT command scheduled scripts. If there are no other entries,
deleting all can be done with command at /delete on the Database Server. If there are other
entries, the deletion must be done only for the related entries. In case of Solaris database
server, similar procedure needs to be performed using the crontab utility. On the other hand,
if database backup is chosen to be done with scheduled scripts, then the database backup
procedures in Automatic Maintenance Procedures Tool should be disabled from use.

The database housekeeping procedures Resource Lock Housekeeper and DB Server Housekeeper
are automatically scheduled to run at the 15th minute of every hour and at 05:59 of every day
respectively.
Tellabs 8000 Intelligent Network Manager SR2.1
Maintenance Procedures
10

76.8021-70211A
2011 Tellabs.

2 Automatic Maintenance Procedures

2.1

Activating Automatic Maintenance Procedures


You can activate the automatic maintenance procedures to be run regularly.
Tool(s) to use

Step 1

Automatic Maintenance Procedures Tool

Make sure that the Automatic Maintenance Procedures Service (AMPS) is enabled and running
on one of the Management Servers.
If it is not enabled, shut down a Management Server, start the Tellabs 8000 intelligent network manager installation program with the re-configure option and select also the Automatic Maintenance Procedures service from the list of services to be installed.

Step 2

Login with superuser privileges to Tellabs 8000 Intelligent Network Manager Toolbox.

Step 3

Start the Automatic Maintenance Procedures Tool.


Select Tools - Automatic Maintenance Procedures Tool in Tellabs 8000 Intelligent Network Manager Toolbox.

Step 4

Select the procedure that you wish to schedule.

Step 5

Click the Schedule button.

Step 6

Login with the dxxdata login name and password and click OK.

Step 7

Complete the following details in the opened AMPT Scheduling window:


Enter a name for the scheduled procedure in Schedule Name.
Select the interval for the frequency of the selected maintenance procedure (Yearly, Monthly,
Weekly, Daily, Hourly, Every Minute).
Select the execution time preferably to a low traffic period like late night-hours and during
weekends etc.
Click Add to add the task to the schedule. The scheduled procedure is shown in the list.
Click Exit.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
11

2 Automatic Maintenance Procedures

Step 8

Repeat steps 4 to 7 for the other procedures keeping in mind the following:
Refer the guidelines provided in Appendix Suggested Weekly Schedule for Automatic Maintenance Procedures for scheduling the various maintenance procedures.
For database maintenance procedures (16), schedule each procedure initially with rather
high interval between them, for example, 23 hours, so that they do not overlap each-other.
As the execution times vary depending on the network factors, such as size, these intervals need to be reassessed and adjusted after observing couple of runs and also periodically
checked as mentioned in the weekly maintenance task.
During scheduling, follow the sequence guidelines for running different procedures with
procedures 1 and 3 in 2 Automatic Maintenance Procedures. See also chapter 5.2 Verifying
Status of Scheduled Procedures in Automatic Maintenance Procedures Tool.
The database backup procedures (Dump Database, Dump Transaction and Dump Master)
should be scheduled only if database backup procedures are not scheduled using scripts.
The suggested scheduling times and intervals of database backup procedures are as follows:
Full database dump at 04:00 everyday; Database Transaction dump at 07:00, 11:00, 15:00,
19:00 and 23:00 everyday; Database Master dump at 06:00 every sunday.
If database backup procedures are being scheduled, the backup directory should be set by
adding the parameter DBBKPDIR = <BACKUP_DIRECTORY> (Sample values are
G:\backup\ in case of windows and /backup/ in case of Solaris) in the NMS.INI file of
the management server running AMPS process and all work-stations where AMP Tool is
used. Note that the AMPS process and the AMP Tool need to be restarted after making the
change to NMS.INI file to take it into effect.
The Resourcelock Housekeeper and Server Monitoring Housekeeper schedules should be
left as pre-configured.
The DB Server Housekeeper should be scheduled to run daily at 07:30.

Step 9

Click Exit to close the Automatic Maintenance Procedures Tool.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
12

76.8021-70211A
2011 Tellabs.

2 Automatic Maintenance Procedures

2.2

Configuring Automatic Maintenance Procedures on Database


High Availability Setup
The following configuration needs to be done to enable automatic maintenance procedures to be
performed on both ASE1 and ASE2 on a database high availability setup.
Step 1

Make sure that the server names are configured correctly on ASE1 as shown below.
Open the Command Prompt window and start an ISQL session as an sa user on ASE1 and
check the server name configuration.
isql -SDBSERVER -Usa Ppassword
1> sp_helpserver
2> go
The output should display the hostname of ASE1 as the local entry and hostname of ASE2
as an ASEnterprise entry.
If any of the two entries are missing, add them as shown below.
1> sp_addserver <server name of ASE1>, local
2> go
3> sp_addserver <server name of ASE2>
4> go
Enable remote access and login for ASE1.
1> sp_configure allow remote access, 1
2> go
3> sp_addremotelogin <server name of ASE2>
4> go

Step 2

Accomplish the previous step on ASE2 keeping in mind to reverse the roles of ASE1 and ASE2, i.e.
ASE2 as local and ASE1 as remote.

Step 3

Restart the Sybase software on both ASE1 and ASE2.

Step 4

Verify on ASE1 and ASE2 that there are entries for server names of both ASE1 and ASE2 through
the Sybase Directory Service Editor tool using the dsedit command. The dsedit tool is a graphical
user interface which requires X display. Also the dscp command, which is a command line interface,
can be used for this purpose.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
13

3 Fault History Maintenance

3 Fault History Maintenance


3.1

Configuring Automatic Deletion of Fault History


Tool(s) to use

Fault Management System

Description

Automatically deletes the faults that are older than a configured period. In
the long run, the fault history table grows so big that it is necessary to delete
part of the old faults. Otherwise the performance of Tellabs 8000 intelligent
network manager system suffers. If you want to save all faults, you must
always save the latest database backup before deletion of the fault history.

Deletion of Fault History creates a large amount of transaction log data. It should be
periodically checked that there is enough free space in log device as mentioned in daily
maintenance tasks. For more information, see 4.2 Monitoring Size of Data Devices and
Transaction Logs.

The Tellabs 8000 manager can be configured to automatically delete fault history older than a
configured period of days. Both element and service faults can be configured separately.
Procedure
Step 1

Open the Configure Fault History Deleting option.


Select the Fault - Configure History Settings menu option in the network window of Fault
Management.
The Configure Fault History Deleting dialog opens.

Step 2

Configure the number of days of Element Fault History to be retained.


Enable the Automatic Element Fault History Deleting enabled option.
Configure the required number of days after which element fault history can be deleted in
text box Delete element faults from Fault History older than days.

Step 3

Configure the number of days of Service Fault History to be retained.


Enable the Automatic Service Fault History Deleting enabled option.
Configure the required number of days after which service fault history can be deleted in
text box Delete service faults from Fault History older than days.

Step 4

Press OK to exit.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
14

76.8021-70211A
2011 Tellabs.

4 Daily Maintenance

4 Daily Maintenance
4.1
4.1.1

Backing Up Tellabs 8000 Intelligent Network Manager Database


General
Time

Late evening or night time

Tool(s) to use

ISQL and scripts

Description

The Tellabs 8000 intelligent network manager database contains all the
information on the network, for example, Tellabs 8100 managed access
system and Tellabs 8600 managed edge system. Therefore, you have to take
a backup of the Tellabs 8000 intelligent network manager database every
day. If the Tellabs 8000 intelligent network manager database is corrupted
or destroyed for some reason, e.g. a hard disk problem, you must restore
the latest database backup.

You should take a backup of the Tellabs 8000 intelligent network manager database on the hard disk
of the database server and on tape at least every day. The backup procedures take the transaction
logs dump every four hours of the day, and a full dump of the database once every day and master
database dump once every week and moves these dumps to tape once every day. Study these
procedures carefully in order to understand them and to be able to check if they function properly on
your computer.
If you have not installed a backup procedure on your Database Server, see Tellabs 8000 Intelligent
Network Manager Software Installation Manual, chapter Database Backup Procedures for Windows
2003/2008 Database Server for installation instructions. If the Database Server is installed on
Solaris server, see chapter Database Backup Procedures for Solaris for installation instructions. If
Database backup procedures are being used through automatic maintenance procedures (AMPS), the
below verification of scheduled scripts is not applicable . The automatic maintenance procedures are
verified as mentioned in 5.2 Verifying Status of Scheduled Procedures in Automatic Maintenance
Procedures Tool

4.1.2

Verifying Scheduled Backups on Windows Database Server in Case


Backup Scripts Are Used
It is recommended to check daily that the scheduled backups are running. To verify running tasks,
run the at command in Command Prompt:
C:\>at

76.8021-70211A
2011 Tellabs.

Status ID

Day

Time

Command Line

Each M

23:00 PM

g:\backup\dumpdb.bat mon 6

10

Each Tu

3:00 AM

g:\backup\dumptran.bat tue 1

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
15

4 Daily Maintenance

11

Each W

3:00 AM

g:\backup\dumptran.bat wed 1

12

Each Th

3:00 AM

g:\backup\dumptran.bat thu 1

13

Each F

3:00 AM

g:\backup\dumptran.bat fri 1

14

Each S

3:00 AM

g:\backup\dumptran.bat sat 1

15

Each Su

3:00 AM

g:\backup\dumptran.bat sun 1

16

Each M

7:00 AM

g:\backup\dumptran.bat mon 2

17

Each T

7:00 AM

g:\backup\dumptran.bat tue 2

18

Each W

7:00 AM

g:\backup\dumptran.bat wed 2

19

Each Th

7:00 AM

g:\backup\dumptran.bat thu 2

The list above is truncated, you should see at least seven full database dumps, five daily transaction
log dumps and one master database dump scheduled.
Apart from checking that the scheduling of backups is running and the actual backups are created, the
log file C:\SYBASE\BACKUP.LOG should be checked for any errors during creation of backups.

It should be verified that the backup scripts are configured correctly with the right database
server and database name. This can be checked in the scripts DUMPDB.BAT, DUMPDB.QRY,
DUMPMSTR.BAT, DUMPMSTR.QRY, DUMPTRAN.BAT and DUMPTRAN.QRY.

4.1.3

Verifying Scheduled Backups on Solaris Database Server in Case Backup


Scripts Are Used

Needed Crontab Entries


The backup script dbdump.sh runs with Solaris cronjob utility. To check the cronjob for the
scheduled backups, enter the following command.
#crontab l
Result:
#ident "@(#)root 1.21 04/03/23 SMI"
#
# The root crontab should be used to perform accounting data
collection.
#
#
10 3 * * * /usr/sbin/logadm
15 3 * * 0 /usr/lib/fs/nfs/nfsfind
30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] &&
/usr/lib/gss/gsscred_clean
#10 3 * * * /usr/lib/krb5/kprop_script ___slave_kdcs___
0 4 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh data"
0 5 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh tape"
55 5 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh clean"
0 6 * * 0 su - sybase -c "/sybase/bkscripts/dbdump.sh master"
Tellabs 8000 Intelligent Network Manager SR2.1
Maintenance Procedures
16

76.8021-70211A
2011 Tellabs.

4 Daily Maintenance

0
0
0
0
0

7 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh tran"


11 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh tran"
15 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh tran"
19 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh tran"
23 * * * su - sybase -c "/sybase/bkscripts/dbdump.sh tran"

Apart from checking that the scheduling of backups is running and that the actual backups are
created, the log file /sybase/log/SYB_BACKUP.log and the log file with relevant timestamp should
be checked for any errors during creation of backups.

It should be verified that the backup script dbdump.sh is configured correctly with the right
database server and database name.

Database Backup Procedure on High Availability Setup


The primary computer takes the backup. If there are any problems with the primary computer, it
should be fixed as soon as possible. If using the High Availability software, the secondary computer
also takes the backups when it is active.
If there are any problems with the fibres or with one disk storage array, the information is read from
the other disk storage array.

4.1.4

Verifying Tape Backups


The tape should be changed every morning. You should have at least 9 tapes: one for each week
day, the 8th tape should be used every second Monday, and the 9th tape should be used once every
month. If there is a need to have backups older than a month, the operator can use more tapes for
this purpose as needed. This procedure eliminates the possibility of total loss of the database if one
tape is faulty. The tapes should be tagged with their day or number according to their usage to avoid
any confusion. Store the tapes in a safe place (protected from fire). You should never store backups
in the same office/place where your Database Server is.

4.2

Monitoring Size of Data Devices and Transaction Logs


Time

Any time

Tool(s) to use

ISQL, NMS Component Fault Monitoring Window

Description

Needed to check that the transaction logs are not filling up and thus
preventing the usage of Tellabs 8000 intelligent network manager. It is also
useful to check how much of the database device is currently used.

Procedure
Step 1

Open the Command Prompt window.

Step 2

Start an ISQL session.


isql -SDBSERVER -Usa -Ppassword

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
17

4 Daily Maintenance

Step 3

Check how much of the transaction log (free Kbytes in nms_log1) is used with the following
command.
1>sp_helpdb db8000
2>go
Result
The transaction log should never fill up if everything is functioning fine because it is dumped
automatically by the backup procedure that you must have set up. If the transaction log fills up, you
must dump it and find out why it filled up, because normally it should not happen.

Step 4

Check how much free space there is in the data device (free Kbytes in nms_db1).
1>sp_helpdb db8000
2>go
Result
The amount of free space should never drop below 10 Mbytes (10240 Kbytes). If this happens,
check if it is caused by an excessive number of stored fault history events. If this is the case, delete a
part of the fault history. Otherwise the network has grown so big that the size of the data device
must be extended (contact your local Technical Support for Tellabs 8000 system). The audit log
table can possibly cause the database to fill up too much. Delete or move the data from the audit
log to another database if necessary.
Alternatively, it is possible to configure Fault Management to show alarms when a data and log
device is 50, 60, 70, 80 or 90 percent full. This requires that the thresh.qry script is modified to
match the database name and that the script is run once every time the database devices are altered
by adding a device or changing the size of a device. This script sets the threshold triggers to the
database for sending an alarm message to the dbstate table. The NCAM_MONITOR_DB setting
must be set to TRUE in the nms.ini file of the Fault Mapping Workstation to enable the polling of the
dbstate table. Finally, the alarm levels must be configured in the Configure Alarm Limits dialog
- Add/Edit Alarm Limits dialog.
Configuring Alarm Levels

Step 1

Start Fault Management.

Step 2

Select the Options - NMS component monitoring menu option in the Fault Management main
window.

Step 3

Double-click on the computer icon at the lower right hand corner of Network View.
The NMS Component Fault Monitoring window opens.

Step 4

In the NMS Component Fault Monitoring window, Select Options Configure Alarm Limits

Step 5

Click Add or Edit button.


Configure the alarms for different data and log devices and set the corresponding limits using
the Add New Alarm Limit window.
After this has been done, FMS shows an alarm message describing which device is filling up and
which limit has been exceeded.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
18

76.8021-70211A
2011 Tellabs.

4 Daily Maintenance

4.3

Checking Disk Space Usage and Logfiles on Database Server

4.3.1

Checking Disk Space


The disk space usage of all file systems of the Database Server should be checked if they are near full.
On Solaris Database Server the command df k can be used to check disk space usage. Necessary
action should be taken to clear space on those file systems which are near full by clearing large and
old logfiles. On a Windows database server, the sybase log files (%SYBASE%\BACKUP.LOG and
%SYBASE%\MS.LOG) should be emptied during the maintenance window when database is down.

4.3.2

Investigating and Removing Core Files from Solaris Database Server


Core files are created when a crash has occurred at the operating system level. So, these core files
should be analyzed with the help of a solaris administrator to understand what caused the problem.
In order to list the core dump files from the Sun Solaris Database Server run the following command.
$ find / name core
After moving the core files to a different location for investigation, they should be deleted on the
server to save space.
$ find / name core print | xargs rm

4.3.3

Truncating Log File on Solaris Database Server


The running Database Server Log file cannot be deleted as ASE loses file handle on it, so it must be
purged by copying null to it as shown below.

4.4

Step 1

Open a terminal connection to the Solaris server.

Step 2

Go to log file location.


$ cd /sybase/log

Step 3

Copy the log file to archive.


$ cp DBSERVER.log DBSERVER.log.old

Step 4

Purge the existing log file.


$ cp /dev/null DBSERVER.log

Step 5

Similarly, repeat the above steps 2 to 4 for /sybase/log/SYB_BACKUP.log.

Step 6

If there is no need of old logs for any investigation, they should be deleted or moved to a different
storage server.

Checking Tellabs 8000 Intelligent Network Manager Servers

76.8021-70211A
2011 Tellabs.

Time

Any time

Tool(s) to use

Fault Management System

Description

Needed to check that Management, Communication and Route Master


Server(s) are functioning.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
19

4 Daily Maintenance

Procedure
Step 1

Start Fault Management.

Step 2

Check the fault status of Tellabs 8000 intelligent network manager servers.
Select the Options - NMS component monitoring menu option in the Fault Management
main window.
Double-click on the computer icon at the lower right hand corner of Network View.
The NMS Component Fault Monitoring window opens.
Check that there are no faults for any Tellabs 8000 intelligent network manager servers.
Result
There should not be any faults in the servers. If there are, try to find out the reason and solve the
problem, or contact your local Technical Support for Tellabs 8000 system.

4.5

Checking Communication Links and Paths to All Network Nodes


Time

Any time

Tool(s) to use

Fault Management System

Description

Needed to check that communication links or paths to all Network Nodes


are working.

Procedure
Step 1

Open the Network Fault Report for Active or Unacknowledged Faults.


Select the Fault - Network Fault Report Active or Unacknowledged Faults menu option in the network window of Fault Management.

Step 2

Check for any alarms indicating that communication links or paths with the Network Elements
are broken.
Result
Check that there are no used Network Nodes with broken communication links or paths.
If any of them has an alarm, check first that the cables from DXX Server(s) or Communication
server(s) to the Tellabs 8000 system network are connected correctly. If this does not solve the
problem, contact your local Technical Support for Tellabs 8000 system.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
20

76.8021-70211A
2011 Tellabs.

5 Weekly Maintenance

5 Weekly Maintenance
5.1

Verifying Database Configuration Parameters


Time

Night time or early in the morning

Tool(s) to use

AMPT, FMS and ISQL

Description

Check the usage of Database configuration parameters like Procedure Cache,


number of open Partitions etc. The values of these parameters need to be
readjusted based on their usage.

In addition to periodic maintenance, note that database server parameters need to be revised
in cases of major changes in network size, number of users, large operations done with
Tellabs 8000 intelligent network manager, upgrades or change of database server hardware.
Monitoring the database performance is quite essential in order to find out about needs to
change database server parameters. Refer to Sybase database server documentation for
further information and contact your Sybase administrator to change database parameters.

Procedure
Step 1

Start the Automatic Maintenance Procedures Tool.


Select Tools - Automatic Maintenance Procedures Tool in Tellabs 8000 Intelligent Network Manager Toolbox.

Step 2

Select the procedure DB Server Housekeeper from the list available.

Step 3

Click Execute.

Step 4

Login with the dxxdata login name and password and click OK.
Wait until the Last exec status is Successful.

Step 5

Inspect the logs and perform corrective actions, if any.


Inspect the logs in output section of the window for any errors.
In the Fault Management System, open the Network Fault Report for Active or Unacknowledged Faults and check for any Prompt Maintenance Alarms (PMA) with problem
type Database Server configuration required.
Take corrective actions to increase parameter values by running ISQL commands as suggested in the Alarming object field description of the above type of alarms.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
21

5 Weekly Maintenance

Step 6

While increasing memory based parameters like Procedure Cache Size, Additional Network
Memory etc, verify that there is enough free memory available to be allocated, as shown below.
Open a Command Prompt window and start an ISQL session as an sa user and check the
max memory parameter.
isql -SDBSERVER -Usa Ppassword
1> sp_monitorconfig max memory
2> go
The Num_free value should be more than the amount of intended increase suggested in
the alarm.
If there is enough free memory available, increase the parameters
1> sp_configure <parameter_suggested_in_the_alarm>,
<value_suggested_in_the_alarm>
2> go

Step 7

If there is not enough free memory available to increase the memory based parameters, decrease
the number of sort buffers.
Check the value of DB parameter number of sort buffers.
1> sp_configure number of sort buffers
2> go
Decrease the number of sort buffers to half of its current value displayed in the field Run
Value.
1> sp_configure number of sort buffers, <Run Value / 2>
2> go

Step 8

5.2

Click Exit to close the ISQL session and Automatic Maintenance Procedures Tool and the FMS.

Verifying Status of Scheduled Procedures in Automatic


Maintenance Procedures Tool
Time

Any time

Tool(s) to use

Automatic Maintenance Procedures Tool

Description

Verify the status of the different scheduled procedures in Automatic


Maintenance Procedures Tool to check if they are running as planned and
their intervals are not conflicting with each other.

If your network grows extensively, e.g. new network elements are added or moved all the time,
the execution times of different procedures scheduled by Automatic Maintenance Procedures
Tool increase and may cause conflicts due to overlapping with each-other if the intervals set
between them are not long enough.

The execution status of all the scheduled procedures should be checked if there were any problems
during execution. Also, the execution times should be verified so that they do not overlap each
other. If there are any overlaps, intervals should be reorganized using steps in 2.1 Activating
Automatic Maintenance Procedures.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
22

76.8021-70211A
2011 Tellabs.

5 Weekly Maintenance

Procedure
Step 1

Start the Automatic Maintenance Procedures Tool.


Select Tools - Automatic Maintenance Procedures Tool in Tellabs 8000 Intelligent Network Manager Toolbox.

Step 2

Select a scheduled procedure from the list available.

Step 3

Verify the Last exec time of the procedure if it matches with the scheduling specified.

Step 4

Verify the Last exec status of the procedure if it is successful.

Step 5

If the Last exec status is Failed, execute the procedure and inspect the logs for any errors.
Select the procedure and click the Execute button.
Login with the dxxdata as login name and password and click OK.
After completion of running the procedure, inspect the log in output section of the window
for any errors.
Take corresponding corrective actions for the errors.

5.3

Step 6

Check the Last exec duration to determine how much time the procedure has taken to execute.
Based on the execution time, it can be determined if there is any overlap between the different
scheduled procedures. If there are any overlaps, intervals should be reorganized using steps in
2.1 Activating Automatic Maintenance Procedures. The intervals can also be reduced if the
execution times are short and the intervals are wider than necessary.

Step 7

Similarly, repeat the steps 26 for every scheduled procedure in the list.

Reorganizing Indexes in Tellabs 8000 Intelligent Network


Manager Database
Time

Night time or early in the morning

Tool(s) to use

Automatic Maintenance Procedures Tool

Description

Reorganizing indexes is necessary for keeping the performance of the


database access as good as possible. If you do not reorganize indexes on a
regular basis, the performance suffers in the long run.

If your network grows extensively, e.g. new network elements are added or moved all the
time, we advise you to reorganize the indexes more often, e.g. twice a week, to increase the
performance of the Database Server.

The execution of the tasks may take some time depending on the size of the network. You can
see the status of the operation in the Last exec status column. When the operation is completed
the Last exec status is Successful.
Procedure
Step 1

Start the Automatic Maintenance Procedures Tool.


Select Tools - Automatic Maintenance Procedures Tool in Tellabs 8000 Intelligent Network Manager Toolbox.

Step 2
76.8021-70211A
2011 Tellabs.

Select the Rebuild Indexes task.


Tellabs 8000 Intelligent Network Manager SR2.1
Maintenance Procedures
23

5 Weekly Maintenance

Step 3

Click Execute.

Step 4

Login with the dxxdata login name and password and click OK.
Wait until the Last exec status is Successful.

Step 5

If the Last exec status is Failed, inspect the logs for errors.
Inspect the logs in output section of the window for any errors.
Take corresponding corrective actions for the errors.

5.4

Step 6

Similarly, repeat the above steps 2 to 5 with the task Reclaim Unused Space.

Step 7

Click Exit to close the Automatic Maintenance Procedures Tool or continue running other
maintenance procedures.

Updating Database Statistics after Reorganizing Indexes


Time

Always after indexes have been reorganized

Tool(s) to use

Automatic Maintenance Procedures Tool

Description

Database statistics update is an operation that is needed to improve the


performance of database access.

The execution of the tasks may take some time depending on the size of the network. You can
see the status of the operation in the Last exec status column. When the operation is completed
the Last exec status is Successful.
Procedure
Step 1

Start the Automatic Maintenance Procedures Tool.


Select Tools - Automatic Maintenance Procedures Tool in Tellabs 8000 Intelligent Network Manager Toolbox.

Step 2

Select the Update Table Statistics task.

Step 3

Click Execute.

Step 4

Login with the dxxdata login name and password and click OK.
Wait until the Last exec status is Successful.

Step 5

If the Last exec status is Failed, inspect the logs for errors.
Inspect the logs in output section of the window for any errors.
Take corresponding corrective actions for the errors.

Step 6

Similarly, repeat the above steps 2 to 5 with the task Update Index Statistics.

Step 7

Click Exit to close the Automatic Maintenance Procedures Tool or continue running other
maintenance procedures.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
24

76.8021-70211A
2011 Tellabs.

5 Weekly Maintenance

5.5

Checking Tellabs 8000 Intelligent Network Manager Database


Consistency
Time

Recommended to be run when the use of the network management system


is light, e.g. early in the morning.

Tool(s) to use

Automatic Maintenance Procedures Tool

Description

This operation is needed for checking the consistency of the database. If the
database has inconsistencies you may have serious problems in Tellabs 8000
intelligent network manager system.

Procedure
Step 1

Start the Automatic Maintenance Procedures Tool.


Select Tools - Automatic Maintenance Procedures Tool in Tellabs 8000 Intelligent Network Manager Toolbox.

Step 2

Select the Check/Fix Table Corruptions task.

Step 3

Click Execute.

Step 4

Login with the dxxdata login name and password and click OK.
Wait until the Last exec status is Successful.

Step 5

If the Last exec status is Failed, inspect the logs for errors.
Inspect the logs in output section of the window for any errors.
Take corresponding corrective actions for the errors.

Step 6

Similarly, repeat the above steps 2 to 5 with the task Check/Fix Index Corruptions.

Step 7

Click Exit to close the Automatic Maintenance Procedures Tool or continue running other
maintenance procedures.

If Database Server is not running in single user mode, you may receive the following error
message:
Table Corrupt: Page is allocated but not linked Msg 2540, Level 16, State 1:
If you want to be sure about the consistency of the database, run the script checkdb in single
user mode.

5.6

Monitoring Size of Performance Data Size

76.8021-70211A
2011 Tellabs.

Time

Any time

Tool(s) to use

8000 Performance Management

Description

The PMS data size should be monitored in order not to let database fill up
too much.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
25

5 Weekly Maintenance

Procedure
Step 1

Start 8000 Performance Management.


From Toolbox select Performance Management 8000 Performance Management.

Step 2

Open Performance Management Configuration.


Select Performance Performance Management Configuration.

Step 3

Check the size of the PMS data:


See tabs 8000 Configuration and 8100 Configuration and check the Database size MB fields.
Result
It is difficult to give a rule about the acceptable size for the PMS data in database. The acceptable
size depends on the overall database size, database server type and memory as well as the impact
on the duration of running of database maintenance scripts. However, tables which are larger than
several megabytes might cause problems in the SQL server and decrease its performance.
You can affect the size of the PMS data by setting the corresponding Interval history and aging
days variables in the Performance Management Configuration window.

5.7

Testing Restoration of Backup


Time

Any time

Tool(s) to use

ISQL

Description

Test the restoration of Tellabs 8000 intelligent network manager database


backup once a week. Use a second database (e.g. sim_secdb8000) for this
purpose.

This procedure is possible only if you have enough space on your Database Server to create a
second database with the same characteristics, e.g. the size of devices, as in your production
database, or if you have another server on which the database dump can be loaded. However,
it is highly recommended that a separate isolated Database Server is used to avoid any
accidental interruption on the production Database Server.

Windows Server 2008 does not natively support tape backups. See Tellabs 8000 Intelligent
Network Manager Software Installation Manual, chapter Database Backup Procedures for
Windows 2003/2008 Database Server for more information. In Windows Server 2008, copy the
backup from the used external media to test restoration of a backup.

Procedure
Step 1

Create a database sim_secdb8000 that has the same characteristics as your network database, e.g.
the size of devices. See Tellabs 8000 Intelligent Network Manager Software Installation Manual
for instructions on creating a database.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
26

76.8021-70211A
2011 Tellabs.

5 Weekly Maintenance

Step 2

From Database Server, restore the database dump from the tape to the disk following the instructions
below.
If you are using a Windows 2003 based Database Server, proceed as follows:
Open the backup program by selecting Start - Programs - Accessories - System Tools Backup.
Open the tape.
Select the dump and click restore.
If using a Solaris based Database Server, type in the shell:
$cd /backup
$mkdir restore
$cd restore
$tar xvf /dev/rmt/0

Step 3

Load the dump into sim_secdb8000.


Open the Command Prompt window.
Start an ISQL session.
isql -SDBSERVER -Usa Ppassword
Load the dump into sim_secdb8000.
If you are using a Windows 2003/2008 based Database Server, enter the following
command:
1> load database sim_secdb8000 from g:\backup\dumpdb.bkp
2> go
If you are using a Solaris based Database Server, enter the following command:
1> load database sim_secdb8000 from /backup/restore/dumpdb.bkp
2> go
Result
There should be no errors occurring during the load operation.

Step 4

In case transaction dumps are being tested too, load the transaction dumps as shown below, one
after the other in the same order that they were created. If transaction dumps are not being tested,
skip this step and proceed from step 5 onwards.
Load the transaction dump into sim_secdb8000.
If you are using a Windows 2003/2008 based Database Server, enter the following
command:
1> load tran sim_secdb8000 from g:\backup\dumptran.bkp
2> go
If you are using a Solaris based Database Server, enter the following command:
1> load tran sim_secdb8000 from /backup/restore/dumptran.bkp
2> go
Repeat the above for each transaction dump, in the same order that they were created.
Result
There should be no errors occurring during the load operation.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
27

5 Weekly Maintenance

Step 5

Bring sim_secdb8000 online and check the NMS table.


1>online database sim_secdb8000
2>go
1>use sim_secdb8000
2>go
1> select * from nms
2> go
Result
The command must have been executed and it should display information of the installed Tellabs
8000 intelligent network manager.

Step 6

Check if the database backup contains up-to-date information using below commands:
1>select max(time) from sim_secdb8000..osrchan
2>go
Result
The result should be close to the time when the database backup was taken. If not it is likely that the
backup is older in time than it is thought to be.

It is very important that you do not perform any hardware actions using this database (e.g.
update parameters in a network element) with a workstation unless you are sure that there
is no connection to the production network from the server running the sim_secdb8000
database. The production database could be damaged and cut off traffic. You can also
create a simulation database instead of a second database in order to avoid this problem.
Completing steps 5 and 6 should be enough to check that the restoration of the database is
functioning properly.

If you fail to restore your Tellabs 8000 intelligent network manager database backup, contact
your local Technical Support for Tellabs 8000 system.

5.8

Checking Disk Space of Tellabs 8000 Intelligent Network Manager


Computers (and Moving Log Files)
Time

Any time

Tool(s) to use

Commands and scripts

Description

Needed to check that there is enough disk space available for log writing.

Procedure
Step 1

On each computer where you are getting log files, e.g. nmstrace.log, you should check that the hard
disk is not filling up. No more than 90 per cent of the disk space should be used. To make this step
easier, you can use the parameter y to cut log files at midnight with the following command.
nmstrace 0123456 * trace.log y

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
28

76.8021-70211A
2011 Tellabs.

5 Weekly Maintenance

Step 2

Check if there are any unnecessary log files and delete them.
If you find any unnecessary log files, select the directory of the file and delete the files.
There may be logs in the %NMSDIR%\LOG folder. In addition, Phoenix.log is located in
\%NMSDIR%\Java\Extern\Phoenix\Logs.
Delete other unnecessary files.

Step 3

5.9

As long as there are no errors or unsolved entries in the nms trace log files, there is no need to keep
them to occupy disk space, so delete or move the log files to another computer via network.

Checking Disk Space on ThinClient Server


Time

Any time

Tool(s) to use

Commands

Description

Needed to check that there is enough disk space available for log writing on
ThinClient Server. ThinClient Server maintains session data ex. logs for all
the users accessing NMS in their corresponding directories. The nms logs
created by these individual sessions should be regularly leaned up in order
to ensure there is enough disk space on the ThinClient Server.

Procedure
Step 1

The disk space usage of all file systems of the ThinClient server should be checked if they are near
full. No more than 90 per cent of the disk space should be used. The following main nms directory
should be checked if there are any excessive log files and they should be deleted.
%NMSDIR%\log

Step 2

Check if there are any excessive log files in all the user sessions created in ThinClient server.
The individual user session nms logs are in the directories C:\shared\users\citrix_users\<username>\Tellabs 8000\log.
Make sure the workstation of the corresponding user is not running and delete all excessive
log files in each directory.

5.10 Checking Disk Space on Route Master


Time

Any time

Tool(s) to use

Commands

Description

Needed to check that there is enough disk space available for log writing on
Route Master server.

Procedure
Step 1

76.8021-70211A
2011 Tellabs.

The disk space usage of all file systems on the Route Master should be checked if they are near full.
The command df k can be used to check disk space usage. No more than 90 per cent of the disk
space should be used. Necessary action should be taken to clear space on those file systems which
are near full by clearing large and old logfiles.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
29

5 Weekly Maintenance

Step 2

Empty excessive logs in the directory /var/log/routemaster, if there are any.


Open a terminal connection to the Route Master server.
Go to log file location.
$ cd /var/log/routemaster
Copy the log file to archive
$ cp ocnm_mon.log ocnm_mon.log.old
Purge the existing log file.
$ cp /dev/null ocnm_mon.log

Step 3

Similarly, repeat the above step 2 for below logs


/var/log/routemaster/nsm.log
/var/log/routemaster/ospfd.log

Step 4

If there is no need of old logs for any investigation, they should be deleted or moved to a different
storage server.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
30

76.8021-70211A
2011 Tellabs.

6 Monthly Maintenance

6 Monthly Maintenance
6.1

Deleting Audit Log


Tool(s) to use

Tellabs 8000 Intelligent Network Manager Toolbox

Description

Delete the Audit Log entries that are older than one month. In the long run,
the auditlog table becomes so big that it is necessary to delete part of the
old entries. Otherwise, the performance of Tellabs 8000 intelligent network
manager system suffers. If you want to save all entries, you must always save
the latest database backup before deleting the Audit Log.

This procedure creates a large amount of transaction log data. Check first that you have
enough free space in your log device before executing this operation.

Procedure
Step 1

Select the Audit Log entry truncation date.


Select the Security - Audit Logs - Manager Audit Log menu option in Tellabs 8000 Intelligent Network Manager Toolbox.
The Audit Log window opens.
Select Options - Refresh.
Select the Options Delete Events menu option.
The Delete dialog opens.
Define the deletion time to one month earlier than the current date.

Step 2

Delete the faults.


Click the Ok button.
The specified Audit Log events are deleted from the database.

6.2

Monitoring Size of Fault Tables

Step 1

76.8021-70211A
2011 Tellabs.

Tool(s) to use

ISQL

Description

It is very important to monitor the size of the tables that can grow a lot in
the database. In some cases, if a table is too big, the SQL server can have
problems creating indexes, or some entries can become corrupted.
The performance of the database decreases a lot if some tables are too big.

Start an ISQL session.


isql -SDBSERVER -Usa -Ppassword

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
31

6 Monthly Maintenance

Step 2

Check the size of the fault tables with the following commands.
1>use db8000
2>go
1>sp_spaceused faulthist
2>go
1>sp_spaceused fault
2>go
Result
It is rather difficult to give a rule about the acceptable size of a table. The acceptable table size
depends on various parameters such as the computer type and the amount of memory. However,
the tables which are bigger than several megabytes might cause problems in the SQL server and
decrease its performance.
If the fault history table is too big, you should keep the entries corresponding to a shorter period
of time as mentioned in chapter 3 Fault History Maintenance .
If the fault table is too big, start clearing up the faults in your network more efficiently.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
32

76.8021-70211A
2011 Tellabs.

6 Monthly Maintenance

6.3

Accounting Management Events


This creates a large amount of transaction log data. Check first that you have enough free
space in your log device using DB command sp_helpdevice before executing this operation.

Time

Any time

Tool(s) to use

Accounting Management

Description

Delete the events that are older than one month. If you want to save all
events, you must always save the latest database backup before deleting the
events. Contact your local Technical Support for Tellabs 8000 system for
more detailed instructions on backing up and restoring a database.

Procedure

6.4

Step 1

Start the Accounting Management tool.

Step 2

Click the Delete button.

Step 3

Type a month-old date. All events that occurred before that date is deleted.

Step 4

Click OK.

Step 5

Click Yes.

Private Subnetwork Trunk Accounting Events for Tellabs 8100


Nodes
This creates a large amount of transaction log data. Check first that you have enough free
space in your log device using DB command sp_helpdevice before executing this operation.

Time

Any time

Tool(s) to use

Private Subnetwork Trunk Accounting

Description

Delete the events that are older than one month. If you want to save all events
you must always save the latest database backup before deleting the fault
history. Contact your local Technical Support for Tellabs 8000 system for
more detailed instructions on backing up and restoring a database. Note that
this operation is needed only if you have defined private subnetworks for
8100 nodes in your Tellabs 8000 system network.

Procedure
Step 1

Start the Private Subnetwork Trunk Accounting tool by selecting Tools Private Subnetwork
Trunk Accounting.

Step 2

Select a private subnetwork and click OK.

Step 3

Open the Delete Events dialog by selecting Options - Delete Events menu option.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
33

6 Monthly Maintenance

Step 4

Define the deletion time in YYYYMMDD format. All events that occurred before that date is
deleted.

Step 5

Click OK.

6.5

Recovery Management System

6.5.1

Spontaneous Fault Reporting (GMH Units)


We recommend that you set spontaneous fault reporting systematically when setting parameters to
trunk interfaces with Node Manager.

Step 1

Open the unit window for a GMH unit using Node Manager.

Step 2

Open the Fault Block Configuration dialog by selecting Unit - Fault block Configuration.

Step 3

Set Event Reporting on.

Step 4

Set Fault Filtering Time according to your needs. The default value is 0.

6.5.2

Spontaneous Fault Reporting (GMU, GMU-A and GMU-M1 Units)


Time

Any time, after adding a new VC-12, VC-2, VC-12-mc or VC-2-mc trunk

Tool(s) to use

Node Manager

Description

It is necessary to set the spontaneous fault sending on in trunk interfaces.


This is needed for faster Tellabs 8000 intelligent network manager recovery
operations. Otherwise the recovery is much slower.

Procedure
Step 1

Open the unit window for the GMU that contains the VC-12, VC-2, VC-12-mc or VC-2-mc TTPs of
the new trunks.

Step 2

Right-click the mouse to open the Fault Block Configuration dialog.

Step 3

Set Event Reporting on.

Step 4

Set Filtering Time according to your needs. The default value is 0.

Step 5

Set Fault Declare Time according to your needs. The default value is 2 s.

Step 6

Set Fault Clear Time according to your needs. The default value is 10 s.

6.5.3

Spontaneous Fault Reporting (Other Units)


Fault reporting can be set on with the same principle as for GMH in XCG (trunk interfaces),
GMM, VMM, and QMH.

1GMU

and GMU-M have been discontinued.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
34

76.8021-70211A
2011 Tellabs.

6 Monthly Maintenance

6.6

Maintenance of Element Software in Database


Time

Whenever new ESW is available

Tool(s) to use

Tellabs 8000 Intelligent Network Manager Toolbox

Description

It should be taken care on a monthly basis or even more often that the relevant
versions of element software are loaded into the database and obsolete
versions are deleted.

Procedure

6.7

Step 1

In Tellabs 8000 Intelligent Network Manager Toolbox select Tools - 8600 Element Software
Management...
The 8600 Element Software Management window opens.

Step 2

Click Add ESW.


The Add ESW File to DB window opens.

Step 3

Click Select File Navigate to the desired path containing the relevant ESW files and select a file
you want to add into the database.

Step 4

Fill in the Version, File Type, Description, Info Model Version and Unit Type fields.

Step 5

Click Copy to DB.

Step 6

When the copying is finished, click Exit.

Step 7

Delete unnecessary files in 8600 Element Software Management window by selecting the file
from the list and clicking Delete.

Node Configuration Backup


Time

Whenever new ESW is available

Tool(s) to use

Tellabs 8000 Intelligent Network Manager Toolbox

Description

In Tellabs 8000 intelligent manager it is possible to store a node configuration


into the database using Node Configuration snapshot or backup.

Procedure
Step 1

In Tellabs 8000 Intelligent Network Manager Toolbox select Network Editor.


The Network Editor window opens.

Step 2

Select Edit Communication Servers


The NWED:Communication Server Parameters dialog opens.

Step 3

Select the relevant communication server from the Name drop-down list, and select the relevant
adapter to parametrize from Communication Adapters window pane.

Step 4

Click Parameters
The NWED:Tellabs 8600 Adapter dialog opens.

Step 5

Click Polling Policies


The NWED:Polling Policy Parameters dialog opens.

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
35

6 Monthly Maintenance

Step 6

Tick the Configuration Backup selection box and click Parameters next to it.
The NWED:Configuration Backup dialog opens.

Step 7

Select either Periodic backup or Daily backup radio button, and fill in the relevant parameters and
the number of backup files per node.

Step 8

Click OK to confirm the parameters and close the dialog.

Step 9

Click Update on the NWED:Polling Policy Parameters dialog to activate the policy.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
36

76.8021-70211A
2011 Tellabs.

7 Additional Tools for Sun Solaris Server

7 Additional Tools for Sun Solaris Server


7.1

Sun Explorer
Sun Explorer is a data collection tool, which is used for collecting information on the
configuration, version and logs to help problem solving or system audits. The data is stored in the
/opt/SUNWexplo/output directory and can be read by the system administrator at any time. The
data can also be transferred to the Sun Microsystems database from where the Tellabs Technical
Support personnel can retrieve them when necessary.
The Explorer software can be downloaded from http://sunsolve.sun.com (click on Explorer Data
Collector under the Diagnostic Tools title). You can download the software by clicking on View
License Agreement and Download. In addition, there is a link to Frequently Asked Questions
where extra information on the Explorer software can be found.

7.2

Installation of Explorer
Explorer is provided by Sun Microsystems as SUNWexplo.tar.Z, which is a compressed tar kit ready
to be installed. The installation commands are as follows.
#cp SUNWexplo.tar.Z /var/tmp
#cd /var/tmp
#zcat SUNWexplo.tar.Z | tar xvf
#pkgadd d . SUNWexplo
The last command starts the installation process. During the installation you have to answer several
questions which determine the operation of Explorer. The questions with example answers can be
found below (Explorer version 4.0).
Agree to these terms? [y, n] Y
Where should this package be installed? [/opt/ SUNWexplo]:
/opt/ SUNWexplo
Contact Information NOTE! Fill all the possible fields.
Geographic Region: EMEA
Please Enter the two characters Country code or enter ? for a
list of countries. []:? Fi
Would you like explorer output to be sent to alternate email
addresses at the completion of explorer? If not, enter a
single only for your reply.
Return address for explorer email output: firstname.lastname@company.com
Are these values ok? [y,n]
Do you wish to run explorer once a week? [y,n] y
At which hour would you like this to run (0 to 23): Select the
one best suited for your needs
At which minute would you like this to run (0 to 59): Select
the one best suited for your needs

76.8021-70211A
2011 Tellabs.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
37

7 Additional Tools for Sun Solaris Server

On which day would you like this to run (0 to 6 with 0=Sunday):


Select the one best suited for your needs
If this is a new install of explorer, please run explorer and
mail the results to Sun. The command to do this would be:
/opt/SUNWexplo/bin/explorer q e
Would you like to do this now? [y,n] y
Do you want to continue with the installation of <SUNWexplo>
[y,n,?] y
The information is stored in the file called /etc/opt/SUNWexplo/default/explorer and you can edit it
at any time.

7.3

Configuration and Service Tracker (CST) Latest Version


Configuration and Service Tracker (CST) is a software that helps the system administrator to register
and archive the changes and activities done in the system. CST also calculates availability and
keeps log of the system architecture.
CST monitors e.g. the following events:
hardware, software and patch installations, removals and upgrades
configuration changes in the operation systems
shut down, reboot, panic and run-level type of status changes
cluster level changes
CST calculates the availability of the system using the above-mentioned events. CST can be
configured to send an e-mail when a change is noticed.
CST software can be downloaded free of charge from Sun Download Center at:
http://www.sun.com/software/download (click first on System Administration, and then on
System Management).
An alternative site is http://srsnetconnect3.sun.com, where the installation manuals and other
documents can be found.

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
38

76.8021-70211A
2011 Tellabs.

Suggested Weekly Schedule for Automatic Maintenance Procedures

Suggested Weekly Schedule for Automatic


Maintenance Procedures
The following is suggested week-wise scheduling of automatic maintenance procedures. These can
only be used as a guideline as the running times of various procedures vary based on the size of
the network, hence they need to be adjusted as per requirement.
On Windows Database Server

76.8021-70211A
2011 Tellabs.

Mon

Tue

Wed

Thu

Fri

Sat

Sun

00:00

tape
backup

tape
backup

tape
backup

tape
backup

tape
backup

tape
backup

tape
backup

03:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction dump

transaction
dump

03:30

rebuild
indexes

check/fix
table
corruptions

04:30

reclaim
unused
space

05:30

update
table
statistics

06:20

update
index
statistics

check/fix
index
corruptions

06:30

server
monitoring
housekeeper

server
monitoring housekeeper

server
monitoring housekeeper

server
monitoring
housekeeper

server
monitoring
housekeeper

server
monitoring
housekeeper

server
monitoring
housekeeper

07:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction dump

transaction
dump

07:30

db
server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

11:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction dump

transaction
dump

15:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction dump

transaction
dump

19:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction dump

transaction
dump

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
39

Suggested Weekly Schedule for Automatic Maintenance Procedures

On Windows Database Server


master
dump

22:00
23:00

full db
dump

full db
dump

full db
dump

full db
dump

full db
dump

full db
dump

full db
dump

Tue

Wed

Thu

Fri

Sat

Sun

00:00

rebuild
indexes

check/fix
table
corruptions

01:00

reclaim
unused
space

02:00

update
table
statistics

03:00

update
index
statistics

On Solaris Database Server


Mon

check/fix
index
corruptions

04:00

full db
dump

full db
dump

full db
dump

full db
dump

full db
dump

full db
dump

full db
dump

05:00

tape
backup

tape
backup

tape backup

tape
backup

tape
backup

tape
backup

tape
backup

05:55

clean up

clean up

clean up

clean up

clean up

clean up

clean up
master
dump

06:00
06:30

server
monitoring
housekeeper

server
monitoring housekeeper

server
monitoring housekeeper

server
monitoring
housekeeper

server
monitoring
housekeeper

server
monitoring
housekeeper

server
monitoring
housekeeper

07:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction
dump

transaction dump

07:30

db
server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

db server
housekeeper

11:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction
dump

transaction dump

15:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction
dump

transaction dump

19:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction
dump

transaction dump

23:00

transaction
dump

transaction dump

transaction
dump

transaction
dump

transaction dump

transaction
dump

transaction dump

Tellabs 8000 Intelligent Network Manager SR2.1


Maintenance Procedures
40

76.8021-70211A
2011 Tellabs.

You might also like