You are on page 1of 47

Internal

M2000V2R6&R8
Fault Management
ISSUE 1.4
M2000 TSD www.huawei.com

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Report and Storage of Alarms

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential Page 2


Alarm Classification

Fault Alarm
With occurrence and clearance time
Event Alarm
With occurrence time and without clearance time

Alarm Definition Relevant Knowledge Impact on the System


Category

Fault alarm The alarm A fault is a physical or logical element Faults may affect the
notification that causes the failure of the system normal operation of the
generated by the operation; for example, high CPU system.
system in the case usage, faulty hard disk, and
of faults disconnected network cable.

Event The alarm An event is known as any situation of Events occur any time
alarm notification the managed object; for example, and anywhere and do
generated by the periodically exporting operation logs not have any negative
system for certain succeeds. impact on the system
events that occur operation.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm States
According to the acknowledgement, event alarms have two states:
Unacknowledged and acknowledged
According to the acknowledgement and clearance, fault alarms have the
following four states:
Unacknowledged and uncleared, acknowledged and uncleared,
unacknowledged and cleared, and acknowledged and cleared
Transition relation between fault alarm states:
The acknowledged and cleared fault alarms are called history alarms.
Fault alarms of other states are called current alarms.

Current alarm
History alarm

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Status Transition
Acknowledgement
Automatic acknowledgement
Real-time automatic acknowledgement: Client > Monitor > Settings > Auto
Acknowledgement Rule
Timing automatic acknowledgement: Client > Maintenance > Task
Management > Others > Alarm Acknowledgement
Manual acknowledgement
Unacknowledgement
Applicable to acknowledged & uncleared alarms.
Right-click an alarm in the alarm browse window, and then choose
Unacknowledge from the shortcut menu.
Clearing
Automatic clearing
Manual clearing
Right-click an alarm in the alarm browse window, and then choose Clear.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Automatic Alarm Clearing

After being automatically or manually recovered, the NE


reports an alarm clearing message to the M2000. The M2000
automatically clears the alarm after receiving the message.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Manual Alarm Clearing
Manual alarm clearing is used for only fault alarms.
The M2000 sends an alarm clearing command (RMV ALM) to NE.
After receiving the command, the NE clears the alarm and sends an
alarm cleared message to the M2000. Then, the alarm on the M2000
is changed to the cleared status.
The NEs that do not support the manual alarm clearing are listed as
follows:
CBSC, CMSC, CT800GLMS, GLMS, PDSNR01, RAC, TSC, iMSC,
tMSC, CBTS3601C, CBTS3606, CBTS3612, RAU3601C, RAU3606,
RAU3612, CHLR, GT800BSC, MSC

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status Transition
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Alarm Report and Storage

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Notification Modes

When an alarm is generated, the M2000 notifies users of the alarm


through:
M2000 client
Alarm board
Speaker
Topology
Alarm box
Remote notification
Short message service (SMS)
Email

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Audible and Visual Alarms

Speaker

Topo
Alarm Board

Alarms
Alarm Box
The following devices can receive only fault alarms:
Alarm box
Speaker
Topology
The following device can receive both fault alarms and
event alarms:
Alarm box

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Remote Alarm Notification

SMS

Email

Alarms

Email
Sent through the SMTP server.
SMS
Sent through the wireless modem, which can be connected to either the M2000 server or
the M2000 client through serial port.
Sent through the SMSC.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status Transition
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Alarm Report and Storage

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Browse, Query, and Statistics

Query
Query the current alarms, history
alarms, event alarms, and
masked alarms.
Browse the alarms in real time.

Browse
Browse the current alarms and
event alarms.
Browse the alarms by type or by
severity.

Statistics
Collect the statistics on the
current alarms, history alarms,
and event alarms.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Query

Select Auto Refresh to browse


alarms in real time.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Query

Click Template to use the


function of alarm query Click Filter to set
template management. alarm query
conditions.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Setting Time Zone DST

Users can set whether to display the alarm time according to the NE time
(NT) or the M2000 server time (ST).
The NEs report alarms according to the local NE time. The M2000 calculates the time
differences according to the time zones and DSTs of both NEs and the M2000, and then
changes the NT to ST or changes the ST to NT.
Configuration path: Right-click an alarm, and then choose a time display mode from the
shortcut menu.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Setting Time Zone DST
Set whether to display the GMT time zone offset and DST flags of the
alarm time on the M2000.
Configuration: Choose System > Preferences > Region Settings > Time.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Browse

Browse by type:
Only the latest 1,000 alarms are displayed for each type.
Browse by severity:
Only the latest 1,000 alarms are displayed for each severity.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Statistics

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status Transition
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Alarm Report and Storage

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Synchronization
The purpose is to ensure the consistency of alarms between the M2000
and NEs to the maximum extent.
The alarm synchronization is classified into automatic alarm
synchronization and manual alarm synchronization according to
triggering modes.
Automatic alarm synchronization
Creating NEs
Reconnecting NEs after disconnection
Resetting NEs
Manual alarm synchronization
Synchronizing NE alarms
Checking single or multiple alarms
The IP NE provides only the function of real-time alarm report instead of
providing the function of NE synchronization.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Synchronizing NE Alarms
Synchronizing NE Alarms
Synchronize all the uncleared alarms on NEs (or active alarms).

Choose Monitor > NE Alarm


Synchronize on the menu bar. In In the topology view, right-click
the displayed dialog box, select an NE whose alarms need to be
one or multiple NEs, and then synchronized, and then choose
click OK. Synchronize NE Alarm from
the shortcut menu.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Check
Alarm Check
Check only the uncleared alarms.
Check whether the uncleared alarms of the M2000 exist on the NEs. If they
exist, the M2000 makes no processing. If they do not exist, the M2000
automatically clears the alarms.

Right-click one or more alarms to be


checked, and then choose Synchronize
from the short-cut menu.
The synchronization result is displayed
in Alarm Detail Information at the
bottom of the screen.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status Transition
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Alarm Report and Storage

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


5. Alarm Setting
5.1 Alarm Severity Redefinition
5.2 Alarm Filtering
5.3 Alarm Correlation

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Severity Redefinition
Alarms are classified into four severities: critical, major, minor, and
warning.
Two methods
Redefining alarm severity of M2000
This method is applicable only to the M2000. The alarm severities of NEs
remain unchanged.
Support all NEs.
Redefining alarm severity of NEs
This method is applicable to both NEs and the M2000.
Support only partial NEs.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Severity Redefinition
1 Redefine alarm severity of M2000
Choose Monitor > Settings >
Severity Redefine.

2 3

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


5. Alarm Setting
5.1 Alarm Severity Redefinition
5.2 Alarm Filtering
5.3 Alarm Correlation

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Filtering
Alarm filtering is also called alarm suppression or alarm masking.
Two methods
NE alarm filtering
The M2000 delivers filtering rules to NEs. The NEs no longer report alarms
to the M2000, and the filtered alarms are not displayed on the NEs.
M2000 alarm filtering
The M2000 saves filtering rules. When receiving the alarms reported by
NEs, the M2000 checks whether the alarms meet the filtering condition. If
the alarms meet the filtering condition, the M2000 discards the alarms. If
the alarms do not meet the filtering condition, the M2000 saves and
displays the alarms.

Alarm Filter

M2000 NE

M2000 Alarm
Filtering Discard

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Setting M2000 Alarm Filtering Rules
1 Setting M2000 alarm filtering rules
Choose Monitor > Settings > Alarm
Filter Rule.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


5. Alarm Setting
5.1 Alarm Severity Redefinition
5.2 Alarm Filtering
5.3 Alarm Correlation

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Correlation
Purpose
To reduce redundancy alarms
Understanding the alarm interval T
Fault alarm

Occur Clear

0 t
T T T T
Event alarm

0 t
T T T

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Correlation
Principle
Two conditions
Alarm interval T
Unit: second
If the interval T of continuous alarms is less than or equal to T, the
first alarm is saved into the fault alarm table or event alarm table.
The following alarms are saved in the Mask Table or directly
discarded according to the correlation rule.
High-frequency alarm threshold N
Unit: times
If an alarm of a NE is received repeatedly, the interval of alarms is
less than or equal to T, and the number of alarms is greater than
or equal to N, the M2000 generates a high-frequency alarm which
equipment serial number is 0, and alarm severity is leveled up.
For the fault alarm, if the alarm interval of the follow-up alarms is
greater than T, the high-frequency alarm changes to cleared
status.
Only same alarm of an NE can be configured.
HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential
How to Query Masked Alarms

Query of masked fault/event


alarms
Alarms masked according to
alarm correlations
Note: The data of both the
alarms filtered by the M2000
and the alarms masked on NEs
is not saved.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status Transition
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Alarm Report and Storage

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


NE Alarm Setting
The following three functions are delivered to NEs:
NE Alarm Customization
Customizing NE alarms and binding custom alarms
NE Alarm Masking
If the alarms are masked on NEs, the alarms are not reported to the LMT and
M2000.
NE support is required.
NE Alarm Severity Redefinition
NE support is required.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


NE Alarm Setting
1 NE Alarm Setting
Choose Monitor > Settings > NE
Alarm Setting.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


NE Alarm Setting

User-Defined Alarm NE alarm masking (filtering) NE alarm severity


with Alarm Definition redefinition
and Alarm Binding tab
pages

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Custom NE Alarms
Definition
Users customize a new alarm, which is bound to a port of the cabinet,
subrack, or slot of the NE. Add an environment monitoring device on the
port. When the monitoring device detects an environment abnormality, the
NE generates a real-time alarm and reports the alarm to the M2000.
The custom environment alarm contains two parts: Alarm Definition and
Alarm Binding.
Supported NEs
NodeB, MSC Server, MGW, and BSC6000
Alarm Definition
In the system, add a definition about the custom alarm, which map the alarm
names to alarm ID..
The properties are saved in the M2000 database. You can query, add, and
modify the information about the custom alarms in the current database.
Alarm Binding
Add the binding relation between the custom alarms and alarm ports in the
system, which map alarm ports to alarm ID.
The binding relation is saved in NEs instead of the M2000 database. You can
query, add, modify, and cancel the custom alarm relations on the M2000.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


1. Alarm Classification and Status Transition
2. Alarm Notification Mechanism
3. Alarm Browse, Query, and Statistics
4. Alarm Synchronization
5. Alarm Setting
6. NE Alarm Setting and Custom Alarms
7. Alarm Report and Storage

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Report
The internal fault alarms and event alarms of the NE and M2000 are
reported to the fault management module of the M2000 server in
real time.
NEs of the GSM/CDMA/WCDMA networks report alarms to the
M2000 through the real-time alarm report port 6001.
The IP NE uses the SNMP protocol, the trap package of the SNMP
in particular, to send real-time alarms to the M2000.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Storage
The alarm database (fmdb) has four tables for saving alarms.
Table of current fault alarms: tbl_cur_alm
Table of history fault alarms: tbl_his_alm
Table of event alarms: tbl_event
Table of masked alarms: tbl_mask_alm

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Alarm Storage

NE

Alarm correlation

Fault alarm Event alarm Discard

fmdb

Acknowledged &
tbl_cur_alm Cleared
tbl_his_alm tbl_event tbl_mask_alm

Dump
Auto
file file

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Capacity Management of Alarm Database

Capacity Management of Alarm Database


You can set the days for saving history alarms, event alarms, and masked alarms in the
database. The alarms exceeding the save days are dumped to files, and then deleted from the
database. In this way, the database capacity can remain in a normal status.
1
Default save days: 90
Choose Maintenance > Task Management.
Overflow Processing Mechanism of Alarm Tables
If the alarm database is full before the save days exceed, the M2000 starts the overflow
processing mechanism of the alarm tables.
This mechanism is automatically processed by the system and need not be configured.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Capacity Management of Alarm Database

4 Double-click

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Overflow Processing Mechanism of Alarm Tables

After the alarm service is started, the system checks whether


the alarm table reaches the upper threshold every 83 seconds.
tbl_cur_alm: a maximum of 100,000 alarms
tbl_his_alm: a maximum of 2,000,000 alarms
tbl_event: a maximum of 2,000,000 alarms
tbl_mask_alm: a maximum of 200,000 alarms
If the quantity of alarm exceeds the upper threshold,
The tbl_cur_alm table forcibly dumps 10,000 alarms to the tbl_his_alm table.
The earliest unacknowledged&cleared alarms are dumped in the first priority. If
the number of these alarms are less than 10,000, dump the earliest
unacknowledged&uncleared alarms.
The tbl_his_alm table sorts the alarms by the alarm occurrence time, dumps
the earliest 20% alarms to file, and deletes these alarms from the data table.
The tbl_event table sorts the alarms by the alarm occurrence time, dumps the
earliest 20% alarms to file, and deletes these alarms from the data table.
The tbl_mask_alm table does not write alarms to files, but directly deletes the
earliest 100,000 alarms from the table.

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential


Thank You
www.huawei.com

HUAWEI TECHNOLOGIES CO., LTD. HUAWEI Confidential