Professional Documents
Culture Documents
Copyright statement Copyright IBM Corporation 2005, 2011. US Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Contents
Overview . . . . . . . . . . . . . . v
What's new and changed in Server Protection for Windows . . . . . . . . . . . . . . . v How to Use Server Protection for Windows Documentation . . . . . . . . . . . . . vii Technical support contacts . . . . . . . . . vii
iii
69
. 69
Overriding parent policies does not give child policies with the parent policy . . . . . . . . Under heavy traffic conditions traffic seems to be bypassing analysis . . . . . . . . . . . . Not seeing any file integrity monitoring alerts . . . Traffic seems to be bypassing analysis . . . . . Refresh agent feature in the SiteProtector System not functioning . . . . . . . . . . . . . . Agent showing as offline . . . . . . . . . . System tray icon disappeared after upgrade . . .
79 80 80 81 82 82 83
Notices . . . . . . . . . . . . . . 85
Trademarks . . . . . . . . . . . . . . 86
Index . . . . . . . . . . . . . . . 87
iv
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Overview
This guide is designed to help you use the IBM Security Server Protection for Windows, Version 2.x agent to protect servers and ensure that they are in compliance with your corporate policy.
Scope
This guide describes the features of Server Protection for Windows agents and explains how to configure policies from SiteProtector and deploy agents.
Audience
This guide is intended for security managers who manage Server Protection for Windows agents from SiteProtector.
You can control whether to use the Server Protection for Windows agent firewall or to use the Microsoft Windows Firewall. This option can be controlled from the Firewall policy. v Agent protection Agent protection is disabled by default. For more information about preventing unauthorized changes to agent files, see Enabling agent protection on page 73. v Event categorization for system integrity monitoring The system integrity monitoring component includes categorized events and the ability to filter events by operating system.
vi
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Related publications
The following documents are available for downloading from the IBM Security product Information Center at : http://publib.boulder.ibm.com/infocenter/sprotect/v2r8m0/index.jsp: v Server Protection for Windows System Requirements v Server Protection for Windows Administrator Guide v Server Protection for Windows User Guide v Server Protection for Windows Custom Parameters Help v IBM Security SiteProtector System Configuration Guide v IBM Security SiteProtector System Installation Guide v IBM Security SiteProtector System Policies and Responses Guide v IBM Security SiteProtector System User Guide for Security Analysts
License agreement
For licensing information on IBM Security Solutions products, download the IBM Licensing Agreement from: http://www.ibm.com/services/us/iss/html/contracts_landing.html
Overview
vii
viii
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Layered security About policy inheritance on page 3
Layered security
Server Protection for Windows provides multiple lines of defense by protecting the network vector, the application vector, and by monitoring files to detect unauthorized changes to the system. This layered protection approach provides greater overall protection for your host.
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Example
You configure the policies for a group of assets called Mail Servers. There are ten subgroups in the Mail Servers group. If all of the subgroups use the same policy settings, the policy settings can pass from the Mail Servers group to all ten subgroups. If certain subgroups in the Mail Servers group require customized policy settings, you can define a customized policy at the subgroup level for that policy without affecting the policy settings for the other subgroups. This approach is much quicker than configuring the policy settings for all ten subgroups.
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Pilot program
Consider running a pilot program before you deploy Server Protection for Windows agents in your production environment. A pilot program is a small scale deployment of agents (usually on non-production systems) that allows you to test different policy settings in relative safety. During the pilot, you can collect information and use it to modify your policies to minimize usability problems when you deploy agents in a production environment.
Topics
Section A: Deployment scenarios Section B: Before you deploy an agent on page 7 Section C: Upgrading a version 1.0 agent to version 2.x on page 11 Section D: Configuring policies on page 19 Section E: Creating and Deploying an Agent Build on page 22
Topics
Deployment scenario checklists on page 6
Updating an earlier 2.x agent to a later 2.x agent using the heartbeat mechanism
U h h h Complete the tasks outlined in... Updating the SiteProtector database on page 9 Updating the Agent Manager on page 9 Configuring policies on page 20
Updating an earlier 2.x agent to a later 2.x agent using the agent build mechanism
U h h h Complete the tasks outlined in... Updating the SiteProtector database on page 9 Updating the Agent Manager on page 9 Configuring policies on page 20
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
U h
Complete the tasks outlined in... Section E: Creating and Deploying an Agent Build on page 22
Updating from a 2.0 agent or a 2.1 agent using the heartbeat mechanism
U h h h h h Complete the tasks outlined in... Updating the SiteProtector database on page 9 Updating the Agent Manager on page 9 Migrating policy settings from version 2.0 or 2.1 on page 13 Process overview for upgrading use the heartbeat mechanism on page 14 Configuring policies on page 20
Updating from a 2.0 agent or a 2.1 agent using the agent build mechanism
U h h h Complete the tasks outlined in... Updating the SiteProtector database on page 9 Updating the Agent Manager on page 9 Migrating policy settings from version 2.0 or 2.1 on page 13 Process overview for upgrading using the agent build mechanism on page 17 h h Configuring policies on page 20 Section E: Creating and Deploying an Agent Build on page 22
Topics
Verifying agent licenses on page 8 Updating the SiteProtector database on page 9 Updating the Agent Manager on page 9 Setting up SiteProtector Groups on page 10
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
10
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
What to do next
Refer to your deployment scenario checklist in Deployment scenario checklists on page 6.
Prerequisites
See Section A: Deployment scenarios on page 5. See Section B: Before you deploy an agent on page 7.
Upgrading methods
You can upgrade Server Protection for Windows agents in the following ways: v Heartbeat mechanism v Agent build mechanism
Topics
Migrating policy settings from version 1.0 to version 2.x on page 12 Migrating policy settings from version 2.0 or 2.1 on page 13 Process overview for upgrading use the heartbeat mechanism on page 14 Task 1: Upgrading the agent version on page 14 Task 2: Upgrading Policies on page 15 Process overview for upgrading using the agent build mechanism on page 17
11
Procedure
1. In SiteProtector, select the Agent view. 2. In the navigation pane, right-click the applicable group, and then select Updates > Migrate Agent Version. The Migrate Agent Version window appears. 3. Click the Details icon, and then select Proventia Server for Windows from the Agent Type list. 4. 5. 6. 7. 8. Click the Upgrade Details icon. In the Migrate From Agent Version list, select 1.0. In the Update To Agent Version list, select a 2.x version. Click OK. Do one of the following: v To complete the upgrade using the heartbeat mechanism, go to Process overview for upgrading use the heartbeat mechanism on page 14 v To complete the upgrade using the Agent Build mechanism, go to Process overview for upgrading using the agent build mechanism on page 17
12
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Procedure
1. In the SiteProtector System, select the Agent view. 2. In the navigation pane, right-click the applicable group, and then select Updates > Migrate Agent Version. The Migrate Agent Version window appears. 3. Click the Details icon, and then select Server Protection for Windows from the Agent Type list. 4. Click the Upgrade Details icon. 5. In the Migrate From Agent Version list, select 2.0 or 2.1. 6. In the Update To Agent Version list, select 2.1 or 2.2. 7. Click OK. 8. Do one of the following: v To complete the upgrade using the heartbeat mechanism, go to Process overview for upgrading use the heartbeat mechanism on page 14 v To complete the upgrade using the Agent Build mechanism, go to Process overview for upgrading using the agent build mechanism on page 17
13
Prerequisites
See Section B: Before you deploy an agent on page 7. See Migrating policy settings from version 1.0 to version 2.x on page 12. See Migrating policy settings from version 2.0 or 2.1 on page 13.
Process overview
Upgrading Server Protection for Windows agents to version 2.0, version 2.1, or version 2.2 using the heartbeat mechanism involves the following tasks:
Task 1 2 Description Upgrade the agent version Upgrade policies Reference Task 1: Upgrading the agent version Task 2: Upgrading Policies on page 15
14
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
6. In the navigation pane, right-click Policy and then select Open. The Policy pane opens in the right pane. 7. Select Administrative Settings > Group Configuration. The Group Configuration pane appears. 8. For 1.0 agents migrating to version 2.0, in the Server protection version list, select the version that starts with 2.0. 9. For 2.0 agents migrating to version 2.1, in the Server protection version list, select the version that starts with 2.1. 10. For 2.1 agents migrating to version 2.2, in the Server protection version list, select the version that starts with 2.2. 11. In the Install path box, type the path to where the agents in this group are installed. 12. Save the policy. 13. Do one of the following to apply the updated policy: v Wait for the agents to send a heartbeat to the SiteProtector System v Select the applicable group, and then select Action > Refresh Agent to force the agents to send a heartbeat to the SiteProtector System immediately Task 2: Verifying the agent version: Procedure 1. In the SiteProtector System, select the applicable group, and then select the Agent view. 2. In the Version column, verify the agent version.
15
Editing policies
Procedure
1. 2. 3. 4. 5. In the SiteProtector System, select the Agent view. In the navigation pane, right-click the applicable group, and then select Manage Policy. In the Agent Type list, select Proventia Server for Windows. For version 2.0 agents, in the Agent Version list, select 2.0. For version 2.1 agents, in the Agent Version list, select 2.1.
6. For version 2.2 agents, in the Agent Version list, select 2.2. 7. In the navigation pane, select the group. The Active Deployments window for the group appears on the right pane. 8. In the right pane, right-click Install and Update Settings, and select Open. 9. In the Agent Version list, select the agent version. 10. In the Install path box, type the path to where the agents in this group are installed. 11. Click the Save icon. 12. From the Save Policy Version window, enter a Comment and then select the Deploy This New Version check box. 13. Click OK. The Deploy Policy window opens. 14. Click Targets. 15. Select the group you want to deploy this policy to. 16. Click OK and then close the tab. 17. To edit additional policies, do the following: v In the left pane, re-select the applicable group v In the right pane, right-click the policy to edit, and then select Open v Edit the policy Reference: See the Help for guidance. 18. Click the Save icon. 19. Repeat Step 12 through Step 18 for each policy you want to edit. 20. Do one of the following to apply the updated policy: v Wait for the agents to send a heartbeat to the SiteProtector System v Select the applicable group, and then select Action > Refresh Agent to force the agents to send a heartbeat to the SiteProtector System immediately
What to do next
v If you are performing the steps to upgrade from version 1.0 to version 2.0, you are finished. v If you are performing the steps to upgrade from version 2.0 to version 2.1 or version 2.2, you must complete the procedure for Migrating policy settings from version 2.0 or 2.1 on page 13.
16
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Prerequisites
See Section B: Before you deploy an agent on page 7. See Migrating policy settings from version 1.0 to version 2.x on page 12. See Migrating policy settings from version 2.0 or 2.1 on page 13.
Process overview
The following table provides an overview of the stages involved in upgrading agents using the agent build mechanism:
Task 1 2 3 Description Upgrade policies Generate an Agent Build Install agents using the Agent Build Reference Upgrading Policies Generating an Agent Build Installing an agent
17
Upgrading Policies
About this task
Policy settings determine the behavior of agents. As part of the upgrade you must, at a minimum, configure the Install and Update Settings policy. This policy defines the Agent Version setting which defines both the version of the agent and the version of the agent build. This policy also defines the installation directory for the agent on the host system. Tip: You can also configure the settings for any other policies. You can edit policies at the parent group level (settings are inherited by any child groups), or you can override the parent group policies and edit policies at the child-group level (settings are specific to the child group only).
What to do next
See Section E: Creating and Deploying an Agent Build on page 22
18
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Process overview for configuring policies Determining your policy inheritance needs Overriding parent policies on page 20 Configuring policies on page 20
Process overview
The following table outlines the tasks to perform to configure policies:
Task 1 2 3 Description Determine your policy inheritance needs Implement your policy structure Configure the policies Reference Determining your policy inheritance needs Overriding parent policies on page 20 Configuring policies on page 20
Considerations
To maximize efficiency, you should configure the parent policy, override the parent policy from the child, and then customize the child policy. To make configuration changes easier to implement, inherit as many policies as possible from the parent group. At a minimum, consider inheriting policies that define the software version and installation path information for the agent; you may want to override policies that define protection settings so you can have customized protection for child groups.
Chapter 2. Deploying Server Protection for Windows agents
19
Procedure
1. In the navigation pane, right-click a group and select Manage Policy. 2. In the Agent Type list, select Server Protection for Windows. 3. For agent version 2.0, in the Agent Version list, select 2.0. 4. For agent version 2.1, in the Agent Version list, select 2.1. 5. For agent version 2.2, in the Agent Version list, select 2.2. 6. In the navigation pane, select the child group you want custom policies for. Note: The Active Deployments reflects the current policies assigned to this child group. The Active Deployments window for the group appears in the right pane. 7. In the right pane, select the policy that you want to override. Tip: Use the Inheriting From column to determine which parent policies the child currently inherits. 8. Right-click the selected policies, and then click Deploy. 9. Click OK. The selected policy appears under the child group folder in the navigation pane.
Configuring policies
After you have implemented your policy inheritance structurethat is, you have decided which policies to inherit from the parent level and which policies to override at the child levelyou must configure the policies so the agent can protect your system.
Procedure
1. 2. 3. 4. 5. In the navigation pane, right-click a group and select Manage Policy. In the Agent Type list, select Server Protection for Windows. For version 2.0 agents, in the Agent Version list, select 2.0. For version 2.1 agents, in the Agent Version list, select 2.1. For version 2.2 agents, in the Agent Version list, select 2.2.
6. In the navigation pane, expand the group you are configuring policies for.
20
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
7. If you are configuring policies for a parent group, select Default Repository. Important: If the group you are configuring does not contain an Update Settings policy (either by inheriting or by having a custom version), you must create this policy before you can deploy your agent. See Update Settings in the table below for more information. The default policy types appear in the right pane. 8. Configure the appropriate policies based on the following information: Tip: As you configure each policy, press F1 if you need Help.
For this policy... Administration Application Compliance BOEP Bypass Filters File Integrity Monitoring Firewall Group Settings do this... see Chapter 10, Configuring administrative settings, on page 71 see Chapter 6, Enforcing antivirus compliance, on page 47 and Chapter 7, Enforcing application control, on page 51 see Chapter 5, Protecting against buffer overflow exploits, on page 43 see Chapter 9, Configuring bypass filters, on page 69 see Section C: Auditing files and directories on page 64 see Chapter 3, Firewall configuration, on page 25 1. Select the Agent Manager List tab. 2. If the Agent Manager for this group is not in the list, click Add. 3. Click Choose an Agent Manager, select the Agent Manager for this group, and then click OK. 4. Select a trust level as follows: v Trust all SiteProtector trusts the server and does not try to validate the certificate v First time trust SiteProtector trusts the first certificate it receives from the server and stores this certificate locally. The client uses this certificate to validate all future communication with this server. v Explicit trust The certificate for the server must reside in a local SiteProtector directory before the agent can initiate communication with the server. Typically, the certificate is transferred to the client outside the standard communication channels. 5. Click OK. Update Settings Tip: Consider inheriting this policy from the parent to ensure all agents are running the same version and build of the software and that all agents are installed in the same directory. 1. If you do not see this policy, expand Policy Types Not Created, right-click the policy, and then select New Policy. 2. In the Agent Version list, select the agent version and build. 3. In the Install path box, type the location where the agent will be installed on the host system. Registry Integrity Monitoring Security Events System Integrity Monitoring see Section B: Auditing registry entries on page 62 see Chapter 4, Protecting against intrusions, on page 33 see Section A: Auditing operating system logs on page 59
9. Click the Save icon. 10. From the Save Policy Version window, enter a Comment and then select the Deploy This New Version check box. 11. Click OK. The Deploy Policy window opens. 12. Click Targets.
Chapter 2. Deploying Server Protection for Windows agents
21
13. Select the applicable group to which you want to deploy the policy. 14. Click OK. 15. Do one of the following: v If you are deploying policies using an agent build, go to Section E: Creating and Deploying an Agent Build Note: If you are deploying an agent for the first time, you must use the agent build mechanism to deploy the agent. v If you are deploying policies using the heartbeat mechanism, do one of the following: Wait for the agents to send a heartbeat to SiteProtector Select the applicable group, and then select Action > Refresh Agent to force the agents to send a heartbeat to SiteProtector immediately
Prerequisites
Review the appropriate deployment scenario checklists before using the information in this section to create and deploy an agent build. Reference: Deployment scenario checklists on page 6
Topics
Generating an Agent Build on page 23 Installing an Agent on page 24
22
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
23
Installing an Agent
About this task
After you have created the agent build, you can access the build from the host system and install the agent on the server.
Procedure
1. On the computer where you want to install the Server Protection for Windows agent, start Internet Explorer. 2. Type the URL to the SiteProtector system using the following format: http://Agent_Manager_IP_address:8085 The Agent Manager Available Downloads Web page opens. 3. Select the agent build you want to install, and then click Download. 4. Save the agent build program file to the local system, and then double-click the file to begin the installation process.
What to do next
After you have installed Server Protection for Windows, you can use the following list to verify the installation completed successfully:
Look here... System tray Task manager to verify that... the Server Protection for Windows icon is glowing (if you installed the local user interface) v on Windows Server 2000 and 2003 32 bit systems, blackd, blackice, RapApp, and ph processes are running v on Windows Server 2000 and 2003 64 bit systems, blackd, blackice, and ph processes are running v on Windows Server 2008, blackd, blackice, and ph processes are running Server Protection for Windows Services the agent name, the IP address, and an Active status are showing in the Agent view v on Windows Server 2000 and 2003 32 bit systems blackice, RapApp, and IBM Proventia services have started v on Windows Server 2000 and 2003 64 bit systems blackice, and IBM Proventia services have started v on Windows Server 2008 blackice, and IBM Proventia services have started Events tab in the local UI an alert indicates agent-startup was successful (if you installed the local user interface)
If no services and processes start, look in the Windows directory for the AgentUpdateYour_System_Name.log for possible issues with the installation.
24
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Key concepts for firewall Protection levels on page 26 Firewall blocking on page 27 Firewall rules on page 28 Opening a port through the firewall on page 30 Advanced firewall configuration on page 31
Level of configuration
The level of configuration required for the firewall varies depending on the level of protection you want to enforce. Choosing a protection level quickly implements perimeter security. Defining firewall rules provides more granular control.
25
Protection levels
A protection level is a set of predefined firewall rules that provide a certain level of security. Protection levels allow or restrict access to the system based on port and protocol. For example, at the Paranoid level, the agent blocks incoming traffic on all TCP and UDP ports. Important: Protection levels do not block outbound traffic. To block outbound traffic, you must define outbound firewall rules.
Cautious
Trusting
Considerations
Keep the following in mind when you set up protection levels: v Ports 137 and 138 accept network traffic when Network Neighborhood lookup is enabled and they block traffic when the feature is disabled. v Ports 139 and 445 accept network traffic when file sharing is enabled and they block traffic when file sharing is disabled. v For computers running Windows Server 2000, Windows Server 2003, or Windows Server 2008 both ports 139 and 445 are used for the SMB file sharing protocol. Therefore, when default firewall settings from the management server are applied to the local agent, accept rules for both port numbers are automatically created. v If configuration sharing is enabled, the end user can enable or disable Network Neighborhood lookup and file sharing through the local agent console. The Protection Level may also be set through the local console.
Navigation
Locate the policy you want to edit and then click Firewall.
26
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Firewall blocking
A firewall can reduce, but not eliminate, exposure introduced by networked hosts. Firewall technology can help prevent attacks by limiting access to ports, IP addresses or ranges of IP addresses required for legitimate system activity. A firewall is one of several approaches you can use to block malicious traffic. See Chapter 4, Protecting against intrusions, on page 33 for a different approach.
Automatic blocking
When the agent detects an attack that poses an immediate threat to the system, it can automatically instruct the firewall to block the intruder. By default, automatically created blocks expire after 24 hours. Because the firewall controls transmission at the TCP/IP level, intruders cannot get around a block from the agent. Automatically created blocks have precedence over manually created firewall rules.
Manual blocking
Security managers can manually configure the firewall to block or accept traffic from any IP address or TCP/UDP port for a period of time or permanently.
Blocking
The agent can block traffic identified by IP address, port, or protocol: v For rules based on an IP address, the agent checks the source IP of inbound communications, and the destination IP for outbound communications. v For rules based on a port number or protocol, the agent checks the port the communication is destined for. Protocol numbers refer to the 8-bit field in the IP header that specifies protocols such as TCP, UDP, or ICMP.
27
Firewall rules
You can create firewall rules to block inbound, outbound, or bidirectional traffic based on IP type or address, port number, or ICMP type or code. In addition, the agent can accept or reject inspected packets based on the source IP address.
TCP port
Navigation
Locate the policy you want to edit and then click Firewall > Firewall Rules.
Firewall precedence
Use the Firewall Rules tab in the Server Protection for Windows agent's Firewall policy to add firewall rules based on IP type, IP address, UDP port, TCP port, and ICMP type and code. Note: Any Intrusion Prevention rules takes precedence over firewall rules.
Order of precedence
The agent applies firewall rules to network traffic based on traffic type and action requested. The agent processes rules in the order indicated in the following table:
Priority 1 Traffic Type IP Address Protocol IP Address and Port/ICMP/Protocol combination Action Accept
28
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Priority 2
Action Reject
3 4 5
Scenario 1
You want to block all outbound UDP packets except for outbound UDP DNS packets, and you have the following UDP Rules in place: v UDP Rule 1: All, 53, ACCEPT, OUTBOUND v UDP Rule 2: All, All, REJECT, OUTBOUND Important: If the IP address setting for a UDP Rule or a TCP Rule is configured as All (and not specified numerically), the firewall interprets the rules as a Port/ICMP Traffic Type. In this scenario, Rule 1 would be priority 4 and Rule 2 would be priority 3. As a result, all outbound UDP packets would be rejected because a priority 3 rule (Rule 2) is processed before a priority 4 rule (Rule 1). Suggested approach Use the following rules: v UDP Rule 1: 0.0.0.0-255.255.255.255, 53, ACCEPT, OUTBOUND v UDP Rule 2: All, All, REJECT, OUTBOUND While this approach does not change the logic of the rule (0.0.0.0-255.255.255.255 is the same as All), it does change the priority of UDP Rule 1 from priority 4 to priority 1 because the firewall interprets the numeric IP addresses as an IP Address Traffic Type not a Port/ICMP Traffic Type. This syntax change allows outbound DNS queries while blocking all other outbound UDP packets.
Scenario 2
You want to block unused service ports on a computer. Your first thought might be to do both of the following actions: v Use IP accept rules (Priority 1) to allow access to the computer from your corporate network v Use IP and port reject rules (Priority 2) to block unused ports This approach is not effective because of the order in which the agent applies firewall rules. The IP allow rules are processed first, thereby allowing all IP traffic through. All packets would match the IP allow rules; therefore, the IP and port reject rules are never used. Suggested approach Use both of the following rules:
Chapter 3. Firewall configuration
29
v IP Port accept rules (Priority 1) for all services that you want to allow v IP reject rules (Priority 2) on all others This approach is effective because the IP Port accept rules cover only services that you specifically allow. Not all packets would match these rules, therefore the IP reject rules are applied, denying access to all others.
Time
Position
30
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Navigation
Locate the policy you want to edit and then click Firewall > Advanced Configuration. Reference: See the Server Protection for Windows Custom Parameters Help for detailed information about firewall parameters.
31
32
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Protection provided
The agent prevents the following from being used to attack a system: v Known exploits v Unknown exploits against known vulnerabilities v Unknown exploits that attempt to exploit a system by abusing a specific protocol
Topics
Understanding security events Intrusion prevention or intrusion detection Security events Trusted addresses on page 36 Tuning the agent on page 37 Event filtering on page 39Event filtering Advanced security event configuration on page 40 Back tracing on page 40 Packet logging on page 41 Evidence Logging on page 42
33
34
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Navigation
Locate the policy you want to edit and then click Security Events.
Security events
The Security Events tab lists hundreds of rules that can protect your system against attacks. You can view security rules and edit their settings individually. You can control the following settings for each security check: v Whether the check is enabled or disabled v Severity level v Whether the event is blocked or only reported Important: The Block settings for events are enforced only when the agent is running in intrusion prevention mode. In intrusion detection mode, intrusions are only reported, regardless of the block status shown for events. See Intrusion prevention or intrusion detection for information.
Navigation
Locate the policy you want to edit and then click Security Events > Security Events.
35
Trusted addresses
When you specify a trusted IP address, the Server Protection for Windows agent ensures that traffic associated with that addresses is not blocked by the security event rules. Trusting ensures that incoming traffic from useful or helpful addresses is not blocked automatically. Important: Consider trusting or ignoring only those addresses that do not pose a threat to your system. Keep in mind that an intruder can spoof, or fake, the IP addresses of an internal system. Note: Event filters give you more granular control over trusted addresses. The trusted addresses feature is provided to support trusted addresses configured in Version 1.0.
36
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Because there are several ways for the agent to filters traffic, it is possible to enter filters that make other filters redundant. Although the effect on the system is similar (traffic is allowed to circumvent the protection offered by the agent), you might not see the expected behavior if you later clear a filter. If you are unaware that there are redundant filters configured, you might expect the agent to resume processing certain traffic after you clear a filter; however, there might still be a filter configured that allows certain traffic to circumvent the protection offered by the agent. References: See Event filtering on page 39 and Chapter 9, Configuring bypass filters, on page 69. See Traffic seems to be bypassing analysis on page 81 for troubleshooting advice.
IP address or IP range
You can create trust rules for individual IP addresses or for a range of IP addresses.
Navigation
Locate the policy you want to edit and then click Security Events > Trusted Addresses Detail.
37
Description Allows and ignores all attacks from one or more IP addresses. CAUTION: Use this type of rule carefully! If you create a trust.address rule, the agent can no longer protect the computer if the trusted computer becomes infected by a virus or compromised by an attacker. The agent does not report the attack to SiteProtector.
False positives
If you find that the agent is falsely identifying some types of network traffic in your environment, report the issue to IBM Internet Security Systems. For information about submitting reports, see Reporting a false positive on page 77.
38
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Event filtering
An event filter ensures that traffic associated with useful or helpful addresses is not blocked by rules configured on the Security Events tab. An event filter can reduce the number of alerts displayed in the Console and the number of false positives detected by the agent. Important: Consider filtering events for only those addresses or events that do not pose a threat to your system. Keep in mind that an intruder can spoof, or fake, the IP address of an internal system.
Example
A known user pings you frequently and the agent sends an alert when it detects the ping flood event on the host. As you know this is a harmless event, you configure an event filter for this IP address and this event. When the agent detects the ping flood event from any IP addresses defined in the event filter, it does not block the traffic or send an alert to the console.
How it works
After traffic passes through the firewall, the agent processes the rules configured on the Security Events tab against the traffic. If the traffic triggers a security event rule, the agent would typically take protective action and send an alert to the console to notify you of the potential threat to your system. If, however, you have configured an event filter for this type of event from this IP address, the agent takes no protective action, sends no alert to the console, and allows the packet to continue (even if the packet contains malicious content).
Navigation
Locate the policy you want to edit and then click Security Events > Event Filters.
39
Navigation
Locate the policy you want to edit and then click Security Events > Advanced Configuration. Reference: See the Server Protection for Windows Custom Parameters Help for detailed information about IPS parameters.
Back tracing
Use the back trace feature to trace intrusions from other computers. The agent can trace the source of network traffic by analyzing information in the header of a packet that triggers an event. Back tracing also tries to identify other names for the intruder's computer and the hardware address, which makes viewing event information easier for you.
Type of tracing
The agent can perform the following types of tracing: v Indirect tracing v Direct tracing You can configure an agent to use both direct and indirect tracing.
Indirect tracing
Indirect tracing does not make contact with the intruder's system, but identifies an intruder by querying such sources as NetBIOS and the DNS database. This option returns the intruder's host name and lists it in the Events log.
Direct tracing
Direct tracing follows the network path back to the intruder's system to find out the computer name. In general, direct traces gather more reliable information than indirect traces, but direct traces can alert the intruder of the agent's activity.
40
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Navigation
Locate the policy you want to edit and then click Security Events > Back Tracing.
Packet logging
The agent can store information about all system traffic in log files. When you enable packet logging, the agent generates and maintains a set of packet log files. Important: Before you use this option, decide how you want to collect the logs from servers. To collect the logs, you must have local access rights to the server, or you must have someone at the server send the logs to you.
Navigation
Locate the policy you want to edit and then click Security Events > Packet Log.
41
Evidence Logging
When the agent identifies an incoming packet as a security event, the packet is captured and encoded into an evidence file. Evidence files can provide proof of an intrusion and provide some indication of the intruder's intentions. Important: Before you use this option, decide how you want to collect the logs from servers. To collect the logs, you must have local access rights to the server, or you must have someone at the server send the logs to you.
Navigation
Locate the policy you want to edit and then click Security Events > Evidence Log.
42
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Buffer Overflow Exploit Prevention Understanding buffer overflow exploit prevention (BOEP) on page 44
Default monitoring
When BOEP is enabled, the agent automatically protects the most commonly exploited applications from buffer overflow exploits. You have the option to include additional folders to monitor additional applications. Keep in mind that adding folders may increase the risk of false positives. Specify inclusions by referencing folders or files; specify exclusions by referencing applications.
43
What it does
The BOEP component prevents exploits that attempt to use buffer overflows to run; it does not prevent buffer overflows themselves. Vulnerability to buffer overflows is a characteristic of some applications. The Server Protection for Windows agent monitors system calls commonly used by malicious code. If it detects malicious code attempting to make such a system call, the agent blocks the call and prevents the intended operation. Depending on the response you configure the agent to take if it detects a buffer overflow exploit, the computer may need to be restarted. For example, if the agent is configured to terminate the process where the buffer overflow was detected, and the process is critical to system operation, the system may shutdown.
How it works
BOEP protects host computers from buffer overflow attacks through known and unknown vulnerabilities. As a general rule, a computer should never execute code from writable areas of system memory. By watching the use of Stack and Heap system memory, the agent identifies when a buffer overflow has succeeded and prevents the attack's payload from running. BOEP stops the worm from propagating and prevents the attacker from remotely executing code on the local computer.
44
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Protection modes
You can run buffer overflow exploit prevention in one of the following protection modes:
Setting Fail system call and send alert Terminate process and send alert Alert only Description Blocks the suspicious system call, but allows the program to continue running. Sends an alert to the console. Ends the program and sends an alert to the console. Allows the suspicious call and the program to continue running, but sends an alert to the console.
Tip: When you first start using BOEP, use the Alert only setting. This setting, sometimes called simulation mode, can help you identify false positives without interrupting normal business operations.
45
46
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Level of configuration
Antivirus compliance requires only minimal configuration, including selecting the required antivirus program, setting compliance parameters, and providing message text.
Topics
Understanding antivirus compliance Configuring antivirus compliance on page 48 Advanced antivirus compliance configuration on page 49
47
Out-of-compliance settings
You can define out-of-compliance limits for computers. In addition, you can set the agent to block or limit the computer's network access when the computer is out of compliance.
Compliance checking
You can control how frequently the agent checks the computer for compliance. You can define different frequencies depending on whether the computer is in compliance or out of compliance. In most cases, the agent should check the computer more frequently when the computer is out of compliance. Then, when the user takes appropriate steps to bring the computer back into compliance, the agent detects the update more quickly and drops the restrictions.
Navigation
Locate the policy you want to edit and then click Application Compliance > Antivirus Compliance.
48
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Custom setup
The Server Protection for Windows agent is designed to automatically find the specified McAfee or Symantec antivirus software on the system. However, you can specify alternative methods of finding the information if the initial search is unsuccessful. If the agent does not find the expected antivirus software in the expected location, it checks for custom parameters: v executable_name The name of the program file that runs the antivirus scan program. v virus_definition_disk_folder The absolute path name to the folder where the virus definition files reside. v virus_definition_file_name The name of the virus definition file. Wildcard characters are accepted. Note: If you specify a value for virus_definition_disk_folder, you should also specify a value for virus_definition_file_name, and vice versa.
Navigation
Locate the policy you want to edit and then click Application Compliance > Antivirus Compliance.
Reference
See the Server Protection for Windows Custom Parameters Help for detailed information about antivirus compliance parameters.
49
50
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Key concepts for application control How to use application control on page 52 How to handle unknown applications on page 53 How to handle known applications on page 54 Preparing and importing known applications on page 56
Level of configuration
The level of configuration required for application control varies. Before you implement your application control strategy, you might want to monitor which applications run or connect to a network for a period of time so you can plan the best application control strategy. This approach can help you determine the applications that the system uses so that you can create a list of known application. You can also create or import a list of known applications, but this requires more effort.
51
How?
To use this feature for communication control, you must create a list of authorized programs that should be allowed to access the network (using wildcard characters and system variables, as necessary). Then, block all other applications from accessing the network, or use the learning mode to see what other programs are trying to connect. Reference: How to handle known applications on page 54.
How?
Create a known application rule that does not allow the program to run or access the network. If a user attempts to run a banned application, the agent sends an event to SiteProtector. Reference: How to handle known applications on page 54.
Navigation
Locate the policy you want to edit and then click Application Compliance > Application Control.
52
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Learning mode
It is not always possible to include all possible approved applications to the known applications list. The Server Protection for Windows agent can operate in learning mode to allow you to refine your known applications list. In learning mode, the agent allows an unknown application to connect to a network, but that agent also sends an alert to notify you of the activity. You can use the alert to determine whether the application should be added to the known applications list.
Navigation
Locate the policy you want to edit and then click Application Compliance > Application Control.
53
Note: Known application entries use AND logic; the more fields you define, the more specific the entry. For example, if you enter values for three fields, all of those values must match before an application is considered a match.
54
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Wildcard characters
The following table lists wildcard characters and values you can use as part of a file path for known applications:
Wildcard Character * ? [] Function Matches zero or more characters. Example: *.exe matches any file with the .exe extension, regardless of the file name. Matches exactly one character. Example: b?ss matches bass, bess, boss, b7ss. Used to designate a set of values. Matches exactly one character in the set specified within the bracket [ ] pair unless brackets include ! or ^ characters. Example: [r5.H_] matches r, 5, ., H, or _. Used only as the first character within bracket [ ] pair. It indicates that the characters within the bracket pair should be excluded. This character can be used interchangeably with ^. Example: [!a3B6] matches any character except a, 3, B, 6. Used only as the first character within bracket [ ] pair. It indicates that the characters within the bracket pair should be excluded. This character can be used interchangeably with !. Example: [^a3B6] matches any character except a, 3, B, 6. Used within bracket [ ] pair to indicate a range of numbers or letters. Example: [1-6a-d] matches any number 1 through 6 and any letter a through d.
System variables
You can use system variables as part of the file path for known applications. The following table lists the accepted system variables and their values:
System Variable %ProgramFiles% %systemroot% %systemdrive% %windir% %temp% %tmp% Value Program Files directory computer's root directory drive letter for the computer's system drive Windows installation directory Windows temporary directory Windows temporary directory
Navigation
Locate the policy you want to edit and then click Application Compliance > Application Control.
55
56
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
5. In the Files of type box, select CSV File. 6. Choose a file name and a location, and then save your CSV file.
Procedure
1. Open the CSV file using a spreadsheet tool, such as Microsoft Excel. 2. Delete all columns except Path and MD5. 3. Add the following columns before the Path and MD5 columns: v Rule Name v Description 4. Add the following columns after the MD5 column: v Allow to run v Allow to access network When you are done, the columns should be ordered as follows:
Rule Name Description Path MD5 Allow to run Allow to access network
5. Add the appropriate data for each entry in the table, leaving blank any item you do not want to specify. 6. Delete the row that contains the column headers and any blank rows before the first row of data. 7. Save, and then close the file.
57
Procedure
1. Select the appropriate group, and then select Application Compliance > Application Control. The application control settings appear in the details pane. 2. Select Enable general application compliance. 3. Click Import. A browse box appears. 4. Locate the CSV file. 5. Select the file, and then click Open. The information in the file is imported into the list of known applications.
58
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Sections
Section A: Auditing operating system logs Section B: Auditing registry entries on page 62 Section C: Auditing files and directories on page 64
Topics
Auditing Windows event logs on page 60 Coalescing events on page 60 Identifying a suspicious series of events on page 61 Creating exceptions to audit rules on page 62
59
How it works
The agent monitors the event logs for events that you have specified as events of interest. When it detects one of these events, the agent sends an alert to the Console to notify you of the log activity.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Audit Rules. Locate the policy you want to edit and then click System Integrity Monitoring > User Defined Audit Rules.
Coalescing events
The Server Protection for Windows agent, by default, generates one alert every time an event you monitor for occurs on a system. This means that the number of alerts sent to the Console can be significant. The agent can reduce the number of alerts sent to the console by coalescing (combining) several instances of a similar event within a specified time frame in to one alert. This can speed up the analysis of alerts.
How it works
When you configure an audit rule to monitor the event logs for a specific event, you can set a coalescing threshold. If the audit rule is triggered more than one time within this threshold, the agent reports all instances of the event in one alert. The alert specifies the details of the first occurrence of the event and the total number of times the event occurred. Tip: Use coalescing to notify you of many occurrences of lower severity events where timely notification is less critical.
60
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Tip: Study the timing of events on your system to determine the best threshold setting.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Audit Rules. Locate the policy you want to edit and then click System Integrity Monitoring > User Defined Audit Rules.
How it works
You enable certain audit rules to monitor your Windows event log. You then create a correlation rule that defines a relationship between certain events. If these events occur within a certain timeframe and in a certain order (if you specified the order of the events was important), the correlation rule is satisfied and the agent notifies you.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Correlation Rules.
61
How it works
You enable an audit rule to monitor and add an exception to that rule. You then edit the exception and specify pre-defined fields in an event that you want exception to be checked. If the field that you want an exception to be checked on is not pre-defined you can then specify a custom exception expression. Using custom exception expressions requires that you understand the Windows event data structure. Because a Windows event data structure can differ from one version of the operating system to another, you might have to add multiple exceptions.
Example
You want to audit all logons to the system except logons by administrators. Using exceptions, you can create a rule to audit the event log and then create exceptions to the rule for any administrators that are allowed to access the system.
Navigation
Locate the policy you want to edit and then click System Integrity Monitoring > Audit Rules. Locate the policy you want to edit and then click System Integrity Monitoring > User Defined Audit Rules.
Topics
Monitoring the registry on page 63 Creating exceptions to registry monitoring on page 63
62
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Navigation
Locate the policy you want to edit and then click Registry Integrity Monitoring.
Example
You want to audit registry entries for successful queries unless the query was initiated by a security manager. You create a registry integrity rule to monitor the registry entries of interest and then you create an exception rule to exclude queries initiated by the security manager that you do not want to monitor.
Navigation
Locate the policy you want to edit and then click Registry Integrity Monitoring.
63
Topics
Monitoring directories and files File attributes the agent can monitor on page 65 Importing a list of files to monitor on page 66 Scheduling a baseline comparison on page 67 Updating the baseline on page 67 Excluding directories and files from monitoring on page 68
How it works
The agent creates a baseline of attribute values for the files you want to monitor. The baseline is stored in an embedded SQLite database called iss_fim.db. The file integrity monitoring component detects differences between the current environment and the baseline stored in the database. In realtime mode, the comparison happens as soon as a monitored file is modified; in non-realtime mode the comparison happens at an interval scheduled by you. Note: The initial baseline is of the current system. The agent cannot detect files that are already compromised; the agent will be able to notify you only of file integrity issues from this point forward.
64
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring.
Modification Time Time that the monitored file or directory was modified Note: This option is no longer available in Version 2.1 or Version 2.2. Read Note: In Version 2.1 and 2.2 the Read option is only supported at the local attribute level. Monitors for any accessing of information related to a file or directory Note: You do not have to open a file or directory to read information from it; whenever the system must interpret data related to a file or directory, such as when you open the properties window or when you cause a pop-up window to display information about the file, you have initiated a read.
65
Attribute Modify
Description Monitors for the following modifications to the monitored file or directory: v Content Checksum Monitors for changes to the unique identifier for a file. Note that the agent uses the SHA1 checksum algorithm. v File Type Monitors for changes to the file format associated with the file v Discretionary Access Control List (DACL) Controls which users and groups (trustees) are allowed or denied access to a file/folder v File Size Monitors the size of the file v System Access Control List (SACL) A list used to specify which attempts to access a system object are recorded in the security event log. v Owner Monitors for changes in the Owner of a file/folder. For example, if C:\Dir has "Administrator" as the Owner and then it is modified to "someuser", an event is triggered.
File format
You can only import content from text files (files with a .txt extension), the content must contain the full path to the file or directory, and each item must be on a new line.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Inclusions.
66
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Scheduled Comparisons.
When to update
Update the baseline whenever you make changes to the file integrity monitoring policy. Specify an incremental or complete baseline update as part of any changes you make to the file integrity monitoring policy. Example: You install new software on the host system and you want to audit the integrity of the new files. On the Inclusions tab, you add the new files to the list of files to monitor, you then specify an incremental baseline update on the Reestablish Baseline tab to update the baseline.
67
Important: If the agent is configured for non-realtime monitoring, consider running a scheduled comparison before you make policy changes and update the baseline to reflect those changes. If you do not run a scheduled comparison before you update the baseline, be aware of the following: v Any changes made to monitored files since the last scheduled comparison will never be reported because, as soon as the baseline is updated, the agent can no longer report changes based on the values in the previous baseline v Any attribute values that have changed will be saved to the baseline, whether the changes were legitimate or not
Notification of baselining
The agent sends an alert to the console when baselining begins and when it completes.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Reestablish Baseline.
Navigation
Locate the policy you want to edit and then click File Integrity Monitoring > Exclusions.
68
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topic
Bypass filtering
Bypass filtering
You can configure agents to allow packets from certain IP addresses to bypass analysis by the firewall and the security event rules. For example, you do not need to analyze traffic related to a system backup; you can configure a bypass filter to avoid processing this known data and slowing down the backup process.
How it works
When the agent processes a packet, it checks to see if there is a bypass filter set for packets associated with this IP address or this protocol. If there is a bypass filter configured, the agent does not process any firewall rules or security event rules against the packet.
Consideration
The more bypass filters the agent must process, the greater the impact on performance; consider configuring no more than 32 bypass filters.
Navigation
Locate the policy you want to edit and then click Bypass Filters.
69
70
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Configuring management settings Enabling agent protection on page 73 Protecting during system start-up and shutdown on page 74 Setting up configuration sharing on page 75 Configuring event caching on page 76
71
Note: This feature applies to versions 2.1 and 2.2. Important: Allowing traffic to bypass the protection offered by the agent may impact the integrity of your server.
Network monitoring
You can configure the Server Protection for Windows agent to let all network traffic pass through without being processed against firewall rules or security events (IPS and IDS mode). Note: This feature applies to versions 2.1 and 2.2.
3 4
Reference: For detailed information about configuring a backup management server, see the online Help.
72
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Advanced configuration
The Server Protection for Windows agent supports a set of custom parameters that you can use to override default parameters. Each parameter consists of the following elements: v Parameter name v Expected value v Description Important: Do not edit the parameters file manually. The Management section of the policy provides an easy to use interface for adding custom parameters. Reference: See the Server Protection for Windows Custom Parameters Help for detailed information about management parameters.
Navigation
Locate the policy you want to edit and then click Administration > Management.
Other users
The Stop Agent option is always disabled for users without administrator permissions for that server, regardless of agent protection settings.
73
Best practice
Consider requiring a password. The password option works with the other agent protection options to secure the agent. If you set a password, only a person who knows the password can stop agent services from the local server. If you set a password, even users with administrator permissions must know the password to bypass agent protection settings.
Navigation
Locate the policy you want to edit and then click Administration > Agent Protection.
How it works
The agent computer loads the driver at system startup, even before TCP is loaded. If the Block all network traffic when Server Protection for Windows is not running option is enabled, the driver blocks network traffic until the agent is active. When agent services are disabled, the agent applies a special firewall to temporarily allow the following UDP traffic: v DHCP (67, 68) v NETBIOS neighborhood (137, 138)
Navigation
Locate the policy you want to edit and then click Administration > Agent Protection.
74
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Silent agents
An agent that does not include the local user interface is called a silent or headless agent; these agents are always controlled from SiteProtector. Silent agents are appropriate for environments that need host-based protection without relying on local input.
Sharing control
An agent that includes the local user interface may offer the local user a degree of control over agent settings as follows:
Control Level Total management control Shared management control Result SiteProtector has complete control over the agent. The local user has partial control over configuration settings, and can alter any parameters that you have not explicitly set. Settings configured from SiteProtector override settings created locally. The local user has control over all configuration settings. Although SiteProtector distributes configuration settings to all agents in the group, the local user can override any of those settings.
Navigation
Locate the policy you want to edit and then click Administration > Shared Configuration.
75
How it works
If you enable this feature, the agent saves event alerts when the server is not connected to SiteProtector. When the agent reconnects to SiteProtector, the agent sends saved alerts to SiteProtector. If you choose to have event alerts cached, you can set the cache size. If you clear this option, the agent does not save event alerts when the server is not connected to SiteProtector and alerts for events that occur during this time are not sent to SiteProtector.
Cache size
If you choose to have event alerts cached, set the cache size (in megabytes). If the cache fills up, the agent overwrites older alerts with newer alerts; set the cache size to minimize the loss of alerts.
Navigation
Locate the policy you want to edit and then click Administration > Event Caching.
76
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Topics
Reporting a false positive Not seeing any BOEP alerts on page 78 Is Version 2.0 supported on ISA Server? on page 78 How can I restore custom policy settings for a child group? on page 79 Overriding parent policies does not give child policies with the parent policy on page 79 Under heavy traffic conditions traffic seems to be bypassing analysis on page 80 Not seeing any file integrity monitoring alerts on page 80 Traffic seems to be bypassing analysis on page 81 System tray icon disappeared after upgrade on page 83 Refresh agent feature in the SiteProtector System not functioning on page 82 Agent showing as offline on page 82
Background
Sometimes false positives are a result of a misconfiguration of the agent; however, sometimes, because it is difficult to reproduce all possible network configurations when IBM Security tests its software, an agent reports behavior as malicious or suspicious when it is not.
Solution
Report the issue to IBM Security Solutions Technical Support. Please reference the IBM Software Support handbook for information on "Getting IBM Support." v A screen capture of the false positive event (or events) v A brief summary of why you think this false positive happens v A description of the software and version information or network configuration if the false positive is being triggered by a specific software product or network configuration in your environment
77
Frequently, the following information is absolutely necessary for IBM Security Solutions Technical Support to fix the false positive problem. If you can provide the following information in your report, it would be extremely helpful: v A capture file containing a frame by frame record of network traffic over a specific period of time. v Explicit instructions on how to reproduce the false positive v The name, phone, and email address of someone we can contact if we need assistance to reproduce the false positive
Background
By default, Data Execution prevention (DEP) is enabled on Windows 2003 Server. DEP might block certain buffer overflow exploits before the BOEP module of the Server Protection for Windows agent can analyze them and send an alert.
Solution
Disable DEP if you want the Server Protection for Windows agent to monitor for BOEP events.
Solution
Server Protection for Windows will run in Intrusion Detection mode on ISA Servers after you manually configure a registry key.
78
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Example
To test a new configuration, you temporarily need to have each child group send a heartbeat to SiteProtector more frequently than usual; however, if you change the heartbeat setting in each child group manually, you must make a note of the heartbeat setting for each child group so that you can restore the setting after the testing completes.
Background
If a child group is using custom policies and you remove the overrides to the parent policies, the child group uses the parent group policy settings. Reinstate the custom child policies by overriding the parent policies from the child group.
Solution
Reestablish the custom settings for each child group by simply overriding the parent policy at each child group this restores the previous custom policy settings.
Overriding parent policies does not give child policies with the parent policy
Problem
You want to create new custom policies for a child group but, when you override the parent group policy, the child group policy has custom settings from a previous customization of the child group policy not settings from the parent group policy.
Background
Overriding parent policies from a child group reinstates the previous custom policy of the child group (if one exists).
Solution
To create a new custom child policy based on the current parent group policy settings, you must first copy the parent policy and then paste it in the child group. After you paste the parent group policy into the child group, you can customize the policy to create your new child group policy.
79
Background
The agent uses a buffer to pass packets between user space and kernel space. For each adapter on the system, there is one buffer. The buffer is a circular queue, therefore, as the agent processes packets, buffer space is made available to subsequent packets. If the agent cannot read packets from the buffer as quickly as they are written to the buffer, the buffer fills. In these overload situations, the agent fails open and forwards packets to their destination without processing them against firewall rules or security event rules. When the agent recovers from the overload condition, it resumes normal processing of packetspackets are again processed against firewall rules and security event rules.
Solution
If the fail open behavior is not acceptable in your environment, for agent Version 2.1, on the Management tab of the Administration policy, clear the Allow traffic to pass through (fail open) check box. This will cause the agent to drop packets when the capture buffer is full. If the fail open behavior is not acceptable in your environment, for agent Version 2.0, in the Advanced Configuration tab of the Security Events policy, set the packet.DropIfBufferFull parameter to TRUE. This will cause the agent to drop packets when the capture buffer is full.
Background
Before an agent can monitor for file integrity issues based on the settings you configured in the file integrity monitoring policy, it must update entries in the baseline to reflect the files and values you want to monitor.
Solution
Update the baseline whenever you make changes to the file integrity monitoring policy. Specify an incremental or complete baseline update as part of any changes you make to the file integrity monitoring policy.
80
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
v Complete baselineoverwrites the existing baseline with a new baseline Tip: Use this option if you made significant changes to the policy. v Incremental baselineupdates the current baseline with only the new changes to the policy Tip: Use this option if you made minor changes to the policy. 4. Do one of the following to apply the updated policy: v Wait for the agents to send a heartbeat to the SiteProtector System v Select the applicable group, and then select Action > Refresh Agent to force the agents to send a heartbeat to the SiteProtector System immediately
Background
The agent supports bypass filters, event filters, and trusted address filters. v Bypass filters allow packets from certain IP addresses to bypass analysis by the firewall and the security event rules. v Event filters ensure that traffic associated with useful or helpful addresses is not blocked by security event rules. v Trusted address filters ensure that traffic associated with useful or helpful addresses is not blocked by security event rules.
Scenario 1
It is possible that a bypass filter, an event filter, or a trusted address filter is allowing traffic to pass to the system.
Scenario 2
It is possible that you disabled a bypass filter but you still have an event filter or a trusted address filter that is preventing the security event rules from blocking malicious or suspicious traffic.
Scenario 3
It is possible that you disabled an event filter but you still have one or both of the following problems: v A bypass filter that is preventing the firewall and the security event rules from blocking malicious or suspicious traffic v A trusted address filter that is preventing the security event rules from blocking malicious or suspicious traffic.
Solution
Check for filters that might be preventing the agent from inspecting all the traffic you want inspected. Note: Bypass filters and event filters can only be set from the SiteProtector System, where as trusted addresses can be set from either the SiteProtector System or from the local console (if you are allowing configuration sharing between the SiteProtector System and the local console). If, after checking polices at the SiteProtector System level traffic is still not being processed as you expected, check the settings
Chapter 11. Troubleshooting
81
Problem
You initiate a Refresh Agent request from the SiteProtector System to the agent, but the agent does not initiate a heartbeat to the SiteProtector System.
Background
The Refresh Agent feature in the SiteProtector System uses an ICMP message to contact the agent. If you have an ICMP firewall rule that includes your SiteProtector System address as a source to block, the agent cannot detect the ICMP message and answer the request to heartbeat in to the SiteProtector System.
Solution
If you have an ICMP firewall rule that is blocking the SiteProtector System, you can do one of the following actions: v Redefine your firewall rules to allow ICMP traffic from the SiteProtector System to your agent v Use only the regular agent heartbeat to communicate with the SiteProtector System
Problem
The agent is showing as offline in the SiteProtector System, but the agent is still sending alerts.
Background
The Unresponsive Agent Threshold setting specifies the number of minutes that can elapse between agent heartbeat signals before the agent is considered unresponsive. If the Unresponsive Agent Threshold setting is shorter than the heartbeat interval for an agent, SiteProtector shows your agent as offline when in fact it is available but it has not sent a heartbeat within the threshold period. For example, if your Unresponsive Agent Threshold is set to two hours (the default) and your heartbeat interval is set to six hours, the agent status in the SiteProtector System changes to offline when two hours have passed because the agent has not sent a heartbeat within those two hours. The agent will not send a heartbeat for another four hours based on the heartbeat interval setting.
Solution
Do one of the following actions: v Change the Unresponsive Agent Threshold setting so that it is longer than the heartbeat interval v Send a Refresh Agent command to initiate a heartbeat from the agent to SiteProtector System
82
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Solution
Open the local console from the Start menu; the icon will again be visible in the system tray. This only happens immediately following the update to 2.0.
83
84
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: Intellectual Property Licensing Legal and Intellectual Property Law IBM Japan Ltd. 1623-14, Shimotsuruma, Yamato-shi Kanagawa 242-8502 Japan The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.
85
Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact: IBM Corporation Project Management C55A/74KB 6303 Barfield Rd., Atlanta, GA 30328 U.S.A Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us. All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at Copyright and trademark information at Copyright and trademark information at www.ibm.com/ legal/copytrade.shtml. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.
86
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows
Index A
advanced configuration antivirus compliance 49 firewall 31 management settings 73 security events 40 agent password protection 73 agent build definition 22 Agent Manager backup 72 failover settings 72 primary 72 secondary 72 update 9 Alert only (IDS mode) 35 alerts associated with scheduled comparison 67 caching 76 not in console 76, 78, 80 antivirus compliance advanced configuration 49 custom parameters 49 overview 47 supported antivirus software 47 application control 52 handling known applications 54 learning mode 53 system variables 55 unknown applications 53 application rules 28 applications adding to known list 54 audit events coalescing 60 correlating 61 audit rules exceptions 62 monitoring event log 60 predefined 60 user defined 60 automatic blocking 27 automatic port opening 30 buffer overflow exploit prevention (BOEP) (continued) false positives 43 Buffer Overflow Exploit Prevention (BOEP) and Data Execution Prevention (DEP) 78 not seeing alerts 78 overview 44 simulation mode 45 bypass filter interaction with other filters 69 number of 69 when to use 69 bypassing protection 71 deployment (continued) scenarios 6 deployment scenario checklists deployment scenarios 5 double quotes in CSV file 56 6
E
enforce audit policy 60 event filter 39 interaction with other filters 39 event log auditing 60 events coalescing 60 reducing number of 39 exceptions audit rules 62 registry integrity rules 63 system integrity monitoring policy 62 exclusions file integrity monitoring 68
C
caching alerts 76 coalescing audit events 60 combining, coalescing 60 comma in CSV file 56 communication control 52 control application 52 communication 52 correlating audit events 61 correlation rule alert name 61 maximum number of rules 61 critical files monitoring 64 CSV file handling commas 56 handling double quotes 56 handling spaces 56 custom parameters antivirus compliance 49 firewall 31 management settings 73 security events 40 customer support vii
F
fail open 71, 80 failover settings Agent Manager 72 false positives reporting 77 file integrity monitoring attributes monitored 65 creating initial baseline 64 database name 64 exclusions 68 global attributes 65 importing list of files 66 local attributes 65 non-realtime 64 not seeing alerts 80 overview 64 realtime 64 scheduling and non-realtime monitoring 67 scheduling baseline comparison 67 update baseline 67 when to update baseline 67 files or folders adding for file integrity monitoring 64 filters clearing had no effect 81 interaction among 36, 39, 69, 81 firewall advanced configuration 31 automatic port opening 30 blocking 27 custom parameters 31
B
backup Agent Manager 72 how it works 72 baseline start and stop notification 68 updating 67 Block and alert (IPS mode) 35 BlockWhileAgentStopped option 74 BOEP (Buffer Overflow Exploit Prevention) 43 buffer overflow exploit prevention (BOEP) data execution prevention (DEP) 43 Copyright IBM Corp. 2005, 2011
D
Data Execution Prevention and BOEP 78 disable 78 DEP (Data Execution Prevention) deployment agent build 22 before you deploy 5 prerequisites Agent Manager update 9 create groups 10 database update 9 verify licenses 8 43
87
firewall (continued) interaction with Microsoft firewall 25 manual port opening 31 opening port through 30 opening UDP port 31 protection levels 26 rule processing order 28 types of rules 28 Firewall policy 28, 30 application order 29 firewall rules application order 28 conflicts 30 interaction with trusted addresses 36 processing order 28 Firewall Rules tab 28, 29, 30
L
layered protection 1 learning mode 53 licenses 8
S
scheduled comparison alerts associated with 67 secondary Agent Manager 72 security events advanced configuration 40 custom parameters 40 rules to detect 35 Security Events policy 36 security events tab 36 silent agent 75 simulation mode for BOEP 45 space in CSV file 56 system tray icon not visible 17, 83 system variables for Application Control 55
M
management settings advanced configuration 73 custom parameters 73 manual blocking 27 manual port opening 31 Microsoft Windows firewall 25
N G
group settings policy 21 groups adding 11 advantages of 10 network monitoring 72
O
overload condition 80
T
traffic bypassing analysis 81 troubleshooting 82 filter interaction 81 trust rules 37 trusted address filter interaction with other filters trusted IP address 36, 39
H
headless agent 75 changing to or from 75 heartbeat guidelines for setting 71 understanding 71
P
password protection for agents 73 policies Firewall 28, 29, 30 Security Events 34, 36 policy apply updates immediately 3 group settings 21 inheritance 3 policy inheritance 3 parent settings not at child 79 restore custom policy settings 79 policy updates when applied 3 port opening automatic 30 manual 31 UDP traffic 31 primary Agent Manager 72 protection levels 26
36
U
UDP traffic opening ports 31 unknown applications 53 unresponsive agent threshold setting update settings policy 21 upgrading 11 and local console settings 12, 13 before you upgrade 5 82
I
IBM Security Solutions technical support vii IDS mode 35 intrusion detection mode 35 intrusion prevention mode 35 IP addresses trusting 36, 39 IPS mode 35 ISA Server 78 iss_fim.db 64
V
variables for Application Control verify licenses 8 55
K
known applications accepted wildcard characters adding 54 55
R
realtime monitoring when to use 64 refresh agent 82 registry integrity exceptions to rules 63 monitoring 62 reporting false positives 77
W
wildcard characters for known applications Windows event log auditing 60 55
88
Security Server Protection for Windows: Administrator Guide for Security Server Protection for Windows