Exploring IPS Technologies

Exploring IPS Technologies

In this section, we examine the fundamentals of IPS and IDS technologies, as well as ways to categorize them. We determine where they fit into Cisco’s SelfDefending Network and introduce some exemplary solutions in Cisco’s product portfolio. We conclude with some introductory comments about the Cisco IOS IPS router solution in order to set up for the next section, which concentrates on features of the Cisco IOS IPS.

IDS Versus IPS

It’s simple, right? Intrusion Detection Systems (IDSs) monitor for signs of intrusion but do not take action, but Intrusion Prevention Systems (IPSs) monitor for signs of intrusion and take action, right? Actually … not! Some vendors make this distinction, but Cisco does not. The following is a high-level description of Cisco’s definition of what constitutes an IDS or IPS:

  • IDS. According to Cisco, an IDS works offline to the data stream going through the network, analyzing copies of the traffic instead of forcing the data to flow through it. Specifically:
  • Monitors copies of the data flow.
  • Does not impede network traffic.
  • Allows some traffic deemed malicious to enter or leave the network.
  • IPS. An IPS places itself in the data stream, thus seeing all the traffic
    that goes through that part of the network. Specifically:
  • Monitors traffic and content from layers 2 through 7 in real-time.
  • Must be capable of handling all ingress and egress traffic where it is deployed.
  • Prevents traffic deemed malicious from entering or leaving the network.

Fundamentally, both an IDS and IPS monitor for signs of intrusion but because an IDS take copies of traffic from a data flow, it might miss trigger packets or single-packet attacks. A trigger packet is the very first packet in a malicious data flow. That aside, IDSs and IPSs are complementary in function and have their own strengths and weaknesses. An IDS will often require assistance from other devices such as routers or firewalls to perform the actions dictated by the security policy for the malicious traffic. As we will see, IDS/IPS systems can be deployed as a hardware or software solution depending on the requirement and
can be deployed in a number of different solutions; an IDS/IPS can be:

  • A separate network device
  • An option card in a router or security appliance
  • Integral software (as with the Cisco IOS IPS, pictured in Figure 8.1)

Exploring IPS Technologiesfig8.1

IDS and IPS Categories

As shown in Figure 8.1, where you deploy the system and how you configure it will determine whether it is an IDS or IPS. Here are some examples of Cisco IDS and IPS technologies:

  • The AIP-SSM card can be configured on a Cisco ASA 5500 Series security appliance in either promiscuous (IDS) or inline (IPS) mode.
  • Cisco IOS IPS is configured as a process on a Cisco IOS router.
  • Cisco has a whole product line of rackmount IPS appliances that can be physically implemented in line with the network traffic or configured to look at copies of the traffic only.
  • Software deployed on an IP host such as an application server or a client workstation.
NOTE
IDS and IPS technology is called a sensor. IDS and IPS technologies use policy-based rules called signatures to detect intrusions.

Where you deploy IDS and IPS technologies will dictate what they can see and how they can deal with intrusion attempts (and how quickly). You can deploy them in two places:

  • In the network. Called network IDS and network IPS.
  • On a host. Host-based IPS or HIPS.
    In Figure 8.2, an attacker on the Internet launches a hack on the public HTTP server deployed in the DMZ.

Exploring IPS Technologiesfig8.2

As indicated in Figure 8.2, this attack might be missed by a network IDS because it is not deployed inline with the ingress packets to the network and will probably not be able to react to the trigger packet of the attack The following steps illustrate this logic. The numbers in the list refer to the labels in Figure 8.2:

  1. The trigger packet of the attack arrives on the IOS firewall.
  2. The router is configured to work with a network IDS. One packet is sent to the HTTP server in the DMZ and a copy is sent to the IDS, which is on a separate VLAN.
  3. The packet arrives relatively simultaneously on both the server in the DMZ and the network IDS. The IDS recognizes that an attack is taking place.
  4.  (not pictured) The IDS tells the IOS firewall to block the attack and logs the attack via syslog or SDEE (see the following notes) messages to the MARS appliance.
NOTE
If you haven’t heard about Cisco Security Monitoring Analysis and Reporting System (MARS) yet, you may want to read a description of MARS in Chapter 4, “Implementing Secure Management and Hardening the Router.” MARS is also mentioned in Chapter 1, “Network Insecurity,” and is referenced in Table 8.3.
NOTE
The network design in Figure 8.2 should look familiar. It is essentially the same as the reference design in Figure 5.1 from Chapter 5, “Using Cisco IOS Firewalls to Implement a Network Security Policy,” with the addition of a new VLAN where the IDS resides. As mentioned in that chapter, the IOS firewall is capable of performing deep packet inspection and can mitigate a number of network-borne attacks, although this isn’t discussed until another Cisco CCSP course, SNRS.

SDEE (Security Device Event Exchange) protocol is discussed later on in this chapter of the Exam Cram.

EXAM ALERT
Table 8.1 presents an overview of the relative pros and cons of IDS and IPS. Memorize this table—it’s just the kind of thing that will be on the exam.

Exploring IPS Technologiestb8.1

EXAM ALERT
Table 8.2 presents a broad overview of sensor types as well as their relative pros and cons. Memorize it for the exam.

Exploring IPS Technologiestb8.2

NOTE
Do not infer from Table 8.2 that anomaly-based IDS/IPS sensors are easy to maintain. Yes, the initial configuration is no more complicated than a signature-based sensor, but much ongoing analysis and reconfiguration of what constitutes normal traffic patterns is required for them to be effective. One thing many enterprises have no real data for is what normal network activity actually is.
NOTE
A honey potis a system that you have deployed in your network security architecture with the specific purpose to attract bees. A honey pot is an excellent place to analyze intrusion attempts to your network, and they are often used in conjunction with more mainstream technical solutions, such as Cisco IDS sensors. You must be careful how you deploy them in a legal context, too. Some legal jurisdictions may declare evidence of intrusion attempts on a honey pot as
inadmissible for prosecution because of the fine line between enticementand entrapment. Enticement means that you have made it easier for the bees (the attackers) to conduct their normal activity, whereas entrapment means that you have coerced the bees into conducting an activity to which they are not normally predisposed. We’ll let the lawyers argue over that one! See the “Law and Ethics” section in Chapter 1 for more information about the legal framework.

IPS Attack Responses

Putting the terms together, an example solution might be a network IPS that uses signatures and policies to detect and block attacks. This leads to the next question, which is: “What response and monitoring actions can be employed by IPS technologies?”

NOTE
From this point forward, we will focus on IPS technologies (thus the name of the chapter!) and the term IDS will hardly be breathed. This is mainly because the term IDS is falling somewhat out of favor, and IPS is often used interchangeably, though often incorrectly. Besides, what will sell more boxes, an intrusion detection system or an intrusion protection system?!
EXAM ALERT
So, what are you going to do about that attempted network compromise or intrusion attempt? What response and monitoring actions can be employed by an IPS? Memorize the following list! You may also want to review the theory of TCP sessions found in Chapter 5 before looking at some of these actions.

What follows is a list of some of the actions that we can configure on Cisco sensors:

  • Deny Attacker Inline. Terminates current packet and shuns all subsequent packets from this attacker for a specified time period, regardless of whether they are part of new TCP connections.
  • Deny Connection Inline. Terminates current and future packets in a specific TCP connection.
  • Deny Packet Inline. Terminates the packet only.
  • Log Attacker Packets. Logs all packets with an attacker’s IP address.
  • Log Pair Packets. Logs events in an attack pair that contains the attacker and victim address pair. This is useful for identifying specific flows.
  • Log Victim Packets. Logs all packets with a victim’s IP address.
  • Produce Alert. Sends an alert to the Event Store (internal log buffer).
  • Produce Verbose Alert. Sends an encoded dump (capture) of the offending packet in the alert.
  • Request Block Connection. Sends a request to another device (router, switch, firewall) to block this connection.
  • Request Block Host. Sends a request to another device (router, switch, firewall) to block this attacker.
  • Request SNMP Trap. Instructs the sensor to send an SNMP trap.
  • Reset TCP Connection. Sends TCP RSTs to terminate the TCP connection.
NOTE
Every action that contains the words “log,” “produce,” or “SNMP” sends an alert to the Event Store (internal log buffer), regardless of whether Produce Alert is also selected as an action. This makes sense because they are all essentially alerting type of actions, even if they aren’t called that. Some actions can be combined. For example, the Deny Packet and Deny Flow actions do not reset the connection, preferring to silently discard traffic, making the IPS stealthy. If you do want to reset the connection, add the Reset TCP Connection action. Of course, the IPS is no longer invisible to the attacker, possibly provoking them. This said, the organization’s security policy will dictate the action(s) to take.

Event Management and Monitoring

Back in Chapter 4, we discussed options for secure management and logging, focusing mainly on syslog and SNMP. So, we already have our mind around the concept of sending event-triggered alerts to the various reporting servers in our network. This is handy, because referring to Figure 8.2, the reference network diagram indicates that a MARS (Monitoring, Analysis, and Response System) appliance has been deployed in a separate VLAN inside the Cisco IOS firewall. This section isn’t a discussion about how to log to these servers. It is a discussion of what to log and how to log (management) and real-time monitoring of events as they unfold in the traffic streams through our network. The scale of the solution has to match th scale of the network, both in the number of devices deployed, as well as the volume of traffic that is being moved across the monitored devices. As we have seen, some of those same metrics define whether the IPS is inline to the traffic or not. We need to know the following:

  • What is happening in real-time (monitoring)
  • IPS actions and event information collection (management)
  • Analysis of the archived information (reporting)
NOTE
A little context is in order. You may recall that in Chapter 4, we discussed how secure management and logging needs to be integrated into the organization’s security policy, and this doesn’t just happen—it takes careful design. It was also pointed out that as much as possible, this traffic should be in a separate plane and preferably should not “cross the cables” of the production network. Similarly, monitoring and management should be OOB (out-ofband) with respect to the data plane, perhaps by putting it in a separate VLAN. Keep this in the back of your mind when you look at some of the subsequent high-level discussion about IPS deployment scenarios and technologies in the remainder of this section. We will look at and implement the different logging protocol options The next section in this chapter, “Implementing Cisco IOS IPS,” examines and implements the different logging protocol options.

Cisco makes the following points about event management and monitoring:

  • Management and monitoring comprises two key functions:
  • Real-time event monitoring and management
  • Analysis of the archived event information (reporting)
  • Depending on scale, management and monitoring can be deployed on a single server or on separate servers.
  • Recommended maximum of 25 well-tuned sensors reporting to a single IPS management console (management consoles will be discussed shortly).

Cisco IPS Management Software

Table 8.3 outlines Cisco’s portfolio of IPS management software.

Exploring IPS Technologiestb8.3

NOTE
The day after the IINS version 1.0 course was released, Cisco announced a new, freeware IPS management solution called IME (IPS Manager Express). It fully supports the newer 6.x version of the IPS signatures and also has limited support for version 5.x signatures. For more information on Cisco IME, navigate to this link: http://www.cisco.com/en/US/ products/ps9610/index.html.

Figure 8.3 illustrates the Threat Analysis Console of the Cisco IPS Event Viewer (IEV)

Exploring IPS Technologiesfig8.3

Host IPS

Recall that IPS technology can be either network- or host-based, although they often co-exist as complementary solutions. With host-based IPS (HIPS), software is deployed right on the application server or client. Cisco’s HIPS solution is the Cisco Security Agent (CSA). Here are some features and recommendations about HIPS:

  • HIPS audits files systems, OS registries, log files, and system resources.
  • CSA should be installed on every host that requires HIPS.
  • CSA protects the individual host by detecting intrusion attempts.
  • CSA does not require special hardware.
  • Because CSA is behavior-based, it can stop even newer attacks in realtime whose behavior it identifies as malicious.

Referring to Figure 8.4, if we installed CSA on the public HTTP server that we first saw in Figure 8.2, it might obviate the need to configure the Cisco IOS firewall as a Network IPS. We could then manage the CSA with the CSA Management Center (CSA MC) on the Systems Administrator workstation. This might work on a small network, but imagine if you have a large network of thousands of IP hosts. You would need to install, configure, and manage all of those CSA instances. Often, CSA is deployed in parts of the network—on critical application servers, for instance—and used to complement Network IPS systems if possible.
Exploring IPS Technologiesfig8.4

Figure 8.5 illustrates how CSA works to protect an IP host from malicious applications directly at the kernel level.
Exploring IPS Technologiesfig8.5

The steps in Figure 8.5 are as follows:

  1. An application attempts to call the OS kernel for system resources.
  2. The HIPS intercepts this call, and checks it against the policy.
  3. The request for system resources is either allowed or denied.
    Here are some other points about how HIPS operates:

    • HIPS intercepts both operating system and application calls.
    • Rules control application and network protocol stacks.
    • Processor controls set limits on the following:
    • Buffer overflow.
    • Registry updates.
    • Writes to system directory.
    • Launching of installation programs.
    • HIPS is behavior-based.

Table 8.4 outlines the pros and cons of Host Intrusion Prevention Systems (HIPS).

Exploring IPS Technologiestb8.4

Network IPS

Figure 8.6 illustrates a network IPS deployed on the WAN side of a Cisco IOS firewall. It is deployed in such a way that it will see the inbound traffic (represented as an arrow) from an attacker on the outside. It is an IPS because it is inline with the flow of traffic from the outside. The features of a network IPS are explained more thoroughly next.

Exploring IPS Technologiesfig8.6

Here are some features of a network IPS:

  • Connected to the network where, if properly deployed, a single sensor can monitor traffic for many hosts or at least a small group of important hosts.
  • Sensors are appliances with software that is tuned for its role in intrusion detection analysis:
  • The underlying operating system is hardened.
  • Hardware is optimized for intrusion detection analysis.
  • Scalable—networks are easily protected as they grow:
  • New end system hosts and devices can be added without the need for new sensors (unlike HIPS).
  • New sensors can be easily added to new or existing networks.
NOTE
Network IPSs are not necessarily network appliances, although the appliance versions are rack mount blade servers with a hardened version of a Windows Server OS. Also, if you have an extra slot in your Catalyst 6500 series switch, you can add an IDSM-2 module. (see the “Cisco IPS Appliances” section of this chapter). When we implement Cisco IOS IPS using the SDM in the next section, we see that network appliances aren’t Cisco’s only network IPS solution.

Exploring IPS Technologiestb8.5

HIPS and Network IPS Comparison

Most of the points in Table 8.6 were already made, but it is useful to summarize them in one place to compare HIPS and network IPS.

Exploring IPS Technologiestb8.6

Cisco IPS Appliances

Figure 8.7 illustrates Cisco’s portfolio of IPS appliance.
Exploring IPS Technologiesfig8.7

Here is a list of Cisco’s four solutions for IPS appliances:

  • Cisco IPS 4200 Series Sensors. Standalone rackmount appliance.
  • Cisco ASA AIP-SSM. Advanced Inspection and Prevention Security Services Module for the ASA 5500 Series adaptive security appliances.
  • Cisco Catalyst 6500 Series IDSM-2. Intrusion Detection System Services Module for the Catalyst 6500 Series switch.
  • Cisco IPS AIM. IPS Advanced Integration Module for Cisco 1841, 2800, and 3800 Series Integrated Services routers.

All of these devices share the same code as the Cisco IPS 4200 Series IPS appliance. This was not an accident, as Cisco product planners wanted to ensure commonality of the configuration interface (and code base) as networks grow and needs change requiring more and different sensors. This makes it easier for IT security staff to grow with the product.

NOTE
There is a fifth choice of network IPS, and it is not an appliance. Cisco IOS IPS acts as an inline IPS sensor that can be turned on in Cisco IOS Software router platforms with security feature images. For example, Cisco 800 series Integrated Services routers must have the Advanced IP Services IOS image to enable this feature. We will be examining and configuring the Cisco IOS IPS in the next section, using the Cisco SDM.

IDS and IPS Signatures

IDS and IPS technologies use signatures to detect malicious network activity. This is similar to Anti-X products—virus scanners, for example. A good IPS solution enables you to update the signatures regularly as new threats emerge because the IPS is only as good as the ability of its signatures to detect intrusive activity. Here are the characteristics of IPS signatures:

  • A network IPS signature is a set of rules used to detect intrusive activity.
  • Sensors use signatures to test network packets for signs of known attacks. The signatures have predefined actions to respond to a detected attack.
  • When a signature-based IPS is first installed, there will be false positives. You must fine tune the signatures to reduce the frequency of false positives by modifying certain signature parameters.
  • Built-in signatures cannot be added or deleted, although they may be tuned to reduce false positives (see the following note).
  • Some signatures have sub-signatures nested in them. These sub-signatures may be used by other signatures. Changing a sub-signature affects only that sub-signature and not the signature that uses it.
NOTE
Built-in signatures have been removed from Cisco IOS IPS starting from Cisco IOS Software Release 12.4(11)T (reference: http://www.cisco.com/en/US/prod/collateral/ iosswrel/ps6537/ps6586/ps6634/prod_qas0900aecd806fc530.html).

Signature Micro-Engines

Conceptually, signatures are a database of patterns that match attacks. The signatures don’t do anything by themselves, but they are read-in, cached, and parsed by real-time processing daemons called signature micro-engines (SME). A signature micro-engine is a self-contained parallel process that has the following characteristics:

  • Signature micro-engines support IPS signatures.
  • Signatures that are read into a micro-engine are scanned in parallel for efficiency and maximum throughput.
  • Micro-engines group a category of signatures.
  • Each micro-engine is customizable for the protocol and PDU fields that it is designed to inspect. Micro-engines specify permissible, legal ranges for the items that they are designed to inspect.
  • Micro-engines use router memory to compile, load, and merge signatures (see the following note). These signatures use “regular expressions” (REGEXs) for pattern matching. This will be very familiar to readers from the Unix world where REGEXs are quite common in shell scripts. Signatures are customized or written with REGEX. Compiling takes more RAM than the finished signature, so you must be careful to stay within the RAM requirements of your platform for the number of signatures that will be used.
NOTE
Micro-engines use router memory to compile, load, and merge signatures only when Cisco IOS IPS is used as the IPS is a software process hosted on an IOS router in that case. The four appliances listed previously are self-contained environments with their own flash memory, RAM, and CPUs. It’s not clear whether the exam makes this distinction, so the “right wrong answer” is above.

The third item in the list states that micro-engines group a category of signatures. Table 8.7 shows the list of supported micro-engines and the categories of signatures they group as of Cisco IOS release 12.4(6)T.

Exploring IPS Technologiestb8.7

NOTE
Atomic signatures examine simple packets. “Simple” in this context means that the test is simple—for example, an ATOMIC.TCP signature micro-engine provides for simple TCP packet alarms based on source and destination port numbers and flags (SYN, RST, ACK, PSH, URG, FIN). For example, the ATOMIC.TCP SME would be able to analyze a TCP session and reset the session if the two connection partners aren’t playing by the rules. Remember the Reset TCP Connection action in the “IPS Attack Responses” subsection earlier in the chapter?

Signature Alarms

When a signature is matched due to an attack or a policy violation, the SME in which the signature is compiled will raise an alarm, after which an action is taken to respond. The signature is said to “fire.” The whole efficacy of the IPS solution hinges on the ability of the sensor to react quickly with an IPS action response using the action specified in the signature. Anyone fond of watching Hollywood dramas would quickly state that there are usually more false alarms than there are real ones. In the real world, reacting to false alarms can result in embarrassment, but no real damage is done. If a network IPS blocks legitimate
traffic because of a false alarm (a so-called false positive), there might be real damage done to the throughput and quality of service offered by the network. Alarms are either false or true, negative or positive:

  • False Positive. Normal traffic or a non-malicious action causes the signature to fire.
  • True Positive. An attack is properly detected by the IPS.
  • False Negative. An attack is not detected by the IPS.
  • True Negative. Legitimate traffic does not cause signatures to fire.

Alarm Severity Levels

Some attacks can cause more damage than others. The alarm severity level assigned to the signature should match the severity of the event. Think of the SMEs as a group of patients vying for the attention of medical doctor, the IPS sensor. The IPS sensor (as well as a real-time monitoring, analysis, and reporting system, such as Cisco Security MARS) should be able to prioritize its responses to the attacks based on the alarms from the SMEs. Signatures should be fired by order of severity, not necessarily in the order in which they are received. The sensor triages the patients, treating the most severely ill first, regardless of when they walked into the emergency room of the hospital.

Here are some general principles with respect to implementing alarms in signatures:

  • The alarm level assigned to the signature determines the alarm severity level.
  • Alarm severity levels are: informational, low, medium, and high.
  • The severity level of the signature should be made to be the same as the
    severity level of the alarm. (Remember this particularly when you create your own signatures.)
  • Signatures should be tuned to recognize traffic patterns that are anomalous with respect to your normal network traffic patterns.
    Here is a summary of the meaning of the four severity levels:
  • Informational. Activity is not an immediate threat, but the information provided is useful.
  • Low. Activity is not an immediate threat, but anomalous network activity could be perceived as malicious.
  • Medium. Activity is likely an immediate threat, and the anomalous network activity is likely malicious.
  • High. An immediate threat is extremely likely, and an access or DoS attack has been detected.

Best Practices for IPS Configuration

The following list of best practices is most appropriate in the context of a large network with multiple IPS sensors. Some of these ideas will be familiar to anyone who has had to set up a deployment of new code to client workstations within a large LAN domain. Even the bandwidth of a modern, high-speed Gigabit Ethernet core can be severely strained if some intelligent design is not employed when updating several devices simultaneously. From a security perspective, it makes good sense to perform your updates separate from the data plane of the production network, using the management network. To summarize:

  • Set the sensors to automatically update their signature packs instead of doing a manual upgrade of individual sensors.
  • Employ a dedicated FTP server in the management network from which the sensors can fetch the signature packs.
  • Stagger the time of day for checking and downloading of the signature packs to prevent over-taxing network resources.
  • Make the deployment of updates simpler by grouping IPS sensors together in a few larger profiles.
    Figure 8.8 shows Autoupdate enabled in the Cisco SDM for an IOS IPS configuration. Note that in this example, autoupdate is set up to fetch the signature packs from a Trivial File Transfer Protocol (TFTP) server.
    Exploring IPS Technologiesfig8.8
NOTE
If you are deploying a dedicated server that uses a cleartext (not encrypted) protocol such as FTP or TFTP, it is critical that these servers be deployed in a dedicated VLAN or entirely separate physical network so that this traffic does not cross the cables with normal data traffic.

About the author

Prasanna

Leave a Comment