Conceptualizing a Site-to-Site IPsec VPN

Conceptualizing a Site-to-Site IPsec VPN

One of the advantages of a site-to-site IPsec VPN is that the clients whose traffic is being protected by the VPN don’t need to know anything about VPNs. All
the users know is that something magic happens, and they can access an HQ server on its native IP address, even if that server is on the other side of a hostile network like the Internet. In this section, we look at the separate components of all IPsec VPNs, and then reinforce these concepts by constructing a
site-to-site IPsec VPN, using both the CLI and the SDM.

IPsec Components

IPsec is an Internet Engineering Task Force (IETF) standard that defines a framework of open standards for secure, real-time communication using IP. Like IP, it works at the network layer, and while it is deliberately security algorithmindependent, it does provide an environment into which various encryption, authentication, and keying algorithms can be plugged in. It is described in RFCs 2401–2412 and was designed as a native component of IPv6, but has been ported to IPv4, where our focus lies. IPsec can be recognized inside an IP packet by its protocol IDs, 50, and 51, which are reserved for ESP (Encapsulating Security Payload) and AH (Authentication Header), respectively. IPsec’s framework provides its own C-I-A triad (actually C-I-A-A):

  • Confidentiality. Uses encryption algorithms (encryption is optional with IPsec VPNs. See the following note).
  • Integrity. IPsec policies must implement hashing algorithms to assure date integrity.
  • Authentication. Peers must authenticate during tunnel establishment. Optionally, origin authentication is specified to verify that packets come from an authenticated peer. Authentication is managed by the Internet Key Exchange (IKE) and can take different forms.
  • Anti-Replay. IPsec must provide assurance of each packet’s uniqueness and that it hasn’t been duplicated through sequencing and the use of a sliding window.
Know what the IPsec C-I-A stands for. (C)onfidentiality through encryption, (I)ntegrity through hashing algorithms, and (A)uthentication using PSKs and digital certificates and RSA encrypted nonces, as well as origin authentication using HMACs (Hashing Media Authentication Codes), has been discussed in Chapter 6. We are now taking those tools out of the toolbox and constructing an IPsec VPN with them.
Anti-replay is the last A in C-I-A-A and will not be further discussed in this book. Besides, C-I-A is a lot easier to remember than C-I-A-A, right?
Our focus will be on encryption; or else how private are our Virtual Private Networks?
That said, IPsec provides for either ESP or AH inside IP packets. ESP specifies encryption, whereas AH does not. Do not be surprised if you see little or no discussion about ESP’s forgotten cousin, AH.

IPsec is versatile as it allows for new and better security protocols in every facet of IPsec’s C-I-A because it is security algorithm-independent. Think of the different components that fit into IPsec as modular, hot-swappable plug-ins. Let’s look at the different elements of C-I-A separately.

Confidentiality: Encryption Algorithms

Confidentiality is assured through the use of encryption algorithms (or ciphers) and encryption keys. Only the VPN peers know what encryption algorithms they have negotiated and which keys they have arrived at (calculated). This information is never transmitted with the encrypted payload, so good luck breaking the code if you’re an attacker! Of course, the shorter the key, the easier it is to guess.

The longer the key, the harder it is to guess, and the longer it takes to create encrypted data because the key is used with the encryption algorithm to create encrypted data, or ciphertext. This is where it would be useful for the VPN peer to possess hardware-accelerated encryption as this can be burdensome on the CPU, again pointing at the classic tradeoff between security and usability. This becomes a critical issue when the VPN peer supports hundreds or thousands of VPN connections simultaneously. Figure 7.3 gives a high-level overview of the relationship between cleartext, ciphertext, encryption, decryption, and encryption algorithms and encryption keys.

Conceptualizing a Site-to-Site IPsec VPNFIG7.3

Where do these keys come from? How long are they? We’re getting a bit ahead of ourselves because we haven’t considered how the encryption algorithms and encryption keys are negotiated between the peers. As we will see in the later section, “IKE Phase I: The IKE Protocol,” the keys are either manually configured or dynamically negotiated (calculated) when the VPN is being established during the Diffie-Hellman (DH) key exchange phase. See the next section for a bit more information on DH.
Here are some encryption algorithms and their associated key lengths that VPNs use:

  • DES (Data Encryption Standard). This cipher uses 56-bit keys. It is a symmetric key crypto system.
  • 3DES (“Triple” or “Three” DES). Using DES as a verb, 64-bit blocks of data are DES’d three times. The base key is still 56 bits. Often described as a 168-bit cipher, but we know better, right? 3DES is a symmetric key crypto system.
  • AES (Advanced Encryption Standard). 128-, 192-, or 256-bit keys. AES has stronger security than DES and is computationally more efficient than 3DES. It was adopted by the National Institute of Standards and Technology (NIST) as a replacement for DES and as a result of a competition.
  • RSA (Rivest, Shamir and Adleman). Not used by IPsec for encryption (C), though RSA encryption can be used during peer authentication (A). RSA uses 512-, 768-, 1024-bit, or larger keys. It is an asymmetric key crypto system.
  • SEAL (Software-Optimized Encryption Algorithm). Stream cipher that uses a 160-bit key for encryption.
Creating an Encryption Key: The Diffie-Hellman Key Exchange

Again, we will put off the discussion for where and how the DH key exchange occurs during IPsec tunnel establishment. For now, it is important to know that it is a necessary component of IPsec and is a way for VPN peers to arrive at a share-secret encryption key for symmetric protocols, such as DES and 3DES, which require it. It describes a secure method for VPN peers to calculate a shared-secret key over a public network without having to resort to some outof-band method such as a courier or a phone call for transmission. This sounds impossible, and was in fact considered as such by cryptographic experts for many years. DH makes VPNs more practical because the shared encryption key is negotiated on demand during the VPN setup.

DH algorithms supported on Cisco IOS routers include DH Groups 1, 2, 5, and 7 (DH1, DH2, DH5, and DH7 for short).

Integrity: Hashing Algorithms

If you want to make sure the 1’s and 0’s you put in one end of the VPN are the same 1’s and 0’s that come out the other end, you need to employ a Hashed Message Authentication Code (HMAC), or “hash” for short. Here are the two common HMAC algorithms that are supported on the Cisco IOS router:

  • Message Digest 5 (MD5). HMAC-MD5 uses a 128-bit shared-secret encryption key.
  • Secure Hash Algorithm 1 (SHA-1). HMAC-SHA-1 uses a 160-bit shared-secret encryption key.

The idea is fairly straightforward. Looks like we’re going to use those encryption keys again! Here are the three steps in insuring the integrity of messages:

  1. The VPN peers agree upon the hash they will use, either MD5 or SHA- 1, while the VPN is being set up.
  2. When a message is transmitted from one peer to the other, a hash is made up using the original message and the shared-secret encryption key, and is appended to it.
  3. When the receiving peer receives the message, it hashes the message with the agreed-upon hash and the shared encryption key and compares it with the hash that is appended to the message. If the computed hash differs from the one that is appended to the message, the peer infers that the message must have been altered in transit.

Authentication: PSKs, RSA Signatures, or RSA Encrypted Nonces

This completes the IPsec C-I-A triad. Again, we have yet to look at how these separate elements, Confidentiality, Integrity, and (now) Authentication are brought into play during the VPN establishment. Suffice it to say, they constitute the categories of modular, swappable plug-ins that can fit in the IPsec framework and that the VPN peers negotiate while the VPN is being set up. Peers can be authenticated (their identity verified) using one of three methods:

  • Pre-Shared Key (PSK). A simple password, manually configured in the VPN peers. This is the simplest form of authentication and therefore quite common, but is not very scalable.
  • RSA Signature. Peers are authenticated through the exchange of digital certificates. Each peer has a key pair made up of a private and public key contained in a certificate that has been issued by a certificate authority. Peer A creates a hash and encrypts it with its private key. This encrypted hash signature is appended to a message. Peer B decrypts the encrypted hash with Peer A’s public key (it is public after all!). The decrypted hash is compared with a recomputed hash. If they are the same, the signature is genuine.
  • RSA Encrypted Nonces. Least common of the three authentication methods. Public keys of every peer have to be manually configured on all the peers as part of the configuration process. These keys are used with a nonce, which is a random number generated by the peer to create an encrypted value that is used to validate a peer’s identity.
The pre-shared key (PSK) has nothing whatsoever to do with the shared-secret encryption key that we discussed in the preceding two sections.

IPsec Strengths

In every mode of operation, the payload of the IP packets transmitted between the peers is encrypted and not the IP packet header, where the routing information resides. Because IPsec is just cargo (same as TCP, UDP, ICMP, and so on) from the perspective of an IP network, implementing IPsec is transparent. Other strengths of IPsec include the following:

  • Data Link Layer independent and Application Layer independent. Because IPsec operates at the network layer, it is supported by all data link layer protocols.
  • Extremely scalable. With careful design, IPsec can scale to very large networks.

Constructing a VPN: Putting it Together

Before we start looking at the details, we should look at IPsec at a high level. It’s easy to over-simplify and speak of a single VPN tunnel. In reality, IPsec constructs two tunnels called Internet Key Exchange (IKE) security associations (SAs):

  • IKE Phase I SA. All negotiation and authentication between VPN peers occurs in this security association.
  • IKE Phase II SA. Data is transformed (encrypted, verified, authenticated) and transmitted in this security association. Figure 7.4 depicts a high-level overview of IKE Phase I and IKE Phase II security associations.

Conceptualizing a Site-to-Site IPsec VPNFIG7.4

If you understand how the File Transfer Protocol (FTP) works, then you also have a good idea about how IPsec VPNs work. As with FTP, IPsec creates two security associations; one that manages the control plane (Phase I) and another which manages the data plane (Phase II).

IKE Phase I: The IKE Protocol

Negotiating what? Authenticating what? No surprise that IKE Phase I is where all the separate components of the IPsec framework are negotiated between the peers. Encryption algorithms, encryption keys, HMACs, authentication method, and other parameters are negotiated between the peers. Even how to negotiate is

negotiated! Once the peers agree on the parameters for the Phase I SA, the peers authenticate to one another and Phase I completes. Phase II negotiation can now occur. A good way to remember the separate IPsec policy elements, which are negotiated during Phase I, to remember the acronym HAGLE. Of course, this is a (misspelled) play on words because “haggle” is slang for negotiate and that’s what happens during Phase I.

Table 7.2 explains the separate elements of the HAGLE acronym.

This is an excellent memory aid. Make sure you memorize it and the choices inferred by each of the letters in HAGLE.

Conceptualizing a Site-to-Site IPsec VPNtb7.2

The IPsec policy elements in the table should look familiar by now. Let’s break down what happens during IKE Phase I a bit further by looking at a list of the
negotiation and authentication tasks that IPsec uses IKE to accomplish:

  • Automating the encryption key exchange process by:
  • Negotiating SA parameters.
  • Automating key generation.
  • Automating key refreshing.
  • Providing for manual configuration.
  • IKE specifies three modes (in order) for negotiating parameters and establishing security associations:
  • Main mode (MM): Negotiating IKE Phase I SA (see the following note).
  • Aggressive mode (AM): Negotiating IKE Phase I (see the following note).
  • Quick mode (QM): Negotiating IKE Phase II.
As was pointed out a bit earlier, IKE is responsible for determining the rules that will be followed during negotiation. Main mode and aggressive mode are two different protocols for negotiating Phase I parameters and leading to the forming of a IKE Phase I SA. They are like Robert’s Rules of Order for IPsec. One or the other is chosen at the beginning of IKE Phase I negotiation and establishes the rules that subsequent negotiation are bound to follow.
Some vendors conduct their own, proprietary, exchanges after MM or AM complete. Cisco’s Mode Configuration, which is used with Easy VPN, is an example of this. This is outside the scope of this Exam Cram.
As its name implies, aggressive mode specifies fewer exchanges than main mode. Each exchange has fewer packets so more negotiating is done in a shorter timeframe that equates to more efficiency. The security tradeoff is that aggressive mode is considered less secure than main mode by some security experts.

Figure 7.5 diagrams a main mode exchange between two VPN peers, Peer A and B, which in this example are both IOS VPN routers.

Conceptualizing a Site-to-Site IPsec VPNFIG7.5
As can be seen, main mode uses three two-way exchanges:

  • Exchange One. VPN peers propose and agree upon encryption algorithms, authentication method, and HMACs that will be used to secure subsequent IKE communications. This is where HAGLE occurs. The elements of HAGLE are called policy sets.
  • Exchange Two. VPN peers employ Diffie-Hellman (DH) to create share-secret encryption keys. This share-secret encryption key is the base for generation of all other encryption and authentication keys. A secure channel (the SA) is now established.
  • Exchange Three. Authentication occurs. If successful, the SA remains up; otherwise, it is torn down by the initiator.

After these three exchanges complete successfully, a secure and authenticated communication path (that is, the Secure Association) is established that is used for all subsequent exchanges between the peers. It’s useful to note that exchange three is secure because the initiator will not send its own authenticators (PSK, RSA digital signature, or RSA encrypted nonce) until it has verified the responder’s.

Breaking it down a bit more, here is additional information you need to know about what happens inside main mode exchanges one and two.

Exchange One

The separate elements of the IKE proposals (HAGLE) are grouped in policy sets. A VPN peer may have several policy sets, numbers indicating their priority, in a policy suite. Why not just one policy set, you might ask? A single VPN peer might need several policy sets because it is configured to support several
other VPNs, both site-to-site and remote-access. Each one of those other VPN peers may require a different policy set for successful VPN establishment.

refer to the amount of keying material that results. The more keying material, the more secure (unbreakable) the encryption keys are that are generated for use
with encryption algorithms. Cisco IOS software supports four DH groups:

  • DH Group 1 (DH1). 768 bits, usually chosen with DES encryption algorithm.
  • DH Group 2 (DH2). 1024 bits, usually chosen with 3DES encryption algorithm.
  • DH Group 5 (DH5). 1536 bits, usually chosen with AES encryption algorithm.
  • DH Group 7 (DH7). 163-bit elliptic curve cryptography (ECC), usually
    chosen with low CPU-powered devices such as PDAs and therefore typically only remote-access VPNs.

IKE Phase II: Quick Mode

At a high level, IKE Phase II performs these functions:

  • Negotiates IPsec Transform sets.
Transform Sets Versus Policy Sets
IKE Phase II Encryption algorithms and HMACs are grouped in something called a transform set. Because the transform set dictates how the data is transformed through the Phase II SA, this should be easy to remember for an exam. This also implies something quite correct, which is that the encryption algorithm and HMACs in Phase II transform sets are independent (and are often different) than those negotiated in IKE Phase I policy sets.
  • Establishes IPsec SAs (at least two; see Figure 7.6).
  • Periodically renegotiates IPsec SAs.
  • (Optional) Performs an additional DH key exchange.

Remember that all negotiating and authentication occurs within IKE Phase I. So, where do you think IKE Phase II negotiations occur? They are conducted in the secure channel established during IKE Phase I negotiations (AM or MM) and use Quick Mode (QM). For example, during QM, policies such as the following are negotiated:

  • Which networks will be protected by the VPN (for example, Net A’ <—> Net B’).
  • Encryption algorithm for the Phase II SA.
  • HMACs.
  • Transport mode versus tunnel mode (discussed in the “Transport Mode Versus Tunnel Mode” section later).
  • ESP versus AH.
    Figure 7.6 depicts IKE Phase II SAs. Using the FTP analogy again, this is the data plane of the IPsec VPN and is protected by the encryption algorithms contained in the transform sets.

Conceptualizing a Site-to-Site IPsec VPNFIG7.6

Interestingly, an IKE Phase II SA is unidirectional, meaning that you need two SAs for bidirectional data flow between VPN peers. This is indicated in Figure 7.6. The SAs are identified by their Security Parameter Index (SPI) numbers. For example, once the VPN was established, Peer B would have SPI AB as an inbound SPI and SPI BA as an outbound SPI in its Security Association Database (SAD).

We now examine some of IPsec’s data transport features, protocols, and algorithms.

Authentication Header (AH)

If your VPN solution does not require confidentiality, you can choose Authentication Header (AH) alone for transport of IPsec data. AH enables you to choose algorithms to assure data integrity and data authentication but not confidentiality. No C-I-A here if you use AH alone! AH is most often used by ESP to provide for optional data integrity and data authentication and not used by itself. If AH is used by itself, AH is encapsulated in an IP packet as protocol 51.

AH features are as follows:

  • Assures data integrity.
  • Origin authentication (ensures that all the 1’s and 0’s came from an authenticated peer).
  • Uses a keyed-hash mechanism.
  • No provision for encryption.
  • Anti-replay protection.

Encapsulating Security Payload (ESP)

If your VPN solution requires confidentiality, you must choose Encapsulating Security Payload (ESP) for transport of IPsec data. As indicated previously, AH can be used with ESP to provide for data integrity and data authentication; thus covering all the letters in C-I-A. ESP is encapsulated in an IP packet as protocol 50. ESP features are as follows:

  • Data confidentiality through encryption.
  • Assures data integrity.
  • Origin authentication.
  • Anti-replay protection.

Referring to Figure 7.7, an IP packet with source address A’ and destination address B’ arrives at peer A. Peer A applies C-I-A services to the packet as required

Conceptualizing a Site-to-Site IPsec VPNFIG7.7

Depending on the transform sets negotiated between the peers, not all the facets of C-I-A will be applied to the packet that will travel in the IPsec VPN between the peers. Assuming that confidentiality will be applied as a minimum, then we must use ESP for encapsulation. This also enables us to optionally apply integrity and authentication services to the packet. If you were to look at the IPsec packet on a protocol analyzer, you will see that the order of the extra fields added while encapsulating the original IP packet are C (ESP Header), I (ESP Trailer), and A (ESP Authentication Trailer) = C-I-A.

  • C. Peer A encrypts the packet and places it into a new IP packet, prepending the encrypted payload with an ESP header. This provides for confidentiality.
  • C and I. If integrity assurance is required, Peer A will add an integrity checksum (a hash) in the form of an ESP trailer before encrypting the original packet and placing it in the payload of the new IP packet.
  • C and I and A. If origin authentication is required, an authentication checksum (similar to a frame check sequence in a data link layer frame) will be added to the payload of the new IP packet but will not be encrypted.
Have you noticed that the order of the fields in the IPsec payload of the IP packet is CI-A, making it simple to remember?

Tunnel Mode Versus Transport Mode

The example described with Figure 7.7 uses tunnel mode. Simply put, all the original IP packet from A’ to B’ is encrypted, including the IP header where the source and destination addresses reside, before it is tunneled inside a new IP packet. Thus, we have layer 3 inside layer 3 and that equals a tunnel. The new (or outer) IP packet has the routing information necessary to route the packet across the Internet.

In some circumstances, it is either not desirable or needed to wrap the IPsec payload (either AH or ESP) inside a new IP packet. In this case, you may opt for transport mode. With transport mode, only the original IP packet’s payload (example: TCP) is encrypted, leaving the original IP header intact. If the IP addresses in the original IP packet are private, RFC 1918, addresses this might be problematic, except where the VPN between peers A and B are connected across a private intranet. There is less overhead associated with transport mode, but it isn’t very common, particularly where the VPN peers are connected across the Internet, where tunnel mode is pretty much mandatory

About the author


Leave a Comment