Introduction to Cisco IOS IPSec

Extranet VPNs Extranet VPNs allow an organization’s suppliers, partners, and customers to be connected to the corporate network in a limited way for business-to-business (B2B) communications.

Now you’re interested, huh? And because VPNs are inexpensive and secure, I’m guessing you’re really jonesing to find out how VPNs are created, right? Well, there’s more than one way to bring a VPN into being. The first approach uses IPSec to create authentication and encryption services between endpoints on an IP network. The second way is accomplished via tunneling protocols, allowing you to establish a tunnel between endpoints on a network. The tunnel itself is a means for data or protocols to be encapsulated inside another protocol—clean!

I’m going to go over the first way to create a VPN (using IPSec) shortly, but first, I really want to describe four of the most common tunneling protocols in use:

Layer 2 Forwarding (L2F) Layer 2 Forwarding (L2F) is a Cisco Proprietary tunneling protocol. It was Cisco’s initial tunneling protocol, and it was created for virtual private dial-up networks (VPDNs). VPDNs allow a device to use a dial-up connection to create a secure connection to a corporate network. L2F was later replaced by Layer 2 Tunneling Protocol (L2TP), which is backward compatible with L2F.

Point-to-Point Tunneling Protocol (PPTP) Point-to-Point Tunneling Protocol (PPTP) was created by Microsoft to allow the secure transfer of data from remote networks to the corporate network.

Layer 2 Tunneling Protocol (L2TP) Layer 2 Tunneling Protocol (L2TP) was created by Cisco and Microsoft to replace L2F and PPTP. L2TP merged the capabilities of both L2F and PPTP into one tunneling protocol.

Generic routing encapsulation (GRE) Generic routing encapsulation (GRE) is another Cisco Proprietary tunneling protocol. It forms virtual point-to-point links, allowing for a variety of protocols to be encapsulated in IP tunnels.

Now that you’re clear on exactly what a VPN is and the various types of VPNs available, it’s time to dive into IPSec.

Introduction to Cisco IOS IPSec

Here’s the $64 question: What is IP Security (IPSec)? No, it doesn’t have anything to do with an airport screener. Simply put, IPSec is an industry-wide standard suite of protocols and algorithms that allows for secure data transmission over an IP-based network, and it functions at Layer 3 (the Network layer) of the OSI model.

Did you notice I said, “IP-based networks?” That’s really important, because by itself, IPSec can’t be used to encrypt non–IP traffic. This means that if you run into a situation where you have to encrypt non–IP traffic, you’ll need to create a GRE tunnel for it and then use IPSec to encrypt that tunnel.

218 Chapter 7 Understanding Cisco IOS IPSec Support

IPSec runs by utilizing transforms such as protocols and algorithms that give IPSec its direction. There are five steps to IPSec operation. This section will discuss the IPSec transforms and how IPSec works.

IPSec Support in Cisco Systems Products

Numerous Cisco devices can be utilized in the creation of IPSec VPNs. Cisco routers can be used to create router-to-router VPN solutions and client VPN solutions. When attempting to configure IPSec VPNs on routers, you should really be sure you have the correct feature set.

The CiscoSecure VPN Concentrator series can be used for remote-user VPN access to your network. The PIX Firewall, discussed in the Appendix, “Introduction to the PIX Firewall,” can be used as an endpoint for VPN connections and provide you with the protection of a firewall.

IPSec Transforms

An IPSec transform specifies a single security protocol with its corresponding security algorithm, and without these transforms, IPSec wouldn’t be able to give you its glorious encryption technologies. It’s important to be familiar with these technologies, so let me take a second to define the security protocols and briefly introduce the supporting encryption and hashing algorithms that IPSec relies on.

Security Protocols

The two primary security protocols used by IPSec are Authentication Header (AH) and Encapsulating Security Payload (ESP).

Authentication Header (AH)

The AH protocol provides authentication for the data and IP header of a packet using a one-way hash for packet authentication. It works like this: The sender generates a one-way hash, and then the receiver generates the same one-way hash. If the packet has changed in any way, it won’t be authenticated, and so, it is dropped. Basically, IPSec relies on AH to guarantee authenticity. AH checks the entire packet, but it doesn’t offer any encryption services.

Encapsulating Security Payload (ESP)

No, ESP won’t tell you when the NASDAQ’s gonna bounce back in spades. But ESP does provide confidentiality, data origin authentication, connectionless integrity, anti-replay service, and limited traffic-flow confidentiality by defeating traffic-flow analysis. Which is almost as good!

There are four components of ESP:

Confidentiality Confidentiality is provided through the use of symmetric encryption algorithms such as DES or 3DES. Confidentiality can be selected separately from all other services, but the confidentiality selected must be the same on all endpoints of your VPN.

Data origin authentication and connectionless integrity Data origin authentication and connectionless integrity are joint services offered as an option in conjunction with the likewise optional confidentiality.

Anti-replay service You can only use the anti-replay service if data origin authentication is selected. Anti-replay election is based on the receiver, meaning that the service is effective only if the receiver checks the sequence number. In case you were wondering, a replay attack occurs when an attacker snags a copy of an authenticated packet and later transmits it to the intended destination. When the duplicate, authenticated IP packet gets to the destination, it can disrupt services or cause other ugly consequences. The Sequence Number field is designed to foil this type of attack.

Traffic flow For traffic flow confidentiality to work, you have to have tunnel mode selected. Tunnel mode is most effective if it’s implemented at a security gateway where tons of traffic amass—a condition that can mask the true source-destination patterns of bad guys trying to breach your network’s security.

Although both confidentiality and authentication are optional, at least one of them must be selected.

Encryption and Hashing Algorithms

Encryption algorithms are used to encrypt and decrypt data, and there’s a large number of both encryption and hashing algorithms available for IPSec. A hashing algorithm is used to create a (surprise) hash—a one-way encryption algorithm that takes an input message of random length and creates a fixed-length output message.

Cisco actually uses a hash variant known as Hash-Based Message Authentication Code (HMAC), which provides an extra level of hashing. This section will discuss Data Encryption Standard (DES), Triple DES, and some variations on the HMAC algorithms.

Data Encryption Standard (DES)

Data Encryption Standard (DES) is known as a symmetric key algorithm, meaning that a single key is used to encrypt and decrypt data. DES utilizes cipher block chaining (CBC) to connect a series of cipher blocks for encrypting data. It uses a 64-bit fixed-length cipher block and a 56-bit key, stored as a 64-bit (eight-octet) quantity with the least significant bit of each octet used as a parity bit. The key used by DES is the same on both sides of the connection. DES functions as follows:

1.The data to be encrypted arrives at the device.

2.DES uses its 56-bit key to encrypt the data.

3.The encrypted data is transmitted to its destination.

4.The encrypted data is received by the decrypting device.

5.The decrypting device uses its 56-bit DES key to decrypt the data.

220 Chapter 7 Understanding Cisco IOS IPSec Support

Triple DES (3DES)

Do you need more security muscle? For a bigger bouncer, try Triple DES (3DES). It’s a much beefier version of DES; the main difference is in how traffic encryption takes place. Instead of encrypting data only once before sending it, 3DES does the following:

1.3DES encrypts the traffic using one 56-bit key.

2.3DES then decrypts the traffic using another 56-bit key.

3.3DES then encrypts the traffic once more with another 56-bit key and finally sends the traffic to its destination.

So basically, it’s the number of times that 3DES encrypts and decrypts traffic before sending it that makes it such a force, and so much stronger than DES.

Hash-Based Message Authentication Code-Message Digest 5 (HMAC-MD5)

Hash-Based Message Authentication Code-Message Digest 5 (HMAC-MD5), also known as HMAC-MD5-96, is a hashing algorithm that creates a 128-bit secret key. It works by producing a 128-bit authentication value that is truncated using the first 96 bits—hence the name HMAC- MD5-96. This truncated value is then inserted into the authenticator field of AH or ESP and sent to the peer. The peer then computes its own 128-bit authentication value and compares the first 96 bits of it to the truncated value stored in the authenticator field of the packet it just received. If the values match, the device is authenticated.

Hash-Based Message Authentication Code-Secure Hash Algorithm-1 (HMAC-SHA-1)

Hash-Based Message Authentication Code-Secure Hash Algorithm-1 (HMAC-SHA-1), also known as HMAC-SHA-1-96, is HMAC-MD5’s big brother. It’s a stronger hashing algorithm that creates a 160-bit authenticator value, which is truncated at 96 bits. It works similarly to HMAC-MD5 in that the truncated value is then stored in the AH or ESP authenticator field of the packet and sent to the peer. The peer then computes its own 160-bit authenticator value and compares the first 96 bits of it to the truncated value stored in the authenticator field of the packet it has just received. If the values match, the device is authenticated.

IPSec Operation

I’ll be going into greater detail on all the components of IPSec in a bit, but for now, understand that the operation of IPSec can be broken down into five steps:

1.IPSec process initiation

2.IKE phase 1

3.IKE phase 2

4.Data transfer

5.IPSec tunnel termination

IPSec Process Initiation

IPSec process initiation does pretty much what the name implies—it initiates the operation of IPSec. It works a lot like making a phone call. First, you decide who you’re going to call, and then you look up that person’s phone number. With IPSec, who is the traffic that needs to be encrypted, and the phone number is where that traffic encryption needs to take place. Once those decisions have been made, a policy specifying the traffic to encrypt needs to be manually created and then applied to the devices that will form the VPN and encrypt the traffic. These devices are known as IPSec peers. So what happens after the policy has been implemented? The answer is simple—when traffic that needs to be encrypted is detected on one of the IPSec peers, IKE phase 1 negotiation begins.

IKE Phase 1

Right now, all you need to know about IKE (Internet Key Exchange) is that it’s used to form the IPSec encrypted tunnel. I’ll discuss IKE in much more detail later in this chapter. IKE phase 1 is the term used to describe the process of determining your IKE policy. During this second step of IPSec operation, the goal is to authenticate IPSec peers and to form the IKE tunnel.

What’s actually happening in IKE phase 1? Well, first, IKE security associations (SAs) are negotiated on the IPSec peers. The IKE SAs are used to specify the type of peer authentication and which Diffie-Hellman (DH) group to use.

Next, an authenticated Diffie-Hellman Key Agreement (discussed later in this chapter) of matching keys is used to authenticate and protect the identities of IPSec peers. And finally, the IKE tunnel is formed for IKE phase 2 negotiation.

It’s also worth mentioning that IKE can use either main mode or aggressive mode for phase 1 negotiation:

Main mode In main mode, IKE uses three two-way handshakes for phase 1 negotiation:

1.During the first exchange, the security algorithms IKE will use are decided on for the IKE SAs. The security algorithms are specified in the IKE policy that has been configured on the IPSec peers. (Chapter 8 discusses the configuration of IKE policies.)

2.During the second exchange, the Diffie-Hellman Key Agreement is used to generate the shared keying information, which is then used to generate the shared secret keys. Finally, the shared secret keys are used to validate the IPSec peers’ identities.

3.During the third exchange, the peer device’s identity is verified.

Aggressive mode Aggressive mode is faster than main mode because it uses only the following two exchanges:

1.During the first exchange, aggressive mode performs both steps 1 and 2 of main mode.

2.During the second exchange, the receiving peer sends back information that is needed to complete the exchange. Finally, the initiator sends back a confirmation.

As you can see, IPSec peer authentication is a priority of IKE phase 1. The completion of IKE phase 1 signals the beginning of the next step, IKE phase 2.

222 Chapter 7 Understanding Cisco IOS IPSec Support

IKE Phase 2

The third step in the IPSec operation is called IKE phase 2, which is the process of creating the IPSec policy. The final outcome of IKE phase 2 is the negotiation of IPSec SAs. An IPSec SA is a unidirectional connection established between IPSec peers and is used to determine the IPSec services that will be offered. To say this another way, the IPSec SA specifies the type of encryption and IPSec services that are offered in one direction. Because an IPSec SA is unidirectional, two SAs must be set up: one from the sender to the receiver and one from the receiver to the sender. The IPSec SAs operate over the secure IKE tunnel that was set up in IKE phase 1.

Figure 7.1 illustrates two devices setting up SAs with one another.

F I G U R E 7 . 1 Two devices setting up security associations

R1 R2

SA

SA

Let me explain the process illustrated in Figure 7.1:

1.R1 sends an SA to R2.

2.R2 accepts the SA.

3.R2 sends the SA back to R1.

4.Two unidirectional SAs are set up.

It’s important to remember that a device has an SA for every IPSec device it peers with. These SAs are stored in the devices’ security association database (SAD) and are indexed by their Security Parameter Index (SPI).

The SPI is a unique identification mechanism for each SA on a device. When an IPSec packet arrives, the device checks the SPI contained in the packet and compares the SPI in the packet to the SPIs in the device’s SAD to determine which IPSec policy is in effect.

Also, make note of the fact that each SA has a unique triple identity consisting of an SPI, an IP destination address, and a security protocol (AH or ESP) identifier. The completion of this step is marked by the formation of an IPSec tunnel. The IPSec tunnel is then used to transport the encrypted traffic.

Data Transfer

Finally, we’ve reached the step in which traffic will begin to flow. Once the IPSec SAs have been negotiated and the IPSec tunnel has been formed, traffic can begin passing over the IPSec tunnel. The traffic that’s allowed to enter the IPSec tunnel is encrypted and decrypted based on the information contained within the IPSec SA.