VPN Interview Questions and Answers: Core IPsec Architecture

Virtual Private Network (VPN) creates a secure network connection over a public network such as the internet. It allows devices to exchange data through a secure virtual tunnel. It uses a combination of security features like encryption, authentication, tunnelling protocols, and data integrity to provide secure communication between participating peers.

• Authentication: It verifies that the packet received is actually from the claimed sender. In short, it verifies the authenticity of the sender. Pre-shared Key, Digital Certificates are some methods that can be used for authentication.
• Integrity: It ensures that the contents of the packet have not been altered or modified in between by man-in-the-middle. Hashing algorithm includes MD5, SHA.
• Confidentiality: It encrypts the message content through encryption so that data is not disclosed to unauthorized parties. Encryption algorithms include DES (Data Encryption Standard), 3DES (Triple-DES), AES (Advanced Encryption Standard).

In Symmetric encryption, a single key is used both to encrypt and decrypt traffic. It is also referred to as a shared key or shared secret encryption. Symmetric encryption algorithms include DES, 3DES, AES.

In Asymmetric encryption two keys are used to encrypt and decrypt traffic, one key is used for encryption and other key is used for decryption. The most common asymmetric encryption algorithm is RSA.

Authorization is a security mechanism used to determine user privileges or access levels related to network resources, including firewalls, routers, switches and application features. Authorization is normally preceded by authentication and during authorization, the system verifies an authenticated user's access rules and either grant or refuse resource access.

Digital signature is an attachment to an electronic message used for security purposes. It is used to verify the authenticity of the sender. The digital signature allows a computer or organization to safely exchange information over the internet.

IP Security Protocol VPN means VPN over IP Security. It allows two or more users to communicate in a secure manner by authenticating and encrypting each IP packet of a communication session. IPSec provides data confidentiality, data integrity and data authentication between participating peers.

IPSec secures traffic at the Layer 3 (Network Layer) of the OSI model.

IPSec offers the following security services:
1. Peer Authentication.
2. Data confidentiality.
3. Data integrity.

IPSec only supports unicast IP traffic.

A site-to-site VPN allows offices in multiple locations to establish secure connections with each other over a public network such as the Internet.

Remote Access VPN allows users to connect to the company's network from a remote location through a secure tunnel that is established over the Internet. The remote user is able to access internal, private web pages and perform various network tasks. There are two primary methods of deploying Remote Access VPN:
1. Remote access IPSec VPN.
2. Remote access Secure Sockets Layer (SSL) VPN.

In remote access VPN, only the client can initiate the connection. In site to site VPN, anyone can initiate the connection.

1. Authentication Header (AH).
2. Encapsulating Security Payload (ESP).
3. Internet Key Exchange (IKE).

1. Encapsulating Security Payload (ESP) - It is an IP-based protocol which uses port 50 for communication between IPSec peers. ESP is used to protect the confidentiality, integrity, and authenticity of the data and offers anti-replay protection. Drawback - ESP does not provide protection to the outer IP Header.

2. Authentication Header (AH) - It is also an IP-based protocol that uses port 51 for communication between IPSec peers. AH is used to protect the integrity and authenticity of the data and offers anti-replay protection. Unlike ESP, AH provides protection to the IP header also. Drawback - AH does not provide confidentiality protection.

Both ESP and AH protocols provide anti-replay protection based on sequence numbers. The sender increments the sequence number after each transmission and the receiver checks the sequence number and rejects the packet if it is out of sequence.

It is a hybrid protocol that implements Oakley and SKEME key exchanges inside the Internet Security Association and Key Management Protocol (ISAKMP) framework. It defines a mechanism for creating and exchanging keys. IKE derives authenticated keying material and negotiates SAs like hashing algorithm, encryption mechanism, Diffie-Hellman group, authentication method.

IKE uses UDP port 500.

The SAs define the protocols and algorithms to be applied to sensitive packets and specify the keying material to be used by the two peers. SAs are unidirectional and are established per security protocol (AH or ESP).

Security Association Database stores the IPSec security associations and Security Policy Database stores the IPSec Policies. As soon as the crypto map is applied to an interface, two databases are created for that interface. These databases are used to determine whether a packet belongs to interesting traffic (VPN) or not and the corresponding encryption and authentication information to use.

SAD holds the information such as SPI, secret keys, IPsec Profile for the SA, IPsec mode (Tunnel or Transport), Peer address, lifetime and sequence counters. SPD holds policy information, interesting traffic information and SPIs.

An outbound packet is first examined against the SPD and if it belongs to the interesting traffic (should go via VPN), the packet is assigned to the SA and the details for SAD are applied next. If the packet is incoming (inbound), the SPI in either the AH or ESP header is used to determine the entry in SAD first and then the SPD is checked for policy information and so on.

SPI is a 32-bit random number which is used by the destination device to identify which SA (security association) to use in order to check the security of the received packet. It acts like an identification tag added to the header while using IPsec VPN for sending the IP traffic. Basically, The SPI is used to map the incoming packet to a SA at the destination device.

IKE is a two-phase protocol:

Phase 1:
1. It protects the phase 1 communication itself (using crypto and hash algorithms).
2. It generates a session key using Diffie-Hellman groups.
3. Peers will authenticate each other using pre-shared, public key encryption, or digital signature.
4. It also protects the negotiation of phase 2 communication.
There are two modes in IKE phase 1: Main mode (six messages) and Aggressive mode (three messages). At the end of phase 1, a bidirectional ISAKMP/IKE SA (phase 1 SA) is established for IKE communication.

Phase 2:
IKE phase 2 protects the user data and establishes a security association for the IPSec tunnel. There is one mode in IKE phase 2: Quick mode. In this mode, three messages are exchanged to establish the phase 2 IPSec security associations. At the end of phase 2 negotiations, two unidirectional IPSec SAs (Phase 2 Security Associations) are established for user data—one for sending and another for receiving encrypted data.

Total of six messages are exchanged in the main mode. Before the message exchange takes place both initiator and responder will calculate the cookies which will be used as a session identifier. A cookie is created by performing hash of the source IP address, destination IP address, source port, destination port, locally generated random number, time and date:
`Cki = md5{S.IP, D.IP, S.Port, D.Port, random no, time, date}`
`Ckr = md5{S.IP, D.IP, S.Port, D.Port, random no, time, date}`

1. The first message is sent from initiator to responder. It contains the initiator's cookie, responder's cookie is set to zero and IKE policy proposal or security associations like Encryption mechanism, Hash algorithm, Authentication method, Diffie-Hellman group and lifetime of IKE SAs.
2. Upon receiving the first message from the initiator, the responder will try to match the received IKE policies with its own locally configured IKE policies. Assuming the responder finds a match, it responds with the second message which contains the IKE policies accepted basically the SAs with which the responder agrees. It also contains both the initiator's and responder' cookie.
Now both the peers will calculate their DH public key and nonce value. The initiator will calculate its DH public key: `Xi = G^pr_i mod p`. Similarly, the responder calculates: `Xr = G^pr_r mod p`. Both calculate the nonce value which is a very large random number used in keying material.
3. The third message contains the Diffie-Hellman public key of initiator in key exchange payload and nonce value of initiator in the nonce payload. On receiving this responder will calculate its shared secret: `SSr = Xi^pr_r mod p`.
4. The responder sends its DH public key value and nonce value in the fourth message. The initiator calculates its shared secret: `SSi = Xr^pr_i mod p`. Both peers now have a shared secret which should be the same. After this both the peers will calculate three session keys using a shared secret, nonce value and pre-shared key.
`SKEYID = PRF (Preshared key, Ni/Nr)`
`SKEYID_d = PRF(SKEYID, shared secret, Cki, CKr, 0)`
`SKEYID_a = PRF(SKEYID, SKEYID_d, shared secret, Cki, CKr, 1)`
`SKEYID_e = PRF(SKEYID, SKEYID_a, shared secret, Cki, CKr, 2)`
Up till now, all messages were clear text. Now subsequent messages are encrypted.
5. The initiator sends hash and identification values. Identification payload basically contains the information about the identity of initiator like IP address or Hostname. Hash is used for authentication purpose. Message 5 was encrypted using SKEYID_e.
6. The sixth message is sent from responder to initiator and contains the identity of responder and hash of responder. If the generated hash matches, authentication succeeds.

Three keys are generated at the end of message 4 and will be used to authenticate two peers to each other as well as to encrypt subsequent IKE message exchange:
1. SKEYID_d - used to calculate subsequent IPSec keying material.
2. SKEYID_a - used to provide data integrity and authentication to subsequent IKE messages.
3. SKEYID_e - used to encrypt subsequent IKE messages.

Aggressive mode is faster than main mode but less secure. Only three messages are exchanged in phase 1.

1. The first message contains initiator's cookie, IKE Policies, the nonce value of initiator, DH public key of initiator, ID_i (identity of initiator). On receiving this, the responder generates the shared secret and then the session keys. Having generated the session keys it uses the SKEYID to generate hash_r for authentication.
2. The second message contains both initiator and responder cookie, accepted IKE policy, the nonce value of responder, DH public key value of responder, ID_r (identity of responder), and hash_r used by the initiator to authenticate responder. Message 2 is only hashed but not encrypted. Upon receiving message 2, the initiator calculates the shared secret and session keys. Then it checks hash_r to authenticate the responder and creates hash_i.
3. The third message contains the hash_i which allows the responder to complete authentication.

It is less secure because in aggressive mode, both the peers exchange their identities in the first two messages which are unencrypted.

Three messages are exchanged in quick mode. All messages are encrypted and hashed with the keys generated in phase 1.

1. The first message contains a hash, IPSec proposals, a nonce, optionally another Diffie-Hellman public value, and identities. Hash authenticates the message. Proposals specify protocol (ESP/AH), mode (tunnel/transport), encryption, and hashing. DH public value is sent if PFS is set. Nonce prevents replay attacks. Identities (proxy identities) define what traffic should be encrypted.
2. The second message contains the hash, accepted IPSec Proposals, Nonce value, DH value, and identities serving the same purpose.
3. The third message is sent by the initiator containing a hash to verify that the responder is alive.

Tunnel mode: Protects data in network-to-network or site-to-site scenarios. In Tunnel mode, which is the default mode on Cisco routers, the original source and destination IP addresses are encrypted and an ESP header is added followed by a new IP header.

`[New IP HDR] [ESP HDR] [IP HDR] [Data] [ESP Trailer] [ESP Authentication]`

Transport mode: Protects data in host-to-host or end-to-end scenarios. In transport mode, no new IP header will be added because the tunnel is in between public IP's unlike tunnel mode where the tunnel is between private IPs. In transport mode, the IP header is not protected (IPsec protects the payload of the original IP datagram by excluding the IP header). IPSec protocols AH and ESP can operate either in tunnel or transport mode.

`[IP HDR] [ESP HDR] [Data] [ESP Trailer] [ESP Authentication]`

DH is a public-key cryptography protocol that allows two parties to establish a shared secret over an insecure communication channel. Diffie-Hellman is used within IKE to establish session keys and is a component of Oakley.

Each side has a private key which is never passed and a Diffie-Hellman Public Key used for encryption. When both sides want to do a key exchange they send their Public Key to each other. For example, Side A gets the Public Key of Side B, then using RSA it creates a shared key which can only be opened on Side B with Side B's Private Key. Even if somebody intercepts the shared key, they cannot reverse engineer it as only the private key of Side B can open it.

• Transform set: An IKE transform set is a combination of security protocols and algorithms. During the IPSec SA negotiation, the peers agree to use a particular transform set to protect a particular data flow.
• Crypto access lists: Crypto access lists specify which IP traffic is protected by crypto and which traffic is not protected by crypto. To protect IP traffic "permit" keyword is used in an access list. If the traffic is not to be protected then "deny" keyword is used.
• Crypto map: Used to pull together the various parts used to set up IPSec SAs including: 1. Traffic to protect (crypto access list), 2. Where traffic should be sent (remote peer), and 3. What IPSec SA should be applied (transform sets). Multiple interfaces can share the same crypto map set in case we want to apply the same policy to multiple interfaces. If more than one crypto map is created for a given interface then use the sequence number of each map entry to rank the map entries, the lower the seq-num argument the higher the priority.

Static Crypto Maps are used when peers are predetermined. It is basically used in IPSec site to site VPNs.

Dynamic crypto maps are used with networks where the peers are not always predetermined. It is basically used in IPsec Remote Access VPNs.

Normally for phase 2 key is regenerated based on SKEY_ID (Session key ID) of phase 1 but if we will enable PFS then again Diffie-Hellman will take place and totally new keying material will be generated for phase 2.

IPSec VTI is the concept of using a dedicated IPSec interface called IPSec Virtual Tunnel Interface for highly scalable IPSec-based VPNs. IPSec VTI provides a routable interface for terminating IPSec tunnels. VTI also allows the encrypting of multicast traffic with IPSec.

We use following commands to check the status of tunnel phases:
Phase 1: show crypto isakmp sa
Phase 2: show crypto ipsec sa