
- •Table of Contents
- •Preface
- •What This Book Covers
- •What You Need for This Book
- •Conventions
- •Reader Feedback
- •Customer Support
- •Errata
- •Questions
- •The Need for Cryptography
- •Privacy
- •Security
- •A History of the Internet
- •Holding the Internet Together
- •The Creation of ICANN
- •ICANN Bypassed
- •The Root Name Servers
- •Running the Top-Level Domains
- •History of Internet Engineering
- •The Internet Engineering Task Force (IETF)
- •RFCs—Requests For Comments
- •IETF and Crypto
- •The War on Crypto
- •Dual Use
- •Public Cryptography
- •The Escrowed Encryption Standard
- •Export Laws
- •The Summer of '97
- •The EFF DES Cracker
- •Echelon
- •The End of the Export Restrictions
- •Free Software
- •Free as in Verifiable
- •The Open Source Movement
- •The History of Openswan
- •IETF Troubles over DNS
- •Super FreeS/WAN
- •The Arrival of Openswan
- •NETKEY
- •Further Reading
- •Using Openswan
- •Copyright and License Conditions
- •Writing and Contributing Code
- •Legality of Using Openswan
- •International Agreements
- •International Law and Hosting Openswan
- •Unrecognized International Claims
- •Patent Law
- •Expired and Bogus Patents
- •Useful Legal Links
- •Summary
- •A Very Brief Overview of Cryptography
- •Valid Packet Rewriting
- •Ciphers
- •Algorithms
- •Uniqueness
- •Public-Key Algorithms
- •Exchanging Public Keys
- •Digital Signatures
- •Diffie-Hellman Key Exchange
- •Avoiding the Man in the Middle
- •Session Keys
- •Crypto Requirements for IPsec
- •IPsec: A Suite of Protocols
- •Kernel Mode: Packet Handling
- •Authentication Header (AH)
- •Encapsulated Security Payload (ESP)
- •Transport and Tunnel Mode
- •Choosing the IPsec Mode and Type
- •The Kernel State
- •Encryption Details
- •Manual Keying
- •Final Note on Protocols and Ports
- •Usermode: Handling the Trust Relationships
- •The IKE Protocol
- •Phase 1: Creating the ISAKMP SA
- •Phase 2: Quick Mode
- •The NAT Problem
- •Summary
- •Linux Distributions
- •Debian
- •SuSE
- •Slackware
- •Gentoo
- •Linux 'Router' Distributions
- •Deciding on the Userland
- •Pluto
- •Racoon
- •Isakmpd
- •More Reasons to Pick Pluto
- •Choosing the Kernel IPsec Stack
- •KLIPS, the Openswan Stack
- •ipsecX Interfaces
- •First Packet Caching
- •Path MTU Discovery
- •KLIPS' Downside
- •NETKEY, the 2.6 IPsec Stack
- •The USAGI / SuSE IPsec Stack
- •Making the Choice
- •GPL Compliance and KLIPS
- •Binary Installation of the Openswan Userland
- •Checking for Old Versions
- •Installing the Binary Package for Openswan
- •Building from Source
- •Using RPM-based Distributions
- •Rebuilding the Openswan Userland
- •Building src.rpm from Scratch
- •Openswan Options
- •Building the Openswan Userland from Source
- •Downloading the Source Code
- •Configuring the Userland Tools
- •Optional Features
- •Compile Flags
- •File Path Options
- •Obscure Pluto Options
- •Compiling and Installing
- •Binary Installation of KLIPS
- •Building KLIPS from Source
- •Kernel Prerequisites
- •Identifying your Kernel's Abilities
- •Using Both KLIPS and NETKEY
- •The Kernel Build Options
- •Required Kernel Options
- •Desired Options
- •NETKEY Stack Options
- •KLIPS Stack Options
- •L2TP Options
- •Patching the Kernel
- •NAT-Traversal Patch
- •KLIPS Compile Shortcut
- •Activating KLIPS
- •Determining the Stack in Use
- •Building KLIPS into the Linux Kernel Source Tree
- •Building a Standard Kernel
- •NAT Traversal
- •Patching KLIPS into the Linux Kernel
- •Verifying the Installation
- •Summary
- •Manual versus Automatic
- •PSK versus RSA
- •Pitfalls of Debugging IPsec
- •Pre-Flight Check
- •The ipsec verify Command
- •NAT and Masquerading
- •Checking External Commands
- •Opportunistic Encryption
- •The ipsec livetest Command
- •Configuration of Openswan
- •The ipsec.conf File
- •Host-to-Host Tunnel
- •Left and Right
- •The type Options
- •The auto Option
- •The rsasigkey Options
- •Bringing Up the IPsec Tunnels
- •Listing IPsec Connections
- •Testing the IPsec Tunnel
- •Connecting Subnets Through an IPsec Connection
- •Testing Subnet Connections
- •Testing Properly
- •Encrypting the Host and the Network Behind It
- •Employing Advanced Routing
- •Creating More Tunnels
- •Avoiding Duplication
- •The Also Keyword
- •KLIPS and the ipsecX Interfaces
- •Pre-Shared Keys (PSKs)
- •Proper Secrets
- •Dynamic IP Addresses
- •Hostnames
- •Roadwarriors
- •Multiple Roadwarrior Connections
- •Dynamic IP and PSKs
- •Mixing PSK and RSA
- •Connection Management
- •Subnet Extrusion
- •NAT Traversal
- •Deprecated Syntax
- •Confirming a Functional NAT-T
- •Dead Peer Detection
- •DPD Works Both Ways
- •Configuring DPD
- •Buggy Cisco Routers
- •Ciphers and Algorithms
- •Using ike= to Specify Phase 1 Parameters
- •Using esp= to Specify Phase 2 Parameters
- •Defaults and Strictness
- •Unsupported Ciphers and Algorithms
- •Aggressive Mode
- •XAUTH
- •XAUTH Gateway (Server Side)
- •XAUTH Client (Supplicant Side)
- •Fine Tuning
- •Perfect Forward Secrecy
- •Rekeying
- •Key Rollover
- •Summary
- •X.509 Certificates Explained
- •X.509 Objects
- •X.509 Packing
- •Types of Certificates
- •Passphrases, PIN Codes, and Interactivity
- •IKE and Certificates
- •Using the Certificate DN as ID for Openswan
- •Generating Certificates with OpenSSL
- •Setting the Time
- •Configuring OpenSSL
- •Be Consistent with All Certificates
- •OpenSSL Commands for Common Certificate Actions
- •Configuring Apache for IPsec X.509 Files
- •Creating X.509-based Connections
- •Using a Certificate Authority
- •Using Multiple CAs
- •Sending and Receiving Certificate Information
- •Creating your own CA using OpenSSL
- •Creating Host Certificates with Your Own CA
- •Host Certificates for Microsoft Windows (PKCS#12)
- •Certificate Revocation
- •Dynamic CRL Fetching
- •Configuring CRL
- •Online Certificate Status Protocol (OCSP)
- •Summary
- •History of Opportunistic Encryption
- •Trusting Third Parties
- •Trusting the DNS?
- •OE in a Nutshell
- •An OE Security Gateway
- •DNS Key Records
- •Forward and Reverse Zones
- •The OE DNS Records
- •Different Types of OE
- •Policy Groups
- •Internal States
- •Configuring OE
- •Configuring Policies
- •Full OE or Initiate-Only
- •Generating Correct DNS Records
- •Name Server Updates
- •Verifying Your OE Setup
- •Testing Your OE Setup
- •The trap eroute
- •The pass eroute
- •The hold eroute
- •Manipulating OE Connections Manually
- •Advanced OE Setups
- •Caveats
- •Summary
- •Where to Firewall?
- •Allowing IPsec Traffic
- •NAT and IPsec Passthrough
- •Configuring the Firewall on the Openswan Host
- •Firewalling and KLIPS
- •Firewalling and NETKEY
- •Packet Size
- •Summary
- •Microsoft Windows
- •Layer 2 Tunneling Protocol (L2TP)
- •Assigning an IP for VPN Access
- •L2TP Properties
- •Pure IPsec versus L2TP/IPsec
- •Client and Server Configurations for L2TP/IPsec
- •The L2TP Openswan Server
- •Configuring Openswan for L2TP/IPsec
- •Linux Kernel Runtime Parameters for L2TP/IPsec
- •Protecting the L2TP Daemon with IPsec using iptables
- •Choosing an L2TP Daemon
- •Configuring L2TPD
- •Configuring User Authentication for pppd
- •Microsoft Windows XP L2TP Configuration
- •Microsoft Windows 2000 L2TP Configuration
- •Apple Mac OS X L2TP Configuration
- •Server Configuration for X.509 IPsec without L2TP
- •Openswan Configuration for X.509 without L2TP
- •Client Configuration for X.509 IPsec without L2TP
- •Microsoft's IKE Daemon
- •Microsoft's Certificate Store
- •Clients using Microsoft Native IPsec Implementation
- •The ipsec.exe Wrapper
- •The Linsys IPsec Tool (lsipsectool)
- •Securepoint IPsec Client
- •TauVPN (iVPN)
- •The WaveSEC Client
- •Third-Party Replacement Clients for Windows
- •The GreenBow VPN Client
- •Astaro Secure Client
- •Mac OS X IPSecuritas
- •VPNtracker
- •Manual Racoon Configuration
- •Importing X.509 Certificates into Windows
- •Importing X.509 Certificates on Mac OS X (Tiger)
- •Summary
- •Openswan as a Client to an Appliance
- •Preparing the Interop
- •The Human Factor
- •Terminology
- •Preparation
- •IPsec Passthrough
- •Tunnel Limitations
- •Anticipate Known Problems
- •Update the Firmware
- •GUI Issues
- •Keepalives
- •ISP Filtering
- •Frequently used VPN Gateways
- •Webmin with Openswan
- •Cisco VPN 3000
- •Cisco PIX Concentrator
- •Nortel Contivity
- •Checkpoint
- •WatchGuard Firebox
- •Symantec
- •Frequently used VPN Client Appliances
- •ZyXEL
- •DrayTek Vigor
- •The Vigor Web Interface
- •Windows Logon Issues
- •Other Vigorisms
- •Unresolved Issues
- •NetScreen
- •Known Issues
- •SonicWALL
- •BinTec
- •LANCOM
- •Linksys
- •Lucent Brick
- •NETGEAR
- •KAME/Racoon
- •Aftercare
- •Summary
- •Methods of Encryption
- •Host-to-Host Mesh
- •Host-to-Gateway Setup
- •Single IP Extrusiautomation or L2TP
- •Opportunistic Encryption in the LAN
- •Non-OE-Capable Machines
- •Designing a Solution for Encrypting the LAN
- •Design Goals
- •Separation of WiFi and Crypto
- •Link Layer Protection
- •The Logical Choice: IPsec
- •Hotspot
- •WaveSEC
- •Full WaveSEC
- •Catch 22 Traffic
- •Building a WaveSEC Server
- •DHCP Server Setup
- •DNS Server Setup
- •Openswan Server Setup
- •Catch 22 Traffic Setup
- •Building a WaveSEC Client
- •DH Client Setup
- •Openswan Setup
- •Testing the WaveSEC
- •Starting the WaveSEC Connection
- •Known Issues with WaveSEC
- •WaveSEC for Windows
- •Design Limitations
- •Building a WaveSEC for Windows Server
- •Obtaining the Certificate and Client Software
- •Our Prototype Experiences
- •Openswan Issues
- •Windows Kernel Issues
- •Summary
- •Cipher Performance
- •Handling Thousands of Tunnels
- •Managing Large Configuration Files
- •Standard Naming Convention
- •The also= Parameter
- •The include Parameter
- •Openswan Startup Time
- •Limitations of the Random Device
- •Other Performance-Enhancing Factors
- •Logging to Disk
- •Disable Dead Peer Detection
- •Reducing the Number of Tunnels
- •OSPF Setup
- •BGPv4 Setup
- •High Availability
- •Heartbeat
- •Xen Migration
- •Using Anycast
- •Summary
- •Do Not Lock Yourself Out!
- •Narrowing Down the Problem
- •Host Issues
- •Configuration Problems
- •Connection Names
- •Interoperability
- •Hunting Ghosts
- •Rekey Problems (After an Hour)
- •Openswan Error Messages
- •IKE: Unknown VendorIDs
- •Network Issues
- •Firewalls
- •MTU and Fragmentation Issues
- •Debugging IPsec on Apple Mac OS X
- •Debugging IPsec on Microsoft Windows
- •Oakley Debugging
- •Debugging ipsec.exe
- •Microsoft L2TP Errors
- •You Suddenly Cannot Log in Anymore over the VPN
- •Software Bugs
- •Userland Issues: Assertion Failed or Segmentation Faults
- •Kernel Issues: Crashes and Oopses
- •Memory Issues
- •Common IKE Error Messages
- •Common Kernel-Related Error Messages
- •Common Errors when Upgrading
- •Using tcpdump to Debug IPsec
- •Situation A: No Communication on Port 500
- •Situation B: Failure at Third Exchange
- •Situation C: QUICK Mode Initiates, but Never Completes
- •Situation D: All IKE Messages Occur, but no Traffic Flows
- •A Final tcpdump Example
- •User Mode Linux Testing
- •Preparing the Openswan for the UML Build Process
- •Running the UMLs
- •Writing a UML Test Case
- •Debugging the Kernel with GDB
- •Asking the Openswan Community for Help
- •Internet Relay Chat (IRC)
- •The Openswan Mailing Lists
- •Posting to the Lists
- •Research First, Ask Later
- •Free, as in Beer
- •Do not Anonymize
- •Summary
- •Linux Kernel Developments
- •Kernel API Changes between 2.6.12 and 2.6.14
- •Red Hat Kernel Developments
- •Fedora Kernel Source/Headers Packaging Change
- •MD5 Insecurities
- •Discontinuation of Openswan 1 by the End of 2005
- •Update on UML Testing Suite Installation
- •Openswan GIT Repositories
- •Openswan on Windows and Mac OS X Updates
- •Known Outstanding Bugs
- •Vulnerability Fixes in Openswan 2.4.4
- •The OSI Model and the IP Model
- •No Layers, Just Packets
- •The Protocol
- •IP Network Overview
- •IP Address Management
- •The Old IP Classes
- •Classless IP Networks
- •The Definition of a Subnet
- •Calculating with Subnets: The Subnet Mask
- •The Rest of the Network
- •Linux Networking Commands
- •Routing
- •Routing Decisions
- •Peering
- •Network Address Translation
- •Port Forwarding
- •Openswan Links
- •Community Documentation
- •Generic Linux Distributions Containing Openswan
- •Specialized Linux Distributions Containing Openswan
- •Overview RFCs
- •Basic Protocols
- •Key Management
- •Procedural and Operational RFCs
- •Detailed RFCs on Specific Cryptographic Algorithms and Ciphers
- •Dead Peer Detection RFCs
- •NAT-Traversal and UDP Encapsulation RFCs
- •RFCs for Secure DNS Service, which IPSEC May Use
- •RFCs Related to L2TP, Often Used in Combination with IPsec
- •RFCs on IPsec in Relation to Other Protocols
- •RFCs Not in Use or Implemented across Multiple Vendors
- •Index

Chapter 3
Openswan Options
For almost everyone, the default options and features for Openswan should be fine. At present, all the binary and source RPM packages assume the build only needs to supply RPMs that use the standard options and features. This might not always be the case. Some of the options cannot be set from the spec file at this point, so to change options, you need to either write a patch against Makefile or Makefile.inc, or manually build and install Openswan outside the packaging system. Always check the spec file to see if options can be changed from there.
You can further change these compile-time options by changing the MODULE_DEF_INCLUDE variable to point to a file that differs from the supplied packaging/linus/config-all.h file.
Building the Openswan Userland from Source
Use of a package manager is not essential: Openswan can of course also be built without one. Though this method offers you greater control, there is less protection from unwise decisions. When building from source fails, it will be more difficult to figure out what went wrong and why.
This section covers the process of building Openswan from source for those who do chose this option, explaining some of the advanced options not previously discussed.
Downloading the Source Code
First, we download and verify the source:
#wget http://www.openswan.org/code/openswan-2.4.1.tar.gz
#wget http://www.openswan.org/code/openswan-2.4.1.tar.gz.asc
#gpg --verify openswan-2.4.1.tar.gz.asc
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: Signature made Fri 13 Aug 2004 12:34:30 AM CEST using RSA key ID B7E82DF8 gpg: Good signature from "Openswan Master Signing Key <build@openswan.org>" gpg: checking the trustdb
gpg: no ultimately trusted keys found
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: D450 193B D905 43FE D929 C9C5 0D58 2984 B7E8 2DF8
You should of course check the Openswan website to confirm the currently recommended version. Normally there are full releases, test releases meant only for developers ('dr'), and release candidates ('rc'), which can be tried by both developers and users.
Configuring the Userland Tools
If you build Openswan from source, the default installation directory will be inside /usr/local. This is to distinguish a custom compile from the distribution software, which is typically installed in /usr. It is therefore important to remove any possible *swan package that is installed as part of the distribution. If you do not, older versions might get (partially) used because they appear in the $PATH before /usr/local/sbin. If using the default locations, the ipsec command is installed in
59

Building and Installing Openswan
/usr/local/sbin, and the sub-commands are installed in /usr/local/libexec/ipsec/*. Some
helper applications are installed in /usr/local/lib/ipsec/. Finally, depending on the distribution, the daemon start-stop script (also called ipsec) is installed in /etc/init.d/.
Openswan does not use a GNU-style configure script. Instead, everything is controlled directly by the top level Makefile. Most of the user-configurable options are located in the file Makefile.inc: Makefile itself should not be changed, only Makefile.inc. Unfortunately, Makefile.inc does not just contain those options you can set, but lots of others too. In general, however, the options that are meant to be configurable use the following syntax:
VARIABLE?=value
This is make syntax. The ?= means that if the environment $VARIABLE has not been set, then create one with the specified value, otherwise leave the existing value untouched. This makes it easy to configure the build process without even editing Makefile.inc, as you only need to specify the right shell variables before you run make.
If you are building Openswan on the machine that needs to run it, and it is a full-blown Linux distribution, then you very likely do not need to set or change any of these options anyway.
Optional Features
The following options are supported in Makefile.inc or as shell environment variables:
Option name |
Description |
|
|
USE_LWRES |
Use the ISC BIND version 9 resolv library. This is necessary for |
(true or false) |
various DNSSEC functions, but is not yet available for all Linux |
|
distributions. Enable this if your distribution has a version 9 |
|
BIND package. |
USE_IPROUTE2 |
Use the iproute2 package (and the kernel's advanced routing |
(true or false) |
features). This should only be false for Linux 2.0 and early 2.2 |
|
kernels. |
IPSEC_FIREWALLTYPE |
This should be set to the type of firewall commands that are to |
(iptables, ipchains, or ipfwadm) |
be used. On modern distributions this is iptables. On older |
|
kernels or distributions, this can be ipchains or ipfwadm. |
USE_IKEPING |
This option compiles a small test program called ikeping. This |
(true or false) |
should only be disabled for small embedded devices. |
USE_KEYRR |
This option adds support for the DNS KEY record. See Chapter 6. |
(true or false) |
|
USE_KERNEL26 |
Enables support for the 2.6 kernel. This should only be disabled |
(true or false) |
on small embedded systems running a 2.4 (or older) kernel. |
USE_VENDORID |
This enables sending a vendor ID identifying the software as |
(true or false) |
"Openswan". This is disabled by default for security reasons. |
|
|
60 |
|

Chapter 3
Option name |
Description |
USE_XAUTH (true or false)
USE_XAUTHPAM (true or false)
USE_NAT_TRAVERSAL (true or false)
USE_NAT_TRAVERSAL_TRANSPORT_MODE (true or false)
Builds the userland with XAUTH support. See Chapter 9.
Add PAM support to XAUTH. With this you can use your system's user and password information for XAUTH.
Support NAT traversal. Strongly recommended in today's imperfect world.
Support NAT-T in Transport mode. This is a security problem and should be disabled. However, it is necessary if you wish to be compatible with Microsoft clients using L2TP over IPsec. See Chapter 8.
USE_LDAP (true or false)
USE_LIBCURL (true or false)
USE_SMARTCARD (true or false)
USE_OE
(true or false)
HAVE_THREADS (true or false)
Support fetching Certificate Revocation Lists over LDAP. See Chapter 5.
Use libcurl instead of native code for fetching over LDAP.
Add smartcard support. This requires openct and opensc.
Enable Opportunistic Encryption per default. See Chapter 6.
Include support for POSIX threads. This is necessary for XAUTHPAM and LDAP. It is recommended to disable this if you don't include support for PAM or LDAP.
Compile Flags
The following are compile options:
Option name |
Description |
|
|
KERNELSRC |
The location of the Linux kernel source you wish to use. If not |
(for example: /usr/src/linux) |
set, it will first try /usr/src/linux-2.6, then |
|
/usr/src/linux-2.4, then /usr/src/linux. |
RH_KERNELSRC |
The location of the Red Hat modified Linux kernel source tree. |
(for example: /usr/src/linux-2.6.8- |
This option is only used for the make rpm target. See |
1.520) |
packaging/redhat/openswan.spec. |
RH_KERNELSRC_POOL |
The location of the Red Hat kernel source pool. This is used for |
(for example: /vol/bigstorage/) |
automating the compilation of a large number of binary RPM |
|
packages for our official binary Red Hat releases. See |
|
packaging/redhat/openswan.spec for details. |
|
|
|
61 |

Building and Installing Openswan
Option name |
Description |
|
|
USERCOMPILE |
The compiler flags for building the Openswan userland. |
(for example: -O3 or -g) |
|
KLIPSCOMPILE |
The compiler flags for building the KLIPS kernel module. |
(for example: -O3 or -g) |
|
|
|
Do not put -L, -I, or other cross-compile options in these compile flag variables. The above options are only meant for generic options such as -O3 for optimizing, or -g, which adds debug information into the binary object code.
File Path Options
The following options affect the paths used within the scripts and binaries:
Option name |
Description |
|
|
DESTDIR |
This is the exact location where the files will be installed. If |
(for example: /vol/bigspace/arm-port) |
unset, it is ignored. |
INC_USRLOCAL |
This is the exact location where the files consider themselves |
(for example: /usr/local or /usr) |
installed for the running system. Various scripts will have this |
|
path hard coded in them. |
PUBDIR |
The location of the ipsec command. This should be within the |
|
$PATH of the root user. Note that some systems, such as Red |
|
Hat and Fedora, call the startup script ipsec as well. These |
|
are entirely different commands though! |
FINALCONFFILE |
The location of ipsec.conf. Normally this is |
|
/etc/ipsec.conf, but Gentoo for instance uses |
|
/etc/ipsec/ipsec.conf. |
INC_RCDIRS |
This contains a list of directories in decreasing preference for |
|
the daemon start-stop script. The first directory found on the |
|
system in this list is used. This option is mainly to recognize |
|
new distributions automatically. It is unlikely you'll need to |
|
change this. |
|
|
There are a few more settings that fine-tune the paths where files are installed. See the comments in Makefile.inc for details. Usually, if compiling Openswan for the host system itself, these do not need to be changed.
Obscure Pluto Options
There are three more options hidden in Pluto's Makefile at programs/pluto/Makefile. It is very unlikely that you need to change these. These options haven't made it into Makefile.inc yet, but will probably move there in the future.
62