Policy Based Routing example: route one subnet via ISP A and another via ISP B

Goal

Setup a network with a Cisco router that routes one local subnet via internet connection A and another local subnet via internet connection B.

In this example we’ll use one LAN side, which has two subnets:

  • 192.168.1.0/24
  • 192.168.2.0/24

We want traffic destined for the internet, originating from the 192.168.1.0/24 network, to be sent to ISP A, which is connected to FastEthernet0

We want traffic destined for the internet, originating from the 192.168.2.0/24 network, to be sent to ISP B, which is connected to FastEthernet1

Both internet connections get their IP address via DHCP.

Prerequisites

For this  configuration you’ll need:

  • One dual WAN router, such as a 1811
  • Two internet connections (or simulated ones)
  • 2 nodes, one for the 192.168.1.0/24 subnet and one for the 192.168.2.0/24 subnet. These can be two different computers, or two virtual machines, etc…
    • ClientA, Windows 7, will connect to 192.168.1.0/24 and surf the net using ISP A
    • ClientB, Windows XP, will connect to 192.168.2.0/24 and surf the net using ISP B

Configure the router

Let’s start by defining our WAN interfaces first, which are FastEthernet0 and FastEthernet1 for a Cisco 1811.  Ports FastEthernet2 to 9 are the integrated switch. Both WAN interfaces use DHCP to obtain an IP address.

Setting up the WAN interfaces

r3(config)#interface FastEthernet0
r3(config-if)#ip address dhcp
r3(config-if)#ip nat outside
r3(config-if)#no shutdown
r3(config-if)#interface FastEthernet1
r3(config-if)#ip address dhcp
r3(config-if)#ip nat outside
r3(config-if)#no shutdown
r3(config-if)#exit
r3(config)#

Verifying WAN operation

We can verify our WAN connection by running the following commands and observing their output.

! Display a list of interfaces and their IPv4 addresses
r3#show ip interface brief
Interface     IP-Address    OK? Method Status                Protocol
Async1        unassigned    YES unset  down                  down
FastEthernet0 213.193.228.1 YES DHCP   up                    up
FastEthernet1 88.9.5.1      YES DHCP   up                    up
FastEthernet2 unassigned    YES unset  up                    up
FastEthernet3 unassigned    YES unset  up                    down
FastEthernet4 unassigned    YES unset  up                    down
FastEthernet5 unassigned    YES unset  up                    down
FastEthernet6 unassigned    YES unset  up                    down
FastEthernet7 unassigned    YES unset  up                    down
FastEthernet8 unassigned    YES unset  up                    down
FastEthernet9 unassigned    YES unset  up                    down
NVI0          unassigned    YES unset  administratively down down
Vlan1         192.168.1.254 YES NVRAM  up                    up
r3#! Display IPv4 routes
r3#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 213.193.228.254 to network 0.0.0.0

S*    0.0.0.0/0 [254/0] via 213.193.228.254
                [254/0] via 88.9.5.254
      88.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        88.9.5.0/24 is directly connected, FastEthernet1
L        88.9.5.1/32 is directly connected, FastEthernet1
      192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.1.0/24 is directly connected, Vlan1
L        192.168.1.254/32 is directly connected, Vlan1
      192.168.2.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.2.0/24 is directly connected, Vlan1
L        192.168.2.254/32 is directly connected, Vlan1
      213.193.228.0/24 is variably subnetted, 2 subnets, 2 masks
C        213.193.228.0/24 is directly connected, FastEthernet0
L        213.193.228.1/32 is directly connected, FastEthernet0
r3#! Ping ISP A's gateway
r3#ping 213.193.228.254
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 213.193.228.254, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
r3#! Ping ISP B's gateway 
r3#ping 88.9.5.254
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 88.9.5.254, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms
r3#! Ping known-to-reply-to-ping address on the internet 
r3#ping 8.8.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/21/24 ms
r3#

Setting up the local network

We’re going to keep this simple. As described we have two internal IP ranges, we’ll assign an IP address to Vlan1 for each of the IP ranges. Of course this scenario is of little use in the real world, you would probably have two VLAN’s, etc… However, this is just a theory of operations exercise, we’ll just configure the computers connecting to the router manually.

r3(config)#interface Vlan1
r3(config-if)#ip address 192.168.1.254 255.255.255.0
r3(config-if)#ip address 192.168.2.254 255.255.255.0 secondary
r3(config-if)#ip nat inside
r3(config-if)#no shutdown
r3(config-if)#exit
r3(config)#

Verifying local network operation

Once we’ve configured the Vlan1 interface, we can check again with show ip interface that we have correctly configured the respective interface:

r3#show ip interface Vlan1

Now we configure our client machines. We will statically configure:

  • ClientA with IP address 192.168.1.5
  • ClientB with IP address 192.168.2.5

After applying the  network configuration, verify that we can ping the routers.

Configuring NAT

We configure two access-lists:

  • One for the 192.168.1.0/24 that should get translated to FastEthernet0’s IP address
  • One for the 192.168.2.0/24 that should get translated to FastEthernet1’s IP address
r3(config)#ip access-list standard 1
r3(config-std-nacl)#permit 192.168.1.0 0.0.0.255
r3(config-std-nacl)#exit
r3(config)#ip access-list standard 2
r3(config-std-nacl)#permit 192.168.2.0 0.0.0.255
r3(config-std-nacl)#exit
r3(config)#
r3(config)#ip nat inside source list 1 interface FastEthernet0 overload
r3(config)#ip nat inside source list 2 interface FastEthernet1 overload
r3(config)#

Basically what we’re telling here is that, when the packet’s source address gets translated, it should get the source address of the interface as specified. Thus, when packets from 192.168.1.0/24 get NAT’ed, they get the source IP of FastEthernet0. When packets from 192.168.2.0/24 get NAT’ed, they get the source IP of FastEthernet1.

Now, when the packets have their correct source address, that doesn’t mean they automatically fly out the right interface. We need to explicitly define that any packets for the interface from 192.168.1.0/24 must go via FastEthernet0 and from 192.168.2.0/24 via FastEthernet1. We do this with a route map.

Building the route-map

Using a route-map we can build policy based  routing. A route map can have several entries, and each entry can have any number of match & set statements. If an entry doesn’t match, the router continues evaluating the route map until it finds an entry that does. When the router finds an entry that does match, it does whatever you say it should do with the set statements.

Matching

The match statements describe on which traffic we should act. In this scenario we have two distinct types of traffic: that for ISP A and that for ISP B. So we’ll need to define two route map entries: one matching the traffic destined for ISP A and one matching the traffic destined for ISP B.

We will do the matching with an access-list. We have two different subnets we want to match, 192.168.1.0/24 and 192.168.2.0/24. But, if you remember correctly, we already have two access-lists matching those exacts subnets (ip access-list 1 & 2). So we’re going to re-use those.

Setting

The set statements describe what should happen on the traffic that matches. Again, we have two distinct types of traffic, that for ISP A and that for ISP B. When we’ve correctly matched the traffic we want, we have to set something, namely, what to do with it. The first route-map entry (matching traffic for ISP A) should make the router send the traffic to FastEthernet0. The second entry (matching traffic for ISP B) should make the router send the traffic to FastEthernet1.

Show me the money

We will call our route-map “ISPSelect”. (IOS calls it a tag).

! Create the first entry in our new route-map called "ISPSelect"
r3(config)#route-map ISPSelect permit 10
! Match the traffic from 192.168.1.0/24 with access-list 1
r3(config-route-map)#match ip address 1
! Set the interface the traffic should go to, to FastEthernet0
r3(config-route-map)#set interface FastEthernet0
r3(config-route-map)#exit

! Create the second entry in our route-map called "ISPSelect"
r3(config)#route-map ISPSelect permit 20
! Match the traffic from 192.168.2.0/24 with access-list 2
r3(config-route-map)#match ip address 2
! Set the interface the traffic should go to, to FastEthernet1
r3(config-route-map)#set interface FastEthernet1
r3(config-route-map)#exit
r3(config)#

Verifying route-map configuration

r3#show route-map ISPSelect
route-map ISPSelect, permit, sequence 10
 Match clauses:
 ip address (access-lists): 1
 Set clauses:
 interface FastEthernet0
 Policy routing matches: 0 packets, 0 bytes
route-map ISPSelect, permit, sequence 20
 Match clauses:
 ip address (access-lists): 2
 Set clauses:
 interface FastEthernet1
 Policy routing matches: 0 packets, 0 bytes
r3#

Applying the route-map

Now where do we apply this route-map? Let’s take the following rule into account:

Packets are first routed, then NAT’ed.

Since we’re talking about a route-map, the route-map influences what happens when the router routes the packets. So we have to apply this map to the interface where the packets are coming in. In this case, this is Vlan1. Using the ip policy route-map statement in the interface configuration, we can define which route-map we want to use there:

r3(config)#interface Vlan 1
r3(config-if)#ip policy route-map ISPSelect
r3(config-if)#exit
r3(config)#

Verifying route-map application

r3#show ip interface Vlan 1
Vlan1 is up, line protocol is up
 Internet address is 192.168.1.254/24
 Broadcast address is 255.255.255.255
 Address determined by non-volatile memory
 MTU is 1500 bytes
 Helper address is not set
 Directed broadcast forwarding is disabled
 Secondary address 192.168.2.254/24
 Outgoing access list is not set
 Inbound access list is not set
 Proxy ARP is disabled
 Local Proxy ARP is disabled
 Security level is default
 Split horizon is enabled
 ICMP redirects are always sent
 ICMP unreachables are always sent
 ICMP mask replies are never sent
 IP fast switching is enabled
 IP fast switching on the same interface is disabled
 IP Flow switching is disabled
 IP CEF switching is enabled
 IP CEF switching turbo vector
 IP Null turbo vector
 IP multicast fast switching is enabled
 IP multicast distributed fast switching is disabled
 IP route-cache flags are Fast, CEF
 Router Discovery is disabled
 IP output packet accounting is disabled
 IP access violation accounting is disabled
 TCP/IP header compression is disabled
 RTP/IP header compression is disabled
 Policy routing is enabled, using route map ISPSelect
 Network address translation is enabled, interface in domain inside
 BGP Policy Mapping is disabled
 Input features: Stateful Inspection, Virtual Fragment Reassembly, Virtual Fragment Reassembly After IPSec Decryption, Policy Routing, MCI Check
 Output features: NAT Inside, Stateful Inspection, NAT ALG proxy
 WCCP Redirect outbound is disabled
 WCCP Redirect inbound is disabled
 WCCP Redirect exclude is disabled
r3#

Testing

Right now you should be able to surf the internet from both subnets. Let’s verify traffic from our subnets are really following the right path. Let’s start by doing a traceroute. I’m going to trace the path to 8.8.8.8 and I’ll be using the -d switch to tracert to not do a reverse lookup on the IP addresses.

Doing a traceroute

Note the second hop.

Result on Windows 7, 192.168.1.0/24 subnet:

Result on Windows XP, 192.168.2.0/24 subnet:

A successful result is when packets from the Windows 7 computer get routed via 213.193.228.254, which is ISP A’s gateway and packets from the Windows XP computer get routed via 88.9.5.254, which is ISP B’s gateway.

Checking the NAT translation table

Further verification, check the NAT translation table on the router using show ip nat translations command. I’ll be pinging 8.8.8.8 from both the Windows 7 and the Windows XP workstation, which will create exactly two NAT entries in the translation table: one from the Windows 7 PC to 8.8.8.8 using ISP A and one from the Windows XP PC to 8.8.8.8 using ISP B.

r3#show ip nat translations
Pro  Inside global   Inside local    Outside local  Outside global
icmp 213.193.228.1:1 192.168.1.5:1   8.8.8.8:1      8.8.8.8:1
icmp 88.9.5.1:512  192.168.2.5:512 8.8.8.8:512    8.8.8.8:512
r3#

Further testing

You can do a packet capture between the router and the ISP’s modems.

You can pull the cables for the internet connections and check which subnets can still reach the internet.

Full configuration

!
! Last configuration change at 11:52:02 UTC Sat Jan 28 2012
!
version 15.1
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname r3
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
crypto pki token default removal timeout 0
!
!
dot11 syslog
ip source-route
!
!
!
!
!
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
license udi pid CISCO1811/K9 sn CENSORED
!
!
!
!
!
!
!
!
!
interface Multilink1
 no ip address
 ppp multilink
 ppp multilink group 1
!
interface FastEthernet0
 ip address dhcp
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
!
interface FastEthernet1
 ip address dhcp
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
!
interface FastEthernet2
!
interface FastEthernet3
!
interface FastEthernet4
!
interface FastEthernet5
!
interface FastEthernet6
!
interface FastEthernet7
!
interface FastEthernet8
!
interface FastEthernet9
!
interface Vlan1
 ip address 192.168.2.254 255.255.255.0 secondary
 ip address 192.168.1.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
 ip policy route-map ISPSelect
!
interface Async1
 no ip address
 encapsulation slip
!
ip forward-protocol nd
no ip http server
no ip http secure-server
!
!
ip nat inside source list 1 interface FastEthernet0 overload
ip nat inside source list 2 interface FastEthernet1 overload
!
logging esm config
access-list 1 permit 192.168.1.0 0.0.0.255
access-list 2 permit 192.168.2.0 0.0.0.255
!
!
!
!
route-map ISPSelect permit 10
 match ip address 1
 set interface FastEthernet0
!
route-map ISPSelect permit 20
 match ip address 2
 set interface FastEthernet1
!
!
!
control-plane
!
!
!
line con 0
 speed 115200
line 1
 modem InOut
 stopbits 1
 speed 115200
 flowcontrol hardware
line aux 0
line vty 0 4
 login
 transport input all
!
end