Suppose you want to route traffic from a certain machine (virtual machine on a
local software bridge) or a certain user through a specified interface
(possibly a VPN), if that interface is down, you want to blackhole the
traffic.

Some things need to be taken care of:

- define names of custom routing tables in /etc/iproute2/rt_tables
- setting up routing, so that marked packets go to the blackhole
- updating the route once the VPN comes up
- make sure that DNS traffic gets sent to the VPN's DNS server
- marking the correct packets
- additional stuff


Define names
~~~~~~~~~~~~

File /etc/iproute2/rt_tables should look like:

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
1 VPN
2 BLACKHOLE

Setting up routing
~~~~~~~~~~~~~~~~~~

Assuming Debian /etc/network/interfaces

Create BLACKHOLE routingtable with defaultroute blackhole:

pre-up ip route add blackhole 0.0.0.0/0 table BLACKHOLE

Let all traffic with fwmark 0x1 use this table:

pre-up ip rule add from all fwmark 0x1 lookup BLACKHOLE

Let all traffic with fwmark 0x1 use table VPN (currenlty empty):

pre-up ip rule add from all fwmark 0x1 lookup VPN

Marked traffic will consult the VPN routing table, will not find anything
there and therefore will consult the BLACKHOLE table and will be sent to the
blackhole.


Updating route once the VPN comes up
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This should be run in the script that establishes the VPN connection.

First, cleanup the routingtable:

ip route flush table VPN

Then add defaultroute:

ip route add default via $ifconfig_local dev tun0 table VPN


Fix DNS
~~~~~~~

Applications send their DNS request by default to the server(s) listed in
/etc/resolv.conf . This DNS server is usually not reachable over the VPN
(unless you use 8.8.8.8).

A chain VPNDNS should be created in 'iptables -t nat'. All packets for
that will go to the VPN and that is directed to udp port 53 should be directed
to this chain. For example:

user: -A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 112  -j VPNDNS

bridged host: -A PREROUTING -p udp -m udp --dport 53 -s $IP -j VPNDNS

If the VPN comes up, the chain VPNDNS should be flushed:

iptables -t nat -F VPNDNS

and a suitable rule (DNAT) should be added to the VPNDNS chain

iptables -t nat -A VPNDNS -j DNAT --to-destination $vpn_dns_server


Marking packets
~~~~~~~~~~~~~~~

All that's left is that we need to mark the packets that should go to the VPN
so that the routing subsystem sends them to the right routing table.

For a user, it is quite simple. Just mark the packets from this user in the
OUTPUT chain of the mangle table.

iptables -t mangle -A OUTPUT -m owner --uid-owner $UID -j MARK --set-xmark 0x1/0xffffffff

local bridge traffic is usually not seen in iptables, so we need ebtables to
mark it and to send it through the router
-p IPv4 -s $MAC --ip-dst ! $NET -j mark --mark-set 0x1 --mark-target CONTINUE
-p IPv4 -s $MAC --ip-dst ! $NET -j redirect --redirect-target DROP
(DROP means: frame must be routed)


Additional stuff
~~~~~~~~~~~~~~~~

Packets from the bridged host must be forwarded:

-t filter -A FORWARD -s $IP -j ACCEPT
-t filter -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT


If the DNS server runs on 127.0.0.1 it is not sufficient to do

iptables -A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 112  -j VPNDNS
iptables -t nat -A VPNDNS -j DNAT --to-destination $vpn_dns_server
iptables -t -A POSTROUTING -o tun0 -j MASQUERADE

It is also needed to:

sysctl -w net.ipv4.conf.tun0.route_localnet=1