This took me a while to figure out, so I might as well post it here
in case it's of use to anyone. Assume you have two network
interfaces, say eth0 and ppp0, with two
totally unrelated IP addresses, say
257.42.0.18 for eth0 (yes, I know, 257 is
impossible, I'm just choosing this to represent a totally arbitrary
address) and 333.64.17.29 for ppp0. You
wish to use one interface for certain connections and the other for
others (there can be plenty of different reasons for that: maybe one
connection is faster but has a stupid firewall so it can't be used
always). Now if you can decide which connection goes where in
function of the destination host('s IP address or
network), then it's easy: just configure your routing tables
appropriately. But what happens if you wish, for example, all
outbound connections to TCP port 80 to go through
eth0 and all others to go through ppp0? We
need a little more work there, and a little magic, but thanks to the
mutant features of the Linux network stack, using
iptables and iproute2, it is possible.
Here's a sample of the command lines that might be useful (just a
guideline, of course: don't ever copy them blindly, please learn about
the programs and understand what each line does), at least if the host
is not a router:
# Ordinary route is via ppp0 (this should probably be done as pppd starts): route add defaut gw 333.64.17.1 dev ppp0 # Routing table 201 (say) is through eth0 (gateway is 257.42.0.1, say): ip route add 257.42.0.0/24 dev eth0 scope link src 257.42.0.18 table 201 ip route add default via 257.42.0.1 dev eth0 table 201 # Use routing table 201 for marked packets: ip rule add fwmark 1 table 201 # Now set up iptable rule to mark packets destined for eth0: iptables -t mangle -A OUTPUT -p tcp -m tcp --dport 80 -j MARK --set-mark 1 # Lastly, we need to do some self-masquerading: iptables -t nat -A POSTROUTING -s 333.64.17.29 -o eth0 -j SNAT --to-source 257.42.0.18
The last line probably deserves extra comments, because it is not
at all obvious: it is needed because otherwise an outbound connection
from the local host on port 80 will have local address
333.64.17.29 (as it appears on the ordinary routing
table) whereas it is sent through the eth0 device, and
this can't work (any router down the stream will reject it as not
being meant for this route, or at best the return packets will come
through the wrong interface).
If you're also trying to open listening (server) sockets on the
eth0 interface, you also need something probably like
this:
# Turn off entry route verification on incoming packets: sysctl -w net.ipv4.conf.eth0.rp_filter=0 # Also mark local packets destined for eth0: iptables -t mangle -A OUTPUT -s 257.42.0.18 -j MARK --set-mark 1
If your box also acts as a router (through some third interface
eth1, say), then at the very least you need to duplicate
the OUTPUT rules (for the mangle table) as
PREROUTING ones and broaden the source address match on
the rules that have one, but more complicated are probably desirable
(of course, it all depends on what kind of addresses you have on
eth1; I'll leave as an exercise for the interested reader
the case where eth1 has private addresses and you wish to
masquerade on forwarding…).