One advantage of Linux™ powerline workstations is the ability to control the low-level networking environment. ISO Layer 2 traffic can be easily directed from one Ethernet interface to another on the same host but Layer 3 traffic is a different matter because routing software merely routes this type of traffic internally.
A Linux™ kernel patch is available that will allow ISO Layer 3 traffic to be routed from one Ethernet interface to another on the same host. With this patch, multiple instances of a traffic generator, like ttcp or iperf, can be effectively deployed on the same host without modification.
This patch is useful for testing on a closed network but it could pose a security risk to the local host when connected to a public network. Kernels having this patch installed should have a special designation such as “linux-2.6.28-send-to-self” so that users are aware that the patch is installed.
Example 2.1. “send-to-self” Patch Description
The following is the full, original patch description.
Send-To-Self interface flag Julian Anastasov <ja@ssi.bg>, July 2003 Patches for different kernels: send-to-self-2.4.21-1.diff send-to-self-2.5.73-1.diff The presented patch implements routing of traffic between local IP addresses externally via ethernet interfaces. This patch is basically the Ben Greear's send-to-self work but reimplemented entirely on routing level. The idea is to return output route via external interfaces if path between two local IP addresses is requested and they are configured on different interfaces with /proc/sys/net/ipv4/conf/DEVNAME/loop set to 1. As result, arp_filter (if enabled - the recommended value) automatically accepts the ARP requests on the right interface. The rp_filter check is modified to accept traffic from such interfaces with local IP as sender, so using loop=1 for interfaces attached to insecure mediums is not recommended. Pros: - it can be used from all existing applications without change - it is not limited to 2 interfaces - you can use it with many IP addresses - does not depend on the rp_filter and arp_filter states, they can be set to 1 - the packets are not altered in any way, useful for QoS testings - the routing result is cached, the routing checks are not per packet Cons: - not possible to use it for interfaces attached to insecure mediums (the rp_filter protection allows saddr to be local IP). By design. Use at your own risk. The usage is simple: # Connect two or more interfaces to same hub or via crossover cable # Enable loopback mode for eth0 and eth1. This even can be # default mode without breaking any other talks. By this way # we allow external routing only between local IPs configured # on the specified interfaces. echo 1 > /proc/sys/net/ipv4/conf/eth0/loop echo 1 > /proc/sys/net/ipv4/conf/eth1/loop # Add some IP addresses for testing, eg. client and server IP ip address add 192.168.1.1 dev eth0 ip address add 192.168.2.1 dev eth1 # Testing with applications that are aware of this binding. # The main thing the apps need to know is what src and dst IP # addresses to use. The client app needs to bind to the src IP # and by this way to request output route to the dst IP. There # is no specific configuration for the server app listening on # 192.168.2.1 ping -I 192.168.1.1 192.168.2.1 # Note that specifying the output device (SO_BINDTODEVICE is # not recommended) # Testing with applications that are not aware of this feature: # for 192.168.1.1 client (the same for the server is not needed). # Note that by default, in local routes the kernel uses the local # IPs as preferred source. This is the safe default mode (if loop=1) # for applications that do not care what src IP will be used # for their talks with local IPs. We try to change that and to # use IPs from different interfaces. ip route replace local 192.168.2.1 dev eth1 scope host src 192.168.1.1 proto kernel # but for any case, here it is and for the "server": ip route replace local 192.168.1.1 dev eth0 scope host src 192.168.2.1 proto kernel # Testing it: ping 192.168.2.1 ping -I 192.168.1.1 192.168.2.1 telnet 192.168.2.1 # Note that by replacing the local route's preferred source IP address # we help the IP address autoselection to select proper IP to the # target, in our case, route via eth
Example 2.2. “send-to-self” Patch Application
The following example illustrates how to use iperf to perform TCP and UDP traffic measurements once this patch is installed. We illustrate the use of iperf but do not necessarily endorse it for traffic measurement. We also illustrate the use of two interfaces but the “send-to-self” patch will support additional interfaces. We also illustrate the use of environment variables so that procedures can execute on different hosts without modification but these environment variables are not required.
First, we define environment variables, IF1 and IF2, for each Ethernet interface and, IP1 and IP2, for their IP addresses. Each interface must be on a separate IP subnet. We export definitions here so that they are accessible to this process and any subprocesses, such as shell scripts. Do whatever is appropriate for your environment.
export IF1=eth1 export IF2=eth2 export IP1=192.168.1.1 export IP2=192.168.2.2
Next, we assign the IP addresses to the interfaces using program ifconfig. There are other ways to do this. Observe that we reference our environment variables on the command line.
ifconfig ${IF1} ${IP1}
ifconfig ${IF2} ${IP2}
Next, we suppress internal routing between local interfaces. The loop propery only exists on kernels that have the “send-to-self” patch installed and have the /proc filesystem mounted. Some systems may not mount this file system.
echo 1 > /proc/sys/net/ipv4/conf/${IF1}/loop
echo 1 > /proc/sys/net/ipv4/conf/${IF2}/loop
Alternately, you could edit file /etc/sysctl.conf, as follows, to set the loop property for each interface during system startup. Again, the loop propery only exists on kernels that have the “send-to-self” patch installed and so errors will occur if you boot another kernel that does not have it installed.
net.ipv4.conf.eth1.loop = 1 net.ipv4.conf.eth2.loop = 1
Open a console window and start iperf as a server. Option -s identifies this instance of iperf as the server. Option -B binds this instance to one host interface by IP address, in this case IP1 defined earlier.
iperf -B ${IP1} -s
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 192.168.1.1
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
Open a second console window and start iperf as a client. Option -c identifies this instance of iperf as a client. Option -B binds this instance to the one interface by IP address, in this case IP2 defined earlier. The server address must also be specified, in this case IP1 bound to the server in the last step.
iperf -B ${IP2} -c ${IP1}
------------------------------------------------------------
Client connecting to 192.168.1.1, TCP port 5001
Binding to local address 192.168.2.1
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.2.1 port 5001 connected with 192.168.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 31.1 MBytes 26.0 Mbits/sec
Example 2.3. “send-to-self” Patch Installation
The “send-to-self” patch exists for several recent Linux™ kernel versions but not all versions. Assuming you have obtained the correct kernel archive and the correct patch version, the following script illustrates the steps needed to apply the patch on Ubuntu 9.04™ and recompile the kernel. Observe that, in this case, the patch version does not match the kernel version because a patch has not been published for that kernel version.
The following script can be used on a Ubuntu Linux distribution to download kernel source, the send-to-self patch, apply the patch then compile and install the resulting kernal image. When the menuconfig screen appears:
Select General Setup on the “Linux Kernel Configuration” screen.
Select Local version - append to kernel release on the “General Setup” screen.
Enter the version suffix “-send-to-self”.
Select ok to return to the “General Setup” screen.
Select Automatically append version information to the version string on the “General Setup” screen.
Select exit to return to the “Linux Kernel Configuration” screen.
Select exit to close the menuconfig program.
Select yes if prompted to save your new kernel configuration. This message does not appear each time.
#!/bin/bash
# file: patches/send-to-self-2.6.28.sh
# ====================================================================
# environment variables;
# --------------------------------------------------------------------
VERSION=2.6.28
CURRENT=9
VARIANT=send-to-self
PACKAGE=linux-source-${VERSION}
ARCHIVE=${PACKAGE}.tar.bz2
PATCH=send-to-self-2.6.26-1.diff
# ====================================================================
# extend version string;
# --------------------------------------------------------------------
if [ ! -z ${CURRENT} ]; then
VERSION+=.${CURRENT}
fi
if [ ! -z ${VARIANT} ]; then
VERSION+=-${VARIANT}
fi
# ====================================================================
# install required software;
# --------------------------------------------------------------------
if [ ! -f ${ARCHIVE} ]; then
wget http://www.ssi.bg/~ja/${PATCH}
apt-get install ${PACKAGE}
# apt-get install ${PACKAGE} --reinstall
apt-get install binutils patch gcc g++
apt-get install ncurses-dev
mv /usr/src/${ARCHIVE} .
fi
# ====================================================================
# confirm archive file exists;
# --------------------------------------------------------------------
if [ ! -f ${ARCHIVE} ]; then
echo "File ${ARCHIVE} is missing or misplaced"
exit 1
fi
# ====================================================================
# confirm patch file exists;
# --------------------------------------------------------------------
if [ ! -f ${PATCH} ]; then
echo "File ${PATCH} is missing or misplaced"
exit 1
fi
# ====================================================================
# remove old kernel source if present;
# --------------------------------------------------------------------
if [ -d ${PACKAGE} ]; then
echo "Removing old source ..."
rm -fr ${PACKAGE}
fi
# ====================================================================
# extract kernel source;
# --------------------------------------------------------------------
tar -vjxf ${ARCHIVE}
if [ ! -d ${PACKAGE} ]; then
echo "Folder ${PACKAGE} does not exist"
exit 1
fi
cd ${PACKAGE}
# ====================================================================
# patch kernel source;
# --------------------------------------------------------------------
patch -p1 < ../${PATCH}
# ====================================================================
# compile kernel source;
# --------------------------------------------------------------------
make mrproper
make menuconfig
make
# ====================================================================
# install kernel source;
# --------------------------------------------------------------------
make modules_install
make install
# ====================================================================
# install kernel source;
# --------------------------------------------------------------------
mkinitramfs -o /boot/initrd.img-${VERSION} ${VERSION}
ln -fs config-${VERSION} /boot/config
ln -fs initrd.img-${VERSION} /boot/initrd.img
ln -fs System.map-${VERSION} /boot/System.map
ln -fs vmlinuz-${VERSION} /boot/vmlinuz
apt-get --reinstall
The apt-get program is only available on Debian-based distributions. If you do not use a Debian-based system then you must find another way to obtain the necessary packages. Option --reinstall instructs apt-get to download the kernel even though it has been installed before. It is not needed on the first script execution but may be needed on subsequent script executions if you have deleted the kernel archive file.
mkinitramfs
This script uses mkinitramfs instead of the mkinitrd. This may differ on other distributions. The kernel source package used here has Ubuntu™ modifications that result in a minor version being appended to the kernel version. This may not happen with other distributions or with kernels obtained directly from kernel.org.
cut-and-paste
This script, or some like it, are included in the ./patches folder of the toolkit. You can also copy and paste this script but remember to edit the environment variables at the top, remove all carriage returns and set correct file permissions with chmod 0755 before executing it on your Linux™ host. Run the script as root user.
grub/menu.lst
If your system uses grub then edit file /boot/grub/menu.lst and add a new reference to the new initrd.img, System.map and vmlinuz files installed in folder /boot by this script. We recommend adding these references as the last ones in the file so that the new kernel does not start by default. Once you are confident that everything works, you can then move the references to the first entry. We also recommend setting the timeout value to 10 for now.
Example 2.4. “send-to-self” Patch Listing
The following “send-to-self” patch is specifically for Linux™ kernel 2.6.30 and is provided for information only. For practical purposes, the patch has not changed much from version to version but the line numbers have changed. Some recent “send-to-self” patches are included in the toolkit ./patches folder.
diff -urp v2.6.30/linux/Documentation/networking/ip-sysctl.txt linux/Documentation/networking/ip-sysctl.txt
--- v2.6.30/linux/Documentation/networking/ip-sysctl.txt 2009-06-13 10:53:29.000000000 +0300
+++ linux/Documentation/networking/ip-sysctl.txt 2009-06-13 15:54:15.000000000 +0300
@@ -637,6 +637,13 @@ accept_redirects - BOOLEAN
forwarding - BOOLEAN
Enable IP forwarding on this interface.
+loop - BOOLEAN
+ By default (loop=0) the traffic between local IP addresses
+ is routed via interface "lo". Setting this flag for two
+ interfaces allows traffic between their IP addresses to
+ be looped externally. This is useful for setups where the
+ interfaces are attached to same broadcast medium.
+
mc_forwarding - BOOLEAN
Do multicast routing. The kernel needs to be compiled with CONFIG_MROUTE
and a multicast routing daemon is required.
diff -urp v2.6.30/linux/include/linux/inetdevice.h linux/include/linux/inetdevice.h
--- v2.6.30/linux/include/linux/inetdevice.h 2009-06-13 10:53:56.000000000 +0300
+++ linux/include/linux/inetdevice.h 2009-06-13 15:54:15.000000000 +0300
@@ -106,6 +106,7 @@ static inline void ipv4_devconf_setall(s
IN_DEV_ORCONF((in_dev), ACCEPT_REDIRECTS)))
#define IN_DEV_ARPFILTER(in_dev) IN_DEV_ORCONF((in_dev), ARPFILTER)
+#define IN_DEV_LOOP(in_dev) IN_DEV_CONF_GET(in_dev, LOOP)
#define IN_DEV_ARP_ANNOUNCE(in_dev) IN_DEV_MAXCONF((in_dev), ARP_ANNOUNCE)
#define IN_DEV_ARP_IGNORE(in_dev) IN_DEV_MAXCONF((in_dev), ARP_IGNORE)
#define IN_DEV_ARP_NOTIFY(in_dev) IN_DEV_MAXCONF((in_dev), ARP_NOTIFY)
diff -urp v2.6.30/linux/include/linux/sysctl.h linux/include/linux/sysctl.h
--- v2.6.30/linux/include/linux/sysctl.h 2009-06-13 10:53:56.000000000 +0300
+++ linux/include/linux/sysctl.h 2009-06-13 15:54:40.000000000 +0300
@@ -491,6 +491,7 @@ enum
NET_IPV4_CONF_PROMOTE_SECONDARIES=20,
NET_IPV4_CONF_ARP_ACCEPT=21,
NET_IPV4_CONF_ARP_NOTIFY=22,
+ NET_IPV4_CONF_LOOP=23,
__NET_IPV4_CONF_MAX
};
diff -urp v2.6.30/linux/kernel/sysctl_check.c linux/kernel/sysctl_check.c
--- v2.6.30/linux/kernel/sysctl_check.c 2009-06-13 10:53:57.000000000 +0300
+++ linux/kernel/sysctl_check.c 2009-06-13 15:55:00.000000000 +0300
@@ -220,6 +220,7 @@ static const struct trans_ctl_table tran
{ NET_IPV4_CONF_PROMOTE_SECONDARIES, "promote_secondaries" },
{ NET_IPV4_CONF_ARP_ACCEPT, "arp_accept" },
{ NET_IPV4_CONF_ARP_NOTIFY, "arp_notify" },
+ { NET_IPV4_CONF_LOOP, "loop" },
{}
};
diff -urp v2.6.30/linux/net/ipv4/devinet.c linux/net/ipv4/devinet.c
--- v2.6.30/linux/net/ipv4/devinet.c 2009-06-13 10:53:58.000000000 +0300
+++ linux/net/ipv4/devinet.c 2009-06-13 15:55:22.000000000 +0300
@@ -1449,6 +1449,7 @@ static struct devinet_sysctl_table {
DEVINET_SYSCTL_RW_ENTRY(ARP_IGNORE, "arp_ignore"),
DEVINET_SYSCTL_RW_ENTRY(ARP_ACCEPT, "arp_accept"),
DEVINET_SYSCTL_RW_ENTRY(ARP_NOTIFY, "arp_notify"),
+ DEVINET_SYSCTL_RW_ENTRY(LOOP, "loop"),
DEVINET_SYSCTL_FLUSHING_ENTRY(NOXFRM, "disable_xfrm"),
DEVINET_SYSCTL_FLUSHING_ENTRY(NOPOLICY, "disable_policy"),
diff -urp v2.6.30/linux/net/ipv4/fib_frontend.c linux/net/ipv4/fib_frontend.c
--- v2.6.30/linux/net/ipv4/fib_frontend.c 2009-06-13 10:53:58.000000000 +0300
+++ linux/net/ipv4/fib_frontend.c 2009-06-13 15:54:15.000000000 +0300
@@ -239,16 +239,17 @@ int fib_validate_source(__be32 src, __be
.tos = tos } },
.iif = oif };
struct fib_result res;
- int no_addr, rpf;
+ int no_addr, rpf, loop;
int ret;
struct net *net;
- no_addr = rpf = 0;
+ no_addr = rpf = loop = 0;
rcu_read_lock();
in_dev = __in_dev_get_rcu(dev);
if (in_dev) {
no_addr = in_dev->ifa_list == NULL;
rpf = IN_DEV_RPFILTER(in_dev);
+ loop = IN_DEV_LOOP(in_dev);
}
rcu_read_unlock();
@@ -258,6 +259,11 @@ int fib_validate_source(__be32 src, __be
net = dev_net(dev);
if (fib_lookup(net, &fl, &res))
goto last_resort;
+ if (loop && res.type == RTN_LOCAL) {
+ *spec_dst = FIB_RES_PREFSRC(res);
+ fib_res_put(&res);
+ return 0;
+ }
if (res.type != RTN_UNICAST)
goto e_inval_res;
*spec_dst = FIB_RES_PREFSRC(res);
diff -urp v2.6.30/linux/net/ipv4/route.c linux/net/ipv4/route.c
--- v2.6.30/linux/net/ipv4/route.c 2009-06-13 10:53:58.000000000 +0300
+++ linux/net/ipv4/route.c 2009-06-13 15:54:15.000000000 +0300
@@ -2521,6 +2521,11 @@ static int ip_route_output_slow(struct n
dev_put(dev_out);
goto out; /* Wrong error code */
}
+ err = -ENETDOWN;
+ if (!(dev_out->flags&IFF_UP)) {
+ dev_put(dev_out);
+ goto out;
+ }
if (ipv4_is_local_multicast(oldflp->fl4_dst) ||
oldflp->fl4_dst == htonl(0xFFFFFFFF)) {
@@ -2588,10 +2593,41 @@ static int ip_route_output_slow(struct n
free_res = 1;
if (res.type == RTN_LOCAL) {
- if (!fl.fl4_src)
- fl.fl4_src = fl.fl4_dst;
+ struct in_device *in_dev;
+ __be32 src;
+
if (dev_out)
dev_put(dev_out);
+ dev_out = FIB_RES_DEV(res);
+ in_dev = in_dev_get(dev_out);
+ src = fl.fl4_src? : FIB_RES_PREFSRC(res);
+ if (in_dev && IN_DEV_LOOP(in_dev) && src) {
+ struct net_device *dev_src;
+
+ in_dev_put(in_dev);
+ in_dev = NULL;
+ dev_src = ip_dev_find(net, src);
+ if (dev_src && dev_src != dev_out &&
+ (in_dev = in_dev_get(dev_src)) &&
+ IN_DEV_LOOP(in_dev)) {
+ in_dev_put(in_dev);
+ dev_out = dev_src;
+ fl.fl4_src = src;
+ fl.oif = dev_out->ifindex;
+ res.type = RTN_UNICAST;
+ if (res.fi) {
+ fib_info_put(res.fi);
+ res.fi = NULL;
+ }
+ goto make_route;
+ }
+ if (dev_src)
+ dev_put(dev_src);
+ }
+ if (in_dev)
+ in_dev_put(in_dev);
+ if (!fl.fl4_src)
+ fl.fl4_src = fl.fl4_dst;
dev_out = net->loopback_dev;
dev_hold(dev_out);
fl.oif = dev_out->ifindex;