[Python] Create a QEMU Bridge Using the Socket Networking Backend

I am trying to create a bridge to an interface in my host, much like the Virtualbox's and VMWare's bridge adapters, in QEMU, using a combination of socket networking and a Python library called Scapy (essentially, relying on WinPcap/Npcap on Windows OSes or libpcap on Unix OSes behind the scenes). Here's the bridge script I created to connect the VLAN created by the socket network backend of QEMU to the outside interfaces:

import argparse
import scapy
import threading
import socket
import struct
import scapy.sendrecv
import scapy.packet
import scapy.config
import scapy.layers.l2

MAX_PACKET_SIZE = 65535

send_lock = threading.Lock()
qemu_senders = set()
iface_senders = set()


def qemu_in_iface_out_traffic_thread_func(iface, mcast_addr, mcast_port, local_addr):
global MAX_PACKET_SIZE
global send_lock
global qemu_senders
global iface_senders

# Create the multicast listen socket.
listener_addr = (local_addr, mcast_port)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(listener_addr)
mcast_group = socket.inet_aton(mcast_addr)
mreq = struct.pack('4sL', mcast_group, socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

# Get the packets from the QEMU VLAN, and send them over to the host's interface.
while True:
data, _ = sock.recvfrom(MAX_PACKET_SIZE)
send_lock.acquire()
eth_pkt = scapy.layers.l2.Ether(data)
if eth_pkt.src not in iface_senders:
qemu_senders.add(eth_pkt.src)
scapy.sendrecv.sendp(eth_pkt, iface=iface, verbose=0)
send_lock.release()


def iface_in_qemu_out_traffic_thread_func(iface, mcast_addr, mcast_port):
global send_lock
global qemu_senders
global iface_senders

# Create the multicast send socket.
mcast_group = (mcast_addr, mcast_port)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
ttl = struct.pack('b', 1)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, ttl)

# Sniff packets from the host's interface, and send them to the QEMU VLAN.
def process_packet(eth_pkt):
send_lock.acquire()
if eth_pkt.src not in qemu_senders:
iface_senders.add(eth_pkt.src)
sock.sendto(scapy.packet.Raw(eth_pkt).load, mcast_group)
send_lock.release()
scapy.sendrecv.sniff(iface=iface, prn=process_packet, store=0)


if __name__ == "__main__":
# Parse the command line arguments.
parser = argparse.ArgumentParser()
parser.add_argument('--iface', '-i', required=True)
parser.add_argument('--mcast-addr', '-a', required=True)
parser.add_argument('--mcast-port', '-p', required=True, type=int)
parser.add_argument('--local-addr', '-l', default='127.0.0.1')
parser.add_argument('--disable-promisc', '-d',
default=False, action='store_true')
args = parser.parse_args()

# Set promiscuous mode.
scapy.config.conf.sniff_promisc = 0 if args.disable_promisc else 1

# Create the traffic threads.
qemu_in_iface_out_traffic_thread = \
threading.Thread(target=qemu_in_iface_out_traffic_thread_func, args=(
args.iface, args.mcast_addr, args.mcast_port, args.local_addr
))
iface_in_qemu_out_traffic_thread = \
threading.Thread(target=iface_in_qemu_out_traffic_thread_func, args=(
args.iface, args.mcast_addr, args.mcast_port
))

# Run the traffic threads, and join them to wait for their exit.
qemu_in_iface_out_traffic_thread.start()
iface_in_qemu_out_traffic_thread.start()
qemu_in_iface_out_traffic_thread.join()
iface_in_qemu_out_traffic_thread.join()



With this bridge source code, I was able to ping back and forth between a device outside of my host (e.g. a Raspberry Pi controller) and my QEMU VM, which both of my controller and my host reside on the same LAN. However, I was not able to do the same thing between my QEMU VM and my host on the same network. I would like to know if the issue has to do with having two different MAC addresses (i.e. my QEMU VM's MAC address and my host's MAC address) having the same interface (the host's interface connected to the LAN) and the traffic to the same LAN gets filtered out, or what am I missing here?

Edit #1:

So, I inspected the packets being sent/received on the network by either of the QEMU VM or my host, and they are being received on all devices on the network. I have connected a router to my network, and another laptop device with a Wi-Fi and Wireshark to join the network through the router. I can see the packets being received on all ends of the network (e.g. on the new laptop on the network), but my host doesn't respond to the ARP packets, for e.g., that were originated by another machine on the same interface such as the QEMU VM.

Here are Wireshark screenshots taken from the host and from the new laptop on the network, respectively:





From the screenshots taken above, any device outside of my host (the router at 192.168.1.1, the Rasberry Pi controller, and the other laptop) can talk to my QEMU VM and replying to ARP requests (and the QEMU VM likewise), but not the host.

Here's an observation of what I am seeing (apologize for the bad drawing):



However, if I replace the QEMU VM with the Virtualbox VM, there would be a connection between the host and the Virtualbox VM. In addition, the QEMU VM cannot talk to any machine inside the host either, such as between the QEMU VM and the Virtualbox VM, which both are connected to the same LAN through the host's interface.

Edit #2:

Another thing I observed is that when ARPing my host from my VirtualBox VMs, I would not see the ARP reply from my host to the VMs on the same interface in Wireshark (just like the symptoms above). I think VirtualBox's NetFilter host driver that is responsible for creating bridges creates another network, which the host's and the VMs's requests and replies are exchanged, while the host and VM's communicate with the outside world on the real interface. Apparently I would need to create my own host driver that would inject the packets into the receive stream that are sent from my QEMU VMs through the bridge.

Continue reading...