Dino Fizzotti

Engineer, maker, hacker, thinker, funner.

Jan 9, 2019 - 17 minute read - Comments - Software Python Raspberry Pi

Running a man-in-the-middle proxy on a Raspberry Pi 3

Raspberry Pi 3 and mitmproxy

In preparation for a training session I will be giving on public key infrastructure (with a focus on TLS and certificates) I wanted to demonstrate how a transparent “man-in-the-middle” (MITM) proxy works.

This post walks through the configuration of a Raspberry Pi 3 acting as a Wi-Fi access point, running a transparent man-in-the-middle proxy (mitmproxy), which can be used to sniff HTTP and https traffic on connected devices.

What is a Man-In-The-Middle Proxy

An MITM proxy is a piece of software running on a device (e.g. a Wi-Fi access point or a network router) in between a client (your phone, your laptop) and the server you intend to communicate with. The proxy is able to intercept and parse the information being sent back and forth between the client and the server. It may even manipulate the request being sent, or modify the information coming back.

Simple example of a MITM proxy

A MITM proxy need not be “malicious”, although I guess this depends on your view of information privacy and the implementation of IT security controls. The majority of large corporate organizations usually employ a MITM proxy to scan and filter digital traffic moving within their internal network and an outside network (such as the internet). This way they could detect someone moving sensitive information outside of the company’s controlled environment, and also attempt to prevent malware from being installed on staff machines.

An example of a truly malicious MITM proxy would be a Wi-Fi access point that you may connect to thinking it is trustworthy. Perhaps you are in a coffee shop and see on your phone that there is an access point called “CoffeeShop FREE Wi-Fi”. It may even have a “fake” portal with the same “correct” password that was given to you by a (legitimate) barista. The problem is this access point has nothing to do with the coffee shop, it’s actually running on a device in the pocket of a nearby hacker, and now any non-HTTPS traffic that is sent between your laptop and the websites you visit are visible to the hacker, and can be recorded for later inspection. With some effort it is also possible that the hacker can be snooping on your HTTPS traffic. In a variation of this scenario the hacker may have previously compromised the CoffeeShop’s real Wi-Fi access point and could be forwarding the traffic from the access point to his machine. In either case the hacker is able to inspect the traffic between your device and the intended destination.

Components

mitmproxy

The free and open-source proxy software that I have chosen to use in my training demonstration is called simply “mitmproxy”. It has features such as both a command-line and a web interface, as a well as a Python API which can be used to write scripts which can read and modify the requests and responses as the flow through the mitmproxy.

Raspberry Pi 3

I wanted to create a near zero-configuration “plug-n-play” experience for my MITM proxy demo. I should be able to give my training session and demonstrations with as little set up as possible. To satisfy this I opted to install and run mitmproxy on a Raspberry Pi 3 Model B, with the intended operation to be as simple as:

  1. Plug in a network cable to the Raspberry Pi
  2. Power on the Raspberry Pi
  3. Observe a new Wi-Fi access point is available.
  4. Connect laptops/phones to this Wi-Fi network.
  5. Observe/modify traffic on the Raspberry Pi as data is sent between the connected devices and the internet.

I could certainly install and configure mitmproxy to run on my laptop, but it would require me to manually manage the network configuration on my laptop between demos.

Using a Raspberry Pi 3 Model B instead of an earlier version has the benefit that it contains two network interfaces: a wired Ethernet interface and an on-board wireless Wi-Fi transceiver. This will allow me to use the Pi’s on-board Wi-Fi as an access point to which client devices can connect. The traffic from these connected devices to the outside internet will then travel over the wired Ethernet connection. Previous versions of the Pi did not have the on-board Wi-Fi, and so a Wi-Fi dongle would have been required to add a second network interface.

Setup

What follows is a walk-through on how to set up and configure mitmproxy on a Raspberry Pi. I am using bits and pieces of various tutorials, blog posts, forum discussions and stack overflow answers. The primary source material for the DHCP and Wi-Fi setup is this Hackaday project from Grégory Paul: Raspberry Pi MITM. The major differences between that write-up and mine is that he is using a Raspberry Pi 2 Model B using 2x Wi-Fi dongles and no wired connection, running an older version of Raspbian, and using a custom script to inject pictures of unicorns in passing web traffic (haha!). My instructions below use updated hardware and OS image, as well as on-board wired and wireless network interfaces in place of his two dongles. In place of the unicorn script I have opted to automatically run the mitmproxy web interface from which I can view the intercepted traffic in a browser on another device.

I recommend that you follow these instructions with the Pi connected to your router with an Ethernet cable, and use a directly connected monitor, keyboard and mouse to enter in the commands and view the results. You can of course set up SSH access, but as you will be messing with network settings you may find yourself unable to connect to the Pi over the network and will need to run commands locally to sort yourself out. You will need internet access available on the Ethernet cable anyway for the MITM proxy operation.

Raspberry Pi Image Installation and Setup

Follow the installation guide and download the latest Raspbian image from the downloads page and flash it to a micro SD card. I used Etcher to flash the Raspbian image to an SD card.

The instructions below were written using Raspbian GNU/Linux 9.6 (stretch)

Once the flashing process is complete, pop the SD card into your Raspberry Pi 3, connect a monitor, keyboard and mouse and power it on using the micro USB port. As you will be downloading software I recommend you connect the Pi to the internet using an Ethernet cable. We will be adjusting the wireless settings and so the wireless connection may become unusable while we configure the access point capability.

On first boot you should walk through the short sequence of set-up dialog boxes, setting the location and language, as well as choosing a new password for the default “pi” user account.

Raspberry Pi Network Configuration

To turn the Pi into a Wi-Fi access point we will be using both the wired network interface and the on-board wireless network interface. On a Raspberry Pi 3 Model B running Raspbian Stretch these will named as follows:

  • eth0 for the wired network interface
  • wlan0 for the wireless network interface

Note that in the settings below I will be specifying an address range of 192.168.42.* for the custom wireless network. Feel free to use another one of the ranges reserved for private networks (see section 3 of RFC 1918).

Configure dhcpcd

“DHCP” stands for Dynamic Host Configuration Protocol and is responsible for managing device addresses on a network. There exists client and server DHCP software:

  • The DHCP client software handles how to configure a network interface once it has received a dynamically assigned address - or to force the network interface to use a fixed static address.

    • You will be modifying the Pi’s existing DHCP client configuration to assign a static IP address to the wlan0 interface, as this network interface will represent the address of the access point for all other devices that will be connecting to the custom Wi-Fi network.

  • The DHCP server software is responsible for handing out network addresses to clients who wish to join the network.

    • You will need to install DHCP software on the Pi, as you will need to assign addresses to connecting clients when they want to join the custom Wi-Fi network (see below)

To modify the Pi’s current DHCP client (dhcpcd) configuration, open up /etc/dhcpcd.conf as root:

 # I'm using nano here as it will be installed by default
 # Feel free to use which ever text editor you are most comfortable with.
 # Note also that I am using the "$" character to indicate that the
 # commands should be typed into the terminal - you would not actually
 # type the "$" character, only what follows it.
 $ sudo nano /etc/dhcpcd.conf

Add the following lines at the bottom of the file:

 interface wlan0
 static ip_address=192.168.42.1/24
 nohook wpa_supplicant

This tells the DHCP client to use a static IP address for the wireless network interface, as well as preventing the WPA supplicant hook from launching on this interface. The ever-useful Arch Wiki page for dhcpcd has some more information on this.

Install isc-dhcp-server and hostapd

Two pieces of software which will be working with these interfaces and are required for the Pi access point are:

  • isc-dhcp-server : Responsible for assigning the devices that will be connecting to the access point IP addresses.
  • hostapd : Responsible for managing the authentication of devices to the access point.

To install this software, open up a terminal on the Pi and issue the following command:

 # Install the required software
 $ sudo apt install hostapd isc-dhcp-server


You may see that the isc-dhcp-server failed to start upon completion of the installation. That’s OK, it will be resolved once you complete the next steps.

Configure isc-dhcp-server

Open up the /etc/dhcp/dhcpd.conf file:

 $ sudo nano /etc/dhcp/dhcpd.conf

Remove the comment on the line stating authoritative;:

 # If this DHCP server is the official DHCP server for the local
 # network, the authoritative directive should be uncommented.
 authoritative;

And then either just after authoritative; or at the end of the file, add the following lines:

 subnet 192.168.42.0 netmask 255.255.255.0 {
         range 192.168.42.10 192.168.42.250;
         option broadcast-address 192.168.42.255;
         option routers 192.168.42.1;
         option domain-name "local";
         option domain-name-servers 8.8.8.8, 8.8.4.4;
 }

These lines configure the DHCP server working on the wireless interface to hand out IP addresses beginning from 192.168.42.10 through to 192.168.42.250, with a broadcast address of 192.168.42.255, as well as specifying the router address to be 192.168.42.1 (note that this is the same static IP address as was configured on the wlan0 interface above).

Next you will need to open up the /etc/default/isc-dhcp-server file:

 $ sudo nano /etc/default/isc-dhcp-server

Change the “INTERFACESv4” line to contain wlan0, and comment out the “INTERFACESv6”:

 INTERFACESv4="wlan0"
 #INTERFACESv6=""

Configure isc-dhcp-server systemd entry

After installation the isc-dhcp-server will automatically run after boot, however if it attempts to start before the network interface is ready it will fail - and not attempt to start again. I obviously wasn’t the first one to come across this issue and found this StackOverflow answer which explains exactly what you need to do to resolve this issue. It is explained quite explicitly and so I won’t reproduce it here.

Configure hostapd

Next you will need to edit the hostapd configuration file - but first we need to copy and extract an example configuration file within which we will make our changes:

 $ cd /etc/hostapd/
 $ sudo cp /usr/share/doc/hostapd/examples/hostapd.conf.gz .
 $ sudo gunzip ./hostapd.conf.gz

Open this file up in the text editor:

 $ sudo nano /etc/hostapd/hostapd.conf

You will need to find the lines in the file which set the configuration variables below. In some cases you will need to modify the value and in others you will need to un-comment and modify the value:

 interface=wlan0
 driver=nl80211  
 ssid=mitmdemo                                  
 macaddr_acl=0
 auth_algs=1   
 wmm_enabled=0  
 wpa=2
 wpa_passphrase=password2600           
 wpa_key_mgmt=WPA-PSK               
 wpa_pairwise=TKIP
 rsn_pairwise=CCMP      

You can set your own name for the SSID and your own pass phrase

Next you need to open the /etc/default/hostapd file:

 $ sudo nano /etc/default/hostapd

And un-comment the line specifying DAEMON_CONF and edit it such that it reads like this:

 DAEMON_CONF="/etc/hostapd/hostapd.conf"

mitmproxy Installation and Setup

The version of mitmproxy which exists in the Raspbian Stretch repository is quite old (0.18). I recommend installing the latest version (currently 4.x) from the source repository.

The current source installation instructions for mitmproxy don’t mention this, but you actually need Python 3.6. The version of Raspbian Stretch I am using has only Python 3.5. So first we will need to upgrade the version of Python on the Pi.

Download, Compile and Install Python 3.7

What follows are instructions which mirror and enhance those as found here: https://www.scivision.co/compile-install-python-beta-raspberry-pi/

Install the build prerequisites

First you will need to make sure that you have all the required build dependencies for compiling Python:

 $ sudo apt install libffi-dev libbz2-dev liblzma-dev libsqlite3-dev libncurses5-dev libgdbm-dev zlib1g-dev libreadline-dev libssl-dev tk-dev build-essential libncursesw5-dev libc6-dev openssl git

Download the latest Python 3 source

As of early January 2019 the latest current release of Python is 3.7.2. To install the latest version of Python copy the link to the .tar.gz file from the the CPython releases page. Navigate to a suitable directory and download the release there (I will be using the latest release at the time of this post - 3.7.2):

 $ cd /home/pi/Downloads
 $ wget https://github.com/python/cpython/archive/v3.7.2.tar.gz 

Next, stay in your download directory and extract the files from the archive:

 $ tar -xf v3.7.2.tar.gz 

This will create a directory called “cpython-3.7.2” in the same location as the tar file.

Compile Python

  1. Open up a terminal and to where the Python release was downloaded and extracted and cd into the cpython directory:

     $ cd cpython-3.7.2
  2. Next run the configure script:

    $ ./configure --prefix=$HOME/.local --enable-optimizations
  3. The configure step should finish without any issues. You can then run the compile step:

    $ make -j -l 4
  4. Once the compilation has completed you can install the newly compiled Python 3 binaries:

    $ make install
  5. Finally you will need to add the Python binary installation directory to your PATH. Open up your ~/.bashrc file in a text editor and add the following line at the end of the file.

    export PATH=$HOME/.local/bin/:$PATH
  6. It is then a good idea to source your .bashrc to load this change:

    $ source ~/.bashrc
  7. You can now test that python3 and pip3 commands refer to your newly installed Python 3 binaries:

    $ which python3
    /home/pi/.local/bin/python3
    $ which pip3
    /home/pi/.local/bin/pip3

    If you see the paths above then you have succesfully installed the latest version of Python 3!

Install mitmproxy

Now that you have the latest version of Python 3, you can install mitmproxy by running:

 $ pip3 install --user mitmproxy

Once this has completed you should be able to verify that mitmproxy is installed correctly with the following command:

 $ mitmproxy --version
 Mitmproxy: 4.0.4
 Python:    3.7.2
 OpenSSL:   OpenSSL 1.1.0j  20 Nov 2018
 Platform:  Linux-4.14.79-v7+-armv7l-with-debian-9.6

Configure mitmproxy to run on start-up

Inspiration taken from this forum post: https://discourse.mitmproxy.org/t/mitm-proxy-on-ubuntu-startup/943/2

The last bit of configuration for mitmproxy is to set it up such that it runs at start-up. As I will be using mitmproxy as part of a live demonstration of a man-in-the-middle proxy I will be running the bundled mitmweb application. mitmweb runs a web app which can be viewed on a browser, from which a user can then view all the intercepted HTTP requests and responses.

To launch mitmweb open up a text editor and save the following contents in a file at the location /home/pi/start_mitmweb.sh:

#!/bin/bash
/home/pi/.local/bin/mitmweb --mode transparent --web-port 9090 --web-iface 0.0.0.0 &>> /var/log/mitmweb.log

Explanation for the command line arguments:

  • --mode transparent : this runs mitmproxy in transparent proxy mode, which forwards the requests and responses on to their intended destinations after being inspected.
  • --web-port 9090 : this tells the mitmweb application on which port to expose the web interface.
  • --web-iface 0.0.0.0 : this tells the mitmweb application that the web interface should be visible and accessible on the Pi’s LAN IP address.

Next we need to create a systemd service file…

 $ sudo nano /etc/systemd/system/mitmweb.service

… with the following contents:

 [Unit]
 Description=mitmweb service
 After=network.target
 
 [Service]
 Type=simple
 User=root
 ExecStart=/home/pi/start_mitmweb.sh
 Restart=on-failure
 RestartSec=5
 
 [Install]
 WantedBy=multi-user.target

Enable this service to run automatically on start-up by running the following commands:

 $ sudo systemctl daemon-reload
 $ sudo systemctl enable mitmweb.service

Network traffic configuration

Modify iptables rules

To correctly intercept and forward traffic between the wireless interface and the wired interface you will need to issue the following commands to set up the “iptables” firewall rules:

 $ sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
 $ sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT
 $ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
 $ sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
 $ sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8080

Next save the iptables configuration to a file:

 $ sudo sh -c "iptables-save > /etc/iptables.up.rules"

Now we will modify a script to issue a command which will restore these rules at start-up. Open up the “rc.local” file:

 $ sudo nano /etc/rc.local

And enter the following statement before the “exit 0” line at the end of the file:

 iptables-restore < /etc/iptables.up.rules

Enable traffic forwarding

By default the operating system will not allow IP traffic to be forwarded from one network to another. To change this behavior run the following command:

 $ sudo sysctl -w net.ipv4.ip_forward=1

And then to ensure that this setting is persisted between boot cycles, open the /etc/sysctl.conf file:

 $ sudo nano /etc/sysctl.conf

And find the commented line which says “#net.ipv4.ip_forward=1” and uncomment it:

 ...
 # Uncomment the next line to enable packet forwarding for IPv4
 net.ipv4.ip_forward=1
 ...

Shutdown / Reboot

If you have made it this far you should be ready to shutdown or reboot the Pi - the next time it starts up it will bring up a wireless access point and run the transparent MITM proxy!

Demo

You will need a phone or a laptop to connect to the access point and another laptop to view intercepted traffic. I am using an iPhone to connect to the access point and my laptop to view the intercept traffic.

Make sure your laptop is on the same network as that where the Raspberry Pi will be plugged into via the Ethernet cable.

Observe HTTP traffic

  1. Run an Ethernet cable from an available port on your router through to the Ethernet port on your Raspberry Pi 3.
  2. Plug in the micro USB cable.
  3. You should see the red (power) and green (activity) LEDs on the Pi light up. The lights in the LAN port should also soon start blinking.

    Raspberry Pi 3 plugged in Raspberry Pi 3 plugged in
  4. Use your router’s admin console (or maybe a search with nmap) to determine the IP address of the Pi on your network.

    Admin console of router showing Pi LAN IP address Admin console of router showing Pi LAN IP address
  5. Open up a browser tab on your laptop and navigate to http://<ip address of RPi3>:9090/

    • You should see the mitmweb application running in the browser.
  6. Wait a few seconds, and then use your phone to search for a new Wi-Fi access point.

    • You should see one called “mitmdemo”.
  7. Connect to the “mitmdemo” access point and use the password “password2600” to join the network.

    mitmdemo Wi-Fi network mitmdemo Wi-Fi network
  8. Once connected, open up a web browser and navigate to http://www.example.com (make sure it is http and not https).

  9. Observe the traffic in the mitmweb application running in the laptop browser tab.

    mitmweb intercepting HTTP traffic mitmweb intercepting HTTP traffic
  10. If you attempt to view an https website you may receive a warning in your browser.

    Warning when browsing https website Warning when browsing https website

Observe HTTPS traffic

  1. While you are still connected to the “mitmdemo” access point, navigate in your phone’s browser to http://mitm.it
  2. You will observe a website giving you a choice of links which will download the root certificate for the relevant platform.

    http://mitm.it http://mitm.it
  3. Tap the icon which resembles the platform you are currently using. It will download a certificate which needs to be installed into your device’s trusted store, the instructions of which vary from platform to platform. See below some screenshots of the iOS experience.

    • Note for iOS: After the certificate has been installed you will need to manually activate it by going to Settings -> About -> Certificate Trust Settings and toggling on the certificate for “mitmdemo”.

    mitmdemo certificate installed mitmdemo certificate installed
    mitmdemo certificate active mitmdemo certificate active

  4. Once you have installed and trusted the certificate, you should be able to visit https websites and view the requests and responses without any warning on your device.

    No further warnings fro https content No further warnings fro https content

I recommend that you remove the certificate from your device’s trusted store once you have completed the demonstration.

Conclusion

I hope that you found this interesting and perhaps learned something from my experience in creating a portable and demo-friendly man-in-the-middle proxy! Let me know in the comments if you have any questions.