Setting up BPF Experiments on MacOS

Preface: This is not an article defining what the Berkeley Packet Filter(BPF) is. The internet is full of resources on the topic and I’m definitely not an expert on it. Check out this talk by Brendan Gregg if you’d like to know more.

One of the biggest pains in the butt to writing BPF code on MacOS is getting the right environment setup. I know, I know, use the cloud… Or get a Linux laptop. Yes, those two options would make things simpler, but I’m old-fashioned and like to run things locally, which means the cloud won’t work for my experiments. And no, I’m not ready to get a Linux laptop, I just have too many things on-the-go to spend hours debugging my wifi drivers.

NOTE: OK, I know things have been more stable since my last attempt in the early 2000’s, but last time I looked into a few months ago, there were still a gazillion posts about tweaking drivers to get Ubuntu running even on officially supported Dell devices. And yes, I know I’m just making excuses and definitely delaying the inevitability that I will go back to a Linux laptop at some point. Suggestions as to which ones are welcome.

So, in short, my requirements are to be able to edit BPF code using VSCode on my Mac with little effort. Easy right? To make this a little bit easier, I put together a GitHub repository with some tools to get a reproducible environment up quickly. Here are a few tools you’ll need installed if you want to follow along:

• VirtualBox: provides support for running virtual machines
• Vagrant: provides command line tooling for setting up and managing virtual machines
• VSCode: a code editor

Apparently I like tools that start with the letter V. Moving on.

Setting up the virtual machine

First, let’s clone the repository, which contains the Vagrant configuration file used to launch an ubuntu/bionic virtual machine. Then we’ll launch the VM using the vagrant command:

git clone https://github.com/codeboten/bpftracing.git && cd bpftracing
vagrant up # this may take a couple of minutes

While we’re waiting for the virtual machine setup to complete, we can take a look at the Vagrantfile.

Vagrant allows you to define a shell provisioning option, which let us define commands in a script. The $script block at the top defines a set of commands that will install all the tools needed to compile BPF programs inside the virtual machine once it’s booted. Once the provisioning is completed, let’s use SSH to connect to our VM and try compiling our BPF program.

vagrant@ubuntu-bionic:/vagrant$ vagrant ssh
vagrant@ubuntu-bionic:/vagrant$ cd /vagrant
vagrant@ubuntu-bionic:/vagrant$ make
clang -O2 -target bpf -c bpf_program.c -I/kernel-src/tools/testing/selftests/bpf -o bpf_program.o
clang -o monitor-exec -lelf -I/kernel-src/samples/bpf -I/kernel-src/tools/lib -I/kernel-src/tools/perf -I/kernel-src/tools/include -L/usr/local/lib64 -lbpf \
        /kernel-src/samples/bpf/bpf_load.c loader.c
vagrant@ubuntu-bionic:/vagrant$ ls -la monitor-exec
-rwxr-xr-x 1 vagrant vagrant 28128 Apr 24 05:40 monitor-exec

If you see the following output and can see a binary file named monitor_exec in the current directory, you’re in business!

Editing in VSCode

Now that we have a VM to allow us to compile our BPF programs, we’re almost there! I’m a big fan of using VSCode as an editor, after years of using vim, VSCode just strikes that nice balance of lightweight enough and a decent feature set to make it the perfect editor for me. As you saw earlier when you ran make inside the VM, Vagrant automatically mounted our code inside the VM. We could edit the code locally using VSCode and compile it inside the VM. But what I would really like, is getting the benefits of being able to see the header files I need while writing that code, to take advantage of things like autocomplete inside the editor.

Thanks to this quick to read and definitely bookmark-worthy article by Liz Rice, we’re going to tie this last piece together. Seriously, go read that article, you’ll be happy you did. Unlike the example, I ended up saving a custom SSH configuration file in my bpftracing directory to make life more “fun”. The following generates the custom configuration, in which I updated the Host to have the more descriptive name bpftracing .

vagrant ssh-config > .vagrant_ssh_config

The only additional step to configure VSCode, which is mentioned in that article, was to go through the “Remote-SSH: Open Configuration File…” → “Settings” options from the ⌘⇧P menu.

Once the remote connection to my bpftracing “remote” host was established, I add the /vagrant folder and voilà, my code is available in my IDE from the remote host! The terminal window in VSCode (^`) also runs inside the VM, which makes writing code and compiling it a breeze.

Cleaning up

Last but not least, we need a mechanism to clean up our experiments. There’s a handy make target that will remove all the compiled files, and destroy the Vagrant environment for us.

make clean-everything

What next?

Great reads

If you’re looking for inspiration, I highly recommend Linux Observability with BPF and BPF Performance Tools, they’re full of information about BPF and code examples. Well that’s pretty much it, you’re now ready to write BPF code from your MacOS environment, have fun!