First step to document my work: have a place to document my work!
This website is running in a Podman pod on a Debian 10 virtual machine.
Why Podman? Well because I’ve drank Red Hat’s kool-aid, (and I’ve been gifted the Podman for DevOps book by a Red Hat instructor).
I wanted Podman 4, the latest release, but Debian unstable is still stuck on 3.
federico@centrostella:~$ podman -v
podman version 3.0.1
So, let’s install it from source.
Installing Podman
Everything is well documented on their website, in the Podman Installation page, but we want Podman 4 with Netavark support: the newest, fastest, rust-yest network driver available
Fire up your terminal and type:
sudo apt-get install \ btrfs-progs \ crun \ git \ golang-go \ go-md2man \ iptables \ libassuan-dev \ libbtrfs-dev \ libc6-dev \ libdevmapper-dev \ libglib2.0-dev \ libgpgme-dev \ libgpg-error-dev \ libprotobuf-dev \ libprotobuf-c-dev \ libseccomp-dev \ libselinux1-dev \ libsystemd-dev \ pkg-config \ uidmap \ curl \ fuse-overlayfs
Then, we’re going to need to install golang, conmon and crun/runc.
I won’t bore you with the deatails of what they do – just run this
# Install conmon cd git clone https://github.com/containers/conmon cd conmon export GOCACHE="$(mktemp -d)" make sudo make podman # Install runc cd git clone https://github.com/opencontainers/runc.git cd runc make BUILDTAGS="selinux seccomp" sudo cp runc /usr/bin/runc
Finally, build and install podman
git clone https://github.com/containers/podman/ cd podman make sudo make install
and just like that:
$ podman -v
podman version 4.3.1
Install Netavark and Aardvark
Next, it’s time divert from their guide and install Netavark and Aardvark.
To do so, you’re gonna need Rust.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Type 1 at the prompt to use all defaults and then type
source "$HOME/.cargo/env"
Luckily a couple days ago aardvark and netavark have been added to the Cargo repository so it’s as easy as typing
cargo install netavark aardvark-dns
We need to copy the build binary in one of the paths that podman will look for it, otherwise it will throw an error. While I haven’t found any documentation for it, strace comes to our help
So we just need to
sudo cp ~/.cargo/bin/aardvark-dns /usr/lib/podman/.
We’re not done yet
If we try to run our first container everything seems fine
$ podman run docker.io/hello-world
Trying to pull docker.io/library/hello-world:latest...
Getting image source signatures
Copying blob 2db29710123e done
Copying config feb5d9fea6 done
Writing manifest to image destination
Storing signatures
Hello from Docker!
This message shows that your installation appears to be working correctly.
Unfortunately, I had a terrible, hidden problem that quickly showed up on my tiny VPS with 25G of disk.
Let’s try to run something bigger, like a postgres container
$ podman container run --rm docker.io/postgres:15
Trying to pull docker.io/library/postgres:15...
Getting image source signatures
Copying blob 025c56f98b67 done
Copying blob 26dc25c16f4e done
Copying blob 813fd6cf203b done
Copying blob a032d8a894de done
Copying blob 8ebb44a56070 done
Copying blob 40dba7d35750 done
Copying blob 7024f61bf8f5 done
Copying blob 23f986b322e8 done
Copying blob 1fb05ff7a8d6 done
Copying blob 74afc7d9bc5c done
Copying blob 7c2c7eebef2f done
Copying blob bdd9df7f1d37 done
Copying blob 33d269a3a052 done
Copying config 4c6b3cc10e done
Writing manifest to image destination
Storing signatures
And now let’s check our disk usage:
$ du -h ~/.local/share/containers/storage
2,8G .
What? 2.8G for a simple container?
Oh yes! Because podman is using the wrong driver to store container images
$ podman system info | grep graphDriver
graphDriverName: vfs
Docker’s documentation defines VFS as:
The VFS storage driver is not a union filesystem; instead, each layer is a directory on disk, and there is no copy-on-write support. To create a new layer, a “deep copy” is done of the previous layer. This leads to lower performance and more space used on disk than other storage drivers.
https://docs.docker.com/storage/storagedriver/vfs-driver/
This is absolutely not what we want. Luckily in the first step I had you install fuse-overlayfs
, which allows for the stratification of images the container system is known for.
sudo curl https://raw.githubusercontent.com/containers/podman/main/vendor/github.com/containers/storage/storage.conf -o /etc/containers/storage.conf
The magic lines, in this case, are the following:
[storage] driver = "overlay"
Give the podman system a nice reset:
podman system reset
If it complains about User-selected graph driver "overlay" overwritten by graph driver "vfs" from database - delete libpod local files to resolve
, try running it again. If it still complains, delete the .config/container/
folder
Let’s try running our postgres container once again
podman run docker.io/postgres:15
Trying to pull docker.io/library/postgres:15...
Getting image source signatures
Copying blob 26dc25c16f4e done
[...]
This time, disk usage has SIGNIFICANTLY reduced:
du -h ~/.local/share/containers/storage
388M .
Much better!
Switching to Netavark networking
After all this work, we are still not one. One last step is enabling our hard-earned, self-compiled Netavark network stack
Copy the default containers.conf file and edit it
sudo cp /usr/share/containers/containers.conf /etc/containers/. sudo nano /etc/containers/containers.conf
Change these lines:
[network] network_backend = ""
To this:
[network] network_backend = "netavark"
Give another reset just to be sure
$ podman system reset
So we can finally see our well-earned goal:
$ podman system info | grep networkBackend
networkBackend: netavark
Next post I’ll explain how to set up a podman pod with wordpress and mysql.