---
layout: post
title: Caching Docker, Python, Nix, and HuggingFace
date: 2026-01-20
tags: [nixos]
---

I like efficiency. I don't like waiting for downloads or builds. I like my LAN
cables to transfer as fast as possible, and I try to always connect by wire to
my network rather than use wifi.

One thing that I despise is downloading the same thing twice.

Particularly if I had downloaded it just a few minutes or hours ago. Some
docker image, some HuggingFace model, the same python dependencies that I just
downloaded on my laptop, downloading on my desktop machine. And most package
ecosystems have immutable or content-addressed artifacts: content addressed
`docker` layers, `nix` store paths derived from inputs, git references for `hf`
models.

I run caches for Docker registries, PyPI and the Nix binary cache. There's
something deeply satisfying about the efficiency of running a `pip install` and
seeing most packages transfer at my 2.5Gbps LAN speed rather than my WAN's
100Mbps connection's 12.5 megabytes per second. I set up authorization and
credentials on the cache once, so that I don't need to update all credentials
when/if they expire.

Most of the time, one assumes that bandwidth is cheap and storage is expensive,
but nowadays 1TB is about U$D 15. Since my connection is 100Mb, every 1Tb that
I duplicate is about twenty hours of wasted idle time waiting for the network.

Check these out in my [NixOS recipes repo](https://github.com/eordano/recipes).
It uses the NixOS test framework to check with multiple VMs that it actually
works. Here are some sources to read about that:

- [My notes on NixOS](https://eordano.com/words/notes-on-nixos/)
- [On the power of VM tests](https://bmcgee.ie/posts/2025/02/nixos-the-power-of-vm-tests/)
- [Source code for NixOS VM test networking infrastructure](https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/testing/network.nix)

To use it, add to the machine that will act as a server:

```nix
modules.services.nix-cache = {
  enable = true;
  domain = "nix-cache.my-custom-domain.com";
  maxCacheSize = "500g";
};
```

And on the other computers:

```nix
behaviors.nix-cache = {
  enable = true;
  cacheUrl = "https://nix-cache.my-custom-domain.com";
};
```

Configuring the `docker` proxy was the most cumbersome. I had to distribute a
CA certificate around in my machines with a `systemd` service to automate that.
The nix store was also quite painful; the existing solutions (nix-serve,
harmonia, attic) didn't quite work for me; so I ended up with a [setup similar
to this one by Solene
Rapenne](https://dataswamp.org/~solene/2022-06-02-nixos-local-cache.html).

After it's all set up, it saves me tens of gigabytes of bandwidth per day. It's
quite satisfying.