Skip to content

nix.md

Latest commit

 

History

History
60 lines (50 loc) · 3 KB

nix.md

File metadata and controls

60 lines (50 loc) · 3 KB

Nix Setup

Elodin uses Nix for building docker images and for CI dependencies. We heavily use Nix flakes which requires a little more setup. The easiest way to use Nix on macOS or Linux is to use the installer located here: https://zero-to-nix.com/start/install

If you want to use the official Nix installer, you will need to follow the instructions located here: https://nixos.wiki/wiki/Flakes

Unified Development Shell

Elodin provides a unified development shell that includes all necessary tools for development:

  • Run nix develop to enter the unified shell
  • This single shell includes tools for Rust, Python, C/C++, cloud operations, documentation, and git-lfs
  • No need to switch between different shells for different tasks
  • git-lfs is included to handle large files in the repository

macOS VM

Often you want to build Linux binaries with Nix on your mac. This guide shows how to setup a VM using OrbStack, that supports remote builds.

  1. Install OrbStack - https://orbstack.dev
  2. Create new NixOS machine called nixos - use whatever the default NixOS version is screenshot of orbstack
  3. Add this to /var/root/.ssh/config on macOS - you may need to create the directory first and run your editor using sudo
Host orb
  Hostname 127.0.0.1
  Port 32222
  1. Still in macOS - Add this to /etc/nix/machines - replacing user with your username. You will also need to edit the file as root
ssh://user@orb x86_64-linux,aarch64-linux /Users/user/.orbstack/ssh/id_ed25519 20 20 nixos-test,benchmark,big-parallel,kvm - -
  1. Run orb to enter the nixos machine
  2. Add this line below the users declaration in /etc/nixos/configuration.nix. You will likely need to add some sort of text-editor. This can be done temporarily with nix-shell -p vim
nix.settings.trusted-users = ["root" "@wheel"];
  1. Run sudo nixos-rebuild switch to rebuild the nixos config
  2. Next sudo ssh -i ~/.orbstack/ssh/id_ed25519 user@orb on macOS replacing user with your username. If everything work this should drop you into your vm.
  3. Test your build by running nix build --impure --expr '(with import <nixpkgs> { system = "x86_64-linux"; }; runCommand "foo" {} "uname > $out")' in macOS
  4. Profit!

OrbStack disk exhaustion

Qt builds (e.g. qtdeclarative) can overflow the default /build tmpfs inside the OrbStack VM and fail with No space left on device while writing assembler output. Move the build dir to disk instead of tmpfs:

  1. In /etc/nixos/configuration.nix on the VM add: 
    nix.settings.build-dir = "/var/cache/nix-build";
systemd.tmpfiles.rules = [ "d /var/cache/nix-build 1777 root root -" ];
  2. Create the directory once (until the tmpfiles rule is active): 
    sudo install -d -m 1777 /var/cache/nix-build
  3. Rebuild the VM configuration and retry your build: 
    sudo nixos-rebuild switch
nix build …

If you prefer tmpfs, increase it instead via tmpfsSize on the build machine, but the disk-backed build-dir above is usually simpler.