Go to file
2024-03-06 01:22:10 -06:00
oci/nonfree Add nonfree Containerfile 2024-02-29 16:32:07 -06:00
src Add compose container image and push to registry 2024-02-28 08:40:58 -06:00
.gitignore Add .gitignore 2024-02-11 08:01:51 -06:00
.gitlab-ci.yml Use buildah for nonfree image 2024-03-06 01:22:10 -06:00
LICENSE Initial commit 2024-02-01 19:25:46 -06:00
ostree-engine Use Bash heredoc for charm.repo 2024-02-28 18:47:54 -06:00
README.org Add doc for Podman-in-Podman 2024-03-05 14:30:24 -06:00

vauxite-build

This is my personalized Vauxite configuration. Vauxite is an unofficial Xfce variant of ostree-based Fedora. I'm currently working with Fedora developers to make it an official variant. There is already a sort-of-official treefile for Vauxite at https://pagure.io/workstation-ostree-config, as well as an experimental Vauxite container image hosted at https://quay.io/repository/fedora-ostree-desktops/vauxite.

The vauxite.json treefile and under the src directory contains my personal customizations and differ from the upstream one, so they may not be suitable for everyone.

How I use this repository

Setup GitLab runner with Podman

  1. Install GitLab Runner.
  2. Create a new runner from the GitLab UI.
  3. Use the authentication token from the GitLab UI to register a new runner on the machine hosting the runner. Select the Docker executor.

    sudo systemctl enable --now gitlab-runner.service
    sudo gitlab-runner register --url https://git.hyperreal.coffee --token <TOKEN>
  4. Add the following lines to /etc/gitlab-runner/config.toml for Podman:

    We need to set privileged = true in order to create new namespaces inside the runner container. See https://github.com/coreos/rpm-ostree/pull/429

    [[runners]]
      environment = ["FF_NETWORK_PER_BUILD=1"]
      [runners.docker]
        host = "unix:///run/podman/podman.sock"
        tls_verify = false
        image = "registry.fedoraproject.org/fedora:latest"
        privileged = true
        volumes = ["/build-repo", "/cache", "/source-repo"]
  5. Restart the gitlab-runner:

    sudo gitlab-runner restart

    We should now be ready to use the Podman runner.

Setup Podman-in-Podman runner

To get the Podman-In-Podman runner working for a self-hosted GitLab instance, we need to clone the repository and build a custom image passing the GitLab instance URL as a build argument.

  1. Login to the right container registry.

    podman login -u hyperreal -p <registry api token> git.hyperreal.coffee:5050
  2. Clone, build, and push to registry.

    git clone https://gitlab.com/qontainers/pipglr
    cd pipglr
    podman build --env GITLAB_URL=https://git.hyperreal.coffee -t git.hyperreal.coffee:5050/fedora-atomic/containers/pipglr:latest .
    podman push git.hyperreal.coffee:5050/fedora-atomic/containers/pipglr:latest
  3. Create a new runner from the GitLab UI.
  4. Use the authentication token from the GitLab UI to create a podman secret.

    echo '<token>' | podman secret create REGISTRATION_TOKEN -
  5. Ensure config.toml exists in the current working directory.

    touch ./config.toml
  6. Use the custom image we pushed to the container registry.

    IMAGE="git.hyperreal.coffee:5050/fedora-atomic/containers/pipglr:latest"
  7. Register the runner.

    podman container runlabel register $IMAGE
  8. Edit config.toml to use the custom Fedora image from this project.

    [runners.docker]
      image = "git.hyperreal.coffee:5050/fedora-atomic/containers/fedora:latest"
  9. Setup the storage and cache volumes.

    podman container runlabel setupstorage $IMAGE
    podman container runlabel setupcache $IMAGE
  10. Create a podman secret using config.toml.

    podman secret create config.toml ./config.toml
  11. Run the runner.

    podman container runlabel run $IMAGE
  12. Enable the runner user to run services after logout:

    sudo loginctl enable-linger $(id -u)

The new Podman-In-Podman runner should now appear in the GitLab UI.

Expand user namespace
  1. pipglr excludes three UID/GIDs from being used by job-level containers. One for each of root, podman, and runner users. Since most distributions set 65536 as the default maximum number of IDs to allocate for user namespaces (via /etc/login.defs), distribution images that assign essential users a high UID/GID will fail to setup the namespace for pipglr jobs. The workaround to this is increasing the UID/GID limit on the host by three. See https://gitlab.com/qontainers/pipglr/-/blob/main/root/setup.sh?ref_type=heads#L86 /etc/subuid and /etc/subgid:

    jas:100000:65539
  2. Make the changes take effect.

    podman system migrate

For debugging purposes:

podman logs --since 0 pipglr

It may be necessary to build a custom pipglr image with more verbose logging. The runner.service and podman.service files have log-level options that can be set to "debug".

Notes about .gitlab-ci.yml

  • The package container-selinux is required for the vauxite-compose-job so SELinux works inside the runner container. I'm considering having a custom image built on a weekly basis from registry.fedoraproject.org/fedora:latest that contains updated packages and the required dependencies, which I would then just use as the runner's container image. The registry.fedoraproject.org/fedora:latest image doesn't seem to be updated at all. See https://git.hyperreal.coffee/fedora-atomic/containers
  • BUILD_REPO and SOURCE_REPO are the directories /build-repo and /source-repo. If these values are changed, then we would need make equivalent changes to the volumes directive in /etc/gitlab-runner/config.toml if we want to keep persistent storage of those repos across pipeline runs. Eventually there will be a conditional in the .gitlab-ci.yml to clean these volumes if another variable (say CLEAN_BUILD) is set to true.

Rebase to the container image

On PCs using Vauxite, rebase to the container image from the registry:

rpm-ostree rebase ostree-unverified-registry:git.hyperreal.coffee:5050/fedora-atomic/vauxite-build/vauxite-nonfree:39
sudo systemctl reboot