In detail

Following the kind of contribution (new additions, fixes or upgrades), these simple steps will allow you to start contributing to guix:

git clone guix itselft
from the guix repository, do:

sh guix shell -D guix -CPW ./bootstrap ./configure make -j$(nproc) ./pre-inst-env guix build hello add and commit your changes, watch the commit message
beware your synopses and descriptions
remember to run the package tests, if relevant
check the license
use an alphabetical order in input lists
no sign off your commits
don’t forget to use lint/style/refresh -l/dependents to check your code

Boring and routinary, right ?

Use sourcehut

Most of all the of the previous can be run automatically with help of sourcehut build farm #ci capabilities. Just simply, push the guix repository to sr.ht. At this point, it is possible to use this manifest file to run the lint/style/refresh -l/dependents testing stages on the yosys package definition, por example:

image: guix
shell: true
environment:
  prj: guix.guix
  cmd: "guix shell -D guix -CPWN git nss-certs -- ./pre-inst-env guix"
sources:
  - https://git.sr.ht/~csantosb/guix.guix
tasks:
  - def_pkg: |
      cd "$prj"
      _pkg=$(git log -1 --oneline | cut -d':' -f 2 | xargs)
      echo "export pkg=$_pkg" >> "$HOME/.buildenv"
  - setup: |
      cd "$prj"
      guix shell -D guix -CPW -- ./bootstrap
      guix shell -D guix -CPW -- ./configure
      guix shell -D guix -CPW -- make -j $(nproc)
  - build: |
      cd "$prj"
      eval "$cmd build --rounds=5 $pkg"
  - lint: |
      cd "$prj"
      eval "$cmd lint $pkg"
  - style: |
      cd "$prj"
      eval "$cmd style $pkg --dry-run"
  - refresh: |
      cd "$prj"
      eval "$cmd refresh -l $pkg"
  - dependents: |
      cd "$prj"
      eval "$cmd build --dependents $pkg"
triggers:
  - condition: failure
    action: email
    to: builds.sr.ht@csantosb.mozmail.com

Submit the manifest with

hut builds submit # --edit

You’ll be able to log into the build farm to follow the build process or to debug it with

hut builds ssh ID

Check the log here. As you can see, it fails: building of yosys succeeds, but building of packages which depend on it (--dependents) fails.

Advanced

Sourcehut provides a facility to automatize patch submission and testing. Using its hub integrator, one may just send an email to the email list related to your project (guix in this case), which mimics guix behavior for accepting patches.
The trick here consists on appending the project name as a prefix to the subject of the message, for example [PATCH project-name], which will trigger the build of previous .build.yml manifest file at the root of the project, after applying the patch. Neat, right ?
If you followed right here, you’ll notice that previous build manifest file is monolithic, affecting always the same package (yosys), which is kind of useless, as we are here interested in testing our patch. Thus, the question on how to trigger a custom build containing an updated $pkg variable related to the patch to test remains open.
To update the contents of the $pkg variable in the build manifest, one has to parse the commit message in the patch, extracting from there the package name. This is not a problem, as guix imposes clear commit messages in patches, so typically something like

* gnu: gnunet: Update to 0.23.0

or

* gnu: texmacs: Add qtwayland-5

Hopefully, parsing these messages to get the package name, and so the value of $pkg is trivial.
Then, it remains to include in our build manifest a first task which updates the contents of "$HOME/.buildenv". This file is automatically populated using the environment variables in the manifest, and its contents are sourced at the beginning of all tasks. This mechanism allows passing variables between tasks.

echo "export pkg=value" >> "$HOME/.buildenv"

Send your contribution

Finally, once your changes go through all the tests,

use git send-email to create and send a patch
consider reviews, if any, updating your patch accordingly with git ammend
resend a new patch including a patch version (v1, v2 ...)

Interested ? Consult the documentation for details, you’ll learn a lot about how to contribute to a common good and collaboration with other people.
#ciseries

Alu

Let’s consider now a variant of the previous example with open-logic. Here, we concentrate on a toy design only for demonstration purposes, a dummy alu emulator, which uses #osvvm as verification framework and relies on a few #openlogic blocs. In this case, its dependencies are defined in a manifest.scm file, including both fw-open-logic and osvvm, among other dependencies.
Install dependencies locally, in a new profile with

cd alu
mkdir _deps
export GUIX_PROFILE=open-logic/_deps
guix install -P $GUIX_PROFILE -m .builds/manifest.scm
. $GUIX_PROFILE/etc/profile

In this case, we will test the design using, first, a custom made makefile. Secondly, we will use hdlmake to automatically produce our makefile. Similarly to previous #openlogic example, two build manifest are used:

profile1
profile2

You’ll realise how some of the tasks are common with the case of previous #openlogic example (update channels, auth and update profile).

osvvm

In this case, we also need to compile osvvm libraries

compile__osvvm, produce a compiled version of #osvvm verification libraries; this is necessary as we are using here the tcl scripts included in the library itself to follow the correct order of compilation. Libraries will appear within the local profile under $GUIX_PROFILE/VHDL_LIBS/GHDL-X.Y.Z

test

test, for a fully custom made testing pipeline; in this case, using a Makefile
Just simply, source the .envrc file where the local $GUIX_PROFILE variable is defined, cd to the ghdl directory and call make to compile the design and run the simulation in two steps: first, clean all and include sources in its corresponding libraries with

sh make __clean_all __include

Then, produce a new Makefile using ghdl.

sh ./makefile.sh # ghdl --gen-makefile ...

Finally, run the simulation with

sh make GHDLRUNFLAGS="--stop-time=4us --disp-time --ieee-asserts=enable" run

This will produce a executable file before running it with the provided parameters.
You may notice that, in this case, you need to produce somehow your own Makefile, or equivalent pipeline, right ?

hdlmake

Wouldn’t it be nice if we had a tool to deploy online which produces makefiles for us ? It exists, and its name is #hdlmake.

test__hdlmake
Source the .envrc file where the local $GUIX_PROFILE variable is defined, cd to the .builds/hdlmake directory where all Manifest.py files are located, and call hdlmake to produce the Makefile. Finally, just run make to compile the design, produce an executable and run it.

Check the resulting logs inline here, for example.

Open logic

Let’s see how in detail using the cookbook as a starting point, and taking as a complete example the fw-open-logic #openlogic firmware package which comes with the electronics guix channel.
Get it with:

guix install fw-open-logic:out

Open logic is a useful #vhdl library of commonly used components, implemented in a reusable and vendor/tool-independent way. As any other #modernhw library, it includes tests sets for any of its components, using the vunit utility in this case.
To run the full tests suite use (user wide using the default $GUIX_PROFILE), install its dependencies, defined in a manifest.scm file (ghdl-clang and python-vunit in this case).

cd open-logic
guix install -m .builds/manifest.scm
cd sim
python3 run.py --ghdl -v

or local to the project, using a profile

cd open-logic
mkdir _deps
export GUIX_PROFILE=open-logic/_deps
guix install -P $GUIX_PROFILE -m .builds/manifest.scm
. $GUIX_PROFILE/etc/profile
cd sim
python3 run.py --ghdl -v

go remote

Now, how do we proceed online using #sourcehut #ci builds facility ? Builds will pop up a new environment based on an up to date guix-system image when we push a commit to git.sr.ht, provided we include a .build.yml build manifest file, or by a .build folder with up to 4 build manifest files, at the root of the git project [1]. Be careful: consider that this image is built daily using a crontab job, which is a good and a bad thing at the same time. From one side, you won’t be using the same environment for your tests, which breaks #reproducibility (see comments section below). On the other side, #guix is a rolling release, and new fancy features and new fixes are added every day. Keep this in mind.
Let’s create a .builds folder in a topic test branch, with the following contents:

manifest.scm, list of dependencies in our project
guix.scm, default guix repository, redundant, included here for convenience
channels.scm, list of guix channels remote repositories, in addition to the default guix repository, from where we pull packages
We will be using here my own electronics channel (no substitutes), as well as the guix science channel (which provides substitutes).
(note how here we load the local guix.scm file, instead of making use of the %default-channels global variable)

scheme (load "guix.scm") ;;; %default-channels key.pub, auth key to access substitutes of packages in guix channels

build manifests

From now on, every new push to the test #git branch will trigger the execution of the tasks defined in the three build manifest files

profile1
profile2
shell1

The two profile build manifest files use a slightly different approach, and are given here for comparison purposes only. The shell build manifest uses an isolated shell container within the image itself to illustrate this feature.
Inside the manifests, I declare the image to use, guix, and the global environment variables sourced before each task is run: prj (project name), srv (list of servers with substitutes), manifest and channels (pointing to the corresponding files) and key (same). It is important to declare a trigger action, to receive an email with all relevant information in case of failure (log, id, commit, etc.).

tasks

What’s interesting here is the list of tasks. Some of them are common to all three manifests

env, useful only for debugging
guix__update__channels, replace the default project local guix.scm file by the output of

sh guix describe --format=channels

The goal here is avoid pulling latest guix upstream, useless and cpu and time consuming, and using the local version instead. Remember that the guix system image we are using here is updated daily.

guix__auth, runs the authorize command to add the key.pub file to guix, so that we will be able to download package substitutes when necessary

sh sudo guix archive --authorize < "$key"

Here, one may opt by doing a

sh guix pull --channels="$channels"

as in profile2, to set the revision of the guix channels we are using (remember channels are nothing but git repositories).
Note how in profile1 and shell1 we opt for a different approach.
guix__update__profile, where we create a _deps folder to be used as a local $GUIX_PROFILE (defined in .envrc).
Then, one of

sh # profile1 guix time-machine --channels="$channels" -- \ package -p "$GUIX_PROFILE" \ --substitute-urls="$srv" \ -m "$manifest"

or

sh # profile2 guix \ package -p "$GUIX_PROFILE" \ --substitute-urls="$srv" \ -m "$manifest"

will install packages in $manifest into the $GUIX_PROFILE. I’m using here the time-machine mechanism to set the revision of the guix channels, depending if guix pull was run in the previous stage or not.
vunit, sets env variables in .envrc and runs python3 run.py --ghdl -v inside sim directory
Note that here, we are using ghdl-clang and python-vunit packages, provided respectively by guix-science and the electronics channel.
guix__shell__test, used by shell1, make use of time-machine (no former guix pull, then), to create a shell container, where to install project dependencies. Then, if calls inmediately run.sh to run the unit tests

sh guix time-machine --channels="$channels" -- shell -C --substitute-urls="$srv" -m "$manifest" -- ./.builds/run.sh

comments

You may check the logs of profile1, profile2 and shell1 manifests, including a section with logs per task, to better understand what’s going on here. Remember that #sourcehut gives ssh access to the builds by connecting to the runners in case of failures, which provides a practical way of debugging the manifest files.
You may see how, using the remove guix image, it is possible to deploy a series of tasks to test our #modernhw design as we develop it: we will get an email in case of failure to pass the tests. Here, I present three approaches: guix pulling to set the repositories revisions on use; time-machine, to achieve the same, and guix shell to create an isolated container. These three alternatives are not necessary here, of course, but are given as a simple and practical demo of what can be achieved with #guix, #sourcehut and #ci.
To conclude this long post, it is important to stress once again that the point on using #guix resides in its reproducibility capabilities. By keeping a couple of #plaintext files, namely the manifest.scm and channels.scm, one can obtain #determinism in the execution of the tests. Even if the guix image is upgraded and rebuilt daily (and so it changes), by fixing the revision of our channels (remember, guix pull or guix time-machine) we obtain always the same products out of our tests, as we run the same (project and tests) code, within exactly the same environment.

[1] It is also possible to automatically submit builds when a patch to a repo with build manifests is sent to a mailing list. This is achieved by appending the project name as a prefix to the subject of the message, for example [PATCH project-name].

Gitlab

The gitlab #gitforge includes tones of features. Among these, a facility called the container registry, which stores per project container images. Guix pack allows the creation of custom #reproductible environments as images. In particular, it is possible to create a docker image out of our manifest and channels files with

guix time-machine -C channels.scm -- pack --compression=xz --save-provenance -f docker -m manifest.scm

Check the documentation for options.
Remember that there are obviously alternative methods to produce docker images. The point on using guix resides on its reproducibility capabilities: you’ll be able to create a new, identical docker image, out of the manifest and channels files at any point in time. Even more: you’ll have the capacity to retrieve your manifest file out of the binary image in case your manifest file gets lost.
Then, this image must be loaded into the local docker store with

docker load < IMAGE

and renamed to something meaningful

docker tag IMAGE:latest gitlab-registry.whatever.fr/domain/group/NAME:TAG

go remote

Finally, pushed to the remote container registry of your project with

docker push gitlab-registry.whatever.fr/domain/group/NAME:TAG

At this point, you have an environment where you’ll run your tests using gitlab's ci features. You’ll set up your gitlab’s runners and manifest files to use this container to execute your jobs.
As an alternative, you could use a ssh executor running on your own fast and powerful hardware resources (dedicated machine, shared cluster, etc.). In this case, you’d rather produce an apptainer container image with:

guix time-machine -C channels.scm -- pack -f squashfs ...

scp this container file to your computing resources and call it from the #gitlab runner.

Github

The github is probably the most popular #gitforge out there. It follows a similar to #gitlab in its conception (pull requests and merge requests, you catch the idea ?). It also includes a container registry, and the set of features if offers may be exchanged with ease with any other #gitforge following the same paradigm. No need to go into more details.
There is a couple of interesting tips about using #github, though. It happens more usually than not that users encounter frequently problems of #reproducibility when using container images hosted on ghcr.io, the hosting service for user images. These images are usually employed for running #ci testing pipelines, and they usually break as upstream changes happen: updates, image definition changes, image packages upgrades, etc. If you read my dependencies hell post, this should ring a bell.
What can be done about in what concerns #modernhw ? Well, we have #guix. Let’s try a differente approach: building an image locally, and pushing it to #github registry. Let’s see how.

in practice

An example repository shows tha way to proceed. Its contents allow to create a docker container image to be hosted remotely. It includes all that’s necessary to perform remote #ci testing of a #modernhw #vhdl design.

docker pull ghcr.io/csantosb/hdl
docker images # check $ID
docker run -ti $ID bash

It includes a couple of #plaintext files to produce a #deterministic container. First, the channels.scm file with the list of guix chanels to use to pull packages from. Then, a manifest.scm, with the list of packages to be install within the container.
The image container may be build with

image=$(guix time-machine --channels=channels.scm -- \
             pack -f docker \
             -S /bin=bin \
             --save-provenance \
             -m manifest.scm)

At this point, it is to be load to the docker store with

docker load < $image
# docker images

Now it is time to tag the image

docker tag IMID ghcr.io/USER/REPO:RELEASE

and login to ghcr.io

docker login -u USER -p PASSWORD ghcr.io

Finally, the image is to be push remotely

docker push ghcr.io/USER/HDL:RELEASE

test

You’ll may test this image using the neorv32 project, for example, with:

docker pull ghcr.io/csantosb/hdl
docker run -ti ID bash
git clone --depth=1 https://github.com/stnolting/neorv32
cd neorv32
git clone --depth=1 https://github.com/stnolting/neorv32-vunit test
cd test
rm -rf neorv32
ln -sf ../../neorv32 neorv32
python3 sim/run.py --ci-mode -v

sourcehut

#Sourcehut, as a collaborative platform, deserves special attention. It departs from mainstream forges, following a different paradigm based on the most robust, distributed and flexible technology at our hands since decades, plain text #email. Git, since its origins, includes a close integration with email, as they both share a distributed philosophy, avoiding central point of failure silos (surprising how mosft git forges tend to concentrate in silos). Sourcehut core architecture is based on mail exchange, patches and #maillists, which turns out to be a much more flexible approach than that of what most forges propose. Their concept of project goes well beyond that of usual workflows, integrating nicely git with email, wikis, bug trackers and build features. They’re still in an alpha stage, so expect the best still to come.

git

We said #git, then, as mandatory when tracking changes (in documentation, project development, taking notes, etc.). Meaningful changes imply new commits (and good commit messages, for what it takes), but this comes along with a risk of introducing issues. Some kind of mechanism is necessary to automatize the execution of a checkout list to be run per new commit. The list is project aware, for sure, but may also be different following the git branch, and even the kind of commit (merges are to be considered differently to regular commits in topic branches, for example). We need to consider what an issue exactly is, and then you’ll need to adopt a different perspective on kinds of checkout lists.

verification

First (ideally), one starts with clear specifications about the goals of current development effort (in practice this never happens in research, and if you ever have it, they’ll evolve with time). These specifications (you’ll figure out where to find them somehow) will define the tests to run. For example, if you need to implement in firmware a deep neural network, you’ll probably have access to a test data set to verify the outcomes are correct. You may tune, improve or even completely change the architecture of your network, at the very end, you’ll have to verify your design with help of the test data set. Additionally, you may define more sophisticated tests: consumption, area, resources, etc. These all fall into the category of verification testing.

unit tests

Secondly, you’ll be running unit tests during your whole design cycle (and they’ll evolve along with it), and target tests (the one we mentioned just before). Does this addition perform correctly ? What if we stress a module with random inputs ? Are we going through all code in a given design unit ? Do we cover all values of some input/output signal in this important module ? These are all unit testing checkouts, and they’ll help us to detect issues in an early stage of design.

codesign

Codesign falls somewhere in between the two previous: as a testing methodology, it includes concepts of verification and unit testing (and can be combined with them). It is way more ambitious and complex, but also more powerful. No matter your testing strategy, the point here is that you’ll be running these tests (fully or partially) automatically at the several different stages of your development cycle. If they fail, you’ll have to be warned.

guix

Guix, as a package manager, provides all necessary software to deploy our tests (and can be extended with additional tooling). It also includes all that's necessary to create a running environment where we will execute our tests. Most importantly, #guix does so in a #deterministic and #reproductible way: we will be able to reproduce our tests in the future under exactly the same conditions. Shell containers, profiles and the time machine mechanism allow the degree of #reproducibility we need here. All it takes is a couple of text files.

Most usually, we will focus on two strategies to seek for issues: local, and remote. Local strategies are greatly based on git hooks, and will be topic of another post. Let’s see now in practice what can be done with help of remote tools, based on #ci, understood as a methodology consisting on automatically executing a set of tests procedures on a digital design.
#ciseries

osvvm

First, we have osvvm. #Osvvm is a modern verification #vhdl library using most up-to-date language constructs (by the main contributor to the vhdl standard), and I’ll mention it frequently in this #modernhw posts series. Well documented and being continuously improved, it provides a large set of features for natively verifying advanced designs, among them, a constrained random facility, transactions, logging, functional coverage, scoreboards, FIFOs, sophisticated memory models, etc. Even some co-simulation capabilities are included here. Refer to the documentation repository for up-to-date details about osvvm.
You’ll be able to install osvvm with

# guix search osvvm
guix install osvvm-uart osvvm-scripts

You have a simple use of the osvvm vhdl library in the #aludesign, where the random feature is used to inject inputs to a dut unit. Testing runs for as long as every combination of two variables hasn’t been fully covered. This provides a means to be sure that all cases have been tested, regardless of random inputs. You’ll see an example simulation log here, using the remote ci builds facility of sourcehut.

vunit

Then, we have Vunit as a complete single point of failure framework. It complements traditional test benches with a software oriented approach, based on the “test early and test often” paradigm, a.k.a. unit testing. Here, a pre-built library layer on top of the vhdl design scans, runs and logs unit test cases embedded in user test benches. This approach seeks for an early way to detect as soon as possible conception errors. It performs random testing, advanced checking, logging, advanced communication and an advanced api to access the whole from python. It may be called from the command line, adding custom flags, and configured from a python script file where one defines libraries, sources and test parameters. Simple, elegant and efficient as a testing framework, if you want my opinion. Check the documentation for details.
Install it as usual with

guix install python-vunit

A clever example of its use is provided by the fw-open-logic firmware package (also included in the electronics channel). When you install it, you’ll need to build the package once, which gets installed in the guix store for you to use. During the process, the whole testing of its constituent modules is performed. You may have an overview of how it goes with:

guix build fw-open-logic:out

By the way, if you need the simulation libraries, they are available too.

guix install fw-open-logic:out
# guix install fw-open-logic:sim  # sim libraries

Additionnaly, #vunit is compatible with running a testing #ci pipeline online, as explained here.

cocotb

Finally, we have the interesting and original cocotb. It groups several construct providing a set of facilities to implement coroutine-based cosimulation of vhdl designs. Cosimulation, you say ? Yes. It requests on demand #ghdl simulation time from software (python, in this case), dispatching actions as the time advances. Afterward, based on events’ triggers, you’ll stop simulation coming back to software. This forth and back dance goes on, giving access to advanced testing and verification capabilities. Flexible and customizable as much as needed, in my opinion. Go read the documentation to understand how powerful cosumulation approach can reveal. By the way, install it with

guix install python-cocotb

From the previous, you’ll have understood that having access to verification, unit testing and cosimulation libraries is paramount in #modernhw digital design. Independly or combined (be careful!), they provide powerful tools to detect issues (of any kind) in your design. And yet, this is not enough, as the question arises about where, and when do we run these tests ? From the previous logs in the examples, you’ll have noticed that tests run online in #ci infrastructure. How it goes ? This is the topic of the ci posts in this series.

read a couple of good references

Pragmatic Version Control Using Git, Version Control with Git, 3rd Edition and Pragmatic guide to git are good examples, but there are many more around. Use them as a reference and as a starting point, and try to go beyond following your needs.
Check the official doc. And remember you have man git-log, man git-config, etc. at your disposal.

use a front end

Yes, again.
Avoid the #cli. And try to make your text editor and your front end as good friends as possible.

coding

Format your code properly, otherwise, diffing becomes useless, and your code diffs will be hidden by formatting diffs. Even worst, people you collaborate with will be unable to read your history. Comply to language standards.

changes

If possible, ask your text editor to have some kind of visual hints on what you have changed, added or removed.
Learn how to inspect diffs between working copy and staging area, between working copy and last commit, and the contents of a commit. Learn how to discard changes.

commits

Commits are lightweight diffs. A commit has one or two parent commits, and is identified by a unique hash.
Stage your changes first, commit them then. Learn how to stage chunks of changes, not all of them.
Remember to commit early, commit often: git is a CVS, not an archival, not a backup system. Never commit binaries (except artifacts: pdf, etc.).
Authentify who authors your developments and gpg-sign your commits.
Group changes in meaningful commits, and remember git history must be read as a novel: write meaningful commit messages. Consider that people spend much more time reading git history than writing it.

branches, tags and releases

A branch is a pointer to a commit, and if your remove the pointer to a commit, you won’t be able to access it anymore. Branches are free (as in beer !), so branch as much as your need.
Tags are fix pointers (labels) to commits (aliases) They identify stages, or important hints in development.
Remember to always store your work hash/tag along with your results, you’ll know what you’re doing, you’ll know which version of your submodules you’re using, and you’ll be able to compare your results.
Releases are numbered tags, identifying accomplishments. Be familiar with semantic versioning
Before merging branches, understand the differences between fast-forward (advance the pointer) and non fast-forward (keep branch history in a feature branch). And learn how to resolve merge conflicts with your frontend !

logs

Check frequently where you are in the log history, you may get backwards in history by just moving a pointer.
Learn to search (and filter searches) in the log messages

workflows

Remember local is decoupled from remote, and that git doesn’t impose any workflow, so everything is possible.
Learn the advised workflow in collaborative development: gitflow, and consider merge / pull requests are just artificial standards of a #gitforge.
At a minimum, use:

main, stable branch (releases only)
devel, working branch (commit here)
feature, topic-specific branch (spin-off)

locally

Everything may be fixed while working locally.
Use .gitignore., locally at $GITDIR/info/exclude and at global level at ./.gitignore. For lazy people you have gitignore.io
git-config your environment before anything else, globally at ~/.gitconfig, at project local at $GITDIR/config.
You’ll find more details about all the previous here.

install

First things first. You need to be root to proceed to a binary guix installation. Just download the installer, and follow the instructions.
After that, you’ll be using guix as a regular user, and all what follows must be run without any special rights beyond accessing to your home directory. Behind the curtains, guix computes what’s necessary through the running guix daemon, handled by your host’s systemd.

packages

Packages are built definitions. At no surprise, they are installed (or removed) with

guix search synthesis
guix install yosys
guix remove python

Definitions, in turn, are guile descriptions on how and where to obtain code source, a precise and unambiguous reference to it, how to process and how to install it, along with the necessary input dependencies, its kind, and how to use them. Definitions may be seen as customized default build templates, which avoids complicated package definitions, simplifying its design. Thus, a default build template called python-build-system exist, for example, for producing python packages. A package definition customizes the way this template is used, modifying its default package, source, etc. fields.
Definitions are built on isolated, minimalistic environments. Once built, packages are deposit in the guix store under /gnu/store. Each package is given a unique hash: changing the definition, or any of its inputs, produces a different package and hash. This is what usually is referred to as functional package management of #dependencies.
A package may have multiple outputs (out, by default, but also doc, etc.). Packages are built locally following the package definition. To avoid long run times and wasting cpu cycles, guix introduces substitutes or pre-built packages available on remote (substitute) servers. When available, substitutes are downloaded, which avoids having to built packages locally. Otherwise, your local computing resources will be put to contribution, which is far from ideal, so better configure your substitute servers before anything else (check your systemd guix-daemon file). It is possible to verify substitute availability with

guix weather ghdl-clang

  https://guix.bordeaux.inria.fr ☀
--> 100.0 % des substituts sont disponibles (1 sur 1)  <--
    6,2 Mio de fichiers nar (compressés)
    39,6 Mio sur le disque (décompressé)
    0,777 secondes par requête (0,8 secondes en tout)
    1,3 requêtes par seconde

It is crucial to understand that a given package built will be identical to any other build of this same package, regardless of the host computer, which is what holds the validity of the very idea of substitutes, and guarantees #reproducibility. This holds for any guix construction, including shell containers (see below).
Keep that in mind.

profiles and generations

After first install, guix will create on your behalf a default profile under ~/.guix-profile. All operations (install, remove) will affect this profile, unless you decide to point somewhere else (with the modifier -p $GUIX_PROFILE).

guix package -p $GUIX_PROFILE --list-installed

coreutils       9.1     out     /gnu/store/fk39d3y3zyr6ajyzy8d6ghd0sj524cs5-coreutils-9.1
git             2.46.0  out     /gnu/store/wyhw9f49kvc7qvbsbfgm09lj0cpz1wlb-git-2.46.0
fw-open-logic   3.0.1   out     /gnu/store/hrgdvswmvqcyai4pqmr7df0kpyyak94j-fw-open-logic-3.0.1
osvvm-scripts   2024.09 out     /gnu/store/xhxr3y1k8838my6mfk992kn392pwszjm-osvvm-scripts-2024.09
osvvm-uart      2024.09 out     /gnu/store/x3pjf95h8p3mbcx4zxb6948xfq3y3vg8-osvvm-uart-2024.09
fd              9.0.0   out     /gnu/store/nx0hz1y3g7iyi4snyza7rl5600z73xyn-fd-9.0.0
make            4.4.1   out     /gnu/store/963iman5zw7zdf128mqhklihvjh6habm-make-4.4.1
tcllib          1.19    out     /gnu/store/443vgrmwac1mvipyhin5jblsml9lplxf-tcllib-1.19
tcl             8.6.12  out     /gnu/store/w2icygvc0h294bzak0dyfafq649sdqvn-tcl-8.6.12
ghdl-clang      4.1.0   out     /gnu/store/sy0ryysxwbkzj6gpfka20fs27knmgmkd-ghdl-clang-4.1.0

Each profile generation will consist on a set of symbolic links pointing to /gnu/store. A new generation is produced when you install or remove something. This will only redefine your profile’s links, and so the status of the profile (and the packages you have access to). Generations are roughly the equivalent of #git commits, if this helps. They are nothing but collections of links pointing to the store, where packages are installed. Each collection defines a generation and so the current status of a guix profile.
You may roll back to previous generations, or move forward, but only linear generation histories are allowed. In you go back n generations, and then create a new one, your previous history is lost.

guix package -p $GUIX_PROFILE --list-generations

In addition to creating links, a profile redefines the environment variables (following the profle contents), appending, prepending or replacing the current ones. This way, the user enters an augmented context, having access to the packages in the profile.
Note that, inside a profile, the user still have access to the external system: for example, the PATH env variable is augmented with the profile bin directory, but former binaries are still there. To get a higher degree of isolation, we need shell containers (see below).

clean

From time to time, don’t forget to clean the store, removing stuff no profile and generation is pointing to, with

guix gc

Before that, remove old generations from your profiles, unless you plan to make use of them at some point.

upgrade

Upgrade guix current profile with

guix pull && guix upgrade

This will create a new generation in your default profile, including updates to all your packages in current profile. Pulling syncs local guix with remote guix repository, fetching updates locally. Upgrade will deploy these updates to your profile.
Remember also to

sudo -i guix pull
sudo systemctl daemon-reload
sudo systemctl restart guix-daemon

to upgrade the system daemon, so that it is never too delayed with respect to your guix in use.

manifest, channels

If not already clear from the previous, remember that it is possible to replicate environments (contexts, profiles, dependencies) using a couple of #plaintext files.
First, the #manifest.scm, which includes the list of packages in the environment. As an example, export your current profile with

guix package -p $GUIX_PROFILE --export-manifest > manifest.scm

Put it somewhere under version control, and replicate your environment somewhere else with

guix package -p $GUIX_PROFILE -m manifest.scm

That’s all it takes to get exactly the same development context in another host, for example.
But you’re right, I see you follow. This is not enough. You also need to freeze which guix version you’re using (guix, as any other package manager, not always installs the same version of some package). You need also a #channels.scm file. It may be produced with

guix describe --format=channels -p $GUIX_PROFILE > channels.scm

and includes the list of channels in use, along with a hash to identify which version of the channels to use, among the whole history of channel revisions (the #git commit of the channel repository). Then, import it somewhere else with

guix pull -C channels.scm

examples

Fixing your channels, its revision and the list of packages is all you need to eliminate any ambiguity, achieving #reproducibility and #determinism. Remember that this is the best advantage of guix, after all. Say you publish something (report, article, paper, blog ticket). If you provide a git repository with these two files, anyone else will be
able, hopefully, to replicate your asserts.
As a typical example, when you have a complex #vhdl design, including a large set of dependencies, you need a means to handle them. Here, we assume the #dependencies may be vhdl compilers, tcl shells, python interpreters, unit testing libraries, verification frameworks. etc. ... but also other vhdl modules, each in its own git repository.
At this point, you’ll need, first, to fix your channels.scm to “freeze” the repositories status. Here we are using, for example, the electronics, guix-science and guix channels, each in a fix release given by a commit hash.

(list (channel
       (name 'electronics)
       (url "https://git.sr.ht/~csantosb/guix.channel-electronics")
       (branch "main")
       (commit
        "2cad57b4bb35cc9250a7391d879345b75af4ee0a")
       (introduction
        (make-channel-introduction
         "ba1a85b31202a711d3e3ed2f4adca6743e0ecce2"
         (openpgp-fingerprint
          "DA15 A1FC 975E 5AA4 0B07  EF76 F1B4 CAD1 F94E E99A"))))
      (channel
       (name 'guix-science)
       (url "https://codeberg.org/guix-science/guix-science.git")
       (branch "master")
       (commit
        "1ced1b3b913b181e274ca7ed2239d6661c5154c9")
       (introduction
        (make-channel-introduction
         "b1fe5aaff3ab48e798a4cce02f0212bc91f423dc"
         (openpgp-fingerprint
          "CA4F 8CF4 37D7 478F DA05  5FD4 4213 7701 1A37 8446"))))
      (channel
       (name 'guix)
       (url "https://git.savannah.gnu.org/git/guix.git")
       (branch "master")
       (commit
        "3e2442de5268782213b04048463fcbc5d76accd7")
       (introduction
        (make-channel-introduction
         "9edb3f66fd807b096b48283debdcddccfea34bad"
         (openpgp-fingerprint
          "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA")))))

Then, you need the list of dependencies necessary to your design. These are provided in a manifest.scm file, as for example in

(specifications->manifest
 (list "ghdl-clang"
       "tcl"
       "tcllib"
       "make"
       "python-vunit"
       "osvvm-uart"
       "osvvm-scripts"
       "fw-open-logic"
       "git"
       "which"
       "findutils"
       "coreutils"))

One may include these two files in the design, in a different testing #git branch, for example. Then, all it takes to run your design in a reproducible way is cloning the design git repository, checking out the testing branch, and running #guix time machine (see next) to replicate a local profile containing all the design’s dependencies. Remember that here we include also all third party firmware modules instantiated in our design.

time machine

How guix guarantees that it is possible to reproduce a profile in the future ? The trick consist on asking current guix version to call a previous guix version (the one we define), to deploy the profile with the packages we need.
For example: let's ask guix-5 to make use of guix-4 to install emacs-30 package, which is only available in the guix-4 repositories, whereas guix-5 only provides emacs-32.
This mechanism is called time-machine. It is used as, for example:

guix time-machine --channels=channels.scm -- package -p $GUIX_PROFILE -m manifest.scm

Here, up-to-date guix uses time machine to roll back to the former guix version defined in channels.scm. Then, former guix calls the package command to install under $GUIX_PROFILE the list of packages defined in the manifest.scm file.
What’s important to understand here is that this will produce exactly the same output regardless of the host and the point in time when we run this command. The profile we produce is always the same, by design. And this is what is relevant for #modernhw.

shell containers

Guix includes a command to create independent environments from the rest of our host system. This provides an increased degree of isolation when compared to profiles, as the later lie on top of, and only augment, our current shell. Shell containers create a new, almost empty by default, minimalistic context for us to install packages.

guix shell --container --link-profile --emulate-fhs coreutils which python-vunit osvvm-uart
guix shell --container --link-profile --emulate-fhs -m manifest.scm

or, if one needs determinism

guix time-machine --channels=channels.scm -- shell --container --link-profile --emulate-fhs -m manifest.scm

The --link-profile flag will link the contents of $GUIX_PROFILE under /gnu/store to ~/.guix-profile.
The --emulate-fs will, well, reproduce the standard file system under /, as some packages expect this layout and fail otherwise.
coreutils and which packages will be helpful, otherwise, not even ls command is present within the container. Minimalistic, I said. I should have use isolated instead.

packs

Great. But. What if guix is not around ? How do I use it in a cluster, or in another host where guix is not yet available ? How do I distribute my dependencies, environments, etc. to a non-yet-guix-user ? No problem, guix pack is intended to be used as a simple way of “packaging” guix contexts (understood as a set of packages), deploying them afterward in a target host. This is next step after profiles and shell containers.
Guix pack comes equipped with several different backends, producing contexts in the most habitual and useful formats. For example, the following command will pack #emacs, #ghdl and #yosys for you to use where you need it.

guix pack -f docker emacs ghdl-clang yosys

In the context of #modernhw, #docker images may be used for #ci tests, uploading the image to a remote #gitforge registry; #apptainer containers can be sent and run in a #hpc cluster; .tar.gz compresses files are a clean may of installing non-existing software in a remote machine. Furthermore, one has the possibility of packaging all the project #dependencies in a manifest.scm file, and distribute it along with the source code to anyone willing to use it. No instructions about the proper environment to run the project, no complicated installation of dependencies. Stop asking third party users in your README to handle your dependencies for you.
A simple docker pull pointing to a #forge image repository is enough when guix is not locally available. Long run times ? Use a #forge #ci custom runner in your own hardware with your #singularity image. Remote work to an #ssh server with obsolete software ? Pack, send and untar your favorite development tools and create a custom profile, no admin rights needed. The possibilities are endless.
And most important: the advantage of this approach over a classical docker or singularity files for producing the images is #reproducibility: every single time you build the image, you’ll get the exact same binary product. Use the --save-provenance flag to store in the image itself the manifest you used to create it.
Good luck trying to achieve the same with a docker file.

importing

Now, guix is not a universal tool for installing anything around. What about this obscure #python package no one uses but you ? You’d absolutely need this #emacs package you just found on #codeberg which guix doesn’t provide. Rust, go ... There are plenty of pieces of code around not being yet packaged along with guix. No problem. Guix incorporates a simple and elegant way of extending the amount of packages you’ll be able to install. Just import them.

guix import pypi itsdangerous
guix import crate becareful
guix import gem wow

Previous commands will issue a new package definition corresponding to a package already handled by the language own package manager. #Dependencies, you say ? Use the --recursive flag. Once you have the definition, you’ll be able to build, install and use the corresponding package.
Check in the documentation the surprising amount of backends available, you’ll be gratefully surprised.

software heritage

Last, but not least.
You have guix, its package definitions and all the fancy tools which come along. But. What if you don’t have access to the source code ? In this case, all the previous becomes meaningless: remember that guix is a fully bootstrapped distribution, being built from the very bottom up. Building a package from its source means having access to the source, which most of the time is hosted in a #gitforge. But #forges disappear, especially proprietary ones, repositories relocate or are just obsoleted and get replaced.
In this case, guix gets you covered by falling back to Software Heritage (#SH). This initiative, with support of the UNESCO, collects, preserves, and shares the source code of all software that is publicly available, including its full development history. The collection is trigger automatically by a crawler, manually with a browser plugin or by guix itself when developers lint package definitions.
If, in ten years, you try to replicate one of your papers, and you plan to recreate your environment to run your code and reproduce your plots, you won’t be bothered by nowadays python 3.10 having being obsoleted, abandon and buried in history of computers by #guile. SH keeps a copy for you to sleep better at night.

channels

Channels, as a feature to extend guix repository of definitions, deserve its own chapter.

git

git.sr.ht/~user/project, #git repositories.
There are no groups to combine them. I use dot notation to group my projects git.sr.ht/~user/group1.prj1.

builds

builds.sr.ht/~user, task building service, with the list of builds by the user, with user’s recent activity.
builds.sr.ht, similar to the previous.
Builds are trigger by a .build.yml manifest, or by a .build folder folder with up to 4 manifest files, at the root of a project.
It is also possible to automatically submit builds when a patch to a repo with build manifests is sent to a mailing list. This is achieved by appending the project name as a prefix to the subject of the message, for example [PATCH project-name].
Check doc for details.

hub

The main tab, sourcehut, gives access to the hub hub.sr.ht/~user (identical to sr.ht/~user).
The hub displays user’s projects.
Projects are groups of git repositories, #maillists and bug trackers. There may be any of them in a project.
Sourcehut itself is organised as a project here https://sr.ht/~sircmpwn/sourcehut, and may be used as an example.

todo

todo.sr.ht/~user, ticket tracking service, with the list of trackers created by the user, with user’s recent activity
todo.sr.ht, similar to the previous

man

man.sr.ht/~user/NAME, #wiki service to create documentation, with the wikis created by the user
man.sr.ht, sourcehut documentation
man.sr.ht/builds.sr.ht, builds service documentation
man.sr.ht/hub.sr.ht, hub service documentation, etc.

lists

lists.sr.ht/~user/NAME, #email #maillists service, with the list created by the user, with user’s recent activity
lists.sr.ht, lists the user follows, with recent activity

Extra services are alse provided:

chat

chat.sr.ht, without the user alias, is a paid service. It provides a bouncer to save history of IRC channels while not connected, and may be used from another IRC client.

paste.sr.ht

Paste hosting service.

srht.site

Once again, the official documentation gives in depth details about the previous. And remember, there is also hut to operate from the #cli.