Alu

Let’s consider now a variant of the previous example with open-logic. Here, we concentrate on a toy design only for demonstration purposes, a dummy alu emulator, which uses #osvvm as verification framework and relies on a few #openlogic blocs. In this case, its dependencies are defined in a manifest.scm file, including both fw-open-logic and osvvm, among other dependencies.
Install dependencies locally, in a new profile with

cd alu
mkdir _deps
export GUIX_PROFILE=open-logic/_deps
guix install -P $GUIX_PROFILE -m .builds/manifest.scm
. $GUIX_PROFILE/etc/profile

In this case, we will test the design using, first, a custom made makefile. Secondly, we will use hdlmake to automatically produce our makefile. Similarly to previous #openlogic example, two build manifest are used:

profile1
profile2

You’ll realise how some of the tasks are common with the case of previous #openlogic example (update channels, auth and update profile).

osvvm

In this case, we also need to compile osvvm libraries

compile__osvvm, produce a compiled version of #osvvm verification libraries; this is necessary as we are using here the tcl scripts included in the library itself to follow the correct order of compilation. Libraries will appear within the local profile under $GUIX_PROFILE/VHDL_LIBS/GHDL-X.Y.Z

test

test, for a fully custom made testing pipeline; in this case, using a Makefile
Just simply, source the .envrc file where the local $GUIX_PROFILE variable is defined, cd to the ghdl directory and call make to compile the design and run the simulation in two steps: first, clean all and include sources in its corresponding libraries with

sh make __clean_all __include

Then, produce a new Makefile using ghdl.

sh ./makefile.sh # ghdl --gen-makefile ...

Finally, run the simulation with

sh make GHDLRUNFLAGS="--stop-time=4us --disp-time --ieee-asserts=enable" run

This will produce a executable file before running it with the provided parameters.
You may notice that, in this case, you need to produce somehow your own Makefile, or equivalent pipeline, right ?

hdlmake

Wouldn’t it be nice if we had a tool to deploy online which produces makefiles for us ? It exists, and its name is #hdlmake.

test__hdlmake
Source the .envrc file where the local $GUIX_PROFILE variable is defined, cd to the .builds/hdlmake directory where all Manifest.py files are located, and call hdlmake to produce the Makefile. Finally, just run make to compile the design, produce an executable and run it.

Check the resulting logs inline here, for example.

Open logic

Let’s see how in detail using the cookbook as a starting point, and taking as a complete example the fw-open-logic #openlogic firmware package which comes with the electronics guix channel.
Get it with:

guix install fw-open-logic:out

Open logic is a useful #vhdl library of commonly used components, implemented in a reusable and vendor/tool-independent way. As any other #modernhw library, it includes tests sets for any of its components, using the vunit utility in this case.
To run the full tests suite use (user wide using the default $GUIX_PROFILE), install its dependencies, defined in a manifest.scm file (ghdl-clang and python-vunit in this case).

cd open-logic
guix install -m .builds/manifest.scm
cd sim
python3 run.py --ghdl -v

or local to the project, using a profile

cd open-logic
mkdir _deps
export GUIX_PROFILE=open-logic/_deps
guix install -P $GUIX_PROFILE -m .builds/manifest.scm
. $GUIX_PROFILE/etc/profile
cd sim
python3 run.py --ghdl -v

go remote

Now, how do we proceed online using #sourcehut #ci builds facility ? Builds will pop up a new environment based on an up to date guix-system image when we push a commit to git.sr.ht, provided we include a .build.yml build manifest file, or by a .build folder with up to 4 build manifest files, at the root of the git project [1]. Be careful: consider that this image is built daily using a crontab job, which is a good and a bad thing at the same time. From one side, you won’t be using the same environment for your tests, which breaks #reproducibility (see comments section below). On the other side, #guix is a rolling release, and new fancy features and new fixes are added every day. Keep this in mind.
Let’s create a .builds folder in a topic test branch, with the following contents:

manifest.scm, list of dependencies in our project
guix.scm, default guix repository, redundant, included here for convenience
channels.scm, list of guix channels remote repositories, in addition to the default guix repository, from where we pull packages
We will be using here my own electronics channel (no substitutes), as well as the guix science channel (which provides substitutes).
(note how here we load the local guix.scm file, instead of making use of the %default-channels global variable)

scheme (load "guix.scm") ;;; %default-channels key.pub, auth key to access substitutes of packages in guix channels

build manifests

From now on, every new push to the test #git branch will trigger the execution of the tasks defined in the three build manifest files

profile1
profile2
shell1

The two profile build manifest files use a slightly different approach, and are given here for comparison purposes only. The shell build manifest uses an isolated shell container within the image itself to illustrate this feature.
Inside the manifests, I declare the image to use, guix, and the global environment variables sourced before each task is run: prj (project name), srv (list of servers with substitutes), manifest and channels (pointing to the corresponding files) and key (same). It is important to declare a trigger action, to receive an email with all relevant information in case of failure (log, id, commit, etc.).

tasks

What’s interesting here is the list of tasks. Some of them are common to all three manifests

env, useful only for debugging
guix__update__channels, replace the default project local guix.scm file by the output of

sh guix describe --format=channels

The goal here is avoid pulling latest guix upstream, useless and cpu and time consuming, and using the local version instead. Remember that the guix system image we are using here is updated daily.

guix__auth, runs the authorize command to add the key.pub file to guix, so that we will be able to download package substitutes when necessary

sh sudo guix archive --authorize < "$key"

Here, one may opt by doing a

sh guix pull --channels="$channels"

as in profile2, to set the revision of the guix channels we are using (remember channels are nothing but git repositories).
Note how in profile1 and shell1 we opt for a different approach.
guix__update__profile, where we create a _deps folder to be used as a local $GUIX_PROFILE (defined in .envrc).
Then, one of

sh # profile1 guix time-machine --channels="$channels" -- \ package -p "$GUIX_PROFILE" \ --substitute-urls="$srv" \ -m "$manifest"

or

sh # profile2 guix \ package -p "$GUIX_PROFILE" \ --substitute-urls="$srv" \ -m "$manifest"

will install packages in $manifest into the $GUIX_PROFILE. I’m using here the time-machine mechanism to set the revision of the guix channels, depending if guix pull was run in the previous stage or not.
vunit, sets env variables in .envrc and runs python3 run.py --ghdl -v inside sim directory
Note that here, we are using ghdl-clang and python-vunit packages, provided respectively by guix-science and the electronics channel.
guix__shell__test, used by shell1, make use of time-machine (no former guix pull, then), to create a shell container, where to install project dependencies. Then, if calls inmediately run.sh to run the unit tests

sh guix time-machine --channels="$channels" -- shell -C --substitute-urls="$srv" -m "$manifest" -- ./.builds/run.sh

comments

You may check the logs of profile1, profile2 and shell1 manifests, including a section with logs per task, to better understand what’s going on here. Remember that #sourcehut gives ssh access to the builds by connecting to the runners in case of failures, which provides a practical way of debugging the manifest files.
You may see how, using the remove guix image, it is possible to deploy a series of tasks to test our #modernhw design as we develop it: we will get an email in case of failure to pass the tests. Here, I present three approaches: guix pulling to set the repositories revisions on use; time-machine, to achieve the same, and guix shell to create an isolated container. These three alternatives are not necessary here, of course, but are given as a simple and practical demo of what can be achieved with #guix, #sourcehut and #ci.
To conclude this long post, it is important to stress once again that the point on using #guix resides in its reproducibility capabilities. By keeping a couple of #plaintext files, namely the manifest.scm and channels.scm, one can obtain #determinism in the execution of the tests. Even if the guix image is upgraded and rebuilt daily (and so it changes), by fixing the revision of our channels (remember, guix pull or guix time-machine) we obtain always the same products out of our tests, as we run the same (project and tests) code, within exactly the same environment.

[1] It is also possible to automatically submit builds when a patch to a repo with build manifests is sent to a mailing list. This is achieved by appending the project name as a prefix to the subject of the message, for example [PATCH project-name].

sourcehut

#Sourcehut, as a collaborative platform, deserves special attention. It departs from mainstream forges, following a different paradigm based on the most robust, distributed and flexible technology at our hands since decades, plain text #email. Git, since its origins, includes a close integration with email, as they both share a distributed philosophy, avoiding central point of failure silos (surprising how mosft git forges tend to concentrate in silos). Sourcehut core architecture is based on mail exchange, patches and #maillists, which turns out to be a much more flexible approach than that of what most forges propose. Their concept of project goes well beyond that of usual workflows, integrating nicely git with email, wikis, bug trackers and build features. They’re still in an alpha stage, so expect the best still to come.

git

git.sr.ht/~user/project, #git repositories.
There are no groups to combine them. I use dot notation to group my projects git.sr.ht/~user/group1.prj1.

builds

builds.sr.ht/~user, task building service, with the list of builds by the user, with user’s recent activity.
builds.sr.ht, similar to the previous.
Builds are trigger by a .build.yml manifest, or by a .build folder folder with up to 4 manifest files, at the root of a project.
It is also possible to automatically submit builds when a patch to a repo with build manifests is sent to a mailing list. This is achieved by appending the project name as a prefix to the subject of the message, for example [PATCH project-name].
Check doc for details.

hub

The main tab, sourcehut, gives access to the hub hub.sr.ht/~user (identical to sr.ht/~user).
The hub displays user’s projects.
Projects are groups of git repositories, #maillists and bug trackers. There may be any of them in a project.
Sourcehut itself is organised as a project here https://sr.ht/~sircmpwn/sourcehut, and may be used as an example.

todo

todo.sr.ht/~user, ticket tracking service, with the list of trackers created by the user, with user’s recent activity
todo.sr.ht, similar to the previous

man

man.sr.ht/~user/NAME, #wiki service to create documentation, with the wikis created by the user
man.sr.ht, sourcehut documentation
man.sr.ht/builds.sr.ht, builds service documentation
man.sr.ht/hub.sr.ht, hub service documentation, etc.

lists

lists.sr.ht/~user/NAME, #email #maillists service, with the list created by the user, with user’s recent activity
lists.sr.ht, lists the user follows, with recent activity

Extra services are alse provided:

chat

chat.sr.ht, without the user alias, is a paid service. It provides a bouncer to save history of IRC channels while not connected, and may be used from another IRC client.

paste.sr.ht

Paste hosting service.

srht.site

Once again, the official documentation gives in depth details about the previous. And remember, there is also hut to operate from the #cli.