In detail

Following the kind of contribution (new additions, fixes or upgrades), these simple steps will allow you to start contributing to guix:

git clone guix itselft
from the guix repository, do:

sh guix shell -D guix -CPW ./bootstrap ./configure make -j$(nproc) ./pre-inst-env guix build hello add and commit your changes, watch the commit message
beware your synopses and descriptions
remember to run the package tests, if relevant
check the license
use an alphabetical order in input lists
no sign off your commits
don’t forget to use lint/style/refresh -l/dependents to check your code

Boring and routinary, right ?

Use sourcehut

Most of all the of the previous can be run automatically with help of sourcehut build farm #ci capabilities. Just simply, push the guix repository to sr.ht. At this point, it is possible to use this manifest file to run the lint/style/refresh -l/dependents testing stages on the yosys package definition, por example:

image: guix
shell: true
environment:
  prj: guix.guix
  cmd: "guix shell -D guix -CPWN git nss-certs -- ./pre-inst-env guix"
sources:
  - https://git.sr.ht/~csantosb/guix.guix
tasks:
  - def_pkg: |
      cd "$prj"
      _pkg=$(git log -1 --oneline | cut -d':' -f 2 | xargs)
      echo "export pkg=$_pkg" >> "$HOME/.buildenv"
  - setup: |
      cd "$prj"
      guix shell -D guix -CPW -- ./bootstrap
      guix shell -D guix -CPW -- ./configure
      guix shell -D guix -CPW -- make -j $(nproc)
  - build: |
      cd "$prj"
      eval "$cmd build --rounds=5 $pkg"
  - lint: |
      cd "$prj"
      eval "$cmd lint $pkg"
  - style: |
      cd "$prj"
      eval "$cmd style $pkg --dry-run"
  - refresh: |
      cd "$prj"
      eval "$cmd refresh -l $pkg"
  - dependents: |
      cd "$prj"
      eval "$cmd build --dependents $pkg"
triggers:
  - condition: failure
    action: email
    to: builds.sr.ht@csantosb.mozmail.com

Submit the manifest with

hut builds submit # --edit

You’ll be able to log into the build farm to follow the build process or to debug it with

hut builds ssh ID

Check the log here. As you can see, it fails: building of yosys succeeds, but building of packages which depend on it (--dependents) fails.

Advanced

Sourcehut provides a facility to automatize patch submission and testing. Using its hub integrator, one may just send an email to the email list related to your project (guix in this case), which mimics guix behavior for accepting patches.
The trick here consists on appending the project name as a prefix to the subject of the message, for example [PATCH project-name], which will trigger the build of previous .build.yml manifest file at the root of the project, after applying the patch. Neat, right ?
If you followed right here, you’ll notice that previous build manifest file is monolithic, affecting always the same package (yosys), which is kind of useless, as we are here interested in testing our patch. Thus, the question on how to trigger a custom build containing an updated $pkg variable related to the patch to test remains open.
To update the contents of the $pkg variable in the build manifest, one has to parse the commit message in the patch, extracting from there the package name. This is not a problem, as guix imposes clear commit messages in patches, so typically something like

* gnu: gnunet: Update to 0.23.0

or

* gnu: texmacs: Add qtwayland-5

Hopefully, parsing these messages to get the package name, and so the value of $pkg is trivial.
Then, it remains to include in our build manifest a first task which updates the contents of "$HOME/.buildenv". This file is automatically populated using the environment variables in the manifest, and its contents are sourced at the beginning of all tasks. This mechanism allows passing variables between tasks.

echo "export pkg=value" >> "$HOME/.buildenv"

Send your contribution

Finally, once your changes go through all the tests,

use git send-email to create and send a patch
consider reviews, if any, updating your patch accordingly with git ammend
resend a new patch including a patch version (v1, v2 ...)

Interested ? Consult the documentation for details, you’ll learn a lot about how to contribute to a common good and collaboration with other people.
#ciseries

sourcehut

#Sourcehut, as a collaborative platform, deserves special attention. It departs from mainstream forges, following a different paradigm based on the most robust, distributed and flexible technology at our hands since decades, plain text #email. Git, since its origins, includes a close integration with email, as they both share a distributed philosophy, avoiding central point of failure silos (surprising how mosft git forges tend to concentrate in silos). Sourcehut core architecture is based on mail exchange, patches and #maillists, which turns out to be a much more flexible approach than that of what most forges propose. Their concept of project goes well beyond that of usual workflows, integrating nicely git with email, wikis, bug trackers and build features. They’re still in an alpha stage, so expect the best still to come.

the what

First and foremost, for the requirements.
In my case, the requisites are simple, even if hard to get when one thinks about.
I need a distracting free environment to concentrate on what really matters: the content I’m willing to share. For sure, I do need to remain within my working environment, that’s to say, #emacs. I need to switch context quickly between any current activity and writing prose when something comes up; while writing, I need to stay focus. Similarly, I want to switch back to previous context when the writing is complete. No doubt, I need to complete previous posts when I have something new to include or to correct, so I need a means of retrieving previous posts quickly. Needless to say, I need a #freesoftware tool I may tune to my needs, fixing issues or including new features.
Last, but not least, I privilege a way to push remotely without complicated compilations of anything at all: a couple of keystrokes, and the post is sent online under its right form, including some markup and images. Updating previous versions if necessary should be that simple too, and I must be able to check the rendering with any web browser, included eww under emacs, so no javascript or fancy stuff involved. That’s it by now.
Easy, right ?

the how

My current choice goes for writefreely as an open, decentralized and free alternative to web publishing on the web, which concentrates on providing a simple reading experience. This solution may be self-hosted, but there are also some friendly communities around helping out. Infosec.press is one of them. Blog posts show up in the #fediverse under the (platform) user account, so that they are easy to follow. Server side, this is more than what I need.
But, client side ?, you must be asking. To write text, I’m using #orgmode, with all of its facilities, and not the markdown supported by default by the platform. Then, writefreely.el takes care of exporting contents, handling the data exchange with the server through the provided api. I had to fix a couple of issues before, mostly trivial side effects. This is one of the biggest advantages of #freesoftware, having the possibility to contribute to bug fixing, improving a common.
As for the question on how to access the blog contents locally, I opt for a different #plaintext file by blog ticket, that I manipulate as any other #orgroam node. Orgroam allows to quickly retrieve, manipulate and insert as links previous notes. When I type the name of a non-existing note, it creates a new one for me, based on a custom template which incorporates the necessary headings, title, tags and the like. It resuls in something like:

:PROPERTIES:
:ID:       ID-6dd1-45d7-a70e-ae5c99c2797a
:END:
#+TITLE: on writting freely
#+OPTIONS: toc:nil -:nil \n:t
#+LINK: srht https://repo/pics/%s
#+filetags: :tag1:tag2:
# a comment
[[srht:image.png]]
Donec neque quam, dignissim in, mollis nec, sagittis eu, wisi ... <!--more-->
Nunc eleifend leo vitae magna.

You’ll figure it out.
Remains the question of images: just simply, they are hosted on an unlisted git repository, from where they are fetched. Now, with my working environment and with a couple of keys, I may pop up a new buffer, write some content, then publish, delete or update a new article within seconds, checking the results with #eww, all without leaving #emacs. You’ll get a set of local variables append to you buffer when you publish for the first time, something like

# Local Variables:
# writefreely-post-id: "ID"
# writefreely-post-token: nil
# End:

which allows to retrieve the post online afterwards.
Put the whole under #git control, and the perfect blogging setup is ready for you to enjoy writing ! Simple, elegant and efficient.
Finally, I have packaged writefreely.el and sent a patch to #guix so that it will hopefully get merged upstream soon.

osvvm

First, we have osvvm. #Osvvm is a modern verification #vhdl library using most up-to-date language constructs (by the main contributor to the vhdl standard), and I’ll mention it frequently in this #modernhw posts series. Well documented and being continuously improved, it provides a large set of features for natively verifying advanced designs, among them, a constrained random facility, transactions, logging, functional coverage, scoreboards, FIFOs, sophisticated memory models, etc. Even some co-simulation capabilities are included here. Refer to the documentation repository for up-to-date details about osvvm.
You’ll be able to install osvvm with

# guix search osvvm
guix install osvvm-uart osvvm-scripts

You have a simple use of the osvvm vhdl library in the #aludesign, where the random feature is used to inject inputs to a dut unit. Testing runs for as long as every combination of two variables hasn’t been fully covered. This provides a means to be sure that all cases have been tested, regardless of random inputs. You’ll see an example simulation log here, using the remote ci builds facility of sourcehut.

vunit

Then, we have Vunit as a complete single point of failure framework. It complements traditional test benches with a software oriented approach, based on the “test early and test often” paradigm, a.k.a. unit testing. Here, a pre-built library layer on top of the vhdl design scans, runs and logs unit test cases embedded in user test benches. This approach seeks for an early way to detect as soon as possible conception errors. It performs random testing, advanced checking, logging, advanced communication and an advanced api to access the whole from python. It may be called from the command line, adding custom flags, and configured from a python script file where one defines libraries, sources and test parameters. Simple, elegant and efficient as a testing framework, if you want my opinion. Check the documentation for details.
Install it as usual with

guix install python-vunit

A clever example of its use is provided by the fw-open-logic firmware package (also included in the electronics channel). When you install it, you’ll need to build the package once, which gets installed in the guix store for you to use. During the process, the whole testing of its constituent modules is performed. You may have an overview of how it goes with:

guix build fw-open-logic:out

By the way, if you need the simulation libraries, they are available too.

guix install fw-open-logic:out
# guix install fw-open-logic:sim  # sim libraries

Additionnaly, #vunit is compatible with running a testing #ci pipeline online, as explained here.

cocotb

Finally, we have the interesting and original cocotb. It groups several construct providing a set of facilities to implement coroutine-based cosimulation of vhdl designs. Cosimulation, you say ? Yes. It requests on demand #ghdl simulation time from software (python, in this case), dispatching actions as the time advances. Afterward, based on events’ triggers, you’ll stop simulation coming back to software. This forth and back dance goes on, giving access to advanced testing and verification capabilities. Flexible and customizable as much as needed, in my opinion. Go read the documentation to understand how powerful cosumulation approach can reveal. By the way, install it with

guix install python-cocotb

From the previous, you’ll have understood that having access to verification, unit testing and cosimulation libraries is paramount in #modernhw digital design. Independly or combined (be careful!), they provide powerful tools to detect issues (of any kind) in your design. And yet, this is not enough, as the question arises about where, and when do we run these tests ? From the previous logs in the examples, you’ll have noticed that tests run online in #ci infrastructure. How it goes ? This is the topic of the ci posts in this series.

Curious about everything
#FreeSoftware and computer science #freedom as a basic right
Doing public #research in France
Using #emacs, what else ?

sr.ht ◦ cv-fr ◦ cv-hal ◦ orcid ◦ bibtex ◦ gitlab.com ◦ perso

Modern digital hardware design (#modernhw) in research implies using the right tools. But this is not necessarily always the case.
An engineer usually has to deal with heavyweight tooling, propitiatory GUIs, buggy software and more generally, a handful of rudimentary toolchains developed just to pretend they might be of some utility. But they are not. Most usually, they contribute to pollute the heart of what it’s relevant: ideas, concepts and how they translate to hardware.
Weird languages, obsolete codes, old-fashioned standards, mainstream OSs, ... the list goes long. On top of that, industry imposes wrong habits and closed code, impossible to track, debug or to share. Engineers are most times forced to use some tool just because that’s the way it is, and everyone’s else is doing the same. No choice.
Hardware digital design lacks all the usual development utilities used by software engineers since ages. No control version, no online forges, no patches, no reproducibility, no documentation, no code comments, no scripting, no code review. Just a set of random zip files in a USB key, including a huge amount of useless data files no one cares about.
Is it possible to afford things differently ? Let’s see. Do we have better tools for the task at hand ? I think so. Could we envision to design, review and publish in such a way as it reveals useful in research ? Let’s try.
This series of #modernhw posts are an attempt to explore a somehow original path towards a reproducible, lightweight way of understanding digital hardware design, based on #freesoftware. Here, I’ll try to describe a handful of tools I find useful, why they're helpful in this context, and how to use them to achieve our goal.

usual suspects

Other than guix channel itself, you may have a feeling of what a guix channel looks like taking a look at the guix-science organization. In there, you’ll find collections of package definitions intended to be used in a context of specific research domains, both in its free and nonfree forms. For one reason or another, they have not their place in the main guix repository, and they live their life in here. Interesting enough, guix-past channel brings old-fashioned software to the present, which reveals useful to reproduce results older than guix itself. Another interesting examples are guixrus, by the ~whereiseveryone collective, providing packages not yet fully tested, nightly releases or alpha quality software; or the guix-hpc channel, with packages related to high performance computing; also guix-bioc, containing bioconductor packages, etc. You get the idea.
To include additional channels, other than the default ones, in your ~/.config/guix/channels.scm file, follow the documentation, and check with guix describe: you’ll get a list of all the channels in use next time you do a guix pull. At this point, you’ll have plenty of new packages to install at your hands ! Just remember the discussion about substitutes: some channels provide substitute servers, some not. This might be an issue, or not, depending of the computing resources necessary to build certain packages.

your first personal channel

Guix channels are simple to create and use. Just put your #guile modules, containing your package definitions, in a git repository. To use them, complete the default guix load path as in

guix install -L ~/guix-channel PACKAGE

You may opt to include this local channel into your ~/.config/guix/channels.scm file, or include the -L flag in your commands. As an alternative, guix searches for package definitions in the $GUIX_PACKAGE_PATH, so you may put your channel path in this variable. Remember that guix builds packages by compiling source code: no substitute server will do the job for you in this case, so plan accordingly with your local resources and patience.

electronics channel

This is about #modernhw after all, so let’s see an example more on detail, my own guix-electronics channel which complements the ./gnu/packages/electronics.scm guix module. This is the channel I’m using daily, where I include all the packages I need and doesn’t yet exist upstream. When they get merged somewhere else, I remove them from here. In the meantime, I’m a happy user of my custom packages.
I classify all my package definitions in #guile modules under ./electronics/packages/MODULE.scm. This is the reason why the compilers module, for example, starts by

(define-module (electronics packages compilers)

Then, proceed as usual (check the guix repo itself for inspiration), putting your packages below

(define-public mypackage
  (package
    (name "mypackage")
    ...
    ))

I create a git repo, commit my changes and push it online to a #gitforge.
I need a .guix-channel file with the url of my channel and a version number. If the code in the channel depends on other channels other than main guix channel, one needs to specify this information in this file too. In this case, for example, several of the package definitions include a dependency on ghdl-clang, provided by guix-science channel, so I need to declare it in here too.
It is also necessary a .guix-authorizations file, with the (GPG key) list of developers with right to push to this channel. This is how guix authenticates channels. Finally, it is important to create a readme file or similar where introduce the channel (first commit, signing key fingerprint). That’s it. Include the channel introduction into your ~/.config/guix/channels.scm file, and you’ll be pulling from your online guix channel.

contributing

The advantage of using a custom channel is to benefit from an increased degree of freedom when using guix, without being dependent of someone’s else time availability to consider your particular needs. Remember, guix is a huge project which follows closely the #freesoftware paradigm, and collaboration works in two directions. You take advantage of other developers contributions to guix, while you participate yourself to improving guix repositories with your package definitions, once they have been tested.
Take for example the list of packages in my electronics channel. I have included here utilities for #vhdl code editing under #emacs, specific to the electronic digital design field, which are not yet available in guix upstream channel. There is also a cosimulation library, #cocotb, which allows coupling #python and #ghdl to simulate a design. I have submitted a patch to include this in guix itself, similarly to python-vunit; once merged, I’ll remove them from here.
Some packages, like open fpga loader are now part of guix. Some others, like the gnat ada compiler and ghdl-clang stable have been merged already on guix-science, as gnat package is obtained from binaries (and so ghdl-clang is also impacted, from guix perspective). I include here a more recent ghdl-clang, non yet released, as it includes some fixes and improvements I am interested in, without having to wait until guix incorporates a new release.
Catch the idea ?
Contributing your code to guix comes to sending #email with your patches attached, it’s that simple. Dont be intimidated by the workflow (this is used by the linux project, for example). Once your patches are submitted, a review of your code folllows, see details. Some tools, like mumi, are helpful to that purpose.
These simple steps will allow you to start contributing to guix:

• git clone guix itselft
  
• add and commit your changes, watch the commit message
  
• don’t forget to use guix lint/style/refresh -l to check your code
  
• use git send-email to create and send a patch
  
• cosider reviews, if any, updating your patch accordingly with git ammend
  
• resend a new patch including a patch version (v1, v2 ...)
  

Interested ? Consult the doc for details, you’ll learn a lot about how to contribute to a common good and collaboration with other people.

Nix

Functional package management is a paradigm pioneer by nix and developed by Eelco Dolstra in his influential PhD Thesis. It is based on building each of the nodes in the graph of dependencies based solely in its inputs, contents, and node definition, producing a new node. The process repeats for every single node in the graph. In what concerns operating systems and software management, this comes to a radical different approach to classical package management. Simply put, every new build lives its life, regardless of the remaining builds. This makes it possible to have access to the kind of advanced utilities which ease our lives: declarative configurations, profiles, rollbacks, generations, etc. Forget the dependency hell.
Guix is founded on a similar approach, keeping its own set of rules as it only packages #freesoftware. There are around 30.000 of them as for today, including all the usual suspects. Within the context of #modernhw, guix is to be understood as a dependency management tool with advanced capabilities. Sure, it handles software, but there is no reason to use it exclusively as a software manager. It may handle IP blocks, documentation, bibliographic references, and more generally, all what concerns #plaintext files in a #gitforge. It understands versions, licences, dependencies, repositories and all kinds of relationships between them. Furthermore, it embeds a pragmatic language, guile, to script package definitions, declaring the behavior of nodes in our graph of dependencies.

Reproducibility

The most relevant feature of guix turns out to be its bootstraping capabilities. Full source bootstrap comes to building the whole dependency graph right from the bottom, based on a core minimum of trusted binary seeds. From that point upwards the whole distribution is self-contained, as all that it builds is included in guix itself. Any available package is founded on a package definition included in guix, its source code available online in a #git repository, and its dependencies. Each of the latest, follows the same rules, down to the bottom of the graph where a trust seed is necessary.
Why is this necessary and useful for #modernhw ? Because it provides #reproducibility for free, as reproducible builds are granted here. Turns out that this is at the very heart of guix and produces #determinism, meaning that same operations will produce same outputs, no matter when, no matter what, no matter where. Game over to ambiguity. Determinism, coupled to its declarative nature, reveals as a simple means to track our dependencies history without ambiguity.
Let’s take an example, and say we have a Vivado project in the form of a set of #tcl files. To build the logic of our favourite #fpga, we require also a couple of external firmware dependencies as IP block in their own git repositories, with tagged revisions, being mutually dependent and incompatible, following the tag in use. Not all of them are compliant with our project. Each of the firmware modules incorporates its own set of VHDL versioned dependencies, along with its associated documentation. We need to provide a python testing framework (you guess it), along with its verification libraries. We need to create a static web site with the instructions on how to download, instantiate, compile and deploy each different version of our project for a couple of thousand users out there, each with a different #gnulinux system, version and configuration of installed software and libraries. Take it for granted, each user need a different version of our project, as they need to guarantee compatibility with their own internal developments. And we need to provide the correct version of every tool, compatible with our code and scripts. The right version, I do mean. Not any random version.
Can you imagine the pain ? Now, suppose that you could describe in a couple of #plaintext manifest files the status of your project. End of the history. Users willing to reproduce your project and dependencies only need to git clone the manifest files and install them locally, regardless of their system, of their present libraries or of their abilities. No problem if the host OS doesn’t provide the necessary software, guix handles the situation. Users just do make and the whole project is deployed, tested and simulated, using the right version of each node in the graph. This is Guix at its best.
Not yet convinced ? Take a look at here.
Feeling like tempted ? Start by a crash course to guix.