Work in progress: so far, a vagga container of Rust tools. Eventually in the container, tools for embedded Rust (Xargo) and my own source project under version control.
A similar endeavour is Japaric’s “cross”. The differences:
- that uses a Docker container, here I use Vagga container
- that might be ready to use, this is an explanatory exploration
- a log of my experience
- well linked
- for audience: developers/programmers.
I have been programming embedded computers in C++. I hate C++. I have also used Python and Swift. I read some background material:
- Rust versus Swift
- Japaric (Jorge Aparicio) has many github repositories about embedded Rust
- Zinc (a dead-end)
- Traits: Composable Units of Behaviour
- Tock: a secure OS for ARM using Rust
So here I try to install Rust. Ultimately I want to program in Rust an embedded ARM mcu on a NRF52 radio chip using the NRF52DK dev board.
Typically a developer knows/remembers how to set up a development machine (how to install the OS, development packages, an IDE, etc.) Typically, you use the GUI, maybe write a shell script, take good notes, iterate when you discover packages missing.
Vagga helps you capture the entire process of setting up a development machine. You capture the process in a vagga configuration file. Which is a text configuration file or script; no GUI.
This blog itself is an annotated record of writing and debugging such a vagga configuration file.
The Rust project moves fast. I don’t want to struggle with keeping up to date. I am not sure I will keep it. So I will install Rust in a container. A container is like a virtual machine, but lighter weight, and only on Linux.
Vagga implements containers. Vagga is targeted for developers i.e. specialized to contain development environments. It seems like a natural fit.
Advantages of containers/virtual machines:
- throwaway , non-invasive: Can’t destroy your computer’s installation of non-development packages, your personal applications such as Gimp, LibreOffice )
- distributable: you can give a container to other developers
Advantages of vagga:
- is platform (Linux distribution) agnostic. Vagga scripts might be portable to other developer’s machines (Linux-like.)
- is a high-level package manager
- userspace (doesn’t require root privileges)
Vagga is written in Rust. (I hoped Vagga might even install Rust for me but no, although written in Rust, I install a binary Vagga, which means I need to install Rust separately, but in a container.) In this case, using Vagga is a form of “eating your own dog food”: if you are going to learn Rust, you might as well use tools that are written in Rust.
Vagga is a high-level package manager. (discusses goals and future.)
Rustup is also a package manager (toolchain manager) exclusively for Rust.
Xargo is also a toolchain manager, exclusively for cross-compiling Rust language programs.
So it seems strange that to combat the proliferation of package managers, we invent yet another higher-level package manager. And here we use a chain/graph of package managers: vagga, rustup, cargo, xargo, your favorite Linux distribution’s package manager.
A high-level package manager makes more sense if you are targeting your app to many Linux distributions. Here, I am only targeting (ultimately) one embedded architecture. But by using a high-level package manager, I can distribute my development environment. And I can easily replicate my home dev machine in other remote physical locations.
Per the above, on Ubuntu, just paste this into a terminal:
echo 'deb [arch=amd64 trusted=yes] https://ubuntu.zerogw.com vagga main' | sudo tee /etc/apt/sources.list.d/vagga.list sudo apt-get update sudo apt-get install vagga
(That adds a repository for vagga, and installs vagga from it.)
Putting an OS in my container
Vagga is configured from a text file, vagga.yaml.
I created this simple directory tree:
With the contents of vagga.yaml:
containers: rustdev: setup: - !Ubuntu yakkety commands: test: !Command description: Test container: rustdev run: [ps]
In other words, a container named “rustdev” and a command “test”.
At at terminal, change directory to “rustdev” and enter “vagga test”. I got (unexpected):
(1/1) Installing alpine-keys (1.3-r0) OK: 0 MiB in 1 packages fetch http://repos.mia.lax-noc.com/alpine/v3.5/main/x86_64/APKINDEX.tar.gz ERROR: http://repos.mia.lax-noc.com/alpine/v3.5/main: No such file or directory
Which seems like a problem with online repositories. I entered “sudo apt-get update”, and tried again. This time vagga seemed to install an OS in the container (in about a minute) and run the command, yielding:
PID TTY TIME CMD 1 ? 00:00:00 exe 2 ? 00:00:00 ps
IOW, there are only a few processes running in the container.
Emphasizing: in my experience I had to do this to get it to work:
cd rustDev vagga test (fails to get alpine-keys) sudo apt-get update vagga test
Installing Rust compiler in the container
From my reading, I know the rust compiler command is “rustc”. Entering that at a command line, expect:
The program 'rustc' is currently not installed. You can install it by typing: sudo apt install rustc
So I know that ubuntu has a package. I don’t want it installed directly, but in the vagga container.
Make this change in vagga.yaml:
- !Ubuntu yakkety - !Install [rustc]
IOW, tell vagga you want to install the package named “rustc” in the container.
Now “vagga test” first checks the container configuration for updates and yields:
Generating locales (this might take a while)... en_US.UTF-8... done Generation complete. Reading package lists... Done E: Method mirror has died unexpectedly! E: Sub-process mirror received a segmentation fault. WARN:vagga::builder::commands::ubuntu: The `apt-get update` failed. You have no mirror setup, and default one is not always perfect. Add the following to your ~/.vagga.yaml: ubuntu-mirror: http://CC.archive.ubuntu.com/ubuntu Where CC is a two-letter country code where you currently are.
So I created a file named “.vagga.yaml” in my home directory (this is a hidden “settings” file. Do not change your yagga configuration file i.e. ~/rustdev/vagga.yaml) with the contents
Now, “vagga test” yields:
... E: Failed to fetch http://security.... ... E: Some index files failed to download. They have been ignored, or old ones used instead. WARN:vagga::builder::commands::ubuntu: The `apt-get update` failed. If this happens too often, consider changing the `ubuntu-mirror` in settings
So I followed this thread to find the “best” mirror, and changed the mirror, but it still fails.
So now I rethink: I really want embedded Rust, which suggests the nightly build of Rust, not the outdated package that Ubuntu provides. I don’t want to install packaged Rust, I want to install rustup….
Rust is usually installed by the “rustup” tool. New goal: install rustup in the container. It seems like Ubuntu does not package rustup separately. So edit vagga.yaml to add the instructions given by Rust.org for installing rustup, wrapped in a shell inside vagga. Neively:
- !Ubuntu yakkety - !Sh "curl https://sh.rustup.rs -sSf | sh"
But those instructions download a shell script and pipe it to a shell and the shell script is interactive. So I hacked some more. Summarizing the struggle:
- curl was absent from the container
- the curl package would not install because of mirrors outdated
- I switched OS version to Xenial (Ubuntu 16.04LTS) hoping the mirrors were more stable
- I switched to wget instead of curl
- the rustup shell script requires curl
- Most developers install an omnibus package “build-essential” that includes all the commands a developer typically uses. IOW, a bare OS unsuited is for developers.
- vagga has its own construct for downloading files
And the yagga configuration file for yagga itself builds a Rust development environment (since vagga is written in Rust.)
Using those examples, I ended up with the script which you can find in my git repository. I don’t include the script here, it may suffer revisions.
If you enter “vagga test” expect:
/work/.home/.cargo/bin/rustup /work/.home/.cargo/bin/rustc rustc 1.15.1 (021bd294c 2017-02-08)
Now I wondering whether I can run my IDE in the container, and how my source code gets into and out of the container (probably git.) The answer seems to be that the directory where you invoke vagga is the “project” directory and is mapped into the container as /work. Your IDE can work outside the container. All artifacts of the build should be in the container and not pollute your project directory?
Create a hidden setup or options file ~/.vagga.yaml with contents: ubuntu-mirror: http://us.archive.ubuntu.com/ubuntu
Until you get your vagga.yml correct, vagga seems to repeatedly download dependency packages. IOW, errors prevent completion of the container. When you achieve a correct container, then vagga knows, and only downloads dependency packages as needed (when the repository publishes a security update or a nightly update? Commands don’t establish dependencies?)
Doing “sudo apt-get update” between iterations seems to help some errors.
The directory where you invoke vagga is the “project” directory and is mapped into the container as /work.
Vagga stores the container in the hidden file .vagga in the project directory (alongside your vagga.yaml.) (To delete a container?)
Continue with Part Two…