Vagga Rust embedded

TL;DR

Work in progress: so far, a vagga container of Rust tools.  Eventually in the container, tools for embedded Rust (Xargo) and my own source project under version control.

A similar endeavour is Japaric’s “cross”.   The differences:

  • that uses a Docker container, here I use Vagga container
  • that might be ready to use, this is an explanatory exploration

A repository of source for this blog.

About

This is:

  • a log of my experience
  • well linked
  • for audience: developers/programmers.

Background

I have been programming embedded computers in C++.   I hate C++.  I have also used Python and Swift.  I read some background material:

So here I try to install Rust.  Ultimately I want to program in Rust an embedded ARM mcu on a NRF52 radio chip using the NRF52DK dev board.

Meta

Typically a developer knows/remembers how to set up a development machine (how to install the OS, development packages, an IDE, etc.)  Typically, you use the GUI, maybe write a shell script, take good notes, iterate when you discover packages missing.

Vagga helps you capture the entire process of setting up a development machine.  You capture the process in a vagga configuration file.  Which is a text configuration file or script; no GUI.

This blog itself is an annotated record of writing and debugging such a vagga configuration file.

Strategy

The Rust project moves fast.  I don’t want to struggle with keeping up to date.  I am not sure I will keep it.  So I will install Rust in a container.  A container is like a virtual machine, but lighter weight, and only on Linux.

Vagga github repository

Vagga implements containers.  Vagga is targeted for developers i.e. specialized to contain development environments. It seems like a natural fit.

Advantages of containers/virtual machines:

  • throwaway , non-invasive:  Can’t destroy your computer’s installation of non-development packages, your personal applications such as Gimp, LibreOffice )
  • distributable: you can give a container to other developers

Advantages of vagga:

  • is platform (Linux distribution) agnostic.  Vagga scripts might be portable to other developer’s machines (Linux-like.)
  • is a high-level package manager
  • userspace (doesn’t require root privileges)

Vagga is written in Rust.  (I hoped Vagga might even install Rust for me  but no, although written in Rust, I install a binary Vagga, which means I need to install Rust separately, but in a container.)  In this case, using Vagga is a form of “eating your own dog food”: if you are going to learn Rust, you might as well use tools that are written in Rust.

More meta

Vagga is a high-level package manager.  (discusses goals and future.)

Rustup is also a package manager (toolchain manager) exclusively for Rust.

Xargo is also a toolchain manager, exclusively for cross-compiling Rust language programs.

So it seems strange that to combat the proliferation of package managers, we invent yet another higher-level package manager.   And here we use a chain/graph of package managers:  vagga, rustup, cargo, xargo, your favorite Linux distribution’s package manager.

A high-level package manager makes more sense if you are targeting your app to many Linux distributions.  Here, I am only targeting (ultimately) one embedded architecture.  But by using a high-level package manager, I can distribute my development environment.  And I can easily replicate my home dev machine in other remote physical locations.

Installing Vagga

Vagga instructions for installation.

Per the above, on Ubuntu, just paste this into a terminal:

echo 'deb [arch=amd64 trusted=yes] https://ubuntu.zerogw.com vagga main' | sudo tee /etc/apt/sources.list.d/vagga.list
sudo apt-get update
sudo apt-get install vagga

(That adds a repository for vagga, and installs vagga from it.)

Putting an OS in my container

Vagga docs on configuration

Vagga is configured from a text file, vagga.yaml.

I created this simple directory tree:

rustdev/
    vagga.yaml

With the contents of vagga.yaml:

containers:
  rustdev:
    setup:
    - !Ubuntu yakkety
    
commands:
  test: !Command
    description: Test 
    container: rustdev
    run: [ps]
 

In other words, a container named “rustdev” and a command “test”.

Testing Vagga

At at terminal, change directory to “rustdev” and enter “vagga test”.  I got (unexpected):

(1/1) Installing alpine-keys (1.3-r0)
OK: 0 MiB in 1 packages
fetch http://repos.mia.lax-noc.com/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
ERROR: http://repos.mia.lax-noc.com/alpine/v3.5/main: No such file or directory

Which seems like a problem with online repositories.  I entered “sudo apt-get update”, and tried again.  This time vagga seemed to install an OS in the container (in about a minute) and run the command, yielding:

PID TTY TIME CMD
 1 ? 00:00:00 exe
 2 ? 00:00:00 ps

IOW, there are only a few processes running in the container.

Emphasizing:  in my experience I had to do this to get it to work:

cd rustDev
vagga test    (fails to get alpine-keys)
sudo apt-get update
vagga test

Installing Rust compiler in the container

From my reading, I know the rust compiler command is “rustc”.  Entering that at a command line, expect:

The program 'rustc' is currently not installed. You can install it by typing:
sudo apt install rustc

So I know that ubuntu has a package.  I don’t want it installed directly, but in the vagga container.

Make this change in vagga.yaml:

    - !Ubuntu yakkety
    - !Install [rustc]

IOW, tell vagga you want to install the package named “rustc” in the container.

Now  “vagga test” first checks the container configuration for updates and yields:

Generating locales (this might take a while)...
 en_US.UTF-8... done
Generation complete.
Reading package lists... Done
E: Method mirror has died unexpectedly!
E: Sub-process mirror received a segmentation fault.
WARN:vagga::builder::commands::ubuntu: The `apt-get update` failed. You have no mirror setup, and default one is not always perfect.
Add the following to your ~/.vagga.yaml:
 ubuntu-mirror: http://CC.archive.ubuntu.com/ubuntu
Where CC is a two-letter country code where you currently are.

So I created a file named “.vagga.yaml” in my home directory (this is a hidden “settings” file.  Do not change your yagga configuration file i.e. ~/rustdev/vagga.yaml) with the contents

ubuntu-mirror: http://us.archive.ubuntu.com/ubuntu

Now, “vagga test” yields:

...
E: Failed to fetch http://security....
...
E: Some index files failed to download. They have been ignored, or old ones used instead.
WARN:vagga::builder::commands::ubuntu: The `apt-get update` failed. If this happens too often, consider changing the `ubuntu-mirror` in settings

So I followed this thread to find the “best” mirror, and changed the mirror, but it still fails.

So now I rethink: I really want embedded Rust, which suggests the nightly build of Rust, not the outdated package that Ubuntu provides.  I don’t want to install packaged Rust, I want to install rustup….

Installing rustup

Rust install instructions.

Rust is usually installed by the “rustup” tool.  New goal: install rustup in the container.  It seems like Ubuntu does not package rustup separately.  So edit vagga.yaml to add the instructions given by Rust.org for installing rustup, wrapped in a shell inside vagga.   Neively:

    - !Ubuntu yakkety
    - !Sh "curl https://sh.rustup.rs -sSf | sh"

But those instructions download a shell script and pipe it to a shell and the shell script is interactive.  So I hacked some more.  Summarizing the struggle:

  • curl was absent from the container
  • the curl package would not install because of mirrors outdated
  • I switched OS version to Xenial (Ubuntu 16.04LTS) hoping the mirrors were more stable
  • I switched to wget instead of curl
  • the rustup shell script requires curl

Now I read vagga examples.  From github:vagga-examples/python  I found:

  • Most developers install an omnibus package “build-essential” that includes all the commands a developer typically uses.  IOW,  a bare OS unsuited is for developers.
  • vagga has its own construct for downloading files

And the yagga configuration file for yagga itself builds a Rust development environment (since vagga is written in Rust.)

Using those examples, I ended up with the script which you can find in my git repository.  I don’t include the script here, it may suffer revisions.

If you enter “vagga test” expect:

/work/.home/.cargo/bin/rustup
/work/.home/.cargo/bin/rustc
rustc 1.15.1 (021bd294c 2017-02-08)

Now I wondering whether I can run my IDE in the container, and how my source code gets into and out of the container (probably git.)  The answer seems to be that the directory where you invoke vagga is the “project” directory and is mapped into the container as /work.  Your IDE can work outside the container.  All artifacts of the build should be in the container and not pollute your project directory?

Brief notes

Create a hidden setup or options file ~/.vagga.yaml with contents: ubuntu-mirror: http://us.archive.ubuntu.com/ubuntu

Until you get your vagga.yml correct, vagga seems to repeatedly download dependency packages.  IOW, errors prevent completion of the container.  When you achieve a correct container, then vagga knows, and only downloads dependency packages as needed (when the repository publishes a security update or a nightly update?  Commands don’t establish dependencies?)

Doing “sudo apt-get update” between iterations seems to help some errors.

The directory where you invoke vagga is the “project” directory and is mapped into the container as /work.

Vagga stores the container in the hidden file .vagga in the project directory (alongside your vagga.yaml.)  (To delete a container?)

Continue with Part Two…

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s