Author Archives: kalikiana

Cross-compiling with snapcraft: iota

Last week I snapped iota, a small editor with emacs and vi modes.

I decided I wanted to try it out on a Raspberry Pi before pushing it to the snappy store, for that I need to cross-compile. Conveniently I recently implemented support for that in snapcraft.

If you didn’t follow the previous step from my previous blog post, to try what I’m about to do, you’ll need a fresh clone of my iota branch.

git clone git@github.com:kalikiana/iota.git
cd iota
git checkout snapcraft

If on the other hand, like me, you built a snap for another architecture before, in my case amd64, I need to clean the build files.

snapcraft clean

Now on to the most difficult step:

snapcraft --target=arch=armhf

Time for a cup of coffee!

Once it’s done, you can use SSH (or whatever your preferred method is) to copy the snap to your Raspberry Pi 2 or 3 running Ubuntu Core.

snapcraft install --dangerous --classic iota_0+git.67643b3_armhf.snap

Now I can use iota as my editor on the Raspberry Pi!

iota

Packaging rust with snapcraft: iota

Iota is a small editor for your terminal, with support for emacs hotkeys or vi-style commands, syntax highlighting, and easily hackable, written in Rust… now wouldn’t it be awesome if I could install it easily as a snap? So I figured I’d contribute a snapcraft YAML in my lunch break. Here’s how I did it.

Head to https://github.com/gchp/iota and hit Fork on the top right. Let GitHub work its magic. Once it’s done I can checkout the code.

git clone git@github.com:kalikiana/iota.git
cd iota
snapcraft init

At this stage I’ve got a local copy of iota and a snap/snapcraft.yaml that was automatically created for me. Some editing is needed.

name: iota
summary: Iota is a terminal-based text-editor written in Rust
description: |
  100% open source
  highly extensible/customisable/scriptable
  fast & efficient - designed with modern hardware in mind
  cross platform - it should work anywhere
  developer friendly - it should just "get out of the way"
  Rust tooling integration

The fields name, summary and description are pretty self-explanatory, especially given the template snapcraft init creates. I just picked some wording from iota’s README.md.

version: git
grade: devel
confinement: classic

I used version: git here because it just picks up the tag or commit hash, and it’s good enough for an unstable build anyway. confinement: classic is much more interesting here because this is an editor, and I want to be able to edit all the files on my system. confinement: strict wouldn’t allow that as it restricts access to specific folders only used by the snap itself.

apps:
  iota:
    command: bin/iota

Note how I’m specifying bin here explicitly. Classic snaps don’t get any path magic unlike their strict counterparts.

parts:
  iota:
    plugin: rust
    rust-channel: nightly

Now as for the actual build instructions I don’t really have to do much. iota specifies all its dependencies in its Cargo.toml so no build-packages or stage-packages are necessary. The only thing I need to specify here is the nightly channel since iota requires unstable rust features.

snapcraft

This takes a while as rustup downloads the toolchain and crates needed behind the scenes. High time to grab that lunch my break is meant for.

Fast-forward, I’m installing the result of that build.

snapcraft install --dangerous --classic iota_0+git.67643b3_amd64.snap

Now I can use iota as my editor!

Last but not least, after some testing:

git add snap/snapcraft.yaml
git commit -p -v
git checkout -B snapcraft
git push origin snapcraft

Finally, I’m opening a pull request on GitHub.

How to cross-compile snapcraft plugins

I implemented support for cross-compiling when using the Go plugin and the Rust plugin (bonus points for using a LXD container) in snapcraft. On the part of the user of snapcraft it’s completely transparent but each plugin needs to explicitly support this, so here’s how you enable your favorite snapcraft plugin to make this work.

snapcraft --target-arch=armhf --debug

This will result in the following error if a part uses a plugin that doesn’t support cross-compilation yet:

Setting target machine to 'armhf'
Building for a different target architecture requires a plugin specific implementation in the 'foo' plugin

Let’s enable the fictional foo plugin.

snapcraft/plugins/foo.py

First of all, we tell snapcraft that we know how to cross-compile by implementing one additional method. We don’t need to do anything here. Although I’ve found one good use of the method to be checking that the requested target is supported by the toolchain. For a language like Rust or Go the target is different to the Debian architecture. Rust even supports using a different libc (glibc or musl). That means the plugin needs to map the target and raise an error if it can’t. In the following example I’ll also be seeting self._target so that it can be used in other steps of the lifecycle, for example to set environment variables or when installing target-specific dependencies of the toolchain.

def enable_cross_compilation(self):
    # Cf. rustc --print target-list
    rust_targets = {
        'armhf': 'armv7-{}-{}eabihf',
        'arm64': 'aarch64-{}-{}',
        'i386': 'i686-{}-{}',
        'amd64': 'x86_64-{}-{}',
        'ppc64el': 'powerpc64le-{}-{}',
    }
    self._target = rust_targets.get(self.project.deb_arch).format(
        'unknown-linux', 'gnu')
    if not self._target:
        raise NotImplementedError(
            '{!r} is not supported as a target architecture when '
            'cross-compiling with the rust plugin'.format(
                self.project.deb_arch))

Second of all, we’ll want to adjust the environment variables. This depends on the build system. Typical candidates are compiler names and pkgconfig – we don’t need to worry about the latter since snapcraft already takes care of it. Go for example would also require GOARCH to be set.

In the following code I’m assuming there’s a self._goarch variable that was set in enable_cross_compilation.

def env(self, root):
    env = super().env(root)
    if self.project.is_cross_compiling:
        env.extend([
            'CC={}-gcc'.format(self.project.arch_triplet),
            'CXX={}-g++'.format(self.project.arch_triplet),
            'GOARCH={}'.format(self._goarch),
          ])
    return env

The example uses the triplet of the target architecture. Commonly useful values here are, using armhf values for demonstration purposes:

cross_compiler_prefix: arm-linux-gnueabihf-
arch_triplet: arm-linux-gnueabihf
deb_arch: armhf
kernel_arch: arm

Note that snapcraft will automatically install the compiler and libc headers needed. Should you need extra packages it’s only one command away:

self.build_packages.append('foo-armhf-cross')

It goes without saying that we’ll want to have an integration test checking that this new code actually continues to work in the future.

integration_tests/plugins/test_foo_plugin.py

You’ll need to have a snap in integration_tests/snaps/foo which you may already have. Otherwise you’ll need to add it. It’s a good idea to try and make use of more advanced use cases that may need special care in the context of cross-compilation, such as using C code in the case of Go.

import snapcraft
from snapcraft.tests.matchers import HasArchitecture
[...]
def test_cross_compiling(self):
    if snapcraft.ProjectOptions().deb_arch != 'amd64':
        self.skipTest('The test only handles amd64 to armhf')

    arch = 'arm64'
    self.run_snapcraft('stage', 'foo', '--target={}'.format(arch))
    binary = os.path.join(self.parts_dir, 'foo', 'install', 'bin',
                          os.path.basename(self.path))
    self.assertThat(binary, HasArchitecture('aarch64'))

You’ll also want to unit test the new use cases supported by the plugin. There will be a snapcraft/tests/plugins/test_foo.py if it’s an existing plugin where you can verify the build environment and commands used for building insofar as they differ (Go would require you to use go build for example instead of go install).

I’m not going to go into all the details here since this is very plugin-specific but one thing you’ll generally want to add is scenarios and mock is_cross_compiling:

scenarios = [
    ('armv7l', dict(deb_arch='armhf', go_arch='arm')),
    ('aarch64', dict(deb_arch='arm64', go_arch='arm64')),
    ('i386', dict(deb_arch='i386', go_arch='386')),
    ('x86_64', dict(deb_arch='amd64', go_arch='amd64')),
    ('ppc64le', dict(deb_arch='ppc64el', go_arch='ppc64le')),
]

def setUp(self):
    super().setUp()

    self.project_options = snapcraft.ProjectOptions(
        target_deb_arch=self.deb_arch)
    patcher = mock.patch('snapcraft.ProjectOptions.is_cross_compiling')
patcher.start()
self.addCleanup(patcher.stop)

If all went well I hope you’re going to propose a PR to snapcraft adding support for the foo plugin – a real one of course, not the fictional one that doesn’t exist

Preparing for Chemnitz 19 March 2016

Freshly printed Midori stickers just arrived, in November the last ones of the old batch were being given out to visitors of the OpenRheinRuhr. So I ordered new ones at FlyerAlarm, a print company based in Würzburg, Austria. This time around I went for circular ones and slightler smaller than they used to be. This size also fits better on phones and tablets for those who like to decorate theirs with stickers. You might also notice that the colors are better and the graphic comes out better – fixes to the SVG will be finding their way into the repository soon.

photo221454754825480153

With the new stickers I’m well-prepared for Chemnitz Linux Days this march, in a little more than a fortnight. There will be a Midori booth there. Be sure to add the weekend of March 19 and 20 to your calendars!

Heads or tails

Midori featuring headerbar and Adblock in the toolbarAs things go we diverged a bit from the original plan of making a big compatibility break with this release. Instead much of the original branch has been broken off into pieces that made their way into trunk. We did however bump WebKit2, libSoup and Zeitgeist dependencies. The most visible change is support for CSD, client side decorations, also called header bars after the API in GTK+3.10 (enabled via GTK_CSD=1). With no further adue here goes Midori 0.5.11 with a whole lot of nice things and a ton of bug fixes! Peek at the change log if you want more details.

 

So download Midori v0.5.11 now (1.2 MB)

The source tarball is up. Windows is going to follow suit shortly, and others are still being refreshed, so be patient if the option of your choice isn’t there just yet.

You are welcome to join #midori on irc.freenode.net and party for the release!

A leap of leopards (or: 0.5.10)

Whilst work on the compatibility breaking branch is coming along nicely it seemed sensible to do another bug fix release. Thus Midori 0.5.10.

So download Midori v0.5.10 now (1.2 MB)

The source tarball is up. Windows is going to follow suit shortly, and others are still being refreshed, so be patient if the option of your choice isn’t there just yet.

You are welcome to join #midori on irc.freenode.net or Slack and party for the release!

Ballet (or: 0.5.9)

Seven months of sweat and tears… oh well, not quite so dramatic. In any event Midori 0.5.9 is out!

We’re already scheming… I mean planning for the next cycle. We want to go WebKit2 and GTK+3 only now and do away with the fourfold compatibility setup. Anyone who finds this thrilling is more than welcome to join in; the fun is going to start soon.

So download Midori v0.5.9 now (1.2 MB)

The source tarball and Windows download are up, while others are still being refreshed, so be patient if the option of your choice isn’t there just yet.

You are welcome to join #midori on irc.freenode.net or Slack and party for the release!

Midori booth at OpenRheinRuhr 2014

At OpenRheinRuhr 2014 we had a Midori booth again. Almost didn’t happen due to the strike of Deutsche Bahn which caused many people to miss the event. Fortunately my humble self made it.

In addition to stickers, this time around we even had edible gummi paws!

Midori booth

Amongst the most posed questions was process isolation, something WebKit2 provides, and which will be available to everyone soon once our plan to move fully towards GTK+3, WebKit2 and Vala is put in motion. The other one was why Midori crashed so often – which is due to the at least 4 primary build configurations currently available. I’m very much looking forward to seeing this go away at some point in the future.

Pretty happy with the results personally, was able to get a decent number of people curious to try out Midori and in one instance even install it on site.

Kitten looking to be adopted

The following unfortunately comes up regularly these days:

  • This Midori looks interesting, maybe I should try it out.
  • Aha, there’s a Debian package for it.
  • sudo apt-get install midori
  • midori (0.4.3+dfsg-0.1)
  • Meh, pretty crashy, I better ask some devs for help.
  • Ah, so 0.5.8 is the latest.
  • So why isn’t this packaged!?

The answer is simple, Midori needs a Debian package maintainer! Thousands of Debian and Ubuntu users are facing the above situation, including lots of Raspberry Pi users. A single person stepping up to it can literally change the world at this point. Now if fame isn’t motivation enough, there may be a t-shirt to get.

Experimental Windows Snapshot for 0.5.9

Time for testing. Fresh build with many great fixes from the current trunk as well as updated dependencies.

This build should resolve startup issues for users of Windows 8. In case of crashes debug files are shipped in the archive and you are encouraged to provide a backtrace.

Known issues

  • Non-working webinspector. Regular View (DOM) Source should still work if you need it.

Grab the experimental Windows build now! Testing can help findings bugs before the next release! Give it a go, let us know how it goes! User feedback is valuable, either in comments, bug reports or #midori on Freenode (webchat).