WebKitGTK+ 2.2.0: It shines and doesn’t blink

With a bit of delay but we have finally released a new stable version of WebKitGTK+. This is the first stable release after the major 2.0 release. It mainly contains a lot of bug fixes and a few important new futures and API additions.

New Web Inspector

WebKitGTK+ now uses the new Web Inspector recently opensourced and upstreamed by Apple. We took advantage of the migration to improve the way inspector resources are distributed by compiling them in the WebKitGTK+ library as GResources. This means that resources are now always available without having to run make install or set environment variables.

WebKitGTK+ new Web Inspector

Initial Wayland support

WebKitGTK+ 2.2 can be built with Wayland target if it’s available (it requires GTK+ 3.10 compiled with Wayland support). Not everything is fully supported yet, but the WebKit layout tests can be run under the Weston compositor and WebGL content works under Wayland displays too. There’s more detailed information in Žan Doberšek’s blog.

Video accelerated compositing support

When accelerated compositing is enabled, the GStreamer media player can play videos using OpenGL. You can find more details in Víctor’s blog.

Custom JavaScript injection

This was one of the major regressions in WebKit2GTK+ compared to the WebKit1 API. WebKitGTK+ 2.2 now allows to inject custom JavaScript code using the JavaScriptCore C API from a WebKit Web Process Extension. New API has been added to also allow running specific JavaScript code in isolated worlds. You can find examples about how to use this API and how to write Web Process Extensions in general in this post.

Improved accessibility support in WebKit2

Accessibility support in WebKit2 has been reworked to not depend on pango and gail, which resulted in several bugs fixed and a much better accessibility support. Check Mario’s blog for all the details.

New API

WebKit2GTK+ Web Process Extensions

The multiprocess architecture of WebKit2 brought us a lot of advantages, but it also introduced important challenges, like how to expose some features that now live in the Web Process (DOM, JavaScript, etc.). The UI process API is fully asynchronous to make sure the UI is never blocked, but some APIs like the DOM bindings are synchronous by design. To expose those features that live in the Web Process, WebKit2GTK+ provides a Web Extensions mechanism. A Web Extension is like a plugin for the Web Process, that is loaded at start up, similar to a GTK module or gio extension, but that runs in the Web Process. WebKit2GTK+ exposes a simple low level API that at the moment provides access to three main features:

  • GObject DOM bindings: The exactly same API used in WebKit1 is available in WebKit2.
  • WebKitWebPage::send-request signal: It allows to change any request before it is sent to the server, or even simply prevent it from being sent.
  • Custom JavaScript injection: It provides a signal, equivalent to WebKitWebView::window-object-cleared in WebKit1, to inject custom JavaScript using the JavaScriptCore API. (Since 2.2)

This simple API doesn’t provide any way of communication with the UI Process, so that the user can use any IPC mechanism without interfering with the internal WebKit IPC traffic. Epiphany currently installs a Web Extension to implement some of its features such us pre-filled forms, ads blocker or Do Not Track using D-BUS for the communication between the Web Extension and the UI Process.

How to write a Web Extension?

Web Extensions are shared libraries loaded at run time by the Web Process, so they don’t have a main function, but they have an entry point called by the WebProcess right after the extension is loaded. The initialization function must be called webkit_web_extension_initialize() and it receives a WebKitWebExtension object as parameter. It should also be public, so make sure to use the G_MODULE_EXPORT macro. This is the function to initialize the Web Extension and can be used, for example, to be notified when a web page is created.

static void
web_page_created_callback (WebKitWebExtension *extension,
                           WebKitWebPage      *web_page,
                           gpointer            user_data)
{
    g_print ("Page %d created for %s\n", 
             webkit_web_page_get_id (web_page),
             webkit_web_page_get_uri (web_page));
}

G_MODULE_EXPORT void
webkit_web_extension_initialize (WebKitWebExtension *extension)
{
    g_signal_connect (extension, "page-created", 
                      G_CALLBACK (web_page_created_callback), 
                      NULL);
}

This would be a minimal Web Extension, it does nothing yet, but it can be compiled and loaded so let’s see how to create a Makefile.am file to build the extension.

webextension_LTLIBRARIES = libmyappwebextension.la
webextensiondir = $(libdir)/MyApp/web-extension
libmyappwebextension_la_SOURCES = my-app-web-extension.c
libmyappwebextension_la_CFLAGS = $(WEB_EXTENSION_CFLAGS)
libmyappwebextension_la_LIBADD = $(WEB_EXTENSION_LIBS)
libmyappwebextension_la_LDFLAGS = -module -avoid-version -no-undefined

The extension will be installed in $(libdir)/MyApp/web-extension so we need to tell WebKit where to find web extensions before the Web Process is spawned. Call webkit_web_context_set_web_extensions_directory() as soon as possible in your application, before any other WebKit call to make sure it’s called before a Web Process is launched. You can create a preprocessor macro in the Makefile.am to pass the value of the Web Extensions directory.

myapp_CPPFLAGS = -DMYAPP_WEB_EXTENSIONS_DIR=\""$(libdir)/MyApp/web-extension"\"

And then in the code

webkit_web_context_set_web_extensions_directory (webkit_web_context_get_default (), 
                                                 MYAPP_WEB_EXTENSIONS_DIR);

The Web Extension only needs WebKit2GTK+ to build, so in the configure.ac you can define WEB_EXTENSION_CFLAGS and WEB_EXTENSION_LIBS using pkg-config macros.

PKG_CHECK_MODULES(WEB_EXTENSION, [webkit2gtk-3.0 >= 2.0.0])
AC_SUBST(WEB_EXTENSION_CFLAGS)
AC_SUBST(WEB_EXTENSION_LIBS)

This should be enough. You should be able to build and install the Web Extension with you program and see the printf message every time a page is created. But that’s a useless example, let’s see how to use the Web Extensions API to do something useful.

Accessing the DOM

The GObject DOM bindings API available in WebKit1 is also exposed in WebKit2 from the Web Extensions API. We only need to call webkit_web_page_get_dom_document() to get the WebKitDOMDocument of the given web page.

static void
web_page_created_callback (WebKitWebExtension *extension,
                           WebKitWebPage      *web_page,
                           gpointer            user_data)
{
    WebKitDOMDocument *document;
    gchar             *title;

    document = webkit_web_page_get_dom_document (web_page);
    title = webkit_dom_document_get_title (document);
    g_print ("Page %d created for %s with title %s\n", 
             webkit_web_page_get_id (web_page),
             webkit_web_page_get_uri (web_page),
             title);
    g_free (title);
}

Using WebKitWebPage::send-request signal

Using the Web Extensions API it’s possible to modify the request of any resource before it’s sent to the server, adding HTTP headers or modifying the URI. You can also make WebKit ignore a request, for example to block resources depending on the URI, by simply connecting to the signal and returning TRUE.

static gboolean
web_page_send_request (WebKitWebPage     *web_page,
                       WebKitURIRequest  *request,
                       WebKitURIResponse *redirected_response,
                       gpointer           user_data)
{
    const char *request_uri;
    const char *page_uri;

    request_uri = webkit_uri_request_get_uri (request);
    page_uri = webkit_web_page_get_uri (web_page);

    return uri_is_an_advertisement (request_uri, page_uri);
}

static void
web_page_created_callback (WebKitWebExtension *extension,
                           WebKitWebPage      *web_page,
                           gpointer            user_data)
{
    g_signal_connect_object (web_page, "send-request",
                             G_CALLBACK (web_page_send_request),
                             NULL, 0);
}

Extending JavaScript

Using the JavaScriptCore API it’s possible to inject custom JavaScript code by connecting to the window-object-cleared signal of the default WebKitScriptWorld. You can get the global JavaScript execution context by calling webkit_frame_get_javascript_context_for_script_world() for the WebKitFrame passed as parameter of the window-object-cleared signal.

static void 
window_object_cleared_callback (WebKitScriptWorld *world, 
                                WebKitWebPage     *web_page, 
                                WebKitFrame       *frame, 
                                gpointer           user_data)
{
    JSGlobalContextRef jsContext;
    JSObjectRef        globalObject;

    jsContext = webkit_frame_get_javascript_context_for_script_world (frame, world);
    globalObject = JSContextGetGlobalObject (jsContext);

    /* Use JSC API to add the JavaScript code you want */
}

G_MODULE_EXPORT void
webkit_web_extension_initialize (WebKitWebExtension *extension)
{
    g_signal_connect (webkit_script_world_get_default (), 
                      "window-object-cleared", 
                      G_CALLBACK (window_object_cleared_callback), 
                      NULL);
}

JIT-enabled new WebKit and new GStreamer for Windows

Upgrading time for our Windows build. While all of these are pretty much under-the-hood changes it has a big impact on your browsing experience. We’re going for the latest stable WebKit with JIT-enabled javascript and WebP support. And we’re switching to a more recent maintained set of GStreamer plugins – in the past these weren’t easily available and always required extra work to add.

midori-win32-graybird

See below for the technical overview and currently known issues. The build still needs some rigorous testing to move from experimental to stable – if it works well it will be used for the next Midori version.

Build details

  • WebkitGtk 2.0.4
  • Spellcheck (you need to install dicts yourself for now)
  • JIT for javascript enabled
  • WebP support
  • GStreamer 1.0 plugins

Known Issues

  • Dark shadow on inactive buttons (gtk3 style issue)
  • Cursor does not change appearance on links/ textarreas (webkitgtk3 issue)
  • Slow load when font cache is missing (fontconfig issue)

Grab the experimental Windows build now and help testing! User feedback is valuable, either in comments, bug reports or #midori on Freenode (webchat).

Back from a successful FrOSCon

As I posted earlier I organized the first proper Midori booth for FrOSCon. From my humble perspective it was a success despite the rainy weather. All sorts of different folks who’d never heard about it got curious about Midori when they walked by the booth, asked what it is and what makes it special. A number of people asked about specific bug reports or availability on distributions. And among them some very interesting feature discussions came up.

CIMG2502

Session management was a recurring point of interest. Opening many tabs and windows, managing multiple sessions, storing away sessions for later and restoring after a crash. This is a good time as Tabby, the new session backend, is about to land, and it will make it much easier to keep many sessions or throw away the last open tabs on startup. One point in particular I’d like to take away here is that restoring tabs is something to be decided after startup, or after a crash instead of a fixed preference as it is now.

Another hot topic was sync. Firefox Sync appears to be quite popular for being easy to setup on a server. In one case the university provides logins to students as part of their infrastructure, which encrypts all session data. A very nice solution I would be keen to see is pluggable backends. Besides Firefox Sync there’re plans for a Midori sync server based on PGP encryption.


As for the booth itself, it appears that stickers were popular. But apart from the poster we had nothing with the URL written on it to hand out. Definitely
for the next Midori we will need business cards. Overall it was very exciting and I am looking forward to the next event in November.One thing different kinds of people asked about is writing extensions. I saw some eyes flashing when I mentioned that using Python will soon be possible, besides Vala and C, thanks to libpeas. The API in general will become easier to use and we brainstormed a bit for a third party extension website.

Midori with its first ever booth at FrOSCon 24/ 25th of August

t-shirts and stickers

t-shirts and stickers

My humble self has been helping with Geany and Xfce booths in the past. Though I originally left the organisational aspects to somebody else I got accustomed to the process. So I figured why not go the whole way. There’s one more person besides me by the looks of it and we’re going to wear these fancy t-shirts you see in the picture. Anyone who’s attending FrOSCon feel free to stop by on the 24th and 25th of August, in Sankt Augustin, it’s easy enough to find and there’s other reasons besides Midori to come!

Apples and Oranges

Time for a Midori 0.5.5 with a whole lot of exciting changes. Let’s have a look together at some of the highlights:

The proxy server support received improvement. Hostname prefetching is automatically disabled when a proxy is in use which plugs a potential privacy leak and the way we hook into the global proxy server settings is using offical API now, which should make it more robust.

Another improvement happened in the cookie department. Since a new enough libSoup is required we’re now sharing the cookie jar implementation (no pun here, that is its technical name). Our maximum cookie age code got much more robust, so when you set eg. 1 Week no cookie will survive longer than that even if a website tries to re-create it within that time (anyone interested in the gory details may use MIDORI_DEBUG=cookies to monitor it).

The context menu was in large parts rewritten. The code re-using built-in default menus from the time before WebKit had a sensible API is gone. It was a source for many buglets. At the same time it just got so much easier to extend context menus in Midori extensions.

The web app (and profile) support received more polish and got better integrated.

For WebKit2 we still have a long way to go, truth be told. But we’re making progress. It is at this point at the bare minimum of browsing the web, though still many small details are missing, not all icons show up, many extensions still don’t work. Anyone interested here will find this is a great area to get involved.

And…drums rolling the NoJS extension is here in all its glory. It surely is one of the most-requested features, which allows you to take control of scripts on a per-domain basis. It’s opt-in or opt-out depending on your personal preference.

The next release is incidentally going to be a 4/ 2 cycle with 4 weeks of features and 2 weeks of freeze, or bug fixes in other words. The pattern has worked very well and we’re not abandoning it but it may be beneficial to have more time to focus solely on existing bugs.

As always see the file Changelog for more details.

So download Midori v0.5.5 now (0.3 MB)

Some downloads are still being refreshed right now, so be patient if you think the option of your choice isn’t there yet.

You are welcome to join #midori on irc.freenode.net and party for the release!

Depanelitis spedalia

Skipping versions is becoming a bit of a habit. Midori 0.5.4 is out, the reason is in part due to the version control move and in part to the new website. Overall nothing to worry about, development is going on as fast as ever as you can see in the changelog.

Error pages are looking quite a bit nicer now and also try to be more informative. Alexander popped out of nowhere to work on it and other little improvements to the speed dial.

Bookmarks are also improving at an increasing pace thanks to André. Several bigger changes are already in the pipeline for the next cycle.

Another worthy highlight are the increased minimum requirements. Glib 2.23.3, GTK+ 2.24, Vala 0.16.0 and WebKit 1.8.3 are required now. If you wonder why that’s good news consider the large chunks of backwards compatibility code we were able to drop, which paved the way for improving thumbnail generation and favicon handling among other things.

As always see the file Changelog for more details.

So download Midori v0.5.4 now (0.3 MB)

Some downloads are still being refreshed right now, so be patient if you think the option of your choice isn’t there yet.

You are welcome to join #midori on irc.freenode.net and party for the release!

A new public face for the little browser

Following up on the little hint I gave in the last post. We skipped a release as some may have noticed. And why you ask, for good reason, because not only did we do a big migration to bzr and Rosetta but thanks to Lewis Goddard we now have a completely new website with decent screenshots, integration of the FAQ, our new Contribute page and a proper downloads page. And much of it is automatically regenerated from the existing contents in the wiki, blog and released packages. So apart from looking nice it also reduces some manual labour related to release management. Even better, the new website is hosted on github and you can file bugs and propose changes.

Switching to bzr for tight Launchpad integration

Judging by what the duck says Midori must be the only project in the world migrating from git to bzr. Or maybe, it was just so surprisingly quick and simple that there’s no need to brag about it. In fact Launchpad itself performed the import automagically and the switch is really about updating documentation and a cuple of hyperlinks. It might’ve been harder if I wasn’t familiar with bzr yet; pushing the new trunk and flipping an option to make it the focus branch was all that was needed.

The old world

Why did we actually make this step? To make project management more efficient. Midori as a project roughly might be split into

  • bug triaging
  • reviewing contributions
  • implementing code
  • design decisions
  • roadmap

It turns out a huge bottle neck was the manual effort standing between these separate aspects. For example anyone doing bug triaging wouldn’t have direct access to code. And code review was always separate from both code and bug management. To the point that people end up waiting on, looking for and blocking on each other. And losing contributors, which is the worst thing to happen to any free software project.

The new world

  • There’s an obvious, easy way for anyone to push branches
  • Code review is integrated with branches and bugs
  • Anyone can become a reviewer, merge and push code

bzr vs. git

I do enjoy a proper emotional project bashing as much as the next guy, especially with a beer in front of me. But really, these days bzr and git both have a comparable share, personally I work on either one on and off. And at the end of the day what counts is the health of Midori as a project. Bazaar may well have the weaker storage efficiency and performance compared to git in a benchmark, but in this case git had the higher human cost.

Go ahead and contribute now

And, with a miserable impersonation of River Song I’ll say “There is more exciting changes to come… spoilers”.

WebKit2, downloads, web apps and clang

With a slightly prolongued freeze period Midori 0.5.1 0.5.2 is out of the door. We made a big leap on the promised WebKit2 support – it’s not fully done yet but very, very usable despite not having context menus and some extensions are to be done. Definitely worth giving a go for anyone curious about multi-process goodness who can live without adblock if need be.

Downloads and web app support have both received major refactoring, making the code much more modular and approachable, and fixing buglets and adding polish along the way. Downloads work more reliably across windows and more quality control across panel and toolbar. The way is paved for even more goodness. For the first time you can manage web apps/ launchers created from within Midori graphically. This is also up for even more improvements.

To round it up a whole lot of code improvements have come from static analysis such as clang (LLVM) and other sources and a number of buglets were squashed. Despite all the bigger changes QA is looking very good. Note to self: having strict feature freeze periods before every release is paying off, even if it’s hard in a small team.

Update: I apparently goofed up the release process and 0.5.1 insists it is still 0.5.0. To reduce confusion this is 0.5.2 now, same thing, but with a proper version.

So download Midori v0.5.2 (1.1 MB) (MD5) (ChangeLog) already!

Also Midori v0.5.2 32-bit Portable for Windows (32 MB) or Midori v0.5.2 32-bit Installer for Windows (34 MB) now. You are welcome to join #midori on irc.freenode.net and help testing pre-releases for Windows. You can make the difference by helping out!