Embedding OS X UI elements in the GTK+ toolkit

At Lanedo, we’ve been working on a project with Xamarin for the last couple of years to improve support for the cross platform UI toolkit GTK+ on OS X. We’ve been doing this to improve Xamarin Studio, a development environment for engineers using Xamarin’s tools to deliver applications for various mobile platforms (like iOS, Android, and Mac). Part of the project with Xamarin was to enable embedding of native NSView controls into a GTK+ UI.

GtkNSView

A cow embedded in a WebView embedded in a GtkNSView embedded in a GTK+ GUI embedded in an NSWindow. Yay!

NSView is the Quartz counterpart to GtkWidget (a base class for all UI elements), all things visible are NSViews. The way GTK+’s Quartz backend handles things is that inside the NSWindow that is created for each top-level GtkWindow, it creates one top-level NSView to draw the entire GtkWidget hierarchy to.

Generally, embedding can be split in two somewhat independent parts: displaying the NSView and delivering events to it. In order to deliver events to something, it has to be on screen first, so making that happen was step one.

Displaying the NSView

We took the obvious approach and created a GtkWidget, named GtkNSView, which does most of the embedding work by placing its NSView at its own allocation. We add the NSView as a subview to the Quartz backend’s top-level NSView (see gdk_quartz_window_get_nsview()), and we take care of showing, hiding and positioning it using the normal GtkWidget::map(), ::unmap() and ::size-allocate() methods.

This makes the NSView follow the GtkWidget’s position and visibility, and for most cases “just works” (we’ll come to the tricky parts later). However it would just sit there and look nice, but not get any events, which sort of limits its practical usefulness.

Delivering Events

In order to deliver events to GTK+ widgets, GDK (the low level drawing functions in GTK+) hijacks the entire native event stream, translates it to GdkEvents, finds the right top-level window to deliver it to, and dispatches it accordingly. When an event is converted successfully, it’s removed from the native event stream because GDK has handled it, making it unavailable for our embedded NSView.

Fortunately, event handling in GTK+ and Quartz are similar: Mouse events are delivered directly to the GdkWindow / NSView they are happening on. Key events get dispatched to the top-level GdkWindow / NSWindow which is in charge of forwarding them to the focus widget / first responder.

Forwarding key events to the embedded view is pretty straightforward: Make GtkNSView accept focus depending on whether the NSView accepts first responder, then implement GtkWidget::key-press() and ::key-release(), get the original NSEvent from the GdkEvent and send it to the NSView. We need some additional code to make sure the focus chain works across GtkWidgets and NSViews and need to special-case Tab and Shift+Tab, but that can all happen inside GtkNSView and requires no further magic.

When it comes to getting mouse events to the embedded view, things get a little more complicated. We could in theory implement GtkWidget::event(), get the native NSEvent from the GdkEvent and deliver it to the view manually, but this would not work properly because the stream of translated GdkEvents doesn’t exactly match the native NSEvent stream:

  • the Quartz backend simply ignores events it doesn’t know
  • there are generated GdkEvents mixed into the stream
  • probably more reasons

Therefore, we need a little support from GDK, this can’t be done exclusively in GtkNSView. We took an existing patch from Paul Davis (of Ardour fame) and changed find_window_for_ns_event() in gdkevents-quartz.c so that it would return the event to Quartz if is was on one of the NSWindow’s subviews (which can only be one of our GtkNSView-embedded NSViews).

Done. It works! :)

So far, so good… but there are some issues left:

  • Click to focus
    Mouse events are going back to Quartz without even entering GTK+, but keyboard focus is managed by GTK+. In order to fix this, we added a new signal GdkWindow::native-child-event() which gets emitted before native events are returned back to Quartz. GtkNSView connects to the signal and can grab the focus if it was a button press.
  • Scrolling
    GtkWidgets don’t know when they are being scrolled around, because this is done on the underlying GdkWindow directly. As for click-to-focus, we added a new signal GdkWindow::move-native-children() which gets emitted where GdkWindow moves around child windows as a result of scrolling. GtkNSView listens to the signal and repositions its NSView.
  • Clipping
    Now this is very ugly. Being able to scroll is nice, but what happens when the GtkNSView scrolls out of a scrolled window’s viewport? It doesn’t know anything about clipping and has to be told explicitly. We override NSView’s drawing function, using an obscure objective-c pattern called “method swizzling“. In the overridden method, we walk up the widget hierarchy, clipping to all scrolled windows’ viewports as we come by them, and then call the original drawing method, using the clipped rendering context.
  • NSViews Obscured by widget stacking
    Doing the clipping for scrolled windows is nice, but what if the widget hierarchy is self-overlapping? Walk the widget hierarchy even more and add more complex clipping, in theory. We are currently working on this, and it’s not done yet…
    Also, clipping doesn’t clip the event stream, you can still click on the invisible NSViews.

TL;DR

As you can see, GtkNSView is not a general-purpose embedding solution yet, and we don’t know it ever will be. However, we can say that it works perfectly fine for simple or medium complex UIs, just avoid scrolling and overlapping, and things should “just work”.

The GtkNSView patchset lives in Xamarin’s bockbuild repo on GitHub. It’s the long patch series, which contains all the GTK+ adaptions we did for Xamarin. We are considering to bring the widget in shape for possible inclusion in upstream GTK+ 3.x.

The work on the widget was done mainly by Kristian Rietveld and yours truly.

If you need consulting or customization on GTK+ or on UI toolkits in general, don’t hesitate to contact us.

Share on Google+Tweet about this on TwitterShare on LinkedInShare on FacebookFlattr the authorBuffer this pageShare on RedditDigg thisPin on PinterestShare on StumbleUponShare on TumblrEmail this to someone

Michael Natterer (Mitch) has been an Imendio/Lanedo hacker since 2005. Before that, he spent some years programming user interfaces (like GTK+ and GNOME) and applications for digital TV set top-boxes. In his free time he hacks on GIMP which he has maintained since 2001. You can contact our team for professional consulting on our contact page.

Tagged with: , , , , , , ,
Posted in Blog, Development, Gnome

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>