[Music]
hi everyone welcome back hope you're all
still
full of energy and excitement um towards
the end of the day i didn't think i
would be but somehow all of these lovely
talks are keeping me going
next up we have emmanuela bassey
telling us about a year in the life of
gtk
so emanuela has been working on gtk and
gnome for over 15 years both as a
volunteer and on commercial products
based on gnome technologies like memo on
nokia hardware
he will be taking questions at the end
so please use the um chat in
point that way in venus not the chat the
questions tab above the chat um to ask
your questions and i will pass them on
to emmanuella at the end um so thank you
very much all over to you
hi and thank you and thanks everyone for
joining me
in this uh presentation and also thanks
for to lca for
uh
giving me the chance to talk about gtk
uh in this conference
so
we're going to talk a little bit about a
year in the life of the gtk project
since we the gta project has been
doing a major release
for the first time in nearly 10 years so
stuff has changed uh in the year before
that and in this year
is changing again uh there's there's
lots of stuff to cover so
let's start
so
i am part of the gdk development team
and i've been working on gtk and gnome
for
about 15 years now
i'm mostly involved in a core
application development platform
so
core libraries
but i also work with
the
developer documentation
the documentation team
i maintain the odd application and i
collaborate with other gnome maintainers
in the release team
my contributions have been part of
both my literal job for years
but i also worked on gnome and gtk on my
spare time so i
i'm
usually able to appreciate the use of
gtk and gnome technologies
as a downstream developer as well as a
volunteer
gdk is a very old project
these days it's nearly 25 years old um
it's been in continuous development
since
roughly 1995 1996.
uh when it was written as part of the
new image manipulation program to
replace motif which was the toolkit used
at the very beginning
over the past 25 years gtk has seen
three major version bumps from gtk 1 to
2 from 2 to 3 and now from 3 to 4
and has been used and deployed to
millions of devices all around the world
the last major release of gtk gtk4 is
probably
the biggest leap in terms of the toolkit
internal design since the 2.0 days
and it was a change driven both by the
evolution of windowing systems on linux
and by the challenges and
the realities of maintaining a
volunteer-driven
free software toolkit with no central
authority
and very minimal corporate backing
so today i'm going to talk to you a
little bit about the changes that led to
gtk 4.0 in 2020
and what happened to the project and
around it in the years since then
um
in 2016 the main topic of discussion
about gdk was whether we could improve
the rendering pipeline for gtk3 which
was the major version at the time
without breaking the api contract
the fact of the matter though was that
we had been changing the internals of
the toolkit a lot more than we
did back in 2006
when we introduced cairo in the
rendering pipeline
and we moved away from the internal
drawing api that
mimicked and wrapped xlib
we're also looking at the transition
period in which we'd say uh goodbye to
x11 and hello to wayland
and on the other side of things we were
seeing the adoption of a high dpi
density displays on both laptops and
desktops as well as dedicated graphical
hardware with 3d acceleration
during the gtk 3 stable cycle we did
change a lot of the rendering pipeline
worked
to allow proper transparency between
widgets for instance or implementing
things like css borders and shadows that
extended outside the widgets area and
blended with the background
these changes required little to
no updates in idiomatic code
which
is good in general
but there was a lot of a non-idiomatic
code in applications that were highly
ported from gtk2 to gtk3 back when we
release gnome 3.0
um
this is a time when we did not make
ourselves any friends with application
developers that consume the platform
and
especially the ones that were consuming
the platforms three years at a time by
updating their development
uh machines
uh using long-term releases
from the distributions like ubuntu
our main problem was effectively cairo
cairo is
was still is a high quality vector-based
2d rendering api
while it introduced this high quality
rendering to graphical user interfaces
it was limited by its design which
reflected the area in which it was
developed so the early 2000s
thanks to cairo we were able to move gtk
to a css based drawing state just like
web rendering engines
like those web branding engines
soon discovered
it is effectively easier to take
whatever state cairo has blow it away
and replace it entirely with the state
of the css machinery that is held
client-side in the toolkit
and this is especially true if you're
running something that looks like a lot
like a gui as opposed to a vector image
another problem at the core of cairo was
its reliance on fast read back pipelines
between the cpu and the system memory
which is something that just does not
exist in modern systems using a gpu
finally the maintenance status of the
project had been in question ever since
intel approached the two people
responsible for maintaining the library
one after the other first uh kyle worth
and then chris wilson
in the end cairo is an api that makes
sense as an internal detail of the
toolkit
sadly we exposed it as part of our api
to application and library developers
and that made it necessary to write
wrappers around it to represent a full
set of rendering operations
in order to hide the fact that
the car api didn't actually
work the way we wanted
but at that point though why would you
be using cairo at all when
cairo didn't give you access to the most
efficient graphical hardware on your
system
well because sadly the actual api that
lets you access the hardware is opengl
which has been known to be a disaster
zone for years especially in linux
luckily the introduction of mainstream
x11 compositors using opengl like
compies in the mid-2000s and gunam shell
in 2011
at the effect of improving the gl driver
support which in turn pushed more people
to make use of gl for applications and
games
and that
it was coupled with improvements in the
gl standard api
which deprecated at first and then
jettisoned the old craft from the 90s
and basically pushed for um
using the gpu as a programmable resource
instead of a completely opaque state
machine
a gl based css
state render is much more tractable than
implementing a fully generic vector api
and you can get a lot of mileage out of
small shaders to do things like shadows
borders
this was demonstrated by projects like
mozilla servos web render
which acted as an inspiration for the
gtk rendering pipeline
for instance once you have a full tree
of rendering operations you can decide
what changed from a previous frame
what didn't and what to remove from the
graph of operations since it will never
be visible
at that point you can re-render
everything
and do it faster than just caching
intermediate states and then
figuring out how to expire that cache
suddenly
by doing this the rendering step of each
frame is not the bottleneck anymore and
that gives you much more time to do
um layout at 60fps
or whatever frame per second is your
screen refresh rate
and also gives more time
to
application logic to
do its own thing
um speaking of pushing at 60fps um
the task of doing so becomes really hard
when um
you're dealing with large resolutions
your cpu
back in the day of the early 2000s
late 90s even
your cpu could deal with pushing 800 by
600 pixels or even 1024x768
per frame
but once you get to 1080p
2k 4k 8k
uh or even multiple displays uh with
very large resolutions then you
definitely need something more powerful
than your cpu
and uh hi idpi displays also require
changing the nature of pixels uh from
something that is physical so it's an
actual
like
uh
point of light on your screen to a
logical unit
so you don't accidentally turn your
desktop into the most challenging level
of mine sweeper ever
and you'll need a magnifying glass to
hit uh
like ridiculously small buttons or three
very very
small text
this requires changing how you load
image assets in your application i use
size windowing system surfaces and how
you align every ui element to the pixel
grid especially text
otherwise you get a blurry mess
alongside the changes in the lower
layers of the stack that impact directly
the hardware
starting from the early 2010s the clock
began ticking for display the old
display server technology available on
linux which is x11
over the years toolkits have moved away
from submitting rendering commands to a
server
and instead they started to send
complete frames generated client side
the x server usually stood in the way of
making applications reliably present
windows to the users for years
um
even though we introduced uh well not me
personally but the x11 developers
introduce extensions like the composite
extension
weyland
as a display server technology promised
to make every frame of painting with no
interference by the display server
sadly gtk 3
just like gtk2 was still very much
designed on top of xlib which meant that
every one of its back ends
was re-implementing the same xlib design
on other windowing system and gdk
exposed a lot of ostensibly independent
api that
only ever made sense or even worked when
using x11
in terms of portability as i was saying
gdk basically re-implemented xlib on
every single platform
um since the
gtk 2.0 release in 2001.
uh the ability to
run a gdk application on different
platforms and different environments was
never meant to be used at ways to do
like cross-platform
write once run anywhere development
it was a way to port gtk applications
from linux
and other unix-like systems to windows
and mac os
and
as i said i was saying this amounted for
implementing x11
everywhere
while this minimized the changes to
application code because after all
everything looked like x
it made everything more complicated to
maintain and to fix and to test inside
the toolkit
every new functionality will need to be
obstructed in ways that didn't always
make sense or they would be just left
unimplemented
plus very very few people could
understand how the back ends actually
worked a simpler windowing system
protocol like weyland would fit in a lot
better within these constraints
so we come to 2016 and the internal
changes that happened during the gtk 3
cycle um
basically just
were not
really
for free anymore
of course most people only remember the
theming issues at the time um
themes which were never under any
stability guarantee the
issues were mostly the result of
downstream distributions of themes
as packages
without any additional qa process that
would tie them to release of the toolkit
they were ostensibly targeting
so people got broken
desktops because they were using a theme
that was not tested with a version of
gtk they was
in theory using
the main impact for application
developers
were not
themes where the changes in the drawing
api especially for applications
supported in the early days of gtk3
uh to avoid that happening again uh just
before of the gtk4 development process
we decided to lay down some fundamental
guarantees for application developers
first and foremost that once the 4.0
release was done the toolkit would be
feature frozen
new api and new features could be added
of course but existing functionality
would never change including the css
selectors
so it's a little bit more strong than an
avia api compatibility
additionally the new development cycle
will contain a roadmap for future major
api work
as well as the support window for
existing stable releases to shorten the
development cycle while providing a
reliable schedule
so
feature-free is one stable
just adding small widgets
uh freeze the theme
shorter api cycles
and to support the api versions um for
a window of to support the api versions
so this is kind of a
graph of
our policy you can see that we have the
gtk3 branch that goes on
um it was we had to release gdk320
to um
ease off the porting window but
basically from 324 we jumped to the 390
development process release four
and then from four we're gonna release
we're gonna start the 490 development
cycle in
uh probably three years time
but as
as soon as gtk
5 is going to be released gtk3 is going
to be end of life and
the only
two supported stable release
versions will be gta 4 and gta 5
and then when gtk 6 is released gta 4
will be end of live so it's much more
explicit
instead of
just being random
whatever we want
so in terms of gtk
internal and api design
the version the major version number
three was still heavily influenced by
the model of a deep object-oriented
forest of types
this design encouraged developers to
inherit from existing complex widgets
which add the side effect of locking
some internal state to avoid breaking
expectations
for gdk4 instead the design mantra has
been to favor composition and delegation
instead of inheritance
we made it easier to create your own
custom composite widgets without
exposing their internals
you can do that with object orientation
and use xml
to describe your ui and
bind it to the uh class structure
automatically so you don't have to deal
with
uh the fiddly bits of um
loading uh uh loading xml yourself and
then assigning pointers and stuff like
that there are a bunch of macros and a
bunch of api that will deal with that
for you
um
another thing that we looked at a little
bit later than we wanted but it was hard
to find uh resources and funding for
that sadly
um was accessibility
like
prometheus gifting humans with the uh
with fire the sans accessibility team
gifted the free and open source software
world with 80 spy back in 2000 2000
2001.
and
alongside it is by gtk acquire all
accessibility api
first as an out of three model
because of dependencies
the legacy of that gift
made things much more complicated over
the following 20 years
the original design was heavily based on
corba
which is many things but a nimble ipc
mechanism isn't one of them
additionally the accessibility stack was
heavily tied to x11 with application
broadcasting the old internal state
and allowing other applications to take
them over
on unsecured sockets
and unprivileged assistive technologies
were able to
get and inject windowing system events
across the whole desktop
this design also by layering the api
across multiple components you add the
toolkit
with which provide an implementation
then you add the set of abstract types
as a separate library
since it they could be theoretically
implemented by multiple toolkits and
libraries except it never happened
well if you
yeah it never actually happened
and
on the other side you had the actual low
level library that sat at both ends of
the ipc wire
on top of that there were there was
sun scaling down the accessibility theme
oracle acquiring sun and basically
killing the entire thing
um and the pool of people working on
making the linux desktop accessible to
people with disabilities
yeah
it got smaller and smaller um
the more time passed
in the meantime web browsers are where
the action is these days which meant
having a whole new accessibility
specification to follow the w3c aria
specification
so for gtk4 we decided to follow that
instead of to eliminate all the layers
uh and just
speak the 80 spy protocol directly and
implement
from an api perspective the
w3c specification for accessibility
so what's new in gdk4
we have a new deferred mode rendering
infrastructure uh instead of being an
immediate mode with cairo so you don't
submit you don't draw directly you
submit rendering commands and then gtk
will take them and
structure them in ways that make more
sense
we use opengl by default and cairo is a
fallback
we use weyland by default even the
design of the api is based on weyland
we have much more simpler backends
so they are easy to
test fix and maintain
we have a reliable release policy and a
stable feature set for every single
new major release of gtk
we use delegation instead of inheritance
making the
api more reliable
and we have a new accessibility api
of course we also have to document all
this stuff
gdk
started off in 1997
with um handwritten tech info pages for
the api reference
and then
the project was migrated to a custom
documentation generator that would take
comments from the source code and turn
them into dot book xml first and then
that turned into html
uh the tool was called gtk doc and it
served the gnome project as all well
enough to be reused for all other
libraries
sadly gtk dock was written in pearl and
its maintenance cost
kind of followed the curve of every
other pearl
project
and at some point since pearl
fell from grace
it got progressively less maintained
there was a last-ditch attempt to port
gtk dr python
as well as trying to reduce the dog
books performance bottleneck
but
basically building the api reference for
gtk took
as much as building the actual toolkit
so
500 000 lines of c
uh plus examples plus demos
uh on our ci pipeline take about
um
six to seven minutes
and generating documentation for that
took the exact same amount of time
uh it was untenable
in the meantime though
since 2010-ish
every gnome library started providing a
machine-readable description of its c
api
as well as the documentation for the for
that api
in form of introspection data generated
at build time
the typical consumer of this data
language bindings
they can use it both at runtime as a
memory mappable binary blob
and they can also use
an xml description for code generation
the xml description contains the
documentation associated to every public
type and symbols and it's possible to
use it to generate an api reference
much more efficiently
the
it also has another advantage of using
the introspection data
the generated api will match as much as
possible the actual api that will be
used
by the
language bindings
so you you'll get something that is a
lot
easier to map between
programming languages
so
i wrote this tool called gi doctrine
which take the introspection
documentation and the introspection data
and generates the documentation and and
creates an uh html website
it's
really fast it takes about a minute to
build the entire thing
and
we publish it as part of our ci pipeline
alongside the
reference for other libraries so gtk
gio pango for text gdk big spot for
image loading
it has stable links
it allows cross linking of symbols using
javascript so you can
tweak the cross linking for the remote
and the local
case
and it makes it easier to update to fix
the documentation because every single
type and every single symbol has a link
to the source code
that defines it so you can literally
jump into the gitlab web ui
and then edit the documentation um
straight from the web browser
uh it also has
client-side searches
on an index that is generated at build
time as well
so
uh you never send your queries to i
don't know google or whatever else
on top of that we also revamped the gta
well i say we i revamped the gtk
documentation website the gnome
developer
documentation there
it's been rebuilt from scratch
we moved from
python to django
kind of haddock uh
application
that literally took html out of the
release tarbles and instead we switched
to
sphinx
which is a python
documentation generator
this lets us do things like
cross-linking automatically
searches and everything else
it's um
also a lot easier to build the
documentation locally and test it before
sending it to gitlab for a merge request
and then we use gitlab
this is the gitlab ci pipeline to
generate the entire documentation and
website and publishing it
so it's a lot easier to contribute there
as well
and outside of the content of the
developer documentation we also added
new
from the old set of tutorials that were
kind of out of date most of the time
we added them to we added more content
there and we also adding beginners
tutorials to go from the human interface
guidelines to the api references
and speaking of interface guidelines
the same website all hosts them
and they are generated using the same
tool
so what happened in
this last year from december 2020 to now
we work on stabilization the main goal
of gdk now becomes bug fixing and
handling feedback from application
developers that are currently porting
their applications to gtk4 from gtk3 or
even from gtk2
but also we want to ensure that gtk4 can
be used by virus environments and
platforms so it's possible to easily
port applications between them we have
aggressively simplified improve our
build system thanks to mison and it's
possible to go from cloning just the gtk
repository to a full build including the
library dependencies on both mac os and
windows and we test that on our ci
pipeline
there are still some issues here and
there especially some core dependencies
on macos but we are much more confident
about the state of the toolkit there
um
the ci pipeline for windows also
um we have two pipelines for windows one
using msys2 so using a gcc tool chain
and then we have a
microsoft uh visual c
uh native tool chain
and then we generate the dlls as um
uh
build artifacts so people can literally
download them from uh for a merger quest
to test if the fix was um
was correct
without having to rebuild the library
locally
but another thing that we kind of worked
on is
trying to make it easier to write
platform libraries
since gtk is used by multiple
environments as a foundational toolkit
we wanted to make it more environment
agnostic
elementary xfc are based on gtk and they
provide their own platform libraries for
writing integrated applications
by making gtk less beholden to gnome
design patterns those libraries can now
avoid
undoing
some of the token internal state
or write completely custom widgets to
replace gdk zone
and since gnome is still based on gtk
and it needs a way to implement
its own ui design patterns for their own
for its own applications
we worked on a library called libadvita
which implements the
gnome interface guideline patterns and
widgets
it will also follow the gnome
schedule essentially so it will
cycle through api a little bit faster
than gdk
um
but also theme changes and things like
that
not every six months but it has uh
definitely a shorter lifetime
uh libat waiter was recently released
libavita 1.0 was recently released
alongside gtk 4.6 and will be available
with gnome 42 when drum 42 is released
in march
so i have a few
resources here
this is the gtk documentation website
and the new developer dot num.org for
the developers documentation
this is the api reference for libadvita
as you can see
most of these apis will are
generated
on git lab using the ci pipeline so
they're always up to date
the job gene introspection documentation
and the documentation for the
documentation generator using job j
introspection
and also we have the gtk development
blog where we
try to
explain some of the internals of the
toolkit and outline changes in the
toolkit that are coming or
they that are planned
and we also have a discourse instance so
you can talk to us without having to
open random bugs on on gitlab and then
filling up the
issue tracker
you can follow me on twitter
and on twitch
on twitch i do
regular streaming where i
document i
either write documentation
like the beginner's tutorials for the
gnome developer documentation
or i work on gtk
so i guess
it's time for me to thank you
and thanks uh the lca organization for
having me again
to talk to you about gdk
and i guess if you have any questions
now would be the time
thank you emmanuel that was like a
really interesting behind the scenes
look into
gtk and what's been going on and um i
appreciated the history
lesson as well like i mean i i
obviously um
i
i used
just had a pronunciation discussion in
the chat so i'm going to say both gnome
and gnome um myself a fair bit but i've
never really thought too much about how
it works sorry the cat is just having a
bit of a scratch behind me there
um
so yeah that was that was really
eye-opening for me thank you
so we've got a few questions
um
let's start with
what are your top bits of advice for
people building and maintaining other
libraries and apis based on what you've
learned with gtk
this is a very complex question
the
the main thing i'd say is
not trying to please everyone
try to
use
try to limit yourself
give you constraint writing api is
almost like
painting or doing any other form of art
more than technology
the trick is in the either in the
negative space or in the limits that you
set your to yourself
the important thing i think is to allow
people to do
complicated things
but not
try to second guess
where they will go
um
and
there's always time to add convenience
api at some point
the other thing that is incredibly
important and i think is fundamental to
everything else is
documentation documentation
documentation always write more
documentation than you will actually
need
i think those are the two main main
lessons that i learned
yeah um i mean i my own experience i
agree and i really like the way you
phrased that um
yeah and definitely i could definitely
hear the years of experience and
thinking about those issues behind there
okay next question
so you're trying to move away from xlib
does that make it easier for the windows
port
yes the windows back end in gtk4 is
incredibly simpler than it used to be
we have more people contributing to it
for that reason
we have more people
filing issues and
helping out
because we reduce the internal surface
and the concepts are a lot easier to
deal with
the whale on backhand is also a lot
easier to deal with because you know i
have to implement weird x11 isms here
and there
the main thing that changed for instance
is the clipboard uh and the
copy and paste api
and that
went from being a weird protocol based
on top of xlib
the xdnd and the selection stuff
to
being uh
passing file descriptors around
that was
incredibly that allows us to make the
api a lot simpler as well for
application developers
people routinely now come to the
matrix channel saying
i replace the entire clipboard handling
that i had in my application with like
three lines of code it cannot possibly
be this simple and we're like no no it
is it is the simple it works oh that's
wonderful
that that that's
a joy to imagine it must make you really
happy to see people say that yeah it
does
okay next question is
css support was a bold move do you think
it'll keep up
or meet the browsers ever
um
we kind of followed the
css specifications as much as we as much
it makes sense for a desktop toolkit
so whenever in doubt on how to implement
the css feature we follow the w3c specs
but
the gtk css is not a web css
you cannot influence layout for instance
using gtk css
because we cannot
allow people to turn every single
application into the css zen garden demo
uh it would be
terrible for documentation purposes or
for accessibility or for
general like reliability
it would be pretty pretty bad
um
but to be fair the css
implementation gave us enough freedom to
do
a lot more
uh interesting user interfaces than
the plain blocky
style of um
uis
i don't know we will probably never
catch up to the web in the sense that it
doesn't always make sense to follow that
the next question is
really interesting to see your revamped
documentation workflow any specific tips
or lessons learned for other projects to
ensure docs are both easy to use and to
contribute to
treat documentation as you treat your
code
if you
make it super easy to contribute code to
your project you should make it equally
easy to contribute documentation
so
if you're generating code from
your source code if you're generating
documentation for your source code
link the
document the
place where that happens from the
documentation itself
you can look at how rust does their
documentation and
i mean it i took liberal inspiration
from ra stock and from cargo uh for the
gtk documentation website
um
definitely definitely follow that again
if you make it easy to contribute to
your code there is no reason whatsoever
you should make it easier to convert
documentation that way
so follow the same
um
the same practices there
allow people to easily open a merge
request with the documentation fixes
especially if you can do that without
leaving the web browser for instance
thanks for that okay we have time for
one more question
how well does gnome slash gnome for work
over a network uh for example ssh-x
um
well if you're using x11 it works
exactly the same if you're using weyland
then
we actually
do remoting using
rdp slash vnc
we have an entire
new um
new layer that does that inside the
inside tonum shell that lets you do
screen sharing
uh so basically you send
complete frames and then you have
protocols that do
you can either
you can either have dumb protocols that
literally take the
the output and just send it over the
wire
with or without compression otherwise
you have smarter protocols that can do
diffing so
what you send is smaller than that
but in general
um
the x11 approach is actually the least
efficient possible way to do things
because it the
nobody uses x11
draw commands anymore um unless you're
using application from the 80s or early
90s
um
and the other thing is that it's very
bandwidth and efficient
it sends a lot of data anyway
interleaved with commands
so if you're using protocols like vnc or
rdp it's usually much more efficient
because they can do compression
or even you don't even need to do that
gtk has a
html backend that literally runs your
application
on the server and then
sends the
the structure of the ui and various
assets to using websocket from a server
to the web browser pointed at a specific
url and port
it literally works as a remoting api and
i know of people using it in production
which scares the hell out of me
but it has that functionality as well
and people use it so
it's not just
that is definitely more efficient than
using x by ssh
all right well thank you we there was
one more question but um we're out of
time so we will transfer that question
over to the post talk chat kaya theater
channel in venulis um for you to have a
bit of a chat with emmanuella after the
talk if that's all right with you
absolutely
great um so
thank you very much emanuela for that
really interesting talk um
and
yeah um
your approach to all of this redesign
work and so on is is
um quite nice to hear about
um before we
um close off the stream for the day
um so this is the end of our normal
talks for today for linuxconfeio
um but
there is there is one more so if you are
a
professional or contributor ticket
holder remember to join us from 6 30
conference time
in venulis it'll be in a different stage
to what we're looking at right now for
the professional delegates networking
session and a really interesting
presentation from anthony green on
election analysis um so if
you are such a ticket holder um or a
speaker and you are going to that don't
miss it it's tonight um and
for those who are going to that and
those who aren't
have a lovely evening and we'll see you
back here tomorrow
bye
[Music]