Transcript of talk: "GStreamer"
KDE Contributor and Developer Conference
Notes taken by: Jonathan Riddell (these are not official material)
More information on the talk can be found hereChristian works for Fluendo who are doing the streaming for this conference using GStreamer, he helps on GStreamer and librsvg. Scott wrote JuK and lots of other bits and works for SAP.
GStreamer is a pipeline based multimedia framework ("more buzzwords make us cooler"). It's a media framework and does most of the thing you'd expect from a media framework. It takes input, MP3 or video etc, works out how to handle it and moves it along until you get some output. GStreamer is plugin based, the core is small compared to the plugins of which there are currently around 80. The plugins handle input, output, codecs and all the exciting stuff.
GStreamer is not a sound server and does not include or have a preferred sound server, it can output to a sound server or to drivers directly. GStreamer is not playback only solution like MPlayer, Xine etc. GStreamer does not require any libraries from either of the two main Free Software desktops, integration is done through plugins.
Design was based on research from Portland University. Started on Sourceforge at the start of 2000. Focus was on server backend and embedded uses. Now there is a higher focus on desktop issues because of the increase in interest from this area. Abstraction of sound system and card mixer has been added, support ALSA, OSS and work is being done on Windows. Descriptive i18n messages added for desktop friendly error handling. Work has been done on getting legal issues out of the way, there are patents involved in multimedia stuff.
The core is based on C and uses glib. Bindings exist for Ruby, Python, Perl, Mono, KDE style C++ and Guile (GNU's scheme based extension language). The core is LGPL licensed.
A pipeline is a buzzword but means it has two ends, a source and a sink (this is a pune or play on words). It is composed of many elements which do various jobs along the way. The source can be a file, a KIO slave (Scott says he needs to fix this) or a physical device. Elements are connected to process this data, decoders, effects (not just adding echo etc but volume, also color correction on video), mixing etc. The output is sent to a sink which can be a file, a sound server, a display or a device.
Does a pipeline work with only audio or only video or both? They are taken in together, split for the middle then put back together for output. It does not magically solve the issues of X and audio/video synchronization.
The focus on making API as easy to use as possible for application developers. Pipelines can be defined in XML. Creating a pipeline is difficult because of the logic involved, it's harder than writing an application that uses GStreamer. Command line tool gst-launch lets you easily create pipelines on the command line.
gst-launch filesrc location=example.ogg ! offdemux name=mux { mux. ! queue ! vorbisdev ! audioconvert! osssink}
There is also a plugin called Spider which magically works out what decoder to use.
GStreamer can offer KDE a complete multimedia framework. It has a large and active development community unlike Arts which is basically unmaintained, GStreamer has 15 to 20 people committing including around 5 working on the core, the devel list is more active than kde-devel. GStreamer is not on it's first maintainer, they don't keep killing their maintainers it just means there is enough energy in the project to carry on. They like Gnome jokes.
It is cross platform like Qt and KDE. Licensing allows GPL as well as proprietary or non-GPL license. A shared implementation between KDE and Gnome reduces redundant work.
GStreamer is not a sound server but could be used to make a sound server. Probably the most likely candidate at the time for a sound server is MAS.
GStreamer is not a Gnome (or KDE) library, it is still in the application area just as with KDE. It is not a Gnome project it is a freedesktop project. Like other libraries it may be part of the Gnome platform just as they take libraries from elsewhere.
Doing 0.8 release series currently. API/ABI stable release. Stable GStreamer releases can be installed alongside each other. Current focus is on playback including improving support for broken files to the level of Xine or MPlayer. They are also focusing on cross playform capabilities and being the best for free formats such as the Ogg codecs with the hope to have people use these formats more and convert restricted formats to Free formats.
All of GStreamer's competitors are too narrow in functionally (e.g. xine which is just a playback framework), unacceptably licenced (GPL'd or proprietary) or too immature (there is a lot of work gone into GStreamer). GStreamer has a lot of mindshare. Arts also has mindshare, or is that "angershare"?
Lots of applications using GStreamer: Totem, Rhythembox, Sound Juicer, Marlin, ScoundScrape, Gnome Mixer, Gnome Sound Recorder etc. Commercial support and development through Fluendo. Sony's television division is looking at using GStreamer in a television camera.
KDE GStreamer usage is fairly basic. Juk and Amarok use it, Juk's GStreamer option was announced at Fosdem about 18 months ago (even though Scott was still hacking on it at the time). KDE style bindings are for 0.6 API currently. Scott ported Kaboodle over in about 3 hours, most of which was spent ripping out Arts based code. Scott is not a GStreamer guru but the ease of porting Kaboodle was due to the good design of GStreamer. Scott is however listed on the credits for Gnome Mixer apparently (possibly because he is a GCC 2.95 patch guru).
Questions is should it be the KDE 4.x media framework? This can mean separating the parts of decoding and sound server (and synthesis which isn't really used). A lot of posting or replacing of official KDE media applications would be required. Gnome had lots of volume controls each with their own way of changing the volume, hopefully KDE won't be so bad.
Current KDE bindings provide a low lever wrapper API for the GStreamer API and a high level binding for simple playback. They are C++, Qt, QObject based.
Simple example using TJansen's bindings was shown consisting of only 5 lines. The full code for JuK using GStreamer is about 150 lines.
Can GSteamer keep binary compatibility? Scott has brought this up a lot on the GStreamer list. The hope is that 1.0 will be a stable API for many years and Gnome 3.0 will be out about the same time as KDE 4.0. There is not specifically a higher priority for Gnome than KDE, the tough question is will it be ready for either one of them when they come. Gnome has a longer binary compatible release cycle traditionally than KDE.
All data is timestamped so it can be synchronized until output by GStreamer.
Is it possible to create plugins to use this as a synthesiser? That's not the target. There is discussion for getting MIDI support in. Jack is made for creating media, it is the home recording sound server. There will remain a split between professional and desktop multimedia, GStreamer is focusing on desktop now to keep 99% people happy.
