The KDE Randa 2014 meeting, in easy-digestible video format!

In case you were wondering what was going on in Randa, here are some first hand impressions. The video was produced by Françoise Wybrecht (alias Morgane Marquis) and Lucie Robin, and the people in it are the actual participants of the event. It was also created using KDenlive, one of the awesome Free Software tools a team has been working on at the Randa meeting itself. The video introduces the faces and personalities of the contributors and their different backgrounds and origins. Many thanks to our brand new ad-hoc media team for producing this video!

(In case the embedded video does not show up, see here: https://www.youtube.com/watch?v=yua6M9jqoEk)


Filed under: Coding, CreativeDestruction, English, FLOSS, KDE, OSS, Qt Tagged: Creative Destruction, FLOSS, free software communities, KDE

Qt Weekly #18: Static linking with Qt


Qt has supported static builds for a long time, but iOS made this way of shipping Qt more widespread. To make the discussion about static linking more practical, we will not discuss things abstractly, but we will look at how to compile the Weather Info example application from the Qt Positioning module statically for iOS, and address questions as they come up.

 

Weather Info example app

Weather Info example app

First of all, to build a static application, we need a statically compiled Qt. This is not difficult — adding the “-static” option to configure mostly does it. However, ensuring that the dependent libraries are also available and linked statically can take some effort. In the case of iOS, Qt is statically built by default, so we can simply use the default Qt packages.

To build the Weather Info application statically, let’s just try building as usual, running qmake and make (or even better, let Qt Creator call them for us).

cd weatherinfo
qmake -r weatherinfo.pro CONFIG+=debug
make

…and… it runs!

So, it seems that we don’t have to do anything special once we have a static Qt.
Great, so is this blog post already finished?

Well, not quite. If we take a look at the linking step, we see a huge command line (EDIR=/Users/fawzi/Qt5.3.1/Examples/Qt-5.3/positioning/weatherinfo):

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -arch armv7 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk -L$EDIR/Debug-iphoneos -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/platforms -L/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk/System/Library/Frameworks -L/Users/fawzi/Qt5.3.1/5.3/ios/lib -L/Users/fawzi/Qt5.3.1/5.3/ios/qml/QtQuick.2 -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/accessible -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/qmltooling -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/position -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/bearer -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/imageformats -F$EDIR/Debug-iphoneos -filelist $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo.LinkFileList -dead_strip -headerpad_max_install_names -stdlib=libc++ -u _qt_registerPlatformPlugin -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/platforms -framework UIKit -L/Users/fawzi/Qt5.3.1/5.3/ios/lib -framework OpenGLES -L/Users/fawzi/Qt5.3.1/5.3/ios/qml/QtQuick.2 -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/accessible -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/qmltooling -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/position -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/bearer -L/Users/fawzi/Qt5.3.1/5.3/ios/plugins/imageformats -lz -lm -miphoneos-version-min=5.0 -lqios_debug -framework Foundation -framework QuartzCore -framework CoreFoundation -framework CoreText -framework CoreGraphics -lQt5PlatformSupport_debug -framework Security -framework SystemConfiguration -lqtquick2plugin_debug -lqtaccessiblequick_debug -lqmldbg_qtquick2_debug -lQt5Quick_debug -lqmldbg_tcp_debug -lQt5Qml_debug -lqtposition_cl_debug -framework CoreLocation -lqtposition_positionpoll_debug -lQt5Positioning_debug -lqgenericbearer_debug -lQt5Network_debug -lqdds_debug -lqicns_debug -lqico_debug -lqjp2_debug -lqmng_debug -lqtga_debug -lqtiff_debug -lqwbmp_debug -lqwebp_debug -lQt5Gui_debug -lqtharfbuzzng_debug -lQt5Core_debug -Xlinker -dependency_info -Xlinker $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_dependency_info.dat -o $EDIR/Debug-iphoneos/weatherinfo.app/weatherinfo

It looks mostly OK, but contains a lot of libraries. System and Qt libraries are expected, but we also have:

  • the platform plugin:-lqios_debug that loads plugins/platforms/libqios_debug.a
  • the QtQuick2 plugin:-lqtquick2plugin_debug that loads qml/QtQuick.2/libqtquick2plugin_debug.a
  • the accessibility plugin for qtquick:-lqtaccessiblequick_debug that loads accessible/libqtaccessiblequick_debug.a
  • the plugin to improve debugging of qtquick2:-lqmldbg_qtquick2_debug that loads plugins/qmltooling/libqmldbg_qtquick2_debug.a
  • the plugin to talk to the QML debugger:-lqmldbg_tcp_debug that loads plugins/qmltooling/libqmldbg_tcp_debug.a
  • the plugin giving the position using the core location framework: -lqtposition_cl_debug that loads plugins/position/libqtposition_cl_debug.a
  • the generic polling plugin for position:-lqtposition_positionpoll_debug that loads plugins/position/libqtposition_positionpoll_debug.a
  • the plugin giving the generic bearer implementation: -lqgenericbearer_debug that loads plugins/bearer/libqgenericbearer_debug.a
  • support for the DDS image format:-lqdds_debug that loads plugins/imageformats/libqdds_debug.a
  • support for the ICNS image format:-lqicns_debug that loads plugins/imageformats/libicns_debug.a
  • support for the ICO image format:-lqico_debug that loads plugins/imageformats/libqico_debug.a
  • support for the JPEG 2 image format:-lqjp2_debug that loads plugins/imageformats/libqjp2_debug.a
  • support for the MNG image format:-lqmng_debug that loads plugins/imageformats/libqmng_debug.a
  • support for the TGA image format:-lqtga_debug that loads plugins/imageformats/libqtga_debug.a
  • support for the TIFF image format:-lqtiff_debug that loads plugins/imageformats/libqtiff_debug.a
  • support for the Wireless Application Protocol Bitmap image format:-lqwbmp_debug that loads plugins/imageformats/libqwbmp_debug.a
  • support for the WebP image format:-lqwebp_debug that loads plugins/imageformats/libqwebp_debug.a

The platform plugin and a couple of others should come as no surprise, but Wireless Application Protocol Bitmap? A format by WAP for black and white images? When WAP has basically disappeared? Does our application really need it?

While, the non-debug version libqwbmp.a is only around 15 KB in size, things like libqjp2.a can be around 600 KB, so this is definitely worth some investigation.

One could hope that unneeded code gets stripped away. After all, that is one of the advantages of static linking which leads to a smaller executable than shipping all dynamically loaded libraries. If a function in a library is not used, that function is not part of the static executable.

Unfortunately, plugins make the situation more complex.

Qt Plugins

Qt internally uses dynamic linking in several places, mainly with plugins. Plugins are used for platform dependent support, image formats, audio and video codecs, sensors, and so on.

If you take a look in the plugin directory (qmake -query QT_INSTALL_PLUGINS) you will see the various plugins that Qt might use.

For example, linking QtPositioning, and QtMultimedia is not enough to have a position fix and being able to display a movie, respectively. The required plugins also need to be loaded.

Calling the Q_IMPORT_PLUGIN marcro from user code will load that plugin (which will thus need to be linked).

When using dynamic linking, the plugins are loaded automatically at runtime when needed.
With static linking, one has to decide which plugins are required already at build time.

Starting with Qt 5.3, thanks to Ossi, a meaningful set of plugins (depending on the Qt modules you added) is automatically loaded.
That is the reason why without any extra effort things just worked in the above example, but that is also the reason we have so many plugins linked to our executable.

qmake does not just add the -l<plugin> to the link flags (which has no effect if not used), but really uses the plugin.
Indeed, if we look at the linked files from $EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo.LinkFileList:

$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/main.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/appmodel.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_qml_plugin_import.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/weatherinfo_plugin_import.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/qrc_weatherinfo.o
$EDIR/weatherinfo.build/Debug-iphoneos/weatherinfo.build/Objects-normal/armv7/moc_appmodel.o

we see weatherinfo_plugin_import.o which comes from the compilation of the autogenerated
$EDIR/weatherinfo_plugin_import.cpp which contains the Q_IMPORT_PLUGIN that takes care of loading the Qt plugins.

This is nice, but we want a lean application, without unnecessary cruft, so how can we get rid of the plugins we do not need?

As described in How to Create Qt Plugins, it is possible to switch off the automatic loading of plugins with CONFIG -= import_plugins, but this disables the generation of the weatherinfo_plugin_import.cpp file, and then we would have to do everything manually: add Q_IMPORT_PLUGIN somewhere in our code, define the QT_STATICPLUGIN preprocessor variable and link the plugin (as one has to do without qmake).

We just want to specify which plugin to load, but still leave the ugly details up to qmake.
How to Create Qt Plugins also states that one can set the list of plugins to load for each class with QTPLUGIN.<className>. The position plugins we will actually need, but the image conversions? We should be able to get rid of all of them.

So let’s add

QTPLUGIN.imageformats=-

to our project file, weatherapp.pro, and indeed no more image formats plugins are linked, the release executable becomes around 1MB smaller (13’492’384 -> 12’539’088 bytes), avoids unneeded plugins initializations, and… still works.

If some imageformat plugins are needed, it is possible to add them to QTPLUGIN.imageformats, or directly to QTPLUGIN.

It is worth noting that for this to work, the plugin creator has to define both PLUGIN_TYPE and PLUGIN_CLASS_NAME to make the automatic loading work. There were some issues with that, but hopefully all have been fixed.

QML Plugins

Our example basically uses just QtQuick 2, but in general one has to scan recursively the imports of all QML files used to find all plugins to load, and to load them. Morten wrote a tool that scans the QML files (qmlimportscanner) and qmake uses it to generate $EDIR/weatherinfo_qml_plugin_import.cpp which gets compiled to the weatherinfo_qml_plugin_import.o file we have previously seen in the linked files.

For this to work, the QML files have to be found by the tool (which might for example require installing Qt in some cases) and the qmldir files have to specify the classname of the plugin.

Conclusions

Static linking avoids the dependency on external libraries and leads to smaller packages for a single self-contained application (all the unused code of a library is stripped).
But for the linker, “used code” means code that might be reached, and to make plugins work, one has to pull in the plugin using Q_IMPORT_PLUGIN and link it.

qmake simplifies this operation by generating special files and linking the modules, but its default choices about which modules your application wants to use are not always correct. Furthermore, reachable code does not necessarily mean code that will be executed, or that does something useful for your application. For example, the initialization of a plugin might be a bit expensive, so doing it unconditionally at startup is not always a good idea.

Thus removing plugins that are not needed is a good way to make your application leaner.
There are also other places where Qt might waste CPU time (to initialize things) or space (both in application size and memory), as for example the MIME DB or Unicode information, but that is another story…

It is important to note that static linking has consequences with respect to the licenses of Qt you might want to use. While you are probably safe if your application is distributed according to the terms of a recognized free and open-source license, you should definitely check the licensing issue for proprietary or closed-source applications. Qt Enterprise license is the recommended one for closed-source applications.

I hope this has made the issue and advantages of static linking with Qt clearer. Happy coding and let me know if you have comments or tips.

qbs 1.3.0 released


We are happy to announce the release of qbs 1.3.0 today. Qbs, or Qt Build Suite, is a general cross-platform build tool that uses a language similar to QML for project description. Qbs can be used for any software project, whether it is written in Qt or not and it simplifies the build process for developing projects across multiple platforms.

If you haven’t tried it out yet, we strongly encourage you to take it for a spin to see how convenient building with Qbs is! Check out the documentation for more details.

What’s new in qbs 1.3?

For this release, we have concentrated on improving the integration with Qt Creator. The main points are:

  • Source files can now be added to (and removed from) qbs products via the project tree, just as for qmake projects. Thanks to Thomas Epting for the initiative!
  • More care is now taken to reload a project only if it is really necessary.
  • In addition, reloading a project has become somewhat faster, particularly in the case where a project file was changed without introducing any semantic differences (e.g. whitespace changes).

We think that these items, combined with a number of important bugfixes, have improved the user experience of working with qbs in Qt Creator a lot.

What else is worth mentioning?

On the language side, it is now possible to set different profiles for particular products. This is important for projects that need to produce binaries for different architectures.
Also, we have once again reduced the memory footprint significantly.

Where do I get it?

Source packages and a binary package for Windows can be found here.
Qt license holders can download the enterprise version of qbs directly from their Qt Account.

Other relevant links

Wiki: http://qt-project.org/wiki/qbs
Documentation: http://qt-project.org/doc/qbs-1.3/index.html
Bug tracker: https://bugreports.qt-project.org/browse/QBS
Mailing list: http://lists.qt-project.org/mailman/listinfo/qbs

How to contribute to the KDE Frameworks Cookbook

The annual KDE Randa Meeting, in itself already shock-ful of awesome, this year hosted the KDE Frameworks Cookbook sprint. Valorie, Cornelius and I already wrote a lot about it. Plenty of attention went into finding the place for the cookbook between the getting-started HOWTOs on KDE Techbase and the full-blown API documentation. Not surprisingly, there is a space and a good purpose for the book. Frameworks developers and maintainer have had to deal with the question of where to put introductions that segue newcomers into how to use the modules many times, and so far, the answer have been unsatisfactory. Newcomers only find the API documentation when they already know about a framework, and TechBase is a great resource for developers, but not necessarily a good introduction. What is missing is a good way to help and learn about what KDE Frameworks have to offer. So there is the purpose of the KDE Frameworks Cookbook – to help developers find and learn about the right tools for the problems they need to solve (and also, consumable on a e-book reader by the pool). For developers and maintainers, this means they need to know how to add sections to the book that cover this information about their frameworks. These tools and workflows will be explained in this post.

Im a way, the book will partly provide an alternative way to consume the content provided by KDE TechBase. Because of that, the HTML version of the book will integrate and cross-link with TechBase. The preferences of what kind of documentation should be in the book or on TechBase are not yet written in stone, and will probably develop over time. The beauty of Free Software is that it also does not matter much – the content is there and may be mixed and remixed as needed.

Two principles have been applied when setting up the tooling for the KDE Framworks Cookbook. The first is that content should stay in the individual frameworks repositories as much as possible. The second is that content taken from the frameworks, like code snippets, shall not be duplicated into the book, but rather referenced and extracted at build time.

KDE Frameworks Cookbook front cover

Keeping content that is part of the book in the frameworks repositories makes it easier for developers and maintainers to contribute to it. A book can grow to ginormous proportions, and keeping track of where its text is related to a specific framework or piece of code will be difficult if the two are separated into different places. However, content that is not specific to individual frameworks may as well be placed in the book repository. Considering that contributions of code and prose are voluntary and compete for the available time of the people working on it, it is important to keep the workflow simple, straightforward and integrated with that of development. Frameworks authors add sections to the book by placing Markdown formatted texts in the framework’s repository. The repository for the book (kde:kf5book) references the frameworks repositories that provide content as Git submodules, and defines the order in which the content is included using a CMake based build system. The target formats of the book, currently PDF, HTML and ePub, are generated using pandoc. Pandoc can also be used by the contributors to preview the text they have written and check it for correct formatting. The book repository already contains various sections pulled in from the frameworks repositories this ways. Interested contributors will probably find it easiest to follow the existing examples for the submodule setup and the build instructions in the CMakeLists.txt file to add their content. The ThreadWeaver repository (kde:threadweaver) contains Markdown files that are part of the cookbook in it’s examples/ folder which can serve as a reference. See below for why the file names end in .in.md.

Avoiding duplication by copy-pasting code into the book is achieved by using a special markup for code snippets and other examples and a separate tool to extract them. Especially example and unit test code that is shown in the book should always be part of the regular, CI tested build of the respective framework. This ensures that all code samples shown in the book actually compile and hopefully work for the reader. The snippetextractor tool processes the input files that only contain references to the code samples and produces complete Markdown files that include the samples verbatim. The input file names end in .in.md. The conversion of the input files is handled by the build system of the kf5book repository, not in the individual frameworks repositories. It is however possible to add steps to produce the final Markdown files to the CMake build files of the repository. This will catch markup errors of snippets during the frameworks build, but does require the snippetextractor tool to be installed.

Setting up continuous builds for the book target formats is currently being worked on. Producing the book will be integrated into KDE’s Jenkins CI, and up-to-date version of it will be available on KDE TechBase. Until then, curious readers can self-produce the book:

  • Install pandoc and the necessary Latex packages to produce PDF output.
  • Build and install snippetextractor using QMake and a recent (>5.2) Qt. Make sure it is in the path before running CMake in the later steps.
  • Clone kde:kf5book, and initialize the Git submodules as described in the README file.
  • In a build directory, run cmake <source directory> and make to produce the book.

Enjoy reading!


Filed under: Coding, CreativeDestruction, English, FLOSS, KDE, OSS, Qt Tagged: Akademy, FLOSS, free software communities, KDE, kde community

How syntax highlighting works

This article was written 1 year ago. I finally found the time to translate it to English

If you are a programmer, you spend a significant amount of your time coding. It doesn’t matter how many buttons or menus or IDE or editor has, the core is a code editor component. Do you know how it works?
This article explains how syntax highlighting in Qutepart (and katepart) works. The article is not about UI, but about the architecture. If you are interested, lets go..

read more

Squish tip of the week: How to find and use dynamic objects

Squish provides many options when working with dynamic objects:

  • Using Wildcards in the Object Map
  • Using Regular Expressions in the Object Map
  • Using Wildcards or Regular Expressions with Object Real Names in a script
  • Building Object Real Names within a script using information retrieved during script execution

Watch the videos below to learn two approaches to working with dynamic objects

The videos demonstrate the Squish for Qt edition and sample Qt application; however the same options apply working with other Squish editions.



Further reading
Click here to request your free 30 day Squish evaluation

Adding LGPL v3 to Qt


20 years ago, Trolltech, the company that created Qt, was founded. One of its founding principles was to release Qt as free software to the open source community. In the early versions, this was limited to Unix/Linux and the X11 windowing system. Over the years, more and more platforms were included into the open source version of Qt.

At the same time, the licenses under which Qt was available evolved. The Qt 1.x source code was still released under a rather restrictive license. With Qt 2, we moved over to the QPL. Some years later, with Qt 4.0, Qt started to embrace the GPL v2, to remove some license conflicts between GPL-based applications and the QPL.

Trolltech was involved in talks with the Free Software Foundation (FSF) when the GPL v3 was created, and we added this license as an optional license for Qt after it was published by the FSF. Finally, in 2009 Nokia added LGPL v2.1 as a licensing option to Qt.

The spirit of all GNU licenses is about a strong copyleft, giving users rather strong access and rights to the source code of application and libraries. It was always meant to protect the users’ freedom to modify the application and underlying libraries and run the modified application.

In many people’s opinion there is, however, a loophole in the LGPL 2.1, where it doesn’t clearly talk about running the applications using a modified version of the library. Even though it violates the spirit and intentions of the LGPL, this loophole has been extensively used by companies that create locked-down devices. If devices use LGPL v2.1 software, the user may not be able to install modified versions of the library on the device and use it together with the other software that is installed on it.

We also consider locked-down consumer devices using the LGPL’ed version of Qt to be harmful for the Qt ecosystem. The device is not open to third party developers and thus doesn’t contribute in extending the size of the Qt ecosystem and the range of devices that can be targeted by software developers using Qt. In addition to not contributing to the ecosystem, it doesn’t fund the further development of Qt.

For these reasons we believe that LGPL v2.1 is not protecting the users’ freedom as it was intended by the Free Software Foundation. To account for this, the FSF created version 3 of the LGPL, a license we feel is legally formalizing the intentions of the earlier version.

Changes in the Qt 5.4 Release with LGPLv3

Because of this, we are now adding LGPL v3 as a licensing option to Qt 5.4 in addition to LGPL v2.1. All modules that are part of Qt 5.3 are currently released under LGPL v2.1, GPL v3 and the commercial license. Starting with Qt 5.4, they will be released under LGPL v2.1, LGPL v3 and the commercial license.

However, there will be a set of new add-ons that will be only released under LGPL v3 (plus GPL v2 or later) or commercial license. These add-ons are listed below. We have discussed with the KDE Free Qt Foundation and have their support to make this change in Qt 5.4. We are also in talks with the KDE Free Qt Foundation about further strengthening the agreement.

New add-ons released under LGPL v3

In Qt 5.4, the new Qt WebEngine module will be released under LGPL v3 in the open source version and under a LGPLv2.1/commercial combination for Qt Enterprise customers.

Adding LGPLv3 will also allow us to release a few other add-ons that Digia before intended to make available solely under the enterprise license. In Qt 5.4, we will add a technology preview for two brand new modules to Qt under the LGPL v3.

The first module, called Qt Canvas3D, will give us full WebGL support inside Qt Quick. It is fully functional, but still marked as a preview because the support for JavaScript typed arrays is still implemented in a slow and not 100% compliant way.

The second module is a lightweight WebView module that will also be released as a technology preview. It supports embedding the native Web engines of the underlying operating system into Qt, and is currently supported on Android.

There is a final add-on that will get released under LGPL v3. This module will give native look and feel to the Qt Quick Controls on Android. This module can’t be released under LGPL v2.1, as it has to use code that is licensed under Apache 2.0, a license that is incompatible with LGPL v2.1, but compatible with LGPL v3.

How does this change affect you as a Qt user?

One of the first questions you might have is, of course, how this affects you as a user of Qt.

This first thing to notice is that if you are using Qt under a commercial license, nothing changes at all.

Also, if you are using Qt under GPL v3, you are unaffected, since LGPLv3 can always be converted to GPLv3.

All modules that existed in Qt 5.3 will still be available under LGPL v2.1. So if you are using Qt under the GPL v2 or LGPL v2.1, nothing changes as long as you don’t use any of the new modules that are only available under LGPL v3. If you start using those, your source code will fall under the conditions given by the LGPL v3 (or GPL v2).

These changes will be effective in Qt 5.4 Alpha. I believe that adding LGPL v3 as a licensing option will help both Qt and the open source ecosystem. It is a lot clearer about the intent of the LGPL license and its use in Free Software.

Please find more information about open source licenses at http://www.gnu.org/licenses/.

If you are not sure what license you should be using in your project, please consult a legal expert.

Digia has opened an email address for specific questions about using Lgplv3 in your project. Please contact us via Qtlicensing@digia.com.

Qt Creator 3.2.0 released


We are happy to announce the Qt Creator 3.2.0 release today. This release adds many smaller and larger features, as well as fixing bugs. A few examples:

  • Block selections in text editors now allow you to do “column editing”, meaning that all selected lines are edited simultaneously (Qt Creator Manual)
  • Context help can now be configured to open in an external window (without disabling Help mode)
  • Support for C99 designated initializers and concatenated strings was added to the C++ code model, as well as improvements to encoding handling and lambda support and many other things
  • More panes are now searchable with Ctrl+F, for example the project tree
  • The QML profiler received many performance and stability improvements again

Enterprise Qt Creator users can now also use the QML profiler to debug their JavaScript memory usage in QML. (This is only available when using Qt 5.4 for the debugged application.)

Also have a look at our change log, for a more complete overview. I also want to thank the more than 50 contributors to this version of Qt Creator.

You find the opensource version on the Qt Project download page, and Enterprise packages on the Qt Account Portal. Please post issues in our bug tracker. You also can find us on IRC on #qt-creator on irc.freenode.net, and on the Qt Creator mailing list.

Note: With Qt Creator 3.2 we drop support for OS X 10.6 (Snow Leopard). The technical reason for this is that Apple does not support any kind of C++11 on that OS version. Of course that does not affect on which platforms you can run your Qt applications on. But it is not possible to run the Qt Creator 3.2 binaries on 10.6, and it also is not possible to compile Qt Creator 3.2 on 10.6 with the tool chains provided by Apple.

The KDE Randa Meeting 2014 in retrospective

Leaving Randa after spending a week there at the KDE Randa Meeting 2014 raises mixed feelings. I am really looking forward to coming home and seeing my family, but at the same time the week was so full of action, great collaboration and awesome people that it passed by in an instant and was over so soon. Carving a work week out of the schedule for a hackfest is not an easy feat, especially during summer school break, so the expectations were high. And they have been exceeded in all aspects. A lot of the credit for that goes to the organizer, Mario Fux, and his team of local supporters. The rest goes to the awesome family of KDE contributors that make spending a week on contributing to Free Software so much fun. And of course to the sponsors of the event.

Randa is a place that grows on you. As a big city dweller, I take pride in organizing my time like clockwork, and in fitting a gazillion things into one day. I sometimes pause when crossing the bridge to Friedrichstrasse station to enjoy the view, but only for a couple of seconds. Because I got stuff to do. As soon as I boarded the Glacier Express train from Visp to Randa, the last leg of the long journey from Berlin by rail, it became obvious that I was in a different place. The train travels slowly, so slowly that sometimes wanderers keep up next to it. Later I learned that it is known as the slowest fast train of the world. It makes up for the relaxed pace with the absolutely magnificent view of the Matter valley. The view of the mountains was so breathtaking it almost made me miss the Randa stop. I arrived at the guest house, boarded a small spartanic room and then joined the group of KDE folks that already had arrived. At first, there was still this nagging feeling that whenever I was idle for 5 minutes, it meant a lack of efficiency, and something had to be done about it. And then everything changed.

The Randa panorama

One day provided enough immersion into the monastery like setting to make the feeling of time ticking and the conveyor belt constantly advancing go away. That is the moment when I was made aware again of the amazing group of people that had gathered around me to work on KDE. Not just the fact that fifty people travelled halfway around the world to create software that is free, but also what kind of wonderful kind of people they are. The attendees were a mirror image of the KDE community at large – young and old, women and men, from dozens of different countries, with all sorts of beliefs, and a unifying passion to contribute to a common good. At a time when not a day passes without news about atrocities in the name of mundane details like the whose prophet is more prophet-like, imagine such a diverse group not just spending a week together without a minute of conflict, but devoting that time to build something, and then to give it away freely without discriminating by use or user. That is the spirit of Free Software for me, and it may explain why it means more than getting stuff for free.

Two year old news
2 year old news

So we went to work. The air was fresh, and there was no distraction (not even the internet, because it broke :-) ), and we spent our days alternating between coding, talking, eating, taking walks and sleeping. A number of “special interest groups” formed rather organically, to work on educational software, the KDE SDK, porting to KDE Frameworks 5, Gluon, KDEnlive, Amarok and the KDE Frameworks Cookbook. Every day at lunch, the groups reported on their progress. As it turnes out, the velocity of the team was quite impressive, even though there were no managers. Or because, who knows. There are plenty of blog posts and details about how the work progressed on the sprint page.

Swiss slow food. Delicious.
2 year old news 2 year old news

Speaking of lunch and food in general – a small band of local supporters catered to our every whim like coffee 24 hours a day and a fridge full of FreeBeer. With an incredible supportiveness and delicious swiss cuisine they helped to make this meeting possible. While they received multiple rounds of applause, I do not think we can thank them enough. Just like the work that Mario Fux does to organize these meetings is priceless. Personally, I am hugely grateful for their commitment, which made this meeting and the previous ones possible. And I very much hope that it will for the next one, and I will do my best to be there again. See you in Randa next year!


Filed under: Coding, CreativeDestruction, English, FLOSS, KDE, OSS, Qt

Creating a PDF from a QtQuick 2 scene in SlideViewer

The Challenge

Previously on this blog, we featured a series of articles about our QML-based presentation tool, SlideViewer. To quickly recap: SlideViewer is a presentation program that allows writing slides entirely in QML.

There are situations in which the slide deck needs to be available in PDF format in addition to the QML source:

  1. For our KDAB trainings, the students get a printed handout. The print shops printing the material require a PDF.
  2. For conferences such as Qt DevDays, conference organizers usually require speakers to send in their slide deck as PDF to make them available as download for everyone
  3. Sometimes the presentation is done on a machine without SlideViewer installed. In this case a PDF version is needed as a PDF viewer is usually available everywhere.

Had we used QtQuick 1, the task of creating a PDF from our QML-based slides would have been easy: QtQuick 1 uses raster-based painting via QPainter. With QPainter, it is simple to redirect rendering to QPrinter, which is able to use a PDF file as its output format.

QtQuick 2 uses an OpenGL scenegraph for rendering, and hence does not use QPainter and therefore does not offer a direct way to render the current scene to PDF.

So how did we solve the challenge of getting SlideViewer, which uses QtQuick 2, to generate a PDF file?

First Approach: Taking Screenshots

Even though one can not access the rendering of QtQuick 2 via QPainter redirection, there is a way to get the final rendered image of a slide: Taking a screenshot of a slide with QQuickWindow::grabWindow(). With that, the algorithm for creating a PDF is trivial:

 - For each slide
     - Display the slide in the main window
     - Take a screenshot with QQuickWindow::grabWindow()
     - Draw the screenshot into a QPrinter with QPainter::drawImage()

This approach however has a literally huge drawback: The file size of the PDF will be gigantic, as one big image per slide is stored in the PDF, instead of a few bytes for the text strings. A PDF of our training material with all optional topics contains much more than 1000 slides, and a PDF with screenshots of that weights several hundred megabytes. This size is obviously too impractical to handle. Furthermore the text in the PDF is not searchable and selectable.

Using screenshots for creating PDFs did not work, but is there another way?

Second Approach: Manual Painting

A Typical Slide

A Typical Slide

Looking more closely at a typical slide, it mainly consists of text and images – that is, the QtQuick Text and Image elements. There are a few non-visual elements as well, for example a Repeater for code snippets – which in the end only creates more Text elements though.

With the insight that a slide contains only 2 or 3 different QtQuick types, it should be possible to paint each item on a slide ourselves, with QPainter. How hard could that possibly be? We can simply iterate over the complete object tree of a slide, call QPainter::drawImage() when we encounter an Image, QPainter::drawText() when we encounter a Text and so on.

A (very simplified) version of our code that does exactly this looks like:

void SlidePrinter::printSlide(QQuickItem *slide)
{
    QPrinter pdf;
    ... // Initialize QPrinter
    QPainter painter;
    painter.begin(&pdf);
    paintItem(slide, &painter);
    painter.end();
}

void SlidePrinter::paintItem(QQuickItem *item, QPainter *painter)
{
    if (!item || !item->isVisible())
        return;

    painter->save();
    painter->setOpacity(item->opacity() * painter->opacity());

    QTransform transform;
    auto priv = static_cast<QQuickItemPrivate*>(QObjectPrivate::get(item));
    priv->itemToParentTransform(transform);
    painter->setTransform(transform, true /* combine */);

    if (item->clip()) {
        painter->setClipping(true);
        painter->setClipRect(QRectF{0, 0, item->width(), item->height()});
    }

    if (item->metaObject()) {
        painter->save();
        const QString className = item->metaObject()->className();
        if (className == "QQuickImage")
        paintImage(item, painter):
    } else if (className == "QQuickRectangle") {
        paintRectangle(item, painter):
    } else if (className == "QQuickText") {
        paintText(item, painter):
        painter->restore();
    }

    for (auto child : item->childItems())
        paintItem(child, painter);

    painter->restore();
}

One can see that besides simply painting each element, one needs to take care of per-item properties like position and clipping. For the former we used private API, QQuickItemPrivate::itemToParentTransform(). Not shown in the simplified version is that child items with negative z values need to be painted before their parent item.

Painting the elements

Now, painting an image is just a QPainter::drawImage() call, right?

void SlidePrinter::paintImage(QQuickImage *image, QPainter *painter)
{
    Qt::AspectRatioMode aspectRatioMode;
    switch (image->fillMode()) {
        case QQuickImage::Pad:
        case QQuickImage::Tile:
        case QQuickImage::TileHorizontally:
        case QQuickImage::TileVertically:
        case QQuickImage::Stretch: aspectRatioMode = Qt::IgnoreAspectRatio; break;
        case QQuickImage::PreserveAspectFit: aspectRatioMode = Qt::KeepAspectRatio; break;
        case QQuickImage::PreserveAspectCrop: aspectRatioMode = Qt::KeepAspectRatioByExpanding; break;
    }

    const QImage original = image->image();
    QSizeF targetSize = original.size();
    targetSize.scale({image->width(), image->height()}, aspectRatioMode);
    QRectF targetRect{0, 0, targetSize.width(), targetSize.height()};
    QRectF sourceRect({0, 0}, original.size());

    if (image->fillMode() == QQuickImage::PreserveAspectCrop) {
        if (targetRect.height() > image->height()) {
            const qreal visibleHeightPercentage = image->height() / targetRect.height();
            const qreal visibleSourceHeight = sourceRect.height() * visibleHeightPercentage;
            const qreal sourceOffset = (sourceRect.height() - visibleSourceHeight) / 2;
            sourceRect.setY(sourceOffset);
            sourceRect.setHeight(visibleSourceHeight);
            targetRect.setHeight(image->height());
        } else if (targetRect.width() > image->width()) {
            ...
        }
    }

    if (image->fillMode() == QQuickImage::PreserveAspectFit) {
        if (targetRect.width() < image->width()) {
            const int space = image->width() - targetRect.width();
            targetRect.translate(space / 2, 0);
        } else if (targetRect.height() < image->height()) {
            ...
        }
    }

    QImage copy({(int)targetRect.width(), (int)targetRect.height()}, QImage::Format_ARGB32);
    copy.fill({0, 0, 0, 0});
    QPainter imagePainter(&copy);
    imagePainter.setOpacity(painter->opacity());
    imagePainter.setRenderHint(QPainter::SmoothPixmapTransform, true);
    imagePainter.drawImage({0, 0, targetRect.width(), targetRect.height()}, original, sourceRect);
    imagePainter.end();

    painter->drawImage(targetRect.x(), targetRect.y(), copy);
}

Turns out it is a bit more complicated than that, as the various different fill modes require a bit of math. That math already exists in QQuickImage::updatePaintNode(), but since we’re rolling our own rendering code, we need to duplicate everything that the QtQuick rendering code does. It turns out that the rendering code actually does a lot, and duplicating everything on our own would be quite a bit of effort. Because of that, we didn’t implement all features of QtQuick and left out, for example, the various tiling modes – we were not using them, and they are much harder to implement in QPainter than in OpenGL.

The Text and Rectangle elements are handled in a similar way.

In the above code snippet, I cheated a bit for simplicity’s sake: There is no QQuickImage::fillMode() method. This can however easily be replaced by calling QObject::property("fillMode"). For generating the PDF, using private API or even finding workarounds for missing private API is something we had to do a lot – QtQuick items aren’t meant to be accessed from the C++ side after all.

Screenshots

What if the slide contains QtQuick elements that are not text, images or rectangles? We do actually have some examples of that, one case is that we display a QtQuick Controls Slider in our QtQuick Controls introduction section. In cases like these, we make an exception and grab a screenshot of just that element, and use QPainter::drawImage() to add the screenshot to the PDF. We only have a handful of such cases, using a screenshot as a fallback in these cases is a reasonable compromise.

Conclusion

The above approach of manually rendering text, images and rectangles with QPainter to generate a PDF works surprisingly well, for our use-case, as it handles thousands of slides without problems.

There are several drawbacks that makes this approach a bit less than perfect for being a generic QPainter-based renderer for QtQuick 2 applications:

  • It uses private API of QtQuick
  • All of the scenegraph rendering code has to be duplicated into QPainter rendering code
  • It only works for a fixed subset of QtQuick types, other types will not be supported
  • It still relies on QQuickWindow, especially to do the polishing of items, for taking screenshots of unsupported types and of course for event handling

So this approach might not be the perfect approach for making QtQuick 2 run on systems without OpenGL, or would at least require some additional effort like duplicating most of the QQuickWindow code.

However, for our use-case of creating PDFs from slides, the above solution works just fine and creates nice small and searchable PDFs.

The post Creating a PDF from a QtQuick 2 scene in SlideViewer appeared first on KDAB.

Qt WebChannel – bridging the gap between C++/QML and the web

Hybrid applications, which mix a UI built with Qt Widgets or Qt Quick with embedded HTML websites, are very popular. In particular, something like an HTML 5 app framework is often requested by customers. Qt WebKit makes it trivial to embed HTML content in any graphical application. But how does one bridge the gap between C++/QML on one side, and HTML/JavaScript on the other side?

A bit of history

In the single-process world of the C++ Qt WebKit API, it was easily possible to publish a QObject instance to the JavaScript client side, thanks the Qt WebKit Bridge. This is a crucial functionality for any application that uses Qt WebKit as the foundation for an HTML app framework or similar.

Up until now though, WebView, the Qt Quick 2 integration of Qt WebKit, was missing this functionality. The reason for that was the multi-process architecture at the core of WebKit2, which is used internally by WebView; in such an environment, the synchronous API of the WebKit bridge cannot be supported, since inter-process communication is inherently asynchronous. Due to that, hybrid QML/HTML applications were notoriously hard to implement.

In October 2011, Noam Rosenthal presented an alternative approach to the problem, which he called Qt WebChannel. His idea was simple and powerful at the same time: By leveraging Qt’s introspection system, he could create mock objects on the JavaScript client side which mirror the API of the server-side QML/QObject objects. For communication between the QML host and the HTML/JavaScript client, he chose WebSockets. But, contrary to the WebKit Bridge, the API provided by the WebChannel is completely asynchronous.

Despite the huge interest in hybrid QML/HTML applications, Nokia and nowadays, Digia, did not prioritize work on this missing functionality. Priorities lay elsewhere and the Qt WebChannel project never left the proof-of-concept stage on Qt Labs.

At the beginning of 2013, a customer approached KDAB and requested the development of a QML application with an HTML 5 application framework. Naturally, we chose Qt WebChannel as a foundation and started polishing it. In this first real-world use-case, numerous bugs and missing functionality were found, fixed and upstreamed to the Qt Labs repository. In the process, I essentially took up maintainership of the Qt WebChannel project. Eventually, the codebase worked reliably and efficiently even on embedded devices. Still, it remained a Qt Labs project and as such awkward to use by others.

Making it Official

Over the last couple of months, I had the pleasure to work on Qt WebChannel exclusively, sponsored by KDAB. The goal was to create a proper Qt module which can be used by hybrid QML/HTML applications. At this year’s Qt Contributor Summit the design was reviewed and approved for inclusion in Qt 5.4. A few days ago now, Qt WebChannel was added as a new module for Qt 5.4. What is left now, is the inclusion of two already approved patches, which provide an easy-to-use integration of the Qt WebChannel functionality into Qt WebKit with a minimum of boiler-plate code.

A QML/HTML hybrid application in Qt 5.4

So, what is necessary to build a hybrid QML/HTML application in the upcoming Qt 5.4? The following shows how to use the new Qt WebChannel module.

QML Server Side

On the QML server side, first import the Qt WebChannel module, as well as the Qt WebKit module and its experimental one:

import QtWebChannel 1.0

import QtWebKit 3.0
import QtWebKit.experimental 1.0

Now, let’s create an object that we want to publish to the HTML/JavaScript clients:

    QtObject {
        id: myObject

        // the identifier under which this object
        // will be known on the JavaScript side
        WebChannel.id: "foo"

        // signals, methods and properties are
        // accessible to JavaScript code
        signal someSignal(string message);

        function someMethod(message) {
            console.log(message);
            someSignal(message);
            return "foobar";
        }

        property string hello: "world"
    }

Publishing the object to the HTML clients in your WebView is as simple as

    WebView {
        experimental.webChannel.registeredObjects: [myObject]
    }

This single line is all you need to publish potentially multiple objects. Internally, this will use the WebKit IPC mechanism to transmit method calls, signals and property update notifications to the HTML clients. You can also create a WebChannel object externally and set it on multiple WebViews if necessary.

HTML/JavaScript client side

On the client side, a bit of boiler-plate code is still required. I tried to minimize it as much as possible, and plan to improve this situation even further in the future. First, include the client-side qwebchannel.js library via its Qt resource URL:

    <script type="text/javascript" src="qrc:///qtwebchannel/qwebchannel.js"></script>

Then, in your JavaScript code, instantiate a QWebChannel object and pass it a callback function. The callback function gets invoked when the initialization of the web channel succeeded. Furthermore, you can pass the navigator.qtWebChannelTransport object to your channel – more on this below.

    new QWebChannel(navigator.qtWebChannelTransport, function(channel) {
        // all published objects are available in channel.objects under
        // the identifier set in their attached WebChannel.id property
        var foo = channel.objects.foo;

        // access a property
        alert(foo.hello);

        // connect to a signal
        foo.someSignal.connect(function(message) {
            alert("Got signal: " + message);
        });

        // invoke a method, and receive the return value asynchronously
        foo.someMethod("bar", function(ret) {
            alert("Got return value: " + ret);
        });
    });

That’s it. I hope you like this new functionality. While it was written with the WebView integration in mind, this new design proved to be much more versatile when compared to the old WebKit Bridge, as it allows you to publish QObject instances to any JavaScript client you can talk to, e.g. via WebSockets. This works in any modern desktop or mobile browser, and no Qt is required on the client side at all. Our friends from Basyskom e.g. wrote a node.js example. Also note that QML is not required at all, you can also create a QWebChannel and publish QObject instances in pure Qt/C++ applications. I will write more on these topics in a future blog post.

The post Qt WebChannel – bridging the gap between C++/QML and the web appeared first on KDAB.

Qt Weekly #17: Linking Qt Classes in Documentation Generated with Doxygen


Qt Weekly is back from vacation with a post from a guest blogger (*applause*).  In this post, Lorenz Haas tells us how to link Qt classes in custom documentation that is generated by using Doxygen.

By mid-2008, Sebastian Pipping introduced doxygen2qthelp for generating Qt Compressed Help files (*.qch) via Doxygen. Already by the end of the year, it was successfully merged into Doxygen. While 1.5.7.1 still had some mirror problems, version 1.5.8 provided stable and comprehensive support for creating QCH files out of the box. See Sebastian’s announcement as well as David Boddie’s article in Qt Quarterly.

Since then, this feature – I guess – has been used a thousand, a million times. Nowadays, I personally can’t imagine working without it. This is because it integrates perfectly into my favorite IDE – Qt Creator. There I can use my own documentation for context-sensitive help that can be triggered by the F1-shortcut:

without_links

 

However, one tiny, but very annoying detail spoils the party. Qt classes don’t get linked and so you can’t conveniently click on a piece of text, such as QString, to open the documentation of QString. In Qt’s own help files that come with Qt Creator, however, you can click every Qt class and you are thus used to doing so. So let’s see how we can rock the party with custom help files.

Before we do so, however, let’s briefly summarize how we generate the custom help file that is shown in the picture.

Generating Help Files

First, we have a small example file – main.cpp – posing as the full documented project:

class SomeClass {

public:

  /**
   * \brief A simple description
   *
   * Here is the documentation body containing references
   * to Qt functions like QPixmap::copy().
   */

  QString getText(const QDomElement &e);
};

Second, we create a basic configuration file for Doxygen by calling:

doxygen -g doxyfile.cfg

We edit the following options:

INPUT                  = main.cpp
GENERATE_QHP           = YES
QCH_FILE               = ../MyDoc.qch
QHP_NAMESPACE          = your.domain.project
QHG_LOCATION           = /path/to/qhelpgenerator

You’ll find detailed documentation for these options – as well as for all the other options – in the generated configuration file.

Third, we actually need to call Doxygen with this configuration file:

doxygen doxyfile.cfg

Finally, we have to open Qt Creator, select Tools > Options > Help > Documentation and add the generated  Qt Compressed Help file there. That’s all.

Creating Links

So, how do we get the links? As a matter of fact, we need to tell Doxygen where it can find information about the Qt classes. This information is stored in so called tag files that Doxygen generates if you specify GENERATE_TAGFILE.

Although Qt Project uses QDoc to generate the documentation, it also generates tag files we can use with Doxygen. They are located in the directory Docs/Qt-5.3 where you have installed Qt. The tag file for each module is then located in the respective subfolder. E.g. the tag file for QtCore is located at Docs/Qt-5.3/qtcore/qtcore.tags. For convenience, we copy all the needed tag files for the modules we reference beside the Doxygen configuration file.

Now, we use TAGFILES to make the tag files usable for Doxygen. The syntax of  TAGFILES entries is as follows:

<path to the tag file>=<path>

The path should be prepended to the (relative) link specified by the tag file.

If you like to use the documentation inside Qt Creator and use the local help files of Qt, the prepend path for QtCore looks like this:

qthelp://org.qt-project.qtcore/qtcore/

The URL schema qthelp:// will cause Qt Creator’s help engine to use the local available documentation, org.qt-project.qtcore corresponds to QHP_NAMESPACE, and qtcore to QHP_VIRTUAL_FOLDER. So, TAGFILES for our example looks like this:

TAGFILES = qtcore.tags=qthelp://org.qt-project.qtcore/qtcore/ \
           qtgui.tags=qthelp://org.qt-project.qtgui/qtgui/ \
           qtxml.tags=qthelp://org.qt-project.qtxml/qtxml/

If we now recreate the documentation and open it, we’ll see that QStringQDomDocument, and
QPixmap::copy() link to the Qt documentation. And even Qt links to the Qt
namespace documentation!

with_links

Last, two little hints:

  • If you’d like to link to a specific version of Qt documentation, you can define that after the namespace. So qthelp://org.qt-project.qtcore.531/qtcore/ would generate links that point to the QtCore documentation for Qt 5.3.1.
  • If you create online documentation, use e.g. qtcore.tags=http://qt-project.org/doc/qt-5/ to create links pointing to the official online documentation of Qt. As for the URL, you do not have to specify the module.

So go on, pimp your docs and happy documenting :)

By Lorenz Haas 

 

sailing in search of fresh waters

I've had a long, quiet time on this blog over the past few years while I've been frantically helping Jolla to launch their self-named product: the Jolla. I've enjoyed (almost) every day I've been there: they really are a great bunch of people and the work has been plentiful and challenging.

But as the saying goes, "this too shall pass". Nothing lasts forever, and it's time for a change: after this week, I will be taking a break from Jolla to get some fresh perspective.

On the bright side, maybe I'll have some more time for writing now :)

If anyone is interested in getting a hold of a C++/Qt/QML/Linux expert with a focus on performance, expertise on mobile, and a wide range of knowledge across other areas who loves open source, please let me know.

KDE Frameworks Book Sprint at the Randa Meeting 2014

A couple of weeks before the KDE Randa Meeting of 2014, the meeting’s organizer Mario Fux suggested to have a book sprint to help developers adopt the newly released KDE 5 Frameworks. In the Open Source spirit, the idea was not to start from scratch, but rather to collect the various bits and pieces of documentation that exist in different places in one location that offers itself more to a developer than before. Valorie Zimmermann stepped up to organize it (many thanks!), and a number of people volunteered to take part. After a week the project completely changed its orientation and struggled and found and also newly defined its place as a part of KDE’s documentation efforts. And it produced an initial version of the book, which is currently circulated on people’s ebook readers around here.

The first question to clarify and find concensus about was that of the audience of the book, and that also made an inherent conflict apparent. The target audience was defined as readers with programming experience in C++ and some Qt. Also, it is supposed to be useful to experienced KDE programmers as an easy way to jump into frameworks and programming mechanisms that they are not familiar with. And the inherent conclict was that the intended contributors to the book are those experienced KDE programmers, ideally the authors of the frameworks, and those do not have an itch to scratch by spending their time writing for newcomers. A rather convincing compromise for everybody is that the book will have fulfill a specific role, and that is to complete the spectrum of development documentation going from very basic questions (of which many today are better answered in the Qt Project forum) to cookbook style feature stories (this is the part that was missing) all the way to the already quite good API documentation.

Hack sprint fuel, the Swiss way
Hack sprint fuel, the Swiss way

The second question was that of tools. The flaky network was just the final blow to the expectation that the en vogue authoring tools (Booktype, Flossmanuals & co) would do the trick. They simply do not integrate well with the workflows of developers and a Free Software community in general, which is almost exclusively based on revision control and infrastructure that ties the pieces together. Information like text articles and code snippets are simply never supposed to be copy-pasted form one place to the next, as this would only create hard-to-spot errors and duplicate work. So while it sounded like a compromise to fall back to using a git repository of Markdown files in combination with pulling in the actual frameworks as submodules, it turned out to be the saving grace. There are still open questions, but the general approach is serving us well so far.

The last big question was that of how to integrate the book into the existing infrastructure. We decided to table a lot of these details, to be able to focus on producing content. In the end, the content is in simple to convert and integrate in all kinds of formats. The vision however is to have the production of the book in HTML, PDF and ePub formats integrated into CI, and the current version of the book available all the time. This also includes to decide in what way to revision and “release” the book. An option is to polish and copy-edit a new edition in the same cadence as the KDE software releases. This way, the common spirit of “let’s get it done and out there” could be beneficial to also keep the cookbook up to date. Let’s see.

If you are interested in jumoing in to help or to see what the current state of the book looks like, Valorie has all the details in her post.


Filed under: Coding, CreativeDestruction, English, FLOSS, KDE, OSS, Qt Tagged: Creative Destruction, defensive publications, FLOSS, free software communities, KDE, kde community, OIN

Defragmenting Qt and Uniting Our Ecosystem


Over the last years, many changes have been happening in the Qt ecosystem. One of the biggest was the creation of Qt Project where Qt is now being developed as an open source project. The Qt Project was created to provide a space open for all to further develop and foster innovation for the Qt technology.

Qt has always been a commercial product. During the Trolltech days licensing income funded development of the product. While Nokia’s motivations were different, at Digia, our goal is to again make sure that Qt thrives for all industries, all companies, all users no matter what platform. That said, we need to make sure the business of selling Qt as a product is successful in order to fund its future development for the good of everyone in our ecosystem. The importance of Digia’s commercial business for securing the future of Qt cannot be underestimated as it drives Qt’s foundation and everyday operations. A look into the commit statistics shows that around 75% of all code submissions to qt-project.org come from Digia employees. In addition, Digia manages the release process and the CI and testing infrastructure, thus covering more than 85% of the costs of developing Qt.

But even though the open source project and the commercial side of Qt are highly dependent upon each other, they have over the last years drifted apart. The installers and product packages for the open source and enterprise versions are different, and there is a complete disconnect between qt-project.org and the commercial pages on qt.digia.com.

In the long term, this split is helping nobody. The fragmentation actually weakens our ecosystem and makes it difficult to position Qt against competing technologies and tools. It also makes it very hard to speak with one voice and consistently highlight the benefits of the technology and product.

Because of the separation between the open source and commercial offerings, we often end up competing against ourselves instead of competing against other technologies. The two web sites do not allow for consistent messaging and it’s impossible to explain the full story around the product in one place and provide accurate information for our various target groups. In addition, the separate packages make it difficult and cumbersome to upgrade from the open source to an enterprise version.

We are now starting a conscious effort to overcome these problems. As you might have read, Digia has decided to move the Qt business into a company of it’s own. Thus we will soon have a company (owned by Digia), that will focus 100% on Qt. At the same time we would like to take the opportunity and retire qt.digia.com and merge it with the content from qt-project.org into a new unified web presence. The unified web page will give a broad overview of the Qt technology, both enterprise and open-source, from a technical, business and messaging perspective.

We are also planning on unifying the Qt packages, to have only one set of installers for both open source and enterprise users. This will not only allow for an easier migration path from the open source to the enterprise version, but it will also significantly simplify our releasing process. With these simplifications we expect to be able to deliver better tested and higher-quality packages to the whole ecosystem.

The plan has already been discussed with and accepted by the main contributors at the Qt Contributor Summit in June. We are now moving forward with it and you will see the first pieces coming together already before the release of Qt 5.4 in October.

I have been with Qt for many years and seen it evolve and grow to a software development framework I am proud of. I believe that this change and next step in Qt’s life will help us strengthen the Qt ecosystem as a whole and lead to a stronger, better and more competitive product.

Our goal is to unify our Qt ecosystem and do things together. So please don’t hesitate to send us any feedback, questions or concerns you might have.

Qt Creator 3.2 RC1 released


We are happy to announce the Qt Creator 3.2 RC1 release today. Since the beta we have been busy fixing bugs. We encourage you to try this RC and give us feedback on its quality.

The beta blog post already mentioned a few new features such as “column editing”, but there are a lot more new features. To mention just a few:

  • The file system locator filter ‘f’ can now be used to create new files.
  • More panes are searchable, e.g. the project tree.
  • The qbs plugin now supports adding and removing files from projects.
  • The C++ code model received a lot of fixes e.g. for editing lambdas.


While the majority of commits for Qt Creator come from Digia employees, I would like to highlight the work of Przemysław Gorszkowski, who fixed various bugs in the C++ code completion. For example, he fixed the find usages for members of a typedefed anonymous union. If you want to know what that is, take a look at the test case in the commit. Also, Orgad Shaneh contributed over 200 commits to this release.

This year is the 5th anniversary for Qt Creator. We released the 1.0 release in March of 2009. In those 5 years, over 35,000 commits by around 300 people were added. We added tons of support for Qt Quick – debugging, designing, profiling -, extended our platform support to Android, IOS, QNX and BareMetal. And we added and later removed support for Maemo, Meego and Symbian. We rewrote parts of the C++ code model multiple times, added refactoring support on top and switched major Qt versions. We built the Qt SDK, later Nokia SDK and now again Qt SDK around Qt Creator. Yet, we are still using the same versioning scheme. :) It’s been exciting 5 years. We will continue providing the best cross platform IDE for Qt development.

The Open Source version is available from the Qt Project download page, while Enterprise customers find their package in the Qt Account Portal. Please report issues in our bug tracker. You also can find us on IRC on #qt-creator on irc.freenode.net, and on the Qt Creator mailing list.

Solving the Unavoidable Race

This is the story how I have (not) solved a race condition that impacts QWaitCondition and is also present on every other condition variable implementations (pthread, boost, std::condition_variable).

bool QWaitCondition::wait(int timeout) is supposed to return true if the condition variable was met and false if it timed out. The race is that it may return false (for timeout) even if it was actually woken up.

The problem was already reported in 2012. But I only came to look at it when David Faure was trying to fix another bug in QThreadPool that was caused by this race.

The problem in QThreadPool

When starting a task, QThreadPool did something along the lines of:

QMutexLocker locker(&mutex);

taskQueue.append(task); // Place the task on the task queue
if (waitingThreads > 0) {
   // there are already running idle thread. They are waiting on the 'runnableReady' 
   // QWaitCondition. Wake one up them up.
   waitingThreads--;
   runnableReady.wakeOne();
} else if (runningThreadCount < maxThreadCount) {
   startNewThread(task);
}

And the the thread's main loop looks like this:

void QThreadPoolThread::run()
{
  QMutexLocker locker(&manager->mutex);
  while (true) {
    /* ... */
    if (manager->taskQueue.isEmpty()) {
      // no pending task, wait for one.
      bool expired = !manager->runnableReady.wait(locker.mutex(), 
                                                  manager->expiryTimeout);
      if (expired) {
        manager->runningThreadCount--;
        return;
      } else {
        continue;
      }
    }
    QRunnable *r = manager->taskQueue.takeFirst();
    // run the task
    locker.unlock();
    r->run();
    locker.relock();
  }
}

The idea is that the thread will wait for a given amount of second for a task, but if no task was added in a given amount of time, the thread expires and is terminated. The problem here is that we rely on the return value of runnableReady. If there is a task that is scheduled at exactly the same time as the thread expires, then the thread will see false and will expire. But the main thread will not restart any other thread. That might let the application hang as the task will never be run.

The Race

Many of the implementations of a condition variable have the same issue.
It is even documented in the POSIX documentation:

[W]hen pthread_cond_timedwait() returns with the timeout error, the associated predicate may be true due to an unavoidable race between the expiration of the timeout and the predicate state change.

pthread documentation describes it as an unavoidable race. But is it so? The wait condition is associated with a mutex, which is locked by the user when calling wake() and that is also passed locked to wait(). The implementation is supposed to unlock and wait atomically.

The C++11 standard library's condition_variable even has an enum (cv_status) for the return code. The C++ standard does not document the race, but all the implementations I have tried suffer from the race. (No implementations are therefore conform.)

Let me try to explain the race better: this code show a typical use of QWaitCondition

Thread 1 Thread 2
  mutex.lock();
  if(!ready) {
      ready = true;
      condition.wakeOne();
  }
  mutex.unlock();
  mutex.lock();
  ready = false;
  bool success = condition.wait(&mutex, timeout);
  assert(success == ready);
  mutex.unlock();

The race is that the wait condition in Thread2 timeout and returns false, but at the same time, Thread1 wakes the condition. One could expect that since everything is protected by a mutex, this should not happen. Internally, the wait condition unlocks the internal mutex, but does not check that it has not been woken up once the user mutex is locked again.

QWaitCondition has internal state that counts the number of waiting QWaitCondition and the number of QWaitCondition that are waiting to be woken up.
Let's review the actual code of QWaitCondition (edited for readability)

bool QWaitCondition::wait(QMutex *mutex, unsigned long time)
{
    // [...]
    pthread_mutex_lock(&d->mutex);
    ++d->waiters;
    mutex->unlock();
 
    // (simplified for briefty)
    int code = 0;
    do {
      code = d->wait_relative(time); // calls pthread_cond_timedwait
    } while (code == 0 && d->wakeups == 0);
    --d->waiters;
    if (code == 0)
      --d->wakeups; // [!!]
    pthread_mutex_unlock(&d->mutex);
    mutex->lock();
    return code == 0;
}

void QWaitCondition::wakeOne()
{
    pthread_mutex_lock(&d->mutex);
    d->wakeups = qMin(d->wakeups + 1, d->waiters);
    pthread_cond_signal(&d->cond);
    pthread_mutex_unlock(&d->mutex);
}

Notice that d->mutex is a native pthread mutex, while the local variable mutex is the user mutex. In the line marked with [!!] we effectively take the right to wake up. But we do that before locking the user's mutex. What if we checked again for waiters under the user's lock?

Attempt 1: check again under the user's lock

bool QWaitCondition::wait(QMutex *mutex, unsigned long time)
{
// Same as before:
    pthread_mutex_lock(&d->mutex);
    ++d->waiters;
    mutex->unlock();
    int code = 0;
    do {
      code = d->wait_relative(time); // calls pthread_cond_timedwait
    } while (code == 0 && d->wakeups == 0);
//    --d->waiters; // Moved bellow
    if (code == 0)
      --d->wakeups;
    pthread_mutex_unlock(&d->mutex);
    mutex->lock();

//  Now check the wakeups again:
    pthread_mutex_lock(&d->mutex);
    --d->waiters;
    if (code != 0 && d->wakeups) {
      // The race is detected, and corrected
      --d->wakeups;
      code = 0;
    }
    pthread_mutex_unlock(&d->mutex);

    return code == 0;
}

And there we have fixed the race! We just had to lock the internal mutex again because d->waiters and d->wakeups need to be protected by it. We needed to unlock it because locking the user's mutex with the internal mutex locked would potentially cause deadlock as lock order would not be respected.

However, we now have introduced another problem: If there are three threads, a thread may be woken up before

//    Thread 1              // Thread 2             // Thread 3
mutex->lock()
cond->wait(mutex);
                            mutex->lock()
                            cond->wake();
                            mutex->unlock()
                                                    mutex->lock()
                                                    cond->wait(mutex, 0);

We don't want that the Thread 3 steal the signal from the Thread 1. But that can happen if the Thread 1 is sleeping a bit too long and do not manage to lock the internal mutex in time before Thread 3 expires.

The only way to solve this problem would be if we could order the thread by the time they started to wait.
Inspired by the bitcoin's blockchain, I created a linked list of nodes on the thread's stack that represent the order. When a thread is starting to wait, it adds itself at the end of the double linked list. When a thread is waking other thread, it marks the last node of the linked list. (by incrementing a woken counter inside the node). When a thread is timing out, it checks if it was marked, or any other thread after him in the linked list. We only solve the race in that case, otherwise we consider it is a timeout.

You can see the patch on the code review tool.

Performance

This patch adds quite a bit of code to add and remove nodes in the linked list, and also to go over the list to check if we were indeed woken up. The linked list is bound by the number of waiting thread. I was expecting that this linked list handling would be negligible compared to the other cost of QWaitCondition

However, the results of the QWaitCondition benchmark show that, with 10 threads and high contention, we have a ~10% penalty. With 5 threads there is ~5% penalty.

Is it worth it to pay this penalty to solve the race? So far, we decided not to merge the patch and keep the race.

Conclusion

Fixing the race is possible, bug has a small performance impact. None of the implementations attempt to fix the race. I wonder why there is even a returned status at all if you cannot rely on it.

Hawaii responsive user interface

This was written after FOSDEM but never had the time to finish it and do the video so with too many months of delay I've got some time in my hands to post about the Hawaii desktop environment responsive user interface.

Hawaii is currently focused on desktops and laptops but it's always been a goal to get ready for other devices as well supporting screens with different DPIs.

Why bother, can't we just do the same user interface and let the controls scale according to DPIs and expand contents to fit the screen size? No, we can't because there are differences between devices.

Desktops and laptops have a keyboard and most of the time a mouse and to complicate things even further they can also have a touchpad, for example I own an Apple Magic Trackpad.  Mobile devices, on the other hand, have touch input, no keyboard and higher DPIs.

The same user interface cannot be used for different devices, the user expects it to suit the device.

On a tablet the user expects to see an application at a time and use several gestures, there is also less screen space so you have to design interfaces with fewer things on screen.
The opposite is true on desktop where you can show multiple windows and some tiling feature is more than welcome and since the mouse is more precise you can put more controls on the screen.

Today there are some devices like the Yoga that can alternatively be turned into either a tablet or a laptop. The only problem is that it's not cheap, I will have to save money before buying one :)

That's why I prefer calling this responsive user interface rather than convergence.

Hawaii should react on events such as screen resolution change, mouse plugged in or touch input device plugged in and load the user interface that is willing to step in and show a user interface for the current setup.

To make development easier the switch happens reading settings with the QtConfiguration library that supports changes notifications. That's enough to simulate an event. In the future I plan to use Solid to determine what's the current setup like, but maybe QScreen and QTouchDevice will be sufficient.

Most things that a desktop environment need can be shared between shells, for example lock screen management, idle state management, notifications, power management and sounds are all features that need to be done once.

With Green Island, the QML compositor, one can also plug in different ways to manage windows and panels all with their Wayland protocol interface.

This is not all. Applications need to switch the user interface too, so I've been experimenting with QFileSelector and QQmlUrlInterceptor.

In the application demo I have used QQmlUrlInterceptor.
The application has a content directory with the QML code for the desktop with the following subdirectories that override files for specific profiles: +phone, +tablet, +touch

The +touch subdirectory contains code for all touch user interface, one example is the list view delegate, +phone and +tablet override the main view (the tablet version has a menu that can be pulled from the left with a gesture while the phone version has it always visible).

For demo purpose, the user has to pass all the relevant profiles to the application so for example if I want the tablet version both touch and tablet are passed as arguments.

Here's a little video showing the switch from a tablet UI to a desktop UI and different layouts for the same application.


As you can see the tablet user interface has two panels (launcher at the bottom edge and a smaller panel with indicators on top), it also have a desktop view with a search bar and application icons. Stacked below there's the background view.  The desktop user interface is the classic Hawaii shell that has a bottom panel with more elements.

Of vectors and scalable things.

rect4220Moving

 away for my original plan, today we will be talking about Vectors.

To start this series of posts I had a main motivator, SVG.  It is a great file format, its the file type I use day in day out and the format I use the most to create all of my images…
But every so often the question about scalable UI’s and Vectors pops up. And someone will say something like “we should just use vectors to scale things”. To that, I will usually say something like, “Scalable Vectors Graphics are scalable but your screen is not“, hoping it will close the conversation just there, and it usually does.

However the above statement is only partly correct and is not the definitive reason why we should avoid off-the-shelf vectors as the source image format for our UI assets.

 

Scalable definition is a bit like being “impassioned”.

The way we define “Scalable” UI’s, as we have seen in the past posts, is very peculiar and we tend to use it the way it suits us best, ignoring the practical differences between the different meanings of the concept. Ergo, like being  impassioned, the target of our focus is more what we want it to be rather than what it really is.
This tends to produce the confusions and misunderstandings that are so common in this area, precisely because the Scalable part in SVG, is for a certain type of the referred concept, and most of the time not the type of scalable we need in UI.

So what does scalable mean for a Scalable Vector Graphic?

An SVG or any other main vector format is formed (mostly) of mathematical information about paths and its control points (Bézier curves), its visual size is only relevant in regards to the render size of the canvas it’s on, and as a result you can zoom an image almost infinitely and will never see pixels. (the pixels are just rendered for a given view area and change accordingly to the section of the vectors in that area).
This makes its a great format to scale images to really huge formats. A rendered 40000×40000 px image that is scaled down to 1000×1000 will look exactly like the image originally rendered in 1000×1000.
Now as we have seen so far, this is often not the type of scalable we want.

  • Firstly, it is X/Y dependent where we want many times X/Y independently scalable elements (think of scaling a rounded rectangle), and for those we will need something like Borderimage components.
  • Secondly, as you zoom out many elements become sub pixel and difficult to see though still there. For example, in maps we may want vectors that render differently depending on the zoom level, like making roads disappear/appear as you zoom out/in, as well as simplifying aspects of the path itself.
  • Thirdly, it ignores pixels, as we mentioned pixels are still very important, in lower definition screens we can make use of the of the pixel to create sharp contrasts, this makes your visual element “POP Out” against the background. However, SVG, in its mostly perfect mathematical description of path positions, completely ignores the rendering grid and as a result can produce unsharp elements that are off the pixel grid by decimals of a pixel. (this is only problematic in rectangular elements that align with the pixel grid disposition, but we do use it as an advantage for designs).

SVG’s in QML.

You can use SVG in QML right now as a source format, but attention, it won’t be re-rendered unless you tell it to do that, the result will be that if you do a scale or even change width height you will end up seeing pixels.  You can create bigger renders that will provide higher definitions that are more zoomable, but at the cost of it taking more time to render it and taking a lot of memory for the cached image.
Also SVGs can be very complex, I have created SVGs that take several hours to render Many of my past wallpapers for KDE were done in outline mode and I would only occasionally look at them with filters and colors on and I do have a powerful desktop to render those; trying similar things on a mobile is not a great idea.
SVG support in QT is limited, many things won’t work, the filters are mostly not working so the look will dramatically change if you expect blur-based drop shadows to work, you will not see those, the same goes for multiply filters, opacity masks, etc, etc…
So, in a nutshell, don’t use SVG as a base source image unless you know its limitations and how it works, it’s a wonderful format if you understand it’s limitations and strengths, and is sometimes useful in QML.

rect4134Vector sunset wallpaper crop several hours to render on my old linux pc.

What about other vector formats? like Fonts?

There is a special Vector format’s that we use all the time and that is also scalable, and its a format that has dealt with this problems many years ago, Fonts…

Fonts are special types of monochromatic vector paths that can have special hints to cater to pixel scalable issues. It does that via 2 methods

  • Font hinting (also known as instructing) is the use of mathematical instructions to adjust the font’s visual appearance so that it lines up with a virtual grid.
  • Font Kerning is moving the glyph on the relative x direction to achieve a more pleasing visual result and sometimes to help with the grid alignment issues…

rect4164

All of this magic is done by your local font render engine and the extent to how much these operations are done depends on your local font rendering definitions…

Now, since the introduction of QML 2, the way fonts are rendered has changed, and, by default font hinting is ignored.  As a result, if you zoom into a font by animating the pixel size you get a nice smooth zoom effect, but slightly more blurry all around fonts, since they are not hinted.
QML does allow you to have native looking fonts by doing

Text {
    text: "Native here!"
    renderType: Text.NativeRendering
}

How ever if you try to do a zoom effect here, via animating the pixel size, you will have a “jumpy” text feeling because, as the size increases, the hinting instructions of the font will keep on trying to adjust to an ever-changing pixel grid.
Also, the default method does a vector like scaling via the distance field method when changing the  scale: property,  where when using native rendering, you see the pixels of the 1:1 scale ratio being scaled.
Another side effect of the distance field method used by default is that if the scale/font.size is very large you start to see the inaccuracies of the method, and this is valid for any font size you pick, the method generates the distance field glyph from a small image based on the font and it’s not updated as you scale it.

rect4307
Also if the font is not well formatted (glyph bounding box smaller than the glyph itself) it might clip the font in some border areas with weird visual results.

So, my advice here is: if you need readable text that you don’t want to do pinch to zoom or any other effect that changes the font size in an animated way, use the native render. If you want something dynamic then go for the default mode. Or, you can even try for a compromise solution where, when animating, you choose default and in the end turn native on. It’s mostly a matter of what you consider most important and more polished at the end of the day.
Also noteworthy is that on the higher DPI screens the hinting instructions lose a lot of their importance since the pixel size and respective sub-pixel antialiasing ‘grays’ become much smaller and relatively less important in relation to the font body. The same is true for non square pixels like many (but not all) AMOLED screens have.

Next!

Next post will return to the subject of making scalable X/Y independent elements that work well with DPI metrics…
By the way, we will be discussing this subjects at the training days of Qt Developer Days 2014 Berlin, if you are interested in this subjects Registration is here.

So see you soon, here on my next post or at DevDays.

The post Of vectors and scalable things. appeared first on KDAB.

meta-kf5 usable

Finally I’ve had the time to work over the final issues in meta-kf5. Right now, I build most tier 1 and tier 2 components. I’ve packaged most functional modules and integration modules from these tiers.

When it comes to integration modules, there might be missing dependencies that need to be added – but that should not be too hard to add.

To be able to create useable cmake files, I had to employ a small hack modifying the cmake-files from KF5 before installing and packaging them. This seems to work (i.e. tier 2 builds), but there might be other sed-expressions that are needed.

Also, the autotests are not built as long at Qt5Test is left out form the build. If you would add Qt5Test, I believe that the unit tests will be included in the same package as the libs. I’ll address this as I integrate the autotests into ptest.

Summing up all of this, I’d say that the meta-kf5 layer now is usable!

That is all for now. As always, contributions are welcome! If you find a use for this, I’d be happy to add your project as a reference to the layer!

What's New in Qt 5.3: QPrinterInfo

The Qt 5.3 release made a lot of "under the hood" improvements to the internals of the Qt printing system. There were also some changes visible at the API level. One of the more noticeable changes was to add enhancements to the QPrinterInfo class. In this blog post, I'll go over this class and present an example application illustrating how to use it.

User Attention and Duration of Engagement in Public Kiosks

Common user experience (UX) wisdom says that you have a short period of time (approximately 3 seconds to 3 minutes) to attract, seduce and convince a user to use your app, site or device. Once you capture their attention, you need to sustain a long-term relationship by offering the user real value (Skype), enduring enjoyment (Candy Crush) or at the very least, an obligation to be a participant in popular experiences (Facebook). Do the same principles apply to public kiosks, such as ticketing, retail and museum kiosks? The answer is yes and no.

Google Labs VoltAir Game Built with Qt


As the dog days of summer carry on, we at Digia, Qt are swatting down flies, mosquitoes and bees while we fan ourselves in the unusual summer heat currently striking Scandinavia … in Oslo … at least.

Meanwhile, on a cool note, the Fun Propulsion Labs at Google announced last week that, VoltAir a single and multi-player game built with Qt is available for download via the Google Play Store and as open source software. Coolbeans!

VoltAir was developed to provide an example of a C++ game designed for both Android and Android TV and the folks at Google also tested it on Nexus 5, Nexus 7, Moto X by Motorola, Android TV, and some Samsung devices.

Check Out VoltAir (Courtesy of Google Developers – YouTube)


 

Google’s Fun Propulsion Labs VoltAir Project page said:

“We looked at various UI frameworks and game engines and decided Qt was not only the more daring choice due to the lack of games developed in it, but also a very interesting one due to the future ability to extend it into a rich editor, support from Digia, and many technical development advantages such as:

  • Provided a platform abstraction that allowed us to also develop and test on desktop, reducing iteration time
  • Included a scene graph and out of the box “serialization” mechanism
  • Allowed for easily building a fluid UI system that was good for games
  • Provided an object model with dynamic JavaScript bindings, properties, as well as callback mechanisms for easy scripting and prototyping
  • Contained a well integrated IDE (QtCreator)
  • Included a flexible LGPL License.”

 

To top it off, our friends at Google gave some more kudos to Qt Quick:

“Qt Quick’s strongest feature by far is its abstraction for building user interfaces. It is, in fact, very well suited for the types of UIs you would expect in a game, where there is a lot of latitude taken in the UI element design and animations are abundant.”

Thanks, amigos from Fun Propulsion Labs Google. We appreciate the kind words and we wish much success to VoltAir.

Go on everybody – learn more about the VoltAir Project and find out why, how they developed the game. Download it now from the Google Play Store or play around with it yourself with the open source code.

meta-kf5 – almost there…

So, as of tonight, all but three tier 1 modules from kf5 are built in meta-kf5. The ones remaining are KApiDox, which does not really apply, and KConfig and Sonnet, which both needs to be part built for the native host environment, and part cross compiled. So, any Yocto hackers out there, please have a look at the issues linked to from the meta-kf5 status page.

meta-kf5 progress report

The meta-kf5 Yocto  layer is coming along nicely. Most of the modules are proving to be fairly easy to integrate, much thanks to the excellent ground work in meta-qt5, including the cmake_qt5 bbclass.

My plan for the summer vacation was to do one module a day, so around 5 would be ok. Until now I’ve done 9. Only KConfig has been providing any resistance (it does not like QT_NO_SESSIONMANAGER). My current pipe for tier 1 modules has 6 more candidates in it, so hopefully I can say that I’ve done 15 modules tomorrow night.

Right now, I have only one big worry – I have no code using the packages, so it is a bit of a if-it-compiles-it-works mentality right now. This should be fixed by integrating the test cases from the KF5 modules with ptest and building a test image. This is something that I’ll have to look at further along the road.

Code highlighting in SlideViewer

A couple of months ago, Jesper Pedersen introduced our presentation tool for trainings, called SlideViewer. As SlideViewer is mainly used for our Qt and Squish trainings, we naturally have many code examples on the slides, ranging from C++ and QML to Python and XML. In this blog post, we will take a closer look at how we use WebKit to power the syntax highlighting of these code examples in the training slides.

In the spirit of not re-inventing the wheel, we looked at existing solutions ranging from the Qt Creator highlighting to what is used on the web. Eventually, we decided upon using the JavaScript based highlighting application SHJS, as that turned out to have the most extensive highlighting out-of-the-box among the solutions we looked at.

The SHJS highlighter does not have a function that you call with a piece of code and get a highlighted version back. Instead, when told to, it traverses the DOM tree of a web page. When it finds a piece of code to highlight, it modifies the DOM tree and inserts <span> elements around the different code fragments. Combining these <span> elements with a stylesheet, we get the desired highlighting of the code.

Of course, in order to provide SHJS with the DOM tree so it can perform the highlighting, we need a web page. The natural solution for that, since we’re in Qt land, is to simply use QWebView and friends. And as we do not need any of the QWidget attributes, our starting point is QWebPage.

To make this work for our code slides, we have a number of steps to take:

  1. Create a web page
  2. Feed the HTML with the code to highlight into the web page
  3. Run SHJS’s highlighting function
  4. Parse the HTML code once SHJS is done

Let’s look at these, step-by-step.

Creating a QWebPage is easy:

m_webPage = new QWebPage(this);

SHJS is modular in its implementation, so for the web page containing the code we want to highlight, we only need to use e.g. the C++ specific parts of SHJS. Looking at the code below, we include both the main script as well as a language-specific script from SHJS, then we wrap our, in this case, C++ code in a <pre class="sh_cpp">...</pre> tag. This tells SHJS that what we want to highlight is C++ code. If we wanted to highlight Python code instead, we would include the “sh_python.js” script and wrap the code in <pre class="sh_python">...</pre> instead.

const QString preHighlightHtml = QStringLiteral("<html>"
                        "<head>"
                        "<script src=\"qrc:///highlight/sh_main.js\"></script>"
                        "<script src=\"qrc:///highlight/sh_cpp.js\"></script>"
                        "</head>"
                        "<body onLoad=\"sh_highlightDocument();\">"
                        "<pre class=\"sh_cpp\">"
                        "%1" // code
                        "</pre></body></html>").arg(code.toHtmlEscaped());

m_webPage->currentFrame()->setHtml(preHighlightHtml);

The code above also takes care of the third item on our list. Notice the onLoad attribute of the <body> tag, telling SHJS to run the highlighting once the page is finished loading.

Now, in order to be notified when the highlighting is completed, we cannot use the loadFinished() signal from neither QWebPage nor QWebFrame. That signal is emitted when the page is done loading, and hence before the highlighting is completed. Remember the highlighting doesn’t run until the page is done loading.

To realize when the highlighting is done, we have a number of options. We could add a timer that checks every 100 ms or so, whether we have any <span> tags present. This might lead to a race though, as some parts of the code might have been highlighted by then, but not all of it.

Thanks to the JavaScript integration, we can make our C++ objects available to the JavaScript context in our web page, and this is the technique we use to get notified when the highlighting is done. So to get notified, we add a callback for the SHJS script to call once the highlighting is completed. This is done by calling QWebFrame::addToJavaScriptWindowObject(..), which makes a C++ QObject available to the frame’s JavaScript context, in this case under the name highlighter.

m_webPage->currentFrame()->addToJavaScriptWindowObject("highlighter", this);

This needs to be done each time after loading a new URL. Thankfully, QWebFrame has a signal telling us when it’s needed: QWebFrame::javaScriptWindowObjectCleared().

connect(m_webPage->currentFrame(),
        &QWebFrame::javaScriptWindowObjectCleared,
        this, &Highlighter::addJSCallback);

The implementation of the callback is straightforward:

// Header file
Q_INVOKABLE void setHighlightCompleted(bool success);

// Source file
void Highlighter::setHighlightCompleted(bool success)
{
    emit highlightCompleted(success ? parseHighlightedHtml() : plainTextCode());
}

As a last step, we modify the SHJS code slightly, to call this function when it’s completed:

highlighter.setHighlightCompleted(true);

The parseHighlightedHtml() function used above, simply parses and returns the HTML code between <pre class="sh_cpp"> and </pre>, which is now full of <span> tags generated by SHJS. Combine this with the SHJS stylesheet and we have our highlighted C++ code!

The above described method is used for highlighting C++, Python, XML, JavaScript and Java code. For QML code, we had to take a different approach, as SHJS has no built-in support for QML. Instead, we use a Python script that, similarly to SHJS above, gives us HTML code with <span> tags to combine with a stylesheet.

The post Code highlighting in SlideViewer appeared first on KDAB.

Adapting SlideViewer to Qt Quick Controls

Several previous posts have introduced our SlideViewer tool which we created for use in the various trainings we deliver. The tool started out as an experiment, created using basic QtQuick 2 Items. Startup configuration was specified by command line arguments, and a simple Keys.onPressed function provided most of the runtime control: both navigating around the slide deck but also more advanced functions such as reloading the slide deck and toggling fullscreen mode.

While the above was sufficient for developers, it was unsuitable if we ever wanted to give SlideViewer to non-technical people as a packaged application – and some developers do prefer to use a real GUI too! Hence the aim was to give SlideViewer a menu-bar and dialogs, using the features available in Qt Quick Controls.

First Attempt

My initial change was to add a Menubar item to our existing Item hierarchy, with some
Menus and MenuItems:

    MenuBar {
        id: menuBar

        Menu {
            title: "&File"
            MenuItem { text:"Open" }
        }

        Menu {
            title: "&Go"

            MenuItem {
                text: "&Next slide"
                shortcut: "Right"
                onTriggered: _slideDeckController.incrementPage(1)
            }

            MenuItem {
                text: "&Last slide"
                shortcut: "End"
                onTriggered: _slideDeckController.gotoSlide(_slideDeckController.total,0)
            }
        }
    }

This actually worked on Mac, where we use native code to create and define the menu-bar. But actually MenuBar is supposed to live within an ApplicationWindow element in QtQuick. ApplicationWindow inherits from Window, but is not an Item – so this entailed some changes in the top-level QML element.

Using an ApplicationWindow

Item {
  id: rootWindow
  Item {
    id: content
    // .... all the SlideViewer window content
  }
}

… becomes this …

ApplicationWindow {
  id: rootWindow
  Item {
    id:content
    // .... all the SlideViewer window content
  }
  MenuBar {
    Menu {
      ...
    }
  }
}

But more importantly, it entailed some changes in C++. QQuickWindow is a QWindow, and expects the QML file it loads to define a root Item. But with the new structure above, the root element in our QML is a Window. We need a QML / QtQuick engine which expects us to load window – and fortunately, it exists: QQmlApplicationEngine.

Setting up the application engine is very similar to a QQuickWindow:

QQmlApplicationEngine appEngine;
appEngine.addImportPath(":/qml");
appEngine.load(QUrl("qrc:///qml/SlideViewer/SlideViewerWindow.qml"));

Because we want to manipulate the application window from C++, we use the rootObjects method of the engine to find our application window:

Q_ASSERT(!appEngine.rootObjects().isEmpty());
QQuickWindow * const rootWin = qobject_cast<QQuickWindow*>(appEngine.rootObjects().front());
Q_ASSERT(rootWin);

Actions, always actions

The next problem is that we’re duplicating logic between MenuItems and the Keys.onPressed logic that already existed. The solution in QtQuick Controls is analagous to that in widget-based applications: actions! QAction from C++ is tied to the widget classes, so it can’t be exposed to QtQuick; instead we have a very similar Action item in Qt Quick Controls. As you would expect, you can supply the name, icon, keyboard shortcut and triggered behaviour of each Action once, and then refer to the Action from a MenuItem. Actions also work in toolbars, but SlideViewer doesn’t use those for now.

Action {
  id: printAction
  text: "Print"
  shortcut: "Ctrl+P"
  onTriggered: _printer.start()
}
MenuBar {
  Menu {
    ...
    MenuItem { action:printAction }
    ...
  }
}

Adding a dialog

Having made these changes, adding some dialogs was straightforward (at least for the
complexity needed by SlideViewer) – here is our SettingsWindow.qml:

ApplicationWindow
{
  ColumnLayout {
    anchors.fill: parent
    Checkbox { id:skipHandoutPages; text: "Skip handout pages" }
    Checkbox { id:showClock; text: "Show clock" }
    RowLayout {
      anchors: { bottom: parent.bottom; right:parent.right}
      Button {
        text: "OK"
        onTriggered: ....
      }
    }
  } // of column layout
}

Composing buttons and checkboxes using column and row layouts, and mixing QtQuick Controls layouts with standard QtQuick anchors, works nicely.

Finishing up

There is one final piece to avoid long startup times and increased memory footprint, as we add more dialogs: in a widgets-based application it would be unusual to create all dialogs at startup. Similarly for QtQuick we want to defer loading the QML for a dialog until it’s shown. The solution to this is of course the trusty Loader item.

The final result is SlideViewer is now approaching something we could package and deploy to non-technical users, without needing to explain command-line arguments to them, or memorise keyboard shortcuts. From the developer side we did need to restructure some things; if the codebase was larger, these changes could have been painful. Transitioning from a prototype to a real application UI is not something to leave until the day before ship!

This was my first experience using Qt Quick Controls, and for this scale of application they worked well – definitiely a viable alternative to using widgets.

The post Adapting SlideViewer to Qt Quick Controls appeared first on KDAB.

Re-introducing QicsTable

QicsTable is a full-featured, high-performance table widget suited for use in industrial-strength applications. It incorporates a model-view-controller architecture, so that multiple table widgets can display the same data. The QicsTable widget has extensive printing capabilities, including the ability to print the table with different visual characteristics than the on-screen table display.

KF5 in Yocto

KDE recently released the first version of KDE Frameworks 5, or shorter KF5. This is a set of add on modules extending and improving Qt, forming the base on which the Plasma Desktop is built. The nice thing is that KF5 is very modular and very reuseable.

Recently I’ve spent some time working with Yocto (yes, the series will continue – I just need time to do a couple of clean builds). So, I thought this was the perfect little summer vacation project for me. So, the plan is to package one module a day of KF5 for Yocto. The layer resides on github as meta-kf5.

This is a release early, release often project, so I’ve just gotten KAchive and ECM integrated. You can follow the progress from the project status page.