Merge of diet branch into master

A while ago we announced the ‘diet’ branch and asked the qooxdoo community to help us test the changes. We received plenty of great feedback including detailed bug reports and pull requests, so a big thank you to everyone who participated. Since most bugs are ironed out and no show-stopping issues popped up, we’re going to go ahead as planned and merge diet into master around the middle of the next week.

CSS gradients and border-radius in IE9

While working on the diet branch, we found ourselves in need of a new solution for an old problem that affects Internet Explorer 9 if border-radius and a gradient filter background are applied to a DOM element. See the recent blog post for details about a smart solution, that allows for a combination of both features without requiring doubled DOM elements.

Config File Branching

In general there is no way of conditional branching in the configuration DSL of the tool chain. qooxdoo configuration files are JSON-based and have no “if” construct of any kind. So there is no way of directly expressing e.g. “If the value of this macro is true include this list of classes, otherwise include a different list”. But you can achieve much of the same by using the value of macros in references to other job names. Here is how to do that.

Bugfixes

For a complete list of tasks accomplished during the last working week, use this bugzilla query.

Consider a div element with rounded corners:

#test {
  width: 200px;
  height: 100px;
  border-radius: 20px;
  border: 5px solid red;
}

To give it a color gradient background, we add a CSS linear-gradient which is supported in most modern browsers (Firefox, Chrome, Opera, IE10):

background-image: linear-gradient(#0000FF 0%, #FFFF00 100%);

Element with CSS linear-gradient (Chrome)

Internet Explorer 9 doesn’t support linear-gradient, but it features the proprietary Filters and Transitions API which we can use to achieve mostly the same result:

filter: progid:DXImageTransform.Microsoft.Gradient(GradientType=0, StartColorStr='#FF0000FF', EndColorStr='#FFFFFF00';);

Element with DXImageTransform gradient filter (IE9)

And there’s the problem: The gradient filter doesn’t respect the border-radius property so the background overlaps the corners. A common solution for this is to wrap two elements, applying the border-radius to the outer element and the gradient to the inner element. qooxdoo’s decorators take this approach as well, but for diet our goal was to eliminate the decorator elements entirely and apply all styles directly to the content element, so we had to come up with a new solution.

Fortunately, IE9 supports Canvas and drawing a gradient is pretty simple:

var startColor = "#0000FF";
var startColorPosition = 0;
var endColor = "#FFFF00";
var endColorPosition = 100;
var width = 1;
var height = 200;
 
var canvas = document.createElement("canvas");
canvas.width = width;
canvas.height = height;
var ctx = canvas.getContext('2d');
 
var lingrad = ctx.createLinearGradient(0, 0, 0, height);
lingrad.addColorStop(startColorPosition / 100, startColor);
lingrad.addColorStop(endColorPosition / 100, endColor);
 
ctx.fillStyle = lingrad;
ctx.fillRect(0, 0, width, height);

In qx.desktop’s theming system the gradient is rendered only once and will then be applied to any widget using the same decorator. Since we don’t know in advance how tall each widget is going to be, we use a fixed size of 1 x 200 pixels. Finally, we convert the drawing into a data URL and apply it like a regular background image, stretching it to fill out the element:

var el = document.getElementById("test");
el.style["background-image"] = 'url(' + canvas.toDataURL() + ')';
el.style["background-size"] = "100% 100%";

Element with Canvas-generated background image (IE9)

And that’s it, border-radius and background gradient combined without a wrapper element.

Includer Jobs

In qooxdoo configurations the general way to inject settings into a job is by using the extend key. So if you want settings from one job to propagate into another job you make the second job extend the first:

"jobA" : {
  "environment" : { "foo" : "bar" }
},

"jobB" : {
  "extend" : [ "jobA" ],
  ... // more job settings
}

So now jobB will get the environment setting from jobA as if you had written them into jobB directly (There is some conflict resolution going on if jobB already has an environment key). This of course makes more sense if you want to have more than one job inherit these settings, like when you substitute “jobB” with “source” and “build”. It’s a common way in qooxdoo configs to maintain multiply used settings in a single place.

As in this example the job names in the extend key can refer to jobs that do nothing on their own and are just provided to hold some setting to be used in other jobs (often referred to as “includer jobs”).

Using Macros in Job Names

What’s also interesting here is that the names in the extend key can contain macros. This allows you to select a job according to the value of some macro.

"jobA1" : {
  "environment" : { "foo" : "bar" }
},

"jobA2" : {
   "environment" : { "foo" : "xyz" }
},

"jobB" : {
  "extend" : [ "job${JobSuffix}" ]
  ...
}

By setting the value of the JobSuffix macro to either “A1″ or “A2″ you now select which job is being included into the extending job, and by that select the configuration keys and values that come with it.

In this example you could either specify the concrete value of JobSuffix in the global “let” section of the configuration file

"let" : {
  "JobSuffix" : "A1"     // or "A2"
}

or you could provide it on the command line when invoking the generator:

generate.py -m JobSuffix:A2 ...

Pairs of Includers

Often you will create pairs of includer jobs like in the above example for the same set of settings, to provide alternative values. To pick up the example from the beginning, to provide two different include lists to a source job you could write:

"oneInclude" : {
  "=include" : [ "foo.ClassA", "foo.theme.ThemeA" ]
},

"otherInclude" : {
  "=include" : [ "foo.ClassB", "foo.theme.ThemeB" ]

"source" : {
  "extend" : [ "${IncJob}Include" ]
}

(Don’t worry about the equal sign in "=include" for the moment). Now you only have to provide the value for the IncJob macro, either "one" or "other", and your source job will use the corresponding include lists.

This basically is it, you can now construct different jobs by just assigning different values to macros. Mind that this kind of selecting includer jobs will only work with values of macros, and only if those values are strings. This also means you cannot e.g. refer to the value of a specific environment setting. But in many cases you can work the other way round and make the value of an environment setting part of an includer job which is then selected by the value of a macro.

Lightweight desktop widgets

You might have already seen the blog post that qooxdoo is on a diet. This announces a large internal change to the widgets of qx.Desktop. By reducing the number of DOM elements usually from about three down to just one, this leads to a more lightweight widget set. Still most of qooxdoo’s advanced features could be retained. As the previous implementation was required to bring fully-featured rich user interfaces to some older browsers, support for legacy browsers had to be reduced for the benefit of leveraging features of modern browsers.

Compile with fast linting

During compile time, i.e. when creating a runnable application from the library classes, the Generator does a restricted lint checking to detect some of the more fundamental issues. Unknown globals is one of them, and those checks have significantly been improved recently. @-hints embedded in JSDoc type code comments are now properly supported. These hints allow you to silence warnings about unknown symbols that you know will be available at runtime. The new implementation results in a fast unknown globals check that adds nearly no performance penalty to the existing compile functionality.

Bugfixes

For a complete list of tasks accomplished during the last working week, use this bugzilla query.

Those additional elements were necessary in older browsers to allow for fully-featured rich user interfaces. Now we can take advantage of native features in modern browsers while trying to implement graceful degradation in older browsers and discontinuing support for some legacy browsers.

Every desktop widget now consists of just one content element. Before, each widget consisted of at least two (often even 3-5) DOM elements and their corresponding JavaScript object representations. So the new element or object count is significantly reduced by about 3:1.

qooxdoo Widget Browser in Firefox' 3D DOM view: master branch (left) vs. diet branch

This work happens in a feature branch appropriately named ‘diet’, which we now consider ready for public testing. We’ve prepared a document explaining the changes in detail. This also includes a migration guide for anyone willing to test the modifications. If you don’t use your own custom themes, layouts or widgets, it should be particularly easy to build your app against the diet branch. Give us feedback on the usual channels, e.g the mailing list or Bugzilla.

If you don’t have the time to migrate your applications, you can still help us out by testing the framework’s demo applications in your favorite (modern) browser.

We’re planning to merge the diet branch into master in about two weeks. Anyone currently using the master branch is strongly encouraged to test their application using diet before this time to prevent any unpleasant surprises.

Since these are major modifications including some API changes, they will be published along with a next major release qooxdoo 3.0 this summer.

During compile time, i.e. when creating a runnable application from the library classes, the Generator does a restricted lint checking to detect some of the more fundamental issues. Unknown globals is one of them, and necessarily so as the Generator uses unresolved globals to add classes to the application build (so called “class list/dependency exploration”). When the new lint module was introduced all this checking was offloaded onto it with a suitable configuration, which worked fine functionally but was considered too slow performance-wise.

With recent master this has been amended. Unknown globals-checking is again done during dependency exploration to take advantage of the already existing logic. This was much faster but a major issue with the old implementation was the lack of support for @-hints embedded in JSDoc type code comments. These hints allow you to silence warnings about unknown symbols that you know will be available at runtime, and to guide further dependency exploration.

In order to add @-hint support and make it fast a new approach was taken. With every syntax tree of a class an adjacent tree of hint nodes is now calculated, derived from the corresponding JSDoc comments. Later when checking unresolved symbols this hint tree can be traversed and inspected very quickly, in order to decide whether an unknown symbols should be pursued, the corresponding class be added to the build, warned about or left alone.

The result is a fast unknown globals check that adds nearly no performance penalty to the existing compile functionality.

Deprecated code removed

Following our established development and release process we have now – being in-between releases – removed all deprecated code in master.
Before running your projects with the latest development snapshots, make sure you no longer encounter deprecation warnings. Such development hints were added with the previous official releases. Check your browser’s console and get rid of the deprecated code for a clean, up-to-date code base.

Bugfixes

For a complete list of tasks accomplished during the last working week, use this bugzilla query.

Stay tuned for some exciting announcement next week …

Your Real-life Example

We just called to the community of qooxdoo app creators, as we seek for an updated and extended overview of real-life examples. Let us all know about your specific use of the qooxdoo framework, thanks.

qooxdoo-contrib 2.0

There has been a lively discussion around the proposed new infrastructure of qooxdoo-contrib. Thanks for your interest and all the input so far. Please feel free to provide more feedback or new aspects, particularly if you happen to join the discussion just now.

Bugfixes

For a complete list of tasks accomplished during the last working week, use this bugzilla query.

Bye.

As an open source project – free but with enterprise-grade aspiration – we seek for an updated and extended overview of real-life examples, apps that you develop and release “into the wild”.

Unlike commercially driven projects qooxdoo puts less stress on marketing, but more on advancing and shipping state-of-the-art technologies for you and us to create modern applications. It aims to support a professional, sustainable development and deployment of apps for a wide range of platforms.

These aspects were probably relevant when you chose qooxdoo, and hopefully it is that what you found when you used it in projects. If this is the case then let us and others know about your specific use of the qooxdoo framework. Sometimes that would be publicly available internet applications; sometimes proprietary solutions only available within your intranet; sometimes even open-source solutions such as frameworks built on-top of qooxdoo.

Even if you think you cannot disclose much about an internal enterprise solution or product, please reconsider as this is valuable input for an open source project like qooxdoo. To be frank, we ourselves could do better here. Thus we will also promote real-life examples that emerge within our vicinity.

In any case, we’d be glad to gather your use cases and share them among the members of the qooxdoo community and potential new users. You are welcome to use either of the two existing input channels: you may add your app to the real-life examples page yourself (as this belongs to the wiki-based parts of the homepage, you simply need to register if you haven’t already). Alternatively, we’d also be fine to receive your input by email. Make sure to supply 1-2 paragraphs describing your app in English, links to your app/company if available, and preferably a screenshot. We will copy edit your input, and get back to you for a quick sync if necessary.

Thanks.

The new approach is not like npm or the package handling of Go but somewhere in the middle. It’s intentionally kept simple, and will mostly work through conventions of the people dealing with it rather than code. Most of this affects maintainers of contributions. Users of contributions will see only little change (affecting their config.json).

Reprise – What we wanted to achieve

The existing system maintains the code for all contributions in a single SourceForge SVN repository where they are also looked up and from where they are downloaded. The main goal with the new design is to decouple contributions from a specific store, so contrib authors would be free to develop their contribution on a platform they choose, whichever source control system and hosting provider they prefer (SourceForge, Github, Launchpad, Bitbucket, …).

Leaving the issue of code maintenance to the hands of the authors, the remaining issues are that of finding and using such distributed contributions. For the finding part we want to establish a qooxdoo-contrib catalog.

 The Catalog – A Github Repo

The catalog is just the index of known contributions with a bit of necessary meta information for each. To make maintenance of the catalog simple and straight-forward, we decided to implement it as a Github repository. It will have the same top-level directory structure you might know from the Sourceforge qooxdoo-contrib repo, namely

• the contribution names top-level
• beneath each contribution a set of directories with version names, and pot.a general Readme
• in each version directory a Manifest.json file that describes this particular version of the contribution

The Manifest.json file is exactly the Manifest file you know from qooxdoo libraries, with a few new keys. Contribution maintainers can copy it straight over from their project to the catalog repo. You need to make sure that all necessary fields in it will be filled, particularly the info/download field. A new Generator job, validate-manifest, will help you with that.

That means maintaining your contribution in the catalog will require you to fork the catalog repo, make necessary changes and then issue a pull request which we will incorporate. (This obviously also means you will need a Github account to be a contribution maintainer, but we hope this requirement will be trivial).

Contributions come as Archives

The important thing here is that the info/download field will point to an archive (.zip or .tar.gz) containing the contribution. This puts an extra load on the contrib maintainers as they need to create archives from their contributions. But this is also more efficient and more robust for downloading than spidering Web-hosted files and directories, as is done in the old system (Anyone who tried to spider a directory tree from decorated HTML knows what I’m talking about). So while you authors are now free to maintain the source code of your contribution wherever you prefer you will need to find a place on the Web where you can place the corresponding archives.

Another new field in the Manifest.json for contributions will be information/checksum which will be a Sha1 checksum over the archive. This will be used by the Generator to both validate the archive after download as well as determine whether a contribution has changed online and needs to be re-downloaded.

Author’s Workflow

So the basic workflow for an author having a new revision of his contribution’s version is:

• Create an archive containing the contribution and put it online.
• Edit the contribution’s Manifest.json to contain the download location and a Sha1 checksum over the archive.
• Copy this Manifest.json to the appropriate path in the catalog repo.
• Send a pull request for the catalog.

Once the pull request has been incorporated your new revision is available to contribution users.

For an author choosing Github to host his contribution all this can be quite easily achieved (except for the checksum thing) by maintaining contribution versions as Git branches and exploiting the archive downloads that Github offers. You just use an URL like https://codeload.github.com/<user>/<contrib>/zip/<branch_name> as the Manifest’s download URL.

 Changes for Contribution Users

Users of contributions will just have to adapt their library/manifest configuration keys in config.json. For the new system they will use http:// URLs as values which point to the contribution and version in the catalog. Here is how that would look like for the popular UploadWidget:

{
  "libraries" : {
    "library" : [
      {
        "manifest" : "https://raw.github.com/qooxdoo/contrib-catalog/master/UploadWidget/0.2/Manifest.json"
      }
    ]
  }
}

That’s it. It looks a bit technical but will also allow hosting libraries online outside of any catalog, or for alternative catalogs e.g. maintained on Intranets. The traditional contrib:// pseudo scheme will continue to work for some time to support contributions living in the old system.

 Outlook

That’s the basic system and how to work with it. As said we encourage feedback, especially from contribution authors who will have the most exposure to it. It’s simple but will hopefully serve the basic needs of authors and users.

It’s easy to envision enhancements, like a nice, good-looking, searchable, sortable, category-driven Web interface to the catalog. It is also a question whether there should be a dedicated download step for the Generator (like generate.py update-contribs), instead of doing it all in the background and while building an application. But let’s discuss those when the basic system is in place. For now we just want to do the first step in the right direction.