Paul Dijou Blog

WTF npm update?!

2015-06-06T23:26:35+02:00

tl;dr

Do not use npm update with any package which use custom dist-tags.

Dist-tags

This package has officially those 3 last versions: 1.0.0, 1.0.1 and 1.0.2. But inside the dist-tags (read more), we have two of them: the classic latest which refers to the latest stable release of the package and a custom one, named canary, indicating the last non-stable release of the package. If you are wondering if real projects are using such tags, the answer is yes, the npm package is using latest and next tags for it's weekly pre-release (read more).

Currently, latest points to 1.0.1 and canary to 1.0.2. Meaning that if you run npm view test-npm-update, you will have something like:

{
        name: 'test-npm-update',
        description '...',
        'dist-tags': { latest: '1.0.1', canary: '1.0.2' },
        versions: ['1.0.0', '1.0.1', '1.0.2'],
        version: '1.0.1',
        ...
      }

Versions

It's important to realize a few things here. 1.0.2 is a released version just as 1.0.0 and 1.0.1 and we will say it's the greatest one (as in the biggest number according to semver) but not the latest one (as in the one tagged with the latest dist-tag). We need to make such distinction to fully understand what will happen after that.

So, when you run npm view test-npm-update, it actually runs npm view test-npm-update@latest, meaning it will grab the informations of the latest version. But maybe some other versions have been released with a custom tag after this one. For me, so far, so good. NPM is doing exactly what I would expect. If I want a custom release such as the canary one, I can run npm view test-npm-update@canary and it will display infos about the 1.0.2 version. In fact, but I might be wrong but I except NPM to always use the latest version (aka the latest dist-tag) by default if I don't specify anything. That's what you can mostly read all over the NPM documentation.

But remember, 1.0.2 is inside the versions array just like any other version. So first warning, if you use such metadata for whatever stuff you are doing, do not assume that the greatest version inside the versions array is the latest one.

Install

Now, what if I run npm install test-npm-update? What would you expect to be installed? 1.0.1right? And of course it will be this version, the latest one. That's normal, after all, latest is the default one. All good here.

What if I clean my folder and then run npm install test-npm-update@^1.0.0? Guess what, 1.0.1 will be installed. And I'm totally ok with that. I asked for the best 1.x.x version and I'm glad to have the latest one since it matches.

package.json

But most of the time, you don't install or update from command line, you have a package.json file with a range inside it. Let's say we have the following one:

{
        "name": "awesome-project",
        "version": "0.0.0",
        "dependencies": {
          "test-npm-update": "^1.0.0"
        }
      }

Pretty classic, right? Now, for the purpose of the demo, let's say we currently have the 1.0.0 version of test-npm-update locally installed. If you want to reproduce, just create an empty folder, then create a package.json inside it with the previous content and run npm install test-npm-update@1.0.0 to force the install of an old version.

Done? Cool, let's move forward. NPM has a command to test if you have outdated versions locally installed. Which is our case. Let's check that by running npm outdated. You should have something like:

Package          Current  Wanted  Latest  Location
      test-npm-update  1.0.0    1.0.2   1.0.1

Wait a minute? I'm ok with current (the locally installed) being 1.0.0 and latest (matching the dist-tag) being 1.0.1 but wanted is supposed to be the best matching version I should install according to package.json. How can it be greater than latest?

Actually, it's all ok according to the NPM documentation. After all, the package.json range is ^1.0.0 which means the greatest possible version without changing the first non-zero digit. And among all our versions (see the versions array from npm view), both 1.0.1 and 1.0.2 match this range, but since 1.0.2 is greater than 1.0.1, the wanted version is 1.0.2.

I didn't expect that to be honest. That's not wrong but I can't help myself finding that strange.

Install again

Quick mention to the fact that if I run npm install with my package.json in an empty folder (aka without the 1.0.0 version already installed), it will still install 1.0.1 version. That's ok according to latest being the default one. Back to our outdated 1.0.0 version.

Update

Things start to get really ugly now. So, npm outdated just told me I have an old local version. I should probably update it, and NPM has a command for that. Let's run npm update. To be honest, I wasn't sure anymore what would be installed locally. I mean, I would have normally expected the 1.0.1 version. My brain was like "It should be the greatest stable version which match the range", with stable meaning lower or equal to the latest tag, but for NPM, it's more like "It should be the greatest version which match the range. Period.". And it makes all the difference. My brain stops at 1.0.1 as the latest stable but NPM browse all version, including any custom dist-tags, including the canary version.

At the end, running npm update will install 1.0.2 version. This is wrong. According to documentation:

This command will update all the packages listed to the latest version (specified by the tag config), respecting semver.

I read that as the latest version according to latest dist-tag. But we just updated to a version beyond this latest version. In any case, this is super dangerous! It means you can update to non-stable versions without even noticing it.

What if we didn't have the 1.0.0 already installed? Since npm update also install missing packages, it will indeed install test-npm-update according to package.json and, of course, to the 1.0.2 version.

Conclusion

IMHO, I think this is way too dangerous, npm update should be capped by the latest version, and so should npm outdated. By default, no command should target versions beyond latest dist-tag. Also, it seems inconsistent to have install and update both capable of installing a missing package from a package.json file but not to the same version.

I raised an issue on Github, we will see. Be careful from now on.

Thanks for reading! Spread the word.

Personal ad

It might be a bit too early to speak about that, but if you need an outdated command which is actually capped by the latest tag and also support other package managers (like Bower), please check my outdated project. It's not ready at all yet but it will be in the next few days, promise.

BrowserSync proxy in top of a Play Framework server

2015-06-06T15:40:57+02:00

TL;DR 'coz I'm a h4k3r

Full source code of the demo is on GitHub with a nice and simple README on how to bootstrap the project.

What are you talking about dude?

Play Framework is an HTTP server written in Scala. It is super cool and you should probably give it a try. Let's say you are using it, you really enjoy how productive it is, only having to refresh your browser to see all your code modifications, but you want to go further, you want live-reloading! There are several tools doing that.

Introducing BrowserSync. It provides 2 killers features. One is live-reloading: it can monitor resources, and when it detects some modifications, it will reload the browser page to load them. Even better, if possible, it will hot deploy them, meaning they will be loaded without reloading the page. That's the case for CSS files which are loaded and the page repainted to display the new design without any refresh. The second one is to keep synchronized several browsers across devices. And we are not talking about resources only but about all user actions. If you scroll on one browser, it will scroll on all connected browsers, same for clicking on a button, and so on…

Let me say it again with a concrete example. You are on your computer, one screen with your source code, another one with Chrome opened and a last one with Firefox. You also have an iPad tablet and an Android smartphone connected to your computer. All of those have your application main page opened. Each time your edit your code, they will all reload and display the new result (hot deploy in case of CSS). When you are ready to test interactions, you just go to, let's say, your Chrome browser and start scrolling and clicking. All your screens will start moving and doing the actions everywhere. Talk about increasing productivity!!

What about assets that require compilation? Such as SCSS, LESS, Stylus, CoffeeScript, etc… No biggy bro (or sis), we have you covered. In this demo, we will use Gulp to monitor those resources, compile them, and pass them to BrowserSync for live-reloading (obviously not losing the hot deploy if possible). You can freely use whatever build tool you want, be it Grunt, Broccoli, …

Magic proxy

BrowserSync could be used as the web server, that's the easiest way to use it, and I'm actually doing it when my Play server is only here to provide a REST API. Consider doing that when you don't need Scala templates or when you cannot use them (for example if you want to embed all your standalone HTML files inside a Cordova / PhoneGap application).

But here, we want to use Play as the web server, meaning it will be responsible to serve all our assets: HTML, CSS and JavaScript. For that, we will need to use the proxy feature of BrowserSync. Long story short, it will start another web server, on its own port, and will display the exact content from the Play server but adding a bit of JavaScript magic in it so it can enable all its features. It is just like opening your Play application, just on a different port. By default, you would use the port 9000 for Play, in this demo, we will set the proxy on port 9001 (it would have been around 3000 by default, but I didn't want anyone to freak out so I put it as close as possible to Play default one).

We also want to live-reload some of our resources. I kept it simple for this demo, but you are free to add as many as your want. We will use the files property of BrowserSync configuration and set and array of files to monitor (it supports wildcards). Here, we are monitoring all CSS files from public/stylesheets, JavaScript files from public/javascripts and HTML files from app/views. There are two important points to notice here. First, we are referencing files based on their path on the source code, not on their actual url. For example, public/stylesheets/main.css will be served by Play as assets/stylesheets/main.css, but BrowserSync only needs to know the real path, after that, you can map it to whatever url you want. Second, we are monitoring Scala templates, and it's fine because when BrowserSync will detect a modification, it will reload the page, and Play will re-compile the template before serving it, so it will display the new version. It works just fine with both run and ~run , the latest being faster.

Here is a super small configuration for BrowserSync to enable such a proxy. You can read the online documentation to extend it.

var browserSync = require('browser-sync');
      
      browserSync({
        // By default, Play is listening on port 9000
        proxy: 'localhost:9000',
        // We will set BrowserSync on the port 9001
        port: 9001,
        // Reload all assets
        // Important: you need to specify the path on your source code
        // not the path on the url
        files: ['public/stylesheets/*.css', 'public/javascripts/*.js', 'app/views/*.html'],
        open: false
      });

What about compiled assets?

Right… right… there are assets that you cannot serve directly, you need to preprocess them. The easiest way to do that is using a build tool, just pick one among the best ones (Gulp, Grunt, Broccoli, Brunch, …) and enjoy. There is a really simple separation of concern here: the build tool only manage assets that are not served directly (LESS, CoffeeScript, …) and compile them into assets that are actually handled by BrowserSync (CSS, JavaScript, images, …) which then live-reload them.

It's so easy that it's nearly frustrating. In the demo, the main.css file is generated from main.less. I have two Gulp task to handle that: one for compilation and one to monitor any modification.

var gulp        = require('gulp');
      var less        = require('gulp-less');
      
      // Registering a 'less' task that just compile our LESS files to CSS
      gulp.task('less', function() {
        return gulp.src('./resources/less/main.less')
          .pipe(less())
          .pipe(gulp.dest('./public/stylesheets'));
      });
      
      // Let's watch our LESS files and compile them at each modification
      gulp.task('watch', function () {
        gulp.watch(['./resources/less/*.less'], ['less']);
      });

Wait, where are sbt-web and webjars?

Yeaaaaah… so lately, Play has introduced all those stuff on managing assets using SBT, some plugins and webjars. It works just fine but for me, it's just as strange as if I would install Play from NPM. It's cool if people want to try to merge two super different worlds (front-end and back-end), be it Node.js or scala.js, but I'm totally not ready for that. As a full stack developer, I want to enjoy each world with its own ecosystem, not trying to bend one into the other.

That's why in all my projects, front-end assets are handled by front-end tooling (mostly Gulp and NPM lately) and my back-end resources by back-end tools (SBT, Ivy, Maven, …). Not only do I get the best of each, but I do not have some limitations. For example, the other day, I came across a bug on a JavaScript library I was using. No biggy: fork the repo, correct the bug, do a pull request. And while waiting for the author to merge it (which can take some time), I am free to directly use my fork inside my package.json so it's not blocking at all. It tooks me less than an hour. I would be curious on how to do that using webjars? Anyway, sorry for not planning any proof of concept using webjars if that's what you are using.

And that's it! As I said at the beginning of the article, the demo is on GitHub, feel free to clone and play with it. Enjoy and thanks for reading.

Prismic.io: responsive images inside a StructuredText

2015-06-06T15:40:57+02:00

Once again, I will not explain what prismic.io is (but it's a tool to manage your website content), I will only focus on one limitation and how to solve it. If you don't know about prismic.io, you should quickly check its website before reading any further.

What's the problem?

In prismic.io, you are creating Documents which are composed of Fragments. A fragment is data organized in a more or less semantic way. For example, a fragment can be a Number, or a Date, … Among them, there is the type Image which is really nice because you can not only fully resize / crop your raw picture, but also add smaller images based on the original one for responsive purpose. You can upload your awesome photo of this beautiful sunset, fully sized at 4000 x 3000 pixels, resize it for desktop at 1920 x 1080, and finally add a mobile version at 320 * 480. Note that the last one doesn't respect the ratio at all, but that's the goal: you are creating a subset of your image with the best match possible regarding the targeted screen.

Another really powerful fragment in prismic is the StructuredText. It allows you to have a nice WYSIWYG editor inside which you will put new fragments (paragraphs, links, titles, images, …). It would be perfect if the image fragment inside a StructuredText was a real Image fragment, but it is not. You cannot specify any thumbnail. Meaning your content cannot be responsive at all. Your choice is: consume all the bandwith for mobiles with awesome full HD images or make the eyes of your desktop users bleed with super compressed images.

You don't want to do that!

And me neither! I want to have responsive images inside my StructuredText for God sake. I love tools which make my life easier, but they never should stand on my way to craft the best website possible. I will not lie to you, I couldn't find any solution that integrates nicely with the WYSIWYG editor since there is no way to extend it. The following solution will be a bit of hacking and your writers might complain about it at first.

So, what's the idea? The only way to have responsive images is to use a real Image fragment, no choice here. So let's create one, and let's put it inside a Group fragment. This way, you can add and remove as many images as you want and they will all be responsive. Wait, you can't do that inside the StructuredText, right? Indeed, that's why we will do outside of it and then find a way to bring back the images inside of it. Here is the mask for such a group:

"Images" : {
        "images" : {
          "type" : "Group",
          "fieldset" : "Images",
          "config" : {
            "fields" : {
              "name" : {
                "type" : "Text",
                "config" : {
                  "label" : "Name"
                }
              },
              "caption" : {
                "type" : "Text",
                "config" : {
                  "label" : "Caption"
                }
              },
              "image" : {
                "type" : "Image",
                "config" : {
                  "thumbnails" : [ {
                    "name" : "mobile"
                  }, {
                    "name" : "tablet"
                  } ]
                }
              }
            }
          }
        }
      }

Pro Tip - When creating thumbnails for an Image fragment, the offical documentation states that both width and height are mandatory. This is wrong, you can specify only one of the two or even none of them. Your UI might be a bit ugly (the resized thumbnails will only display after saving your draft) and sometimes you will have to click twice on a button (this one, I really don't understand) but otherwise, it works just fine. I hope prismic guys will make those attributes optional because… well… how can you anticipate how the writer wants to resize its image?

As you can see, I added two more fields. One is name and we will use it to reference our images inside the StructuredText. The other one is caption, that's because I want to put one on my images but prismic didn't allow me to do that at all by default. Thanks God, using this solution, I can finally do it, killing two birds with one stone. Now we can add as many responsive images as we want, yeah! Let's grab some orange juice to celebrate!

Let's hack some HTML

As I already said it, the next step will be to bring those images inside the StructuredText. We cannot do that directly inside the WYSIWYG editor but we can do it when rendering the final HTML. We will need two more things here:

having a placeholder inside the StructuredText to indicate we should insert an image
extend the default HTML renderer to support such placeholder

As for the placeholder, I decided to use a simple format like {image-[name of the image]}. Meaning that if the content of a paragraph inside my StructuredText is {image-sunset}, it will actually render the image named sunset from the group of images when generating the final HTML. Pretty easy right? Your writers might find it ugly, having to write those strange tags, and not seeing their images directly inside the StructuredText, but just tell them that's for the greater good. You could, of course, use another syntax, eventually support attributes, whatever.

The final task, and probably the hardest one, is to extend the HTML rendering system. Each primisc fragment has its own way to render as HTML. Unfortunately, there is no way to extend it, you can only, eventually, override it. And the one for StructuredText is by far the most complex one. The easiest way to do that is to copy/paste the default one from the kit you are using (if you are not using any kit, you are free to do whatever you want, so it's fine) and edit it. You will then edit the part responsible for rendering a paragraph, test if its content match our placeholder syntax, if so, render an image, if not, just render the text as it is. Since I'm using the JavaScript kit, here is the full copy/paste but that's the important part where we are rendering the image. My model is actually a JavaScript class extracted from the original prismic Document, but you could use the raw Document of course.

Conclusion and limitations

That's pretty much it. Here is a quick summary of what to do:

inside the mask, add a Group of Images with, at least, a Text fragment for the name.
choose a naming convention to write placeholders inside a StructuredText, ex: {image-[name]}
override the StructuredText HTML rendering function with a custom one inserting images from the Document when finding a placeholder
you can add more attributes and render them (like a caption)

What are the bad points / limitations of the solution?

writers need to write placeholders rather than having a nice UI to insert images
images do not display natively in the WYSIWYG editor anymore
you cannot re-order a Group fragment right now, meaning images will be sorted based on their created date and not on their place inside the StructuredText

Doing a conference website in two days

2015-06-06T15:40:57+02:00

Last week, on Wednesday, I had a meeting with some colleagues at Movio about the Scala Downunder conference. It's a Scala conference in Auckland, New Zealand, with speakers from Typesafe and local ones, and some replay from Scala Days. I discovered that we were doing the website for the conference here at Movio, and also that it was due for the following Friday in order to send it to Typesafe for feedback…. Ah, and also I would be crafting it.

(Pssst… okay, there has been some commits on the repo after the deadline, but that's only because the schedule totally changed, not my fault!)

Let's get started!

So, deadline is 2 days from now, the website is only one long page with all classic stuff: presentation, speakers, schedule, location and sponsors. Really good point: both the design and the content are nearly fully done. Otherwise, it would have been impossible.

My first step was to pick my tooling. I truly hate copy/pasting anything so it was out of question to copy/paste HTML when it comes to iterable data. I needed a static HTML website generator. Since the plan was to publish it on GitHub Pages, I choose Jekyll. This way, I would have the build for free, only needing to push on the gh-pages branch my source code (at the end, that didn't work at all, but Jekyll is still a good choice). Bootstraping with Jekyll is super fast as long as you already have Ruby installed on your machine.

Next I picked Gulp as my build tool. I love both Grunt and Gulp, but I'm trying to be more familiar with Gulp lately (at some point, I will switch to Broccoli anyway). Here is one important point: you should have snippets for each Gulp (or Grunt) task ready somewhere so you can do your setup in minutes. For example, here, I wanted to use LESS as CSS preprocessor, Browserify for JavaScript packaging and BrowserSync for live-reloading. I already got all those snippets, so I just needed to copy/paste them once in order to have my server running live and compiling all my assets.

Be semantic with your content

Remember, what's the most important is your content, not the design nor the funky JavaScript effects. And don't try to rush to apply funny colors or amazing transitions before having a real content behind them or you might not see the real behaviour when implementing them. Since I already have it, I wrote only HTML during the first hour. It allows me to be 100% semantic with my content. I don't care about CSS selectors or whatever, if this is a paragraph, I create a <p> tag, if this is a section, let's use a <section> tag. And at the end, you can already see the 0.0.1 version of you website, fully "usable" (just a bit ugly, and tons of typos).

After that, I worked to all dynamic parts of my HTML. I really wanted to have full content before starting doing any LESS. I used the _data folder of Jekyll that allow me to structured data in YAML files. I put all speakers and schedule in it. After that, it's all about the Liquid templating system to render it. You have for loops and if blocks, you have filers to render Markdown or whatever, for such a simple website, it was more than enough.

At this point, we are something like nearly half a day in the project and we have all the content and a nice setup allowing us to be super productive. It could have been faster, but there was quite some content to write and it took me a bit more time than planned to implement all my templating. That's cool!

Do it with style

As you can image, the next step is doing all the website design. If possible, I prefer to write all CSS classes first, staying close to the HTML semantic and then implement them rather than inventing crazy CSS selectors and then bind them to your HTML. It's important to know what's your browser target. Because, trust me, you want to use flexbox but that means IE10+ (using a LESS mixins for old syntax). Lucky me, I could do that. If not, consider using a grid system.

When doing the design, never ever think "Oh, it looks already pretty good… I shouldn't be far away from the end". There is this Pareto principle that apply a lot to CSS in my opinion. It means that in just two hours, I had a decent design, but in the and, it costs me a full day to have the final thing. Why? Because after having done the main style, you will start testing on other browsers, on other devices, find small bugs, and spend way too much time to correct them. Also, at some point, if you can, you should review the website with the designer, and this is vital but sometime really hard. He will probably want to reach pixel perfection and nobody can blame him for that, take time and probably some refactoring, and also some hacks. At the end, you will have spent 20% of your time having a nice (but not perfect) design on Chrome, and 80% having a pixel perfect design on most modern browsers, being responsive, and having a decent fallback on old ones.

It was a pretty simple design so it was quite straightforward. Just remember you can do magic with pseudo-elements. I use them a lot for all those forms (circles, traits, …). Flexbox is doing an awesome job to overcome old CSS limitations (Who said vertical align?) and being responsive. This way, I didn't have do use any open-source CSS project except for my reset, using the awesome normalize.css.

I also decided to use pure SVG rather than a font for my icons. This decision was based on reading great articles from Chris Coyier, like this one, that one and finally this battle one. All worked fine.

Finish the script

During the last half day, I focused on writing the small JavaScript needed for the website. There really wasn't much of it: double sticky navigation bar, nice scrolling between sections and modal panel for each talk description. Let's be honest, since I am targeting IE10+, I totally didn't need jQuery. But if you check the source code, you will find it. Why? I was starting to run out of time and I really wanted to use some nice plugins to leverage all the grunt work. Also, using one of the last version of jQuery, it will be fast enough and will not add too much of a payload. So… it's fine.

I finally had the chance to use Velocity, a jQuery plugin for animations. It works just great, super fast even on mobile. You should try it. It powers both the scrolling and the modal animation. For the stick navs, I choose the Sticky plugin which is simple and great. Nothing more to add. I packaged all that stuff with Browserify, but more because I love this tool rather than because it was needed.

Going further

So, why didn't it work to only push to gh-pages? It did at first, but then I started to use more complex features with Jekyll 2.x and, guess what? GitHub Pages still use a 1.x Jekyll version, so it didn't match. Also, I wanted to do post generation stuff, like usemin to minify all my assets and append then with a hash version, also updating the index HTML file. I just had to wrote a simple deploy script in order to overcome the problem.

I don't really want to go through all the details, but you can see the final result, read the full source code, and here are a few highlights:

integrate a nice Google maps based on Snazzy Maps design.
Diplay and hide the modal for talk descriptions
Package SVG icons, enjoy the result, use them in HTML and style them.
Do crazy templating madness

Hope you liked it! Thanks for reading.

Prismic.io JavaScript kit: from callbacks to promises

2015-06-06T15:40:57+02:00

Holy cow, callbacks!

For the last (and only) website I did using prismic.io, I didn't want any back-end, all client-side, pure JavaScript man. This way, I could host it nearly anywhere for pretty much free (GitHub pages FTW). To ease stuff, I decided to use the JavaScript kit from the prismic.io team.

One choice they made and that I don't really like is using callbacks in it. For example, you have to write stuff like:

// Get the current API
      Prismic.Api('/my/awesome/url', function (err, api) {
        if (err) {
          // handle error
          return;
        }
      
        // Use the API there, retrieving documents
        api.form('articles').ref(api.master()).submit(function (err, articles) {
          if (err) {
            // Also error again
            return;
          }
      
          // But I also need authors...
          api.form('authors').ref(api.master()).submit(function (err, authors) {
            if (err) {
              // One more time, handle error
              return;
            }
      
            // Now I got all my data
            // I can finally display it on my page
          }
        });
      }, 'my_token_far_below_at_the_end');

Can you see it? This ugly thing JavaScript experts name "callback hell"? And that's a super simple example. In 2014, the way to do that is using Promises. That's what's hype (and readable, and nice, and cool, and swag). But I quite understand why they made this choice. Promises are not native everywhere yet, so going with it mean either writing your own implementation or imposing an existing one. Fair enough to have callbacks in the kit then, but we are hackers here, we can override it to use Promises!

I will use the last spec Promise syntax, native way baby!, but it is really important to keep in mind that it has a really poor support right now. You should probably be using a Javascript library as Promise implementation. I will link a snippet of code using the Q library at the end of the article. But I wanted this article to be future proof.

OMG, promises \o/

There are two main callbacks in the kit: when retrieving the API and when making a submit. What I want to write is something like:

Api('/my/awesome/url', 'my_token_right_here_at_the_start')
        .then(function (api) {
          return Promise.all([
            api.form('articles').ref(api.master()).submit(),
            api.form('authors').ref(api.master()).submit()
          ]);
        })
        .then(function (data) {
          var articles = data[0];
          var authors = data[1];
          // Display the results
        })
        .catch(function (error) {
          // Handle error
        });

Sooooo much more readable and swag… Overriding the Api callback is really simple:

// Remove the callback from the function signature
      function Api(url, accessToken, maybeRequestHandler, maybeApiCache) {
        return new Promise(function (resolve, reject) {
          // Put a callback that will only resolve/reject the deferred depending
          // on the presence of an error or not
          Prismic.Api(url, function (err, resolvedApi) {
            if (err) {
              reject(err);
            } else {
              // Save point 1 (see below)
              resolve(resolvedApi);
            }
          }, accessToken, maybeRequestHandler, maybeApiCache);
        });
      }

Overriding the submit call is a bit more tricky because it is a prototype function of the SearchForm object, which is totally hidden inside Prismic clojure. This is a bit (super) sad but we can deal with it by creating a fake form and not submitting it. The easiest way I found so far is doing it right after getting the Api. Let's add the following code at Save point 1 (see previous code).

// You might need a polyfill for 'getPrototypeOf' if you support old browser
      var SearchFormProto = Object.getPrototypeOf(resolvedApi.forms('everything'));
      // We are saving the original function so we can call it inside our override
      var originalSubmit = SearchFormProto.submit;
      
      // Let's override!
      SearchFormProto.submit = function () {
        return new Promise(function (resolve, reject) {
          // As before, the callback will only resolve / reject the deferred
          // depending on the returned value
          originalSubmit.call(this, function (err, docs) {
            if (err) {
              reject(err);
            } else {
              resolve(docs);
            }
          });
        });
      };

And after that, you just need to use the new Api function in place of Prismic.Api. You can see the full code here using Q library. This refers to a particular commit to ensure the line highlight is correct but you can freely switch to the master branch to see the most recent code (which shouldn't be much different… but who knows…).

Thanks for reading. Next time we will talk about responsive images inside StructuredText.

Prismic.io: when happiness met disappointment

2015-06-06T15:40:57+02:00

TL;DR

If you don't know about prismic.io, it's a nice tool providing both a super clean web interface to write content and an API to retrieve it. No front-end provided, eventually some kits to help you and some examples, but no more. Which is cool because, if you are a web developer, you can focus on crafting an awesome optimized website and then easily put some content in it… well, that's the theory…

But for me, after using it for a new website, as good as the idea can be, I have found way too much limitations to consider it production ready. The team behind prismic.io is working on most of them so I really hope the implementation will become as awesome as its concept at some point in the future.

The rest of the article will focus on bashing all those limitations, so there will be way more bad than good, but don't get me wrong, prismic.io has tons of good stuff, and it's still in beta, I just want to talk about what I don't like in this article. And since I'm not all about trolling, I will try to consider how to solve them at the end. I will not explain what prismic.io is exactly, so it might be for the best that you quickly check their website before continuing reading. Enjoy.

Do you remember how happy we were at the beginning?

I will never forget… it all started on this windy saturday… I was looking for a solution to create a new blog about traveling. I didn't want to use an all-in-one tool because I love crafting my UI from scratch, with exactly what I want on it (this point is super important for the rest of the story). I said bybye to Wordpress and all its friends. If I were alone, I would have probably go with raw Markdown or whatever, but I am not… I have other people that travel with me and want to write articles without learning a super complex geek syntax. I will call them… writers. Some colleagues of mine told me about you… that you might be the future of content management, that I would be free for the UI, that you had WISIWIG editors and stuff for… writers, sounds like a perfect match, and so our adventure began.

I still can't believe how fast it all went… in a couple of hours, we were able to write new articles and display them in a custom website hosted in GitHub pages. Using one of your kits, it was so easy to query your API, retrieve the content as JSON and inject it inside Handlebars templates in order to render the final HTML. Oh man… it was so good.

And then fictions appeared

A few days later, I wanted to go further, moving forward to more crazy stuff. I know that sometimes I try to do complex features just for the fun of the challenge it provides me, but still, how could you do that to me… Remember this conversation?

Hey prismic.io, I would like to query documents that does not match this predicate
Ahahah, you so funny, no way…
Thanks! Wait… what? Why not?
I don't have a NO operator or whatever close to it.
Whyyyyyy? You have crazy operators like similar to retrieve similar documents based on God know which algorithm and you cannot do a simple negation?
Nope. Deal with it.
Okay… I think I can manage something if the query match this OR that. That's fine, right?
No.
Cool! Wait… what?
I only have AND operators between predicates.
You gotta be kidding me…

I was a bit sad, but I could manage it, retrieving too much documents and filtering client-side. Still, that was the starting point were all went wrong.

My biggest mistake: using StructuredText

All contents in prismic.io must be typed. This is an image, this is a number, etc… A StructuredText is probably the most powerful type since it allows writers to enter complex content through a WISIWIG editor, using paragraphes, images, lists, embedded stuff, … They can style it using bold, italic, and so on. It looks nice, right? So I decided to use it for the main content of my articles. Fair enough. As always, it was so great to have writers doing their own design using the editor, and so easy to render it automatically as HTML (except for some bugs here and there, but that's what pull requests are for, isn't it?). At some point, I just wanted to have a lightbox on some images: clicking on them should expand it to its full sized version with a caption. I don't know for you, but for me, this is a really basic requirement and you should be able to do it in less than 2 hours.

First problem: I wanted to tag images that should have lightbox with [data-lightbox] HTML attribute, all images didn't need one. Since I was using the JavaScript kit to render my HTML, I couldn't do that at all. There is no way to add custom rendering or plugins in the HTML renderer from the kit. Dammit. So I rewrote the full renderer so I could customize it… Wait, how can I indicate which image has a lightbox and which hasn't inside a StructuredText? Oh, right, I just can't. Inside a StructuredText, you can only drop raw images and resize them, that's it. Does that mean I cannot put nor caption or title? Yep, no way to do it. So I started to hack: if after an image, the following paragraph would begin with [caption], the image would be a lightbox, the content of the paragraph would be its legend, and the paragraph itself would be removed from the final rendering.

Guess what? Writers didn't really like having to manually write [caption], they wanted a nice UI to do that… but that was just impossible… They throw some tomatoes at me and we tried to move on but we couldn't… Solving one problem to discover a new one: when you define a type image in a Document, you can tell that it will have different sizes: one for desktop, one for tablet and one for smartphone for example. And that's awesome because you can fully optimize for mobile, which is super important nowadays. Do you think you can do that with image inside StructuredText? Indeed, the answer is no! Once again, a great idea not fully supported. Is there a workaround? Sadly, no easy one. You can upload the same image several times, one for each size, then find a way to retrieve its generated ID by prismic.io and use the same hack as [caption] to specify it and hack a bit more the HTML renderer. I couldn't find the courage to do it… pretty sure tomatoes would have been replaced by rocks from writers. I can only hope my mobile users have 4G now.

Want more? No biggy, I have tons of stories like that. A writer wanted to have blockquotes: a whole paragraph should be displayed in a custom design and have an author. I had to kill him really fast and bury his body deep. Another one wanted semantic distinction between paragraphs, something like: this one should be red and this one blue just because. Thrown him into a bucket full of piranhas. At some point, a writer became a madman and deleted half of documents… no way to find him since the delete operation does not appear in the timeline. But who cares? I mean, the document are gone anyway. I18N, do you speak it? I hope not because there is no support for it… yet. Want to put some tables? Could you please stop joking, it hurts me to laugh so much. Want do manage hundred of images in the media library? Might be less painful to jump from the top of the Eiffel tower. And so on, and so on…

We didn't fully link anymore

Writers are generally really proud of their work, so they want everybody to know about it. I couldn't just display "Title of the article". If I wanted to stay alive, I needed to display "Title of the article, by an awesome author", and if you would click on the name, you would access to a page saying how incredible the author was. As always, I was like "No biggy dudes, this is gonna be easy". I wasn't bluffing at that point since there is this nice feature that allow you to link documents between them. So every author would now have a dedicated document of type Author, where he could write whatever description he wanted, and each document of type Article would be linked to an Author. So far, so good.

And then the problem: if I get a list of Articles, I only got ids and a link to the Author resources, but I do not actually have the data on the Author document, which means I cannot display the real name of the author. And guess what? There is no aggregation or join possible in the API. You just cannot say that you want the 20 most recent articles with their authors. Solutions are: request all authors and then retrieve the good ones from ids (that's 2 requests, and I did that since I don't have that many authors), or do N+1 request (1 for articles, N for authors, one per author id from articles), or generate dynamically a request for authors using any operator (something like [:d = any(my.author.id, [id1, id2, ...])]). In any case, you need multiple requests, eventually not running in parallel, and with too much payload… mobile first, right?

I never though you would mask it from me…

One core concept in prismic.io is Document Mask. Each document must have one and it represents its pattern, how it will be persisted in prismic.io database and how it will be served by the API. So you start creating masks, easy. It directly provides a slick UI for writers to enter content. Nice! And at some point, you realize you could have done better, the mask can be improved. Just don't do that, keep going with your broken mask, trust me!

Let's say you don't trust me (that's fair enough, you don't even really know me). If you rename any property of the mask, you are screwed. The new property will be empty since it doesn't consider it a renaming but the suppression of the previous property and the creation of a new empty one. But you are super courageous, so you migrate all your documents to the new property, yeah! Now you have duplicate a property in your API. Hell yeah! Just check the raw JSON, you will see the new property (thanks God) but also the old one. Properties just keep adding, prismic.io never forget… never! Your payload will just increase. And be sure that your front-end developers always only rely on the mask definition, never on the data from the API, because… it doesn't mean anything anymore.

My heart was so fragmented…

So, let's summarize what's the biggest problem with prismic.io: you can only do what the tool allow you to do. And right now, it allows you only super basic features. Which are needed obviously, but when you go to the real world, you know, outside of the matrix, those are just not enough. Imagine being Mario, you need to save the princess but you can only move forward, no jump, no strange mushroom, no yoshi… Good luck! Direct consequence, the killer feature that would make prismic.io really powerful, in my humble opinion, is what I will call "custom fragments". The idea is to allow developers to create their own type of fragments and plug them both in the UI of the writing room and in the HTML renderer. That might sounds a bit challenging but it isn't that hard in fact.

So first, how to create a custom fragment? Let's use the mask system. When defining a document, you are creating a mask saying "this document will have an image and some text as title". Why not doing the same for custom fragments? Remember the lightbox problem? I would create a custom fragment lightbox which is composed of an image and some text as its caption. Even better, since I am now using a real image type, I can specify several sizes for the image. Big win here.

Next, inside the writing room, how to render a custom fragments? Well, you already know how to render all its pieces, just group them together with an indentation or a left border or whatever. The lightbox fragment would be an image selector and a text input. Of course, you should be able to put a custom fragment both inside a document (like any other fragment) and inside a StructuredText if possible (just like you are allowing em or h2 inside a StructuredText)

// Let's define our custom fragment for lightbox,
      // just the same syntax as a Document Mask
      {
        "title" : {
          "type" : "Text",
          "fieldset" : "Title",
          "config" : {
            "placeholder" : "A short caption for the image"
          }
        },
        "image": {
          "fieldset" : "Image",
          "type" : "Image",
          "config" : {
            "thumbnails" : [ {
              "name" : "Icon",
              "width" : 100,
              "height" : 100
            }, {
              "name" : "Column",
              "width" : 320,
              "height" : 320
            }, {
              "name" : "Wide",
              "width" : 600,
              "height" : 300
            } ]
          }
        }
      }

Finally, each kit should have a renderFragment(type, json => html) method that allow you, for a specific fragment type (here lightbox), to pass a function that know how to render the final HTML from its raw JSON.

// Define how to render your custom fragment
      // from JSON to HTML (which is a String)
      kit.renderFragment('lightbox', function (json) {
        var html = '<img src="';
        html += json.image.url;
        html += '" data-lightbox title="';
        html += json.title;
        html += '">';
        return html;
      });

Hey, we can do even better, let's add a second argument to the rendering function, this one will be the original rendering function for "native" fragments in prismic.io. This way, you can override just one or two "native" fragments. For example, if you want to render image with a <picture> HTML tag because that's super hype. Just do something like:

// Override an already existing fragment
      kit.renderFragment('image', function (json, original) {
        var html = '';
        html += '<picture>';
        // Keep it easy, only one source...
        html += '<source src="' + json.image.url + '">'
        // Fallback using the default renderer
        html += original(json);
        // And the alt text
        html += '<p>' + json.image.alt + '</p>'
        html += '</picture>';
        return html;
      });

And that's it! You just solved so many limitations…

I didn't understand your semantic anymore…

Ok, next step. Inside a StructuredText, I need to have semantic distinction between fields of the same type. This paragraph is important, this one is a blockquote and all the rest are normal. Let's go the easy way: I just need a hash of key -> value, as strings, to reach a limitless world. This way, I could tag one paragraphe important: true, and another with with both blockquote: true and author: Someone. At last but not least, expose it in the API:

// Since all fields have a common structure to expose their type
      // it's easy to have a new key like 'attributes' at this level
      // to expose the semantic hash
      // (btw, it would be cool to map to primitives rather than only string for values)
      {
        "type": "paragraph",
        "text": "bla bla bla...",
        "spans": [],
        "attributes": {
          "blockquote": true,
          "author": "Someone"
        }
      }

I have no idea how to integrate that in the UI of the writing room, but I know its designer, he will figure out something awesome, no doubt.

By adding that, you leverage tons of possibilities for developers to customize the rendering of a StructuredText without any heavy hack. Writers would still need to manually write each key name, but its way better than paragraphs starting with [caption] or [author]. Following a snippet on how to render a blockquote, using the previous renderFragment method of course:

kit.renderFragment('paragraph', function (json, original) {
        if (json.attributes.blockquote) {
          var html = '<blockquote>';
          // Previously private (at least in the JavaScript kit),
          // I hope this method will be available in all kits
          // it applies all span styles to the raw text
          html += kit.insertSpans(json.text, json.spans);
          html += '<footer>';
          html += json.attributes.author;
          html += '</footer>';
          html += '</blockquote>';
          return html;
        } else {
          return original(json);
        }
      });

This feature is for "quick and dirty" hacks if you could say, and allowing you to customize "native" fragments. Because otherwise, you could also have done that with a custom fragment. Rather than having two attributes, a blockquote could be a custom fragment grouping a StructuredText for the content and a simple text for the author. It is way more intuitive in the UI and it's easier to create its own renderFragment.

I like a lot this renderFragment method, but I think we can make it even better! Right know, we can only specify / override for all the application. That's super cool but limited… and if you have read carefully until now, you should know that I don't like limitations. This method should be at least at 3 different levels:

Document instance
Document type
(Collection?)
Application

Here is the idea: when an instance of a Document wants to render some fields (or fragments) as HTML, it will do the following simple procedure:

Hey, do I have a custom renderer method for this type of field on myself? (yeah, because now each Document instance have the method). If so, use it, if not, continue.
I am a Document of type Article, does this type of Document has such a method on it? (yeah, because now you can call this method for just a particular type of Document). Yes, use, no continue.
Eventually, does my Collection have the method?
Nearly finished, does the method is defined at the application level?
If nothing found so far, if it is a native prismic.io field, render it as it should be, otherwise just crash or return empty string, I don't know, it shouldn't happen anyway.

Holy crap… we just created a "rendering context with awesomeness inheritance" or something like that. Can you imagine all the things you could do with that? Can you ?! I can't… it's too big…

Let's break up

I could say more, but since it's a miracle that you are still reading and it would not be as relevant as what I already said, I will stop here.

My conclusion so far: prismic.io is not ready for the real world and for real websites but there is a lot of work going on by its team so, as they say on half of their responses: stay tuned! It might become super awesome at some point (I hope it will).

Thanks for reading! See you soon for more hacky coding posts.

AngularJS HTML5 routing on PlayFramework

2015-06-06T15:40:57+02:00

The problem

AngularJS provides a nice $location service which allows you to deal with routing and url inside your application. It supports both a fallback mode using hashbang and a real HTML5 routing using the new History API. Since we love bleeding edge technologies, that's obvious we want to use the History API if possible. Lucky us, it appears it's quite easy to do so with a Play Framework application. Let's see how to do that.

The full code of the demo is available on GitHub.

Simple configuration

Let's keep things simple. Creating a new Play Framework application with play new. We will use a Scala application in this article, but it should as easy with a Java one. Then add AngularJS using CDN (see main.scala.html#L19). Then we will create an Angular module for our app and enable the HTML5 routing mode which is disable by default.

var app = angular.module("app", ["ngResource"])
        .config(["$routeProvider", function($routeProvider) {
            return $routeProvider.when("/", {
              templateUrl: "/views/index",
              controller: "AppCtrl"
            }).otherwise({
              redirectTo: "/"
            });
          }
        ])
        .config(["$locationProvider", function($locationProvider) {
            return $locationProvider.html5Mode(true).hashPrefix("!");
          }
        ]);

Nice. Now we will make some modification to the default views generated by Play. We want a single-page application, which means our root url must map to a main page which will contains all resource dependencies and includes the ng-view of AngularJS. I've done that by editing the main.scala.html file as you can see here and add a new Play route:

GET     /                           controllers.Application.main(any = "none")

And a new Action in my Application controller:

def main(any: String) = Action {
        Ok(views.html.main())
      }

Wait. Why is there a String param for the main method? And why do we pass a default any = "none" value? It seems a bit useless… Fair enough, I will explain it later on the article, don't mind it for now. So, we have our main page. Let's add some content for the landing page. As we have specified in the AngularJS routing, our root url, which is defined with $routeProvider.when("/", ...) is pointing to /views/index. So we need a Play route for that… but before, let's talk about all those routes.

Routes, routes, routes…

As we advance in our coding, we can see that the term "route" is used in different contexts. It's important to fully understand them all. Play routes, which are defined in the conf/routes file, are the core of your application. It's those routes which will answer to the HTTP requests, serving either HTML, JSON, or whatever. But you, as the developer, will be the only one to know about them. Users of your application will only use and see AngularJS routes, which are defined in your main module using the $routeProvider.

The idea is that when a user land on your site, it should use AngularJS routing which will then use Play routes to load the template (= the content) of the page. So why not using the exact same url for both the AngularJS route and the templateUrl route? Well, in the case of the root, it's obvious: the Play "/" route is already required for our main page. Ok. But in a more general case, could we do:

$routeProvider.when("/page", {
        templateUrl: "/page",
        controller: "PageCtrl"
      })

Nope, you can't. Well, that's a small lie… you could do that, and it would work if your user always open your main page first. If he does that, then all AngularJS routing is loaded, and when the user will go to "/page" url, it will be catch by AngularJS which will ask for the "/page" on Play routes which will give him the content of the page. So what's the problem? If you user directly land on the "/page" url, because he arrived for the first time on your application after having clicked on a link somewhere which was pointing at "/page", then the Play route will be directly called since there is no AngularJS routing initialized at all… And that's bad, right? The user would have the raw content of the page without any JavaScript or CSS from the main page. It's a fail.

In order to prevent that, I have chosen to use some conventions. There is nothing official, you can use your own, the only thing to keep in mind is that AngularJS routes and Play routes should never be equals. My solution to do that is to prefix all Play routes serving HTML with a "/views/" prefix, all Play routes serving JSON with a "/api/" prefix and, as it is by default, all Play routes serving resources with a "/assets/" prefix. Meaning I can now use all routes I want inside AngularJS routing but never starts them with one of those prefix.

Of course, it will just modify the problem without solving it. Now, if a user land directly on "/page", he will have a huge orange Play error telling him that the route doesn't exist. As before, AngularJS routing isn't loaded, so it's Play routing which try to handle that and, since there is no longer any "/page" route in Play, it crash.

So what's the point of having different routes? The next trick is to redirect all unknow routes to our main page. By doing so, if the user land on "/page", Play will say "I don't know that route, go to the main page", so the user will indeed go to your main page, but then, all AngularJS routing will be loaded and super awesome AngularJS will say "Hey, I do know that route, and I need the /views/page template", and then Play will respond "Oh yeah, I can serve you /views/page, here is the HTML to display", and your user will see the correct "/page1" content, even if he lands directly on it.

Another question? Why didn't we use this trick before and then have the same routes for both AngularJS and Play? Because it's important that the trick apply only to unknow routes. We cannot redirect to the main page an existing Play route because it needs to serve its own content. So by using different routes, we can intercept AngularJS routes inside Play as "unknown routes" and redirect them to the main page so AngularJS can handle them. I hope I'm clear enough here, that might be the hardest part of the article. In order to achieve that, we just need to put a Play route at the end of the route file which intercept all routes and redirect them to the main page:

GET     /*any                       controllers.Application.main(any)

And now you should understand why we had a any: String for our main page at the beginning.

First pages

Enough with all the theory, let's do two concrete pages: creating "page1.scala.html" and "page2.scala.html" files with basic content (see source code, I'm using a template to stay DRY about the title and the menu but that's not important), add then as Play routes (with a "/views/" as stated before) and also add them inside our controller:

@()
      
      @template() {
        <h2>Page1</h2>
      }

GET     /views/page1                controllers.Application.page1

def page1 = Action {
        Ok(views.html.page1())
      }

And then add the AngularJS routing:

.when("/page1", {
        templateUrl: "/views/page1"
      })

That's it. We can now create a link, using a a tag, to point to the url "/page1" and AngularJS will do the rest, loading the "/views/page1" template using Play route which will return the HTML code generated from our Scala template. Nice isn't it? The url is correct: http://localhost:9000/page1, you can use the back button to return to the index page, no ugly hashbang, but all in Ajax with a single-page application.

Handling url parameters

So, we have our url working by using the History API thanks to the HTML5 mode of the AngularJS $routeProvider. But that's only for raw url. What if we want to have dynamic parameters inside them? Just like REST, having url as "/colors/1" and "/colors/2" if we deal with colors (yeah, why not?). Do you think that's easy? Well, you could, after all, both AngularJS and Play know how to handle parameters in their url. We could do something like:

.when("/colors/:id", {
        templateUrl: "/views/color/:id",
        controller: "ColorCtrl"
      })

GET     /views/color/:id             controllers.Application.color(id: String)

Nope, we can't. Why? Because the "templateUrl" cannot handle url parameters. It's just a template. And of course, using templateUrl: "/views/color" will not pass any "id" parameter to the Play route. Damn…

That's not a problem in fact. Just keep to the AngularJS way of doing things: it's not Play who should handle the data any longuer, it's AngularJS who rules the world now, so you don't need any params in your Scala code, just trust AngularJS. Ok, but we need a way to load the correct color depending on the id in the url right? Sure thing. It's the role of AngularJS controller to that. That's why there is a "ColorCtrl" along with our template, and guess what, the AngularJS controller know about the url params. Here is the code of the controller:

app.controller("ColorCtrl", ["$scope", "$routeParams", function($scope, $routeParams) {
        // Thanks to scope inheritance, we can access the "db" from the AppCtrl scope
        $scope.color = $scope.db[$routeParams.id];
        if (!$scope.color) {
          $scope.msg = "There is no color for id "+$routeParams.id;
        } else {
          $scope.msg = undefined;
        }
      }])

And just for fun, I've defined the fake database in the "AppCtrl":

app.controller("AppCtrl", ["$scope", function($scope) {
        $scope.db = {
          1: {
            name: "black",
            hex: "000000"
          },
          2: {
            name: "white",
            hex: "FFFFFF"
          }
        };
      }]);

So as we can see, using the $routeParams service, we can retrieve the url params and then load the correct color in the $scope. At the end, it's just a matter of displaying that color in our view using AngularJS data-binding:

@()
      
      @template() {
        <div data-ng-show="msg">
          <h2>{{msg}}</h2>
        </div>
      
        <div data-ng-hide="msg">
          <h2>Color {{color.name}}: # {{color.hex}}</h2>
        </div>
      }

And that's it! you can now display some data depending of params inside your url (and show an error message in case the data doesn't exist). Cool isn't it?

Want more?

Oh God, you are still reading? You should already be able to do anything you need. But if you want, we can dive in a more complex example, using AngularJS $resource service and serve JSON from Play like a REST API would do. Sounds good for you? Ok, let's do that. First, we will create a "Users" controller and it will have two methods: "all()" and "find(id: String)" which will return an Array of Json and a Json object representing our list of users and one particular user based on its id. I will not use a real database, but something like MongoDB would fit really good in there. Here is the code:

package controllers
      
      import play.api._
      import play.api.mvc._
      import play.api.libs.json._
      
      object Users extends Controller {
        val db = Json.arr(
          Json.obj( "id" -> "1", "name" -> "John" ),
          Json.obj( "id" -> "2", "name" -> "Suzanne" )
        )
      
        def all() = Action {
          Ok(db)
        }
      
        def find(id: String) = Action {
          Ok(db.value.filter(v => (v \ "id").as[JsString].value == id).headOption.getOrElse(new JsUndefined("")))
        }
      }

If you don't fully understand it, you can check the PlayFramework documentation about handling Json (ScalaJsonRequests). I am using the new Json API from Play 2.1 (play.api.libs.json.package). Next we need our "UserCtrl" which will use the `$resource</code> service to retrieve the data:

app.controller("UserCtrl", ["$scope", "$routeParams", "$resource", "apiUrl", function($scope, $routeParams, $resource, apiUrl) {
        var Users = $resource(apiUrl + "/users/:id", {id:"@id"});
      
        if($routeParams.id) {
          $scope.user = Users.get({id: $routeParams.id}, function() {
            if (!$scope.user.id) {
              $scope.msg = "There is no user for id "+$routeParams.id;
            } else {
              $scope.msg = undefined;
            }
          });
        } else {
          $scope.users = Users.query();
        }
      }])

To learn more about AngularJS $resource definition, see: ngResource.$resource. What is that "apiUrl" by the way? It's a constant I've defined in my AngularJS app:

var app = angular.module("app", ["ngResource"])
        .constant("apiUrl", "http://localhost:9000\:9000/api")

Pro tip Why is there the 9000 port twice? That's because if we had written http://localhost:9000/api, AngularJS syntax would have analyzed that as a url with a dynamic parameter named "9000" because it's placed right after a : character. So we need that strange syntax to tell AngularJS that this is not a parameter but a real value in our url.

Next, we are creating our resource by extending this apiUrl with our routing "/users/:id". This time, ":id" is a real parameter. We can now use "get" and "query" methods on the resource, passing or not a value to the id, in order to retrieve our Json code and assign it to the $scope. We will need two Scala views: one for the list and one for the detail of course.

@()
      
      @template() {
        <h2>Users</h2>
      
        <ul>
          <li data-ng-repeat="u in users">
            <a data-ng-href="/users/{{u.id}}">User# {{u.id}}: {{u.name}}</a>
          </li>
        </ul>
      }

@()
      
      @template() {
        <div data-ng-show="msg">
          <h2>{{msg}}</h2>
        </div>
      
        <div data-ng-hide="msg">
          <h2>User# {{user.id}} {{user.name}}</h2>
        </div>
      }

And that's it. We now have a list of users and links to each user detail, and all that data is fetch from a REST API using Json.

Pro tip By the way, since our Play routes doesn't clash with AngularJS routes, you can load them directly in your browser. It will works because the Play route handling the redirection to the main page is at the end of the route file, so any real Play route will be loaded before the redirection. If you have the demo running, check http://localhost:9000/api/users and http://localhost:9000/api/users/1 to see your REST API working nicely.

Conclusion

I hope this article will help you bootstrap your next awesome application using awesome tools like AngularJS and PlayFramework. Your next step (if that's not already the case) would be to learn more about the new PlayFramework Json API so you can have typesafe Json (if I can say so). You would also probably need to plug a database on it. One choice could be a MongDB database since it can store raw Json, and interact with it using a nice driver like ReactiveMongo if you want to go all the way down to asynchronous application.

It's up to you!

Some POCs to share

2015-06-06T15:40:57+02:00

Some POCs

Hi there,

I've been a bit busy lately but I'm back on game. Looking for a job in Paris, I've been asked to do some code over more or less funny subjects and now that it's done, I will share them, maybe it could help people to see some code around some technologies. It's not at all perfect code, more experiments I've done.

Mowing is mainly a CoffeeScript project about giving orders to some automatic mowers and see the result. My goal was to test that new syntax around JavaScript and I have to admit it's quite cool. You can see the result here (don't ever do it in large grids). The source code is here on GitHub

Mowing-java is the Java version of Mowing. So in fact, it's nearly the same syntax since you can create class in CoffeeScript but this one runs on a JVM. See here for source code.

Finally, GSearch is an AngularJS application built using Yeoman. Its goal is to plug on GitHub API and allow you to search through repositories and users and display informations and stats about them. Here is the result and go there for source code.

Of course, any feedback is welcome! I can be bugs, features, questions, anything…

NigthHacking Tour 2012

I've given a small quicky about RichFaces Bootstrap and RichFaces 5 on Paris a few weeks ago. Just so you know…

Are you in Paris?

Let's finish with a personal note. I will be living in Paris from now and for quite some time. So if you living there or just passing by and want to grab a beer and talk about Java EE, HTML5, RichFaces, or anything cool, just let me now! Also if you are giving conferences or doing dojos or katas in the area, be sure to notice me so I can come to the next one. Thanks.

News about RichFaces Bootstrap

2015-06-06T15:40:57+02:00

What's new in RichFaces Bootstrap?

Red Alert! The RichFaces Bootstrap project is still under heavy development, tag and attribute names can change at any time and if you find something missing or buggy, there is high chance it's not a bug but just hasn't been done yet (… or it's a real bug). So only use it for fun and prototype purposes.

Semantic components are a brand new concept. I need to present it first so I can use it when talking next about the new components. Most of the time, for one JSF tag, you have one and only one HTML renderer, it's a oneToOne relation (according to JSF naming convention). It seems logical: for one component, you should always have (nearly) the same generated HTML code. But with HTML5, the web is becoming more semantic and that's good. Why not have the same with RichFaces? But before we dive into it, what is "semantic"? For a real definition see Wikipedia but here, we will say it's when a component serves a global purpose like being a header or a footer but shouldn't always render in the same way, instead the rendered result should depend on the context (like a table header isn't the same as a column header).

With RichFaces, a semantic component is a component with no renderer! Yeah yeah, if you use a semantic component on its own, it will throw an exception because it doesn't know how to render itself. The concept is that a semantic component will ask its parent in the JSF tree: "Hey dad, do you know how I can render myself?", if it knows, the parent will provide the correct renderer to the semantic component which will render it, otherwise, the semantic component will ask one level higher the same question, and so on until the root element. If no one answers yes to the question, it will throw an exception. That means we also have components that accept semantic components in order to provide them the correct renderer.

Let's take a concrete example. The modal component in RichFaces Bootstrap can support 3 semantic components: a header, a body and a footer. You can see that by looking at which interfaces the AbstractModal implements (see GitHub). All interfaces with the syntax Render{1}Capable are components that support the semantic component {1}, so here, it's headerFacet, bodyFacet and footerFacet. And if you look at the rest of the code of the AbstractModal, you will see methods with the syntax public String get{1}RendererType() which are the methods giving the right renderer to use by semantic components. Inside a modal, the headerFacet component will render as a div class="modal-header" according to the renderer provided by the modal. But inside another component, it could have been totally different HTML code.

Pro tip If you take a more accurate look to all current semantic components, you will see that they all follow the same syntax: first their purpose (like header or footer) and then a generic Facet suffix. That give us the full list of semantic components: headerFacet, bodyFacet, footerFacet, menuFacet and positionFacet. Why using a suffix? Because we want to keep the no-suffix name for the real HTML tag. The RichFaces header tag will always generate the HTML header tag like a classic JSF tag (the oneToOne relation) but the headerFacet tag is a semantic component so it can generate anything depending of the context. Also, it makes it easier to see if a RichFaces component is a semantic one or not by just looking at its name.

Warning Even if the suffix is Facet keep in mind that semantic components are not facets, they don't have the same limitations: they have attributes, you can use the same semantic component several time inside the same parent, a semantic component doesn't have to be a direct child of a component supporting it and a semantic component can have several children.

Now, let's dive into the new components. input tag is a basic input with all Bootstrap features like prepend and append and several new attributes in order to support new HTML features. By the way, this input centralizes and supports all HTML5 input types, you will no longer need one JSF component for each type, just use the type attribute (default is "text" of course).

modal is like rich:modalPanel or rich:popupPanel, it will display a popup layout on top of your page, potentially covering your page with a dark layer to block any action outside of the modal. The default usage of the modal has a header, specified by using header attribute or header facet, a footer specified with footer facet and a body which will be the code inside the modal component. Concretely, what's happening is that each part, header - body - footer, will be wrapped inside a div class="modal-xxx" where "xxx" is the name of the section in order to align with the Bootstrap syntax. Using footer as a facet might be limiting because you cannot have a form inside the modal wrapping both body and footer because footer, as a facet, will always be outside. The default usage should be the modal inside the form since you will probably not need several forms inside a modal most of the time.

What if that isn't enough? What if you do want the form inside and not outside? Even if this behavior should be enough in most of use-cases, you can still fully customize your modal the way you want! The moment you use one of the following semantic components, headerFacet - bodyFacet - footerFacet, it considers you are doing a custom modal and it will not generate the div class="modal-xxx" anymore. I'm talking about the modal itself. Because the semantic components will generate the corresponding div. However using real components and not facets will allow you to put both bodyFacet and footerFacet inside a form for example.

tooltip and popover are two new ways to display bonus info when the mouse moves over particular content. The first one is for small texts and labels only, the second one can support custom content and a title. Right now, content can only be text but we are planning to improve this.

Pro tip Even if popover content only supports text, you can still put some light HTML in it. You just need to escape chevrons with < and >.

Finally, "orderingList":http://bootstrap-richfaces.rhcloud.com/component/orderingList/ is the new RichFaces ordering list to allow you to re-arrange the order of some items. It already supports single drag-and-drop, multiple selection and "table" layout. Next features will be multiple drag-and-drop, maybe keyboard selection using SHIFT (CTRL is already supported). Thanks to Brian work, it is the first component mixing the power of the jQuery UI widget factory with Bootstrap design. If we can do it once, we can do it for lots of other widgets!

New EL functions are also there. jQuery and jQuerySelector are part of RichFaces Core but have been created to help RichFaces Bootstrap. They will allow you to retrieve a jQuery object or the jQuery selector from a server-side JSF id.

Next are more specific Bootstrap EL functions. If you have take a look at Bootstrap JavaScript API, you might have noticed that lots of JavaScript components have a set of functions with the following syntax: $(sel).compName('singleParameter');, like for example: $('#myModal').modal('show'); for a modal. The first approach to use it in JSF component was to use the previous jQuery function like: onclick="#{rich:jQuery( 'myModal' )}.modal('show')". It works fine but obviously, that wasn't enough for Lukas since he built a different approach from scratch.

The new concept is that the compName part of the call is nearly useless if you can retrieve it from the component returned from the jQuery call. In the previous example, if you know that #{rich:jQuery('myModal')} is actually a modal component, then you also know that you will have to call the modal function, only remains with importance the singleParameter. So here is the new syntax: #{b:singleParameter(sel)}. The previous example becomes: onclick="#{b:show('myModal')}". Much more concise and readable, isn't it? Right now, RichFaces Bootstrap supports show, hide and toggle functions but others will follow soon.

Pro tip Want to know how the EL function retrieves the name of the component? Easy. When the selector will be used to find the JSF component, it will find a UIComponent (like a UIModal) which, according to RichFaces CDK design, will extend an AbstractComponent (like AbstractModal). And if the AbstractComponent supports Bootstrap EL functions, it will be annotated with @BootstrapJSPlugin (like at line 43 of AbstractModal) and it's the name attribute of that annotation that will give us the componentName. As I told you, really easy!

LESS support is no longer a dream (for those who doesn't know LESS, it's a more powerful way to write CSS, see the project website for more infos). Thanks to Lukas' awesome work, a first prototype of that feature is already working. You can read his post to know more about that. There is still work to do but it's an incredible starting point.

What's coming?

Want more? Great, because we have tons of other plans to improve RichFaces!

New build design is currently under discussion on RichFaces wiki so be sure to take a look and give feedback if you care about the future of RichFaces.

Theming all current components from RichFaces Core with Bootstrap design is planned so you can use both projects at the same time.

The orderingList is just a beginning. More jQuery UI widget factory based components are incoming. Feel free to comment here to propose the ones you would like to see supported in RichFaces Bootstrap.

News about RichFaces CDK

2015-06-06T15:40:57+02:00

RichFaces CDK new features

As the RichFaces Bootstrap project grows, we need more tools to achieve new goals and keep the code clean and readable. More tools means a stronger CDK and here are the last features.

Fragments are small portions of code inside the template that are defined outside of the main implementation but can be called inside it or inside other fragments or even inside itself! Say hello to recursion in RichFaces CDK templates. In term of Java, fragments are methods. So when you write a fragment, it will generates a Java method in the final renderer. Knowing that makes fragments really easy to understand and to use. See the JIRA issue 12226 for a fully explained example.

Pro tip Notice that in the signature of the generated Java method, 3 arguments are always passed without having to specify them inside the template : ResponseWriter, FacesContext and UIComponent.

Warning Currently, if you want to use a fragment1 inside a fragment2, you need to write fragment1 first in your template so its signature has been parsed before calling it in fragment2. Problem is reported in JIRA issue 12326.

cc:renderFacet is a new tag you can use inside a CDK template, equivalent of the same tag from JSF composite component. Its usage is quite straightforward: it will render the facet that you will specify in the name attribute. If you put some content inside the tag, it will be used as default value in case the facet is missing. See JIRA issue 12260 for full description.

varStatus is a new attribute for the c:forEach CDK tag. It will perform the exact same thing as the one in the original c:forEach tag, giving you more tools inside a forEach loop. See JIRA issue 12232.

wildcard can now be used inside cdk:passThrough and cdk:passThroughWithExclusions attributes in order to pass all attributes starting with the same prefix. Especially useful with JavaScript events on*. Wildcard can be use with attribute mapping like onkey*:oninputkey*. See JIRA issue 12200.

component variable is now directly casted to the correct class based on cdk:class tag in the template. You will no longer need to write explicitly the cast inside 95% of your templates. Enjoy less verbose code and see JIRA issue 12248 for more details.

In general, CDK has been improved to be more type-safe which allows to catch more issues at compilte-time.

What's next?

There are still a few points undone in the CDK wish-list and I hope some of them will be realized. One of the most important is probably generating methods from interfaces! See JIRA issue 12339 to fully understand the concept.

In another topic, another post will follow next week to talk about what's new in RichFaces Bootstrap project.

JSF, Bootstrap and calling render=@all

2015-06-06T15:40:57+02:00

The problem

So, the other day, I was playing with Twitter Bootstrap on a JSF application. Everything was fine until I decide to use a render="@all" somewhere in the page in order to refresh all my components after quite an heavy operation. Working fine.

But Bootstrap was no longer fully healthy. The CSS and design were ok, but the JavaScript was all broken : nearly all effects didn't appear anymore. It was a bit strange so I hit F5 and everything was fine, all JavaScript events were back. But the moment my render="@all" was called, they disappear again.

Why?

After some investigations, the reason was that Bootstrap calls a JavaScript function that will attach most of its events to the <body> when the HTML DOM is ready. Its doing so in order to catch all user interactions when they bubble up to the <body> tag which wrap the whole page. With that, you can use Ajax as much as you want and update your DOM since there is no event attach to a particular HTML tag, they are all in the <body> tag.

That's really good but that's also the reason of the problem. I said Ajax is always fine, and it's true as long as you don't touch the @<body>@ tag. The moment you update this tag, it will probably be reset to it's initial state, without any JavaScript event, and since the Bootstrap JavaScript function is only called once when the DOM is ready the first time, JavaScript events will not come back. Guess what, render="@all" will render your <h:body> and so will broke all Bootstrap JavaScript events.

A workaround

The true solution would be to call the Bootstrap JavaScript function after each render="@all" in order to attach all JavaScript events again. But since I have no idea how to do that right now, I have chosen to use an easier workaround.

Just wrapping your whole page in a JSF panel and render it instead of render="@all" should be enough in most use case. It is not exactly the same behaviour, but in most case, when calling render="@all", what you really want is just refreshing your whole page using Ajax. For example:

<!DOCTYPE html>
      <html lang="en-US"
            xmlns="http://www.w3.org/1999/xhtml"
            xmlns:h="http://java.sun.com/jsf/html"
            xmlns:f="http://java.sun.com/jsf/core"
            xmlns:ui="http://java.sun.com/jsf/facelets">
      <h:head>
          <title>Site title</title>
      </h:head>
      <h:body>
          <h:panelGroup id="all">
              ... your code...
          </h:panelGroup>
      </h:body>
      </html>

And replace all your render="@all" with render="all".