PIcking Your Javascript Mobile Framework: Part 5 - Performance
Well, here it is: the article about performance. I won’t leave you guessing: here is the chart with the results, and all the surprises it entails:
Cautionary tale: given the number of frameworks and the boredom of hitting refresh, I only did ten refreshes per framework. Given the spread of the results (not shown here) that could catapult any framework a few rankings up or down, depending on where it goes. So look at these numbers with enough salt to cause you heart trouble!
For the methodology: I used the built-in window.performance.timing object. In particular, I injected the following snippet (as an external resource) into each index.html, just before the </body> tag:
window.setTimeout(function () {
console.log("PERFORMANCE " + document.URL + " " +
(window.performance.timing.domContentLoadedEventEnd -
window.performance.timing.navigationStart)
);
}, 500);
The idea was to capture the time from navigationStart to when the domContent was Loaded. Since the domContentLoadedEventEnd is not recorded until that actually happens, I told the browser to wait 500 milliseconds after loading the script. Fortunately, the timing in the performance.timing object is independent of script loading.
As far as the results, there are several surprises:
- The three demos that used primarily angular had completely different results. Angular with Require was one of the fastest frameworks to load; Angular by itself one of the slowest; Angular with PerfJS in the middle of the pack. Turns out that’s because Angular by itself loads an HTTP resource Firefox doesn’t like it to touch.
- Far from being at the front of the pack, the self-proclaimed super-fast Mithril is middling
- RequireJS, a requirements management script, catapults all of the fastest framework combinations to success.
- Vanilla JavaScript is actually not a bad performer, coming in at the middle of the pack.
- The much-maligned jQuery is in the middle third of the pack, on the slower end.
- React, the Facebook JavaScript framework, is incredibly disappointing.
- Backbone/Marionette/Require is far and away the fastest combination. Interestingly, Backbone and Backbone with Marionette are two of the slowest performers.
This is of course a relatively simple example, and the only thing measured was the load time of the first page. At the same time, this comparison can’t go into it any deeper without resorting to artificial comparisons. Should we measure how the apps handle 5000 todo items? Should we randomly add menus and buttons and see what happens? Any manipulation requires modifying the setup of a large number of frameworks. Besides, we can see from the comparison that the majority of the frameworks manage to load just fine within 2x the fastest.
Why does RequireJS make such a huge difference? Because it prevents the browser from loading a bunch of files and makes it load a single dependency. This is of course of biggest advantage in those frameworks that love to create tons of files. So if that’s your coding style, RequireJS is the one for you.
In fact, using RequireJS is by far the largest contributing factor to speedy loading in our comparison. CanJS, for instance, cuts down load time in half simply by adding Require.
The verdict, in the end, is that the frameworks don’t have big differences in page loads at this scale. Certainly, it would be interesting to see if that’s still true at larger scales, but it’s not realistic to attempt a comparison with this number of frameworks. Since there is no realistic way to pick any of them for comparison based on their performance, we can only eliminate the worst offenders and suggest the slower performers implement Require.
Incidentally, there is something seriously wrong with BackBone: While it leads the pack when using Require, it is one of the slowest performers without. The same is also true for AngularJS, but that was more expected (at least by me).
(The Lumbar that gives such a helpful boost to Thorax, by the way, is Require written by WalMartLabs. Yep.)