JavaScript Framework Battle: ‘Hello World’ in each CLI
I was just wondering, given that most of the big JavaScript frameworks offer a Command-line interface (CLI) tool nowadays — to automate the creation of new projects and the building of production assets — how do they actually compare to each other? I mean surely they must all be hitting the same ‘ballpark’ when it comes to bundle size/render perf right? Maybe it’s not as close as you might think.
I decided to test this out by installing 6 popular CLI tools — Create React App, Angular CLI, Ember CLI (for both Ember & Glimmer), Vue CLI, Create Inferno App and Create *Preact App globally onto my laptop, and then follow the official documentation for each.
I was only interested in the ‘out of the box’ project generation — so there was literally zero application code added by me. I just ran the relevant command to scaffold a project, then I immediately ran the production build…
I think this is an interesting test, because although each framework can give you more or less features by default, the point here is that the authors of the framework must deem this default scaffold+build process to be ‘the best you’re going to get’ out of the tool, and that’s what I found fascinating in the results.
So let’s start in this first post by looking at just two easy metrics, JavaScript bundle size & JS first render time.
Note: see https://github.com/shakyShane/arewereadyformobileyet for the automated results.
JavaScript Bundle Size
For each CLI, I ran the ‘as-documented’ command for producing a production-ready build. T̶h̶e̶n̶ ̶I̶ ̶m̶a̶n̶u̶a̶l̶l̶y̶ ̶g̶z̶i̶p̶p̶e̶d̶ ̶t̶h̶e̶ ̶o̶u̶t̶p̶u̶t̶ ̶(̶u̶s̶i̶n̶g̶ ̶d̶e̶f̶a̶u̶l̶t̶ ̶m̶a̶c̶O̶S̶ ̶g̶z̶i̶p̶ ̶c̶o̶m̶p̶r̶e̶s̶s̶i̶o̶n̶ ̶s̶e̶t̶t̶i̶n̶g̶s̶)̶ ̶t̶o̶ ̶g̶e̶t̶ ̶t̶h̶e̶s̶e̶ ̶r̶e̶s̶u̶l̶t̶s̶.̶
Then, I spin up a lightweight server pointing at the resulting ‘build’ or ‘dist’ directory with Gzip enabled. Finally I pump the URL from this server into Lighthouse & measure the size of all the Javascript combined.
Notes
- Preact (and other micro libs, didn’t want to clutter the chart) obviously come out on top given they are simply a thin layer on top of the DOM. You’ll naturally get more features out of something like Angular & Ember, given they are full frameworks — but I’ve left a micro lib in here because with PWAs becoming so popular/necessary, it’s often the case that Apps are built by composing many of these micro libs together and given the starting point is 8kb, you’d need to add a LOT of them before your code is anywhere near the size of what Ember/Angular are shipping by default.
- Interesting how close React & Vue are nowadays — although I believe Vue can be trimmed further depending on which options you select in the CLI scaffold stage (I just went with whatever was presented as a ‘default’)
- Angular seems to have shed about 50kb since I last tried a few months ago, so they are certainly making progress;
- Ember is off the chart (size-wise) and cannot be considered suitable for mobile at this point (to be fair, I don’t think they claim to be).
- Glimmer (by the Ember team) is the newest, and coming in at 34kb is impressive to say the least. Great work.
First JavaScript Render
Next, I throttled the connection speed in the network panel Chrome Dev Tools (down to regular 3G) and applied the 5x CPU slowdown in the timeline section. Then I hit ‘reload’ with screenshots enabled and then scrubbed through until I spotted something appear on screen that was the result of the framework running.
There are of course many other ways of measuring this more accurately — depending on what type of answer you want. Not to mention the difference that may be made via SSR etc, but still, it’s pretty obvious that more JS === more time. Even if a page were rendered by the server and picked up on the client side by the JS framework — the render times in that chart above would simply become the ‘time to interactive’ as the JS still needs to download/be-parsed/be-executed, whether there’s static HTML there or not. A user ‘seeing something’ on the screen is simply not enough if the core product includes JS interactivity.
Interactive in 2 seconds…
I’ve often heard the mythical goal is ‘interactive in 2s’ (with the 3G throttle and CPU slowdown) and recently at my day-job I set a target on a very large project of having the initial bundle (enough JS for rendering, interactivity and Ajax) to be less than 50kb. Safe to say that hitting this target required me to chose a micro view lib like Preact, which, because of its tiny size, allowed me ‘space’ to bring in a few essential libraries whilst still keeping things below 50kb.
How I came up with the results
Create React App
Commands:
- Install:
yarn global add create-react-app - Scaffold:
create-react-app cra-test - Production Build:
yarn run build - JS Bundle Size:
46kb - First render:
770ms
Angular 2 CLI
Note: Why is aot not automatically part of the target=production configuration? It’s extremely easy to miss and I only knew about it because of a Podcast addiction :p
Commands:
- Install:
yarn global add @angular/cli - Scaffold:
ng new ng-test - Production Build:
ng build --aot --target=production - JS Bundle Size:
92kb (4 files) - First render:
1500ms
Ember CLI (Ember app)
Note: The Ember scaffold process does NOT render any JS by default, which leads to a false ‘first render’ score — so I followed the instructions and added an outlet as directed. This is the only CLI tool I had to do this for.
Commands:
- Install:
- -
yarn global add ember-cli - -
yarn global add bower - Scaffold:
ember new ember-test - Production Build:
ember build --target=production - JS Bundle Size:
198.4kb (4 files) - First render:
4200ms
Ember CLI (Glimmer app)
Note: There also appears to a --suppress-sizes flag, but when I used that, the JS bundle it produces appeared to be the same size + caused an error in the browser
Commands:
- Install:
yarn global add ember-cli/ember-cli - Scaffold:
ember new glimmer-app -b @glimmer/blueprint - Production Build:
ember build --target=production - JS Bundle Size:
34kb - First render:
1000ms
Vue CLI
Note: Vue was by far the most confusing CLI experience — the need to specify a template + lots of questions about tooling — it feels very off-putting to newcomers, where’s the ‘default’?
Commands:
- Install:
yarn global add vue-cli - Scaffold:
vue init webpack vue-cli(followed by lots of questions) - Production Build:
npm run build - JS Bundle Size:
43.48kb (4 files) - First render:
840ms
Create *Preact App
Note: I’m not entirely sure this is well supported, but it’s the first/only CLI tool I could find for a ‘micro’ lib.
Commands:
- Install:
yarn global add create-preact-app - Scaffold:
create-preact-app preact-test - Production Build:
npm run build - JS Bundle Size:
8.8kb - First render:
412ms
Create Inferno App
Commands:
- Install:
yarn global add create-inferno-app - Scaffold:
create-inferno-app inferno-test - Production Build:
yarn run build - JS Bundle Size:
70kb - First render:
737ms
Did you enjoy this? Perhaps you’d enjoy some of my lessons on https://egghead.io/instructors/shane-osbourne — many are free and I cover Vanilla JS, Typescript, RxJS and more.
