Web Truths: We need granular control over web APIs, not abstractions

This is part of the web truths series of posts. A series where we look at true sounding statements that we keep using to have endless discussions instead of moving on. Today I want to tackle the issue of offering new functionality to the web. Should we deliver low-level APIs to functionality to offer granular control? Or should we have abstractions that get people started faster? Or both?

In a perfect scenario, both is the obvious answer. We should have low-level APIs for those working “close to the metal”. And we should offer abstractions based on those APIs that allow for easier access and use.

In reality there is quite a disconnect between the two. There is no question that newer web standards learned a lot from abstractions. For example, jQuery influenced many additions to the DOM specification. When browsers finally got querySelector and classList we expected this to be the end of the need for abstractions. Except, it wasn’t and still isn’t. What abstractions also managed to do is to even out implementation bugs and offer terser syntax. Both of these things resonate well with developers. That’s why we have a whole group of developers that are happy to use an abstraction and trust it to do the right thing for them.

Before we had a standardised web, we had to develop to the whims of browser makers. With the emergence of standards this changed. Web standards were our safeguard. By following them we had a predictable way of debugging. We knew what browsers were supposed to do. Thus we knew when we made a mistake and when it was a bug in the platform. This worked well for a textual and forms driven web. When HTML5 broke into the application space web standards became much more complex. Add the larger browers and platform fragmentation and working towards standards and on the web became much harder. It doesn’t help when some of the standards felt rushed. An API that returns empty string, “possibly” or “maybe” when asked if the current browser can play a video doesn’t fill you with confidence. For outsiders and beginners, web standards are not considered any more the “use this and it will work” approach. They seem convoluted in comparison with other offers and a lot of changes seem to be a lot of work for developers to keep up. Maybe too much work.

Here’s what it boils down to:

• Abstractions shield developers from a lot of implementation quirks and help them work on what they want to achieve instead
• Low-level APIs allow for leaner solutions, but expect developers to know them, keep track of changes and to deal in a sensible way with non-supporting environments (see: Progressive Enhancement)

What do developers want?

As web developers in the know, you want to have granular control. We’ve been burnt too often by “magical” abstractions. We want to know what we use and see where it comes from. That way we can create a lot of different solutions and ensure that what we want to standardise works. We also want to be able to fix and replace parts of our solutions when a part becomes problematic. What we don’t want is to be unable to trace back where a certain issue comes from. We also want to ensure that new functionality of the web stays transparent and secure. We achieve this by creating smaller, specialised components that can get mixed and matched.

As new developers who haven’t gone through the pains of the browser wars or don’t need to know how browsers work, things look different. We want to code and reach our goal instead of learning about all the different parts along the way. We want to re-use what works and worry about our product instead. We’re not that bothered about the web as a platform and its future. For us, it is only one form factor to build against and to release products on. Much like iOS is or gaming plaforms are.

This is also where our market is going: we’re not paid to understand what we do – we’re expected to already know. We’re paid to create a viable product in the shortest amount of time and with the least effort.

The problems is that the track record of the web shows that we often have to start over whenever there is a new technology. And that instead of creating web specific functionality we got caught up trying to emulate what other platforms did.

The best case in point here is offline functionality. When HTML5 became the thing and Flash was declared dead we needed a fast solution to offer offline content. AppCache was born, and it looked like a simple solution to the issue. As it turns out, once again what looked too good to be true wasn’t that great at all. A lot of functionality of AppCache was unreliable. In retrospect it also turned out to be more of a security issue than we anticipated.

There was too much “magic” going on that browsers did for us and we didn’t have enough insight as implementers. That’s how Service Workers came about. We wanted to do the right thing and offer a much more granular way of defnining what browsers cache where and when. And we wanted to give developers a chance to intercept network requests and act on them. This is a huge endeavour. In essence we replicate the networking stack of a browser with an API. By now, Service Workers are doing much more than just offline functionality. They also should deal with push notifications and app updates in the background.

This makes Service Workers tougher to work with as they seemed complex. Add to that the lack of support in Safari (which is now changing) and you lost a lot of developer enthusiasm.

Large room of devs.
"Who has heard of service workers?"
Every hand.
"Who has used service workers?"
3 hands.

— Raymond Camden ? (@raymondcamden) August 12, 2016

There is more use in abstractions like Workbox as they promise you to keep up-to-date whilst the changes in the spec are ironed out. Instead of getting a “here are all the lego bricks, build your own car”, it has a “so you want to build a car, here are some ways to do so” approach.

This is a good thing. Of course we need to define more granular and transparent standards and solutions to build the web on. But there is a reluctance in developers to take part in the definition phase and keep an eye on changes. We can not expect everybody who wants to build for the web to care that much. That is not how the web grew – not everybody had to be a low level engineer or know JavaScript. We should consider that the web outgrew the time where everyone was deeply involved with the standards world.

We need to face the fact that the web has become much more complex than it used to be. We demand a lot from developers if we want them all to keep up to date with standards. Work that often isn’t as appreciated by employers or clients than shipping products is.

This isn’t good. This isn’t maintainable or future facing. And it shouldn’t have come to this. But it is a way of development we allowed to take over. Development has become a pretty exhausting and competitive environment. Deliver fast and in a short cadence. Move fast and break things. If you can re-use something, do it, don’t worry too much if you don’t know what it does or if it is secure to do so. If you don’t deliver it first to market someone else will.

This attitude is not healthy and we’re rubbing ourselves raw following it. It also ensures that diversity in our market it tough to achieve. It is an aggressive game that demands a lot of our time and an unhealthy amount of competitiveness.

We need to find a way to define what’s next on the web and make it available as soon as possible. Waiting for all players to support a new feature makes it hard for developers to use things in production.

Relying on abstractions seems to be the way things are going anyways. That means as standards creators and browser makers we need to work more with abstraction developers. It seems less and less likely that people are ready to give up their time to follow specs as they change and work with functionality behind flags. Sure, at conferences and in our talks everyone gets excited. The hardware and OS configurations we have support all the cool and new. But we need to get faster to market and reach those who aren’t already sold on our ideas.

So, the question isn’t about granular definition of specifications, small parts that work together or abstractions. It is about getting new and sensible, more secure and better performing solutions into production code quicker. And this means we need both. And abstractions should have a faster update cycle to incorporate new APIs under the hood. We should work on abstractions using standards, not patching them.