Web 4.0

It’s safe to say that Web 3 was a flop already right? 😛 Anyways, this post is a vision of what could come next for the web, whatever version we’re on now…

The other day I was trying to find a new React component library. We’re using Radix right now but it’s a “headless” component library where you not only have to style everything yourself but also create wrapper components to encapsulate the complexity of all the sub components and customize them for all the missing behavior we need. It’s no joke to make your own (good) component library that has a consistent interface for things like click events, form labels, validation states, disabled components, accessibility, etc, etc. On top of all that Radix is also missing what I consider pretty table stakes components like a Combobox and multi-select field so you’re once again left to fend for yourself.

So I spent the better part of a day looking for a new library. I ended up combing through 11 of the most popular ones and for one reason or another they all pretty much got ruled out. Some were missing basic components we needed, some didn’t support SSR, some would be a big pain in the butt to customize the styling to match the rest of the site, some were using different styling technology than the rest of the app which would be a maintenance headache.

All this got me thinking… How much time have we wasted re-inventing the wheel? Each of these libraries represents multiple thousands of hours of our collective development time, and there are hundreds of them just for React! Think about all the other frontend frameworks that have their own libraries, and then all the various desktop application frameworks that have been built over the years. And how many different build chains have there been with how many hours sunk into churn?

Wouldn’t it be nice if we had one amazing open source way to develop applications instead of thousands of half baked ways?

And yes, I know, I know…

But it still feels like we’re pre-industrial revolution times with everyone hand crafting their special parts. It’s high time we re-invested in some standardization. So to me that’s creating the next version of the web which puts us at 4.0 for whoever’s making up the version numbers.

There was a good comment on Hacker News the other day that captures the feeling:

The reason the web changes so fast, and there are so many rewrites, is the same reason a puzzle whose pieces don’t fit together keeps getting shifted around and restarted.

People are looking for a satisfying non-leaky abstraction to build upon and they don’t find it with web technologies. They get close, but those last few pieces never quite fit, and we lack the power to reshape the pieces, so we tear out all the pieces and try again. Maybe this next time we’ll find a better way to fit them together.

— Buttons840 on HN

Isn’t this what your OS should do?#

Some of you may be thinking that this is a job for your operating system, and it should be providing the tools to build applications easily on top of it. The problem is that all the operating systems people use day to day are owned by companies with negative incentives to make open source cross platform tooling which means no cross platform apps. The operating systems that are open source have a good history of re-inventing the wheel themselves.

Besides the fact it’s pretty much impossible to create a new OS now, there’s never going to be a time where everyone is running the same OS anyways so we’re always going to need an application layer on top of OS that is one abstraction level higher that can hide the details of running on different operating systems.

Native apps will always exist to push the boundaries of what’s possible on a platform, but most people and business don’t care about that and just want the easiest and most cost effective way to build cross platform apps that are easy to maintain across all the platforms. Plus, we still need a solution that scales from a static site that should load in 10’s of milliseconds all the way up to full featured apps. The OS install and update process is a huge drag for native apps. No one wants to install your personal home page.

So then what?#

I’m primarily a web developer, so what I’m thinking is of course shaped by what I know, but the web has been the most successful cross platform application platform ever created. So what I want is like the web, but better. Re-imagined. And suitable for building applications of all kinds, from the simplest static content sites to the most complex desktop apps you can think of like Photoshop or Excel. All with better performance, reduced development time, and more people rowing in the same direction than what’s happening today.

Backwards compatibility has been one of the web’s greatest features for (checks wikipedia…) 34 years now! But it feels like we’re due for a breaking change reset that incorporates all we’ve learned in the last 3 decades and makes building apps for the next 6 decades faster, less bug prone, and easier than it is now. I’m not a browser developer so I can only imagine how much cruft is hanging around slowing everything down to handle the bajillion edge cases for maintaining backwards compatibility for so long.

So lets start laying down some details, even if they are a little vague because this is daydream not a design document.

Web 4 Overview#

To put it simply: new protocol, new markup language, new scripting language, new styling language, new standard components, new mechanisms for data synchronization.

We’ll first need a new protocol to tell the recipient we’re sending data which needs to be interpreted differently by the browser which also requires new capabilities. There may need to be some new verbs introduced (see the Backend section ) but fundamentally it’ll be very similar to HTTP/3 under the hood.

The rest will be very recognizable as upgrades to the existing way the web works. There will still be markup sent to define layout and maintain accessibility. There will still be a styling language to define the visual presentation. And there will still be an application language controlling both the markup and styling, as well as connecting to a backend to send and receive data.

I just want everything more interoperable and cut from the same cloth. Something that’s cohesively designed to work as well together as possible and lower the barrier to learning.

Let’s start by looking at the markup.

Markup#

Have you ever wondered where HTML got the whole bracket and tag syntax from? I didn’t until just now when I was starting this section but apparently it was adopted from something called Standard Generalized Markup Language that was developed in the 60’s.

Since we’re making a breaking change, now would be a good time to update the syntax to be much closer to the other languages we’ll need to use. It’s tempting to think it’d be nice to just have one language for markup, styling, and application code, but there’s power and brevity in having domain specific languages for each so they can all focus on their individual task and keep separation of concerns. Having said that though, there’s no reason the languages shouldn’t be from the same genealogy and share as much syntax as possible.

Here’s a simple example:

  dsml (lang="en-US") {
    head {
      title {"My First Web 4.0 Document!"}
    }
    body {
      grid(columns=2, gap=8) {
        link(to="/blog") {
          img(
            src="/picture.webp",
            caption="My pretty picture",
            zoomable,
            sizes={{400:"/picture-400px.webp", 1800:"/picture-1800px.webp"}}
          )
        }
        section (component={someNamespace.SectionComponent}) {
          p {"Welcome to my site!"}
          button(variant="secondary") {
            "Press the button, Max"
          }
        }
      }
      /* Oh wow, normal comments! */
      someNamespace.myCoolComponent (data={[4,8,15,16,23,42]}) {
        "That has child text"
      }
    }
  }

There’s a few things to notice here. It’s similar to HTML, but instead of opening and closing tags with attributes, but now everything looks more like function calls with optional named arguments. That’s not a coincidence. As you’ll see in the scripting section, everything is really a component, not just a tag. Each component’s attributes props can be any standard data type including objects (like the image sizes), not just strings. With this setup there’s also no more chance of mismatched closing tags, just mismatched brackets as in most languages but with editor formatters that’s a well handled problem.

Most tags components you’ll use will be from the standard library, and the intention is that you’ll be able to look up the source of each of the standard library components and see exactly how they’re implemented from primitives, as well as all the props they take to customize their functionality to suit your needs.

If you do need a custom component, you’ll just specify the script as the component name which will bind it up to put whatever it renders in the right place in the DOM. The children and props specified are the initial values the component scripts will use before they re-render.

With Style#

Another big change from HTML/CSS as we know it is that here all style attributes can be passed as normal props. This has some pretty cool advantages we’ll see in the scripting section but also allows seamlessly customizing components both in look and functionality in the same way. Of course there will have to be good tooling support showing the reserved style attributes so you don’t accidentally create naming collisions with your custom props. This format is also along the same lines of what Tailwind is trying to accomplish but having first class support support for it means you won’t have to learn two languages to do the same thing.

Inline with my thoughts on Tailwind, there’s a breaking point for readability with how many props/classes should be on a component. After about 10 or so, the complexity is sufficient to justify pulling the styles out into a separate styling section where you have both more power and more capability to organize and document the code for readability. Styles should be scoped to components by default and only have the extra syntax to globally style the app for the more rare case when you need to do that. For example, when you want to install a theme. I can imagine sites popping up where you could download theme style sheets that completely change the look and feel of the standard library components that everyone should be using (anyone remember CSS Zen Garden?).

Styles should also be able to override and customize every part of native components like the icons for checkboxes and the circles and dots for radio elements without completely replacing them like you have to do now. Like I mentioned in the markup section you should be able to see the source for native components and see exactly what elements they’re rendering so you can style them. Using real primitives everywhere should also reduce or completely eliminate the need for pseudo-elements like ::before & ::after.

  /* Variables with the same syntax as your scripts */
  var padding-size = 8px;

  grid {
    background: gray-800;
    border: 2px solid gray-900;
    padding: padding-size;
    /* Inline calculations with variables */
    margin: padding-size * 2;

    /* Conditionals also with the same syntax as your scripts */
    if(screen_width < 400px){
      padding: 4px;
    }
  }

  /* Selectors based on component props */
  img(zoomable){
    rounded: 100%;
  }

Here you can see the styling language looks similar to the markup language like selecting elements to style based on their current props. But it should also look like a subset of the scripting language and allow you to use the same syntax for variables and conditionals. TBD whether or not it’d be a good idea to allow loops in the style sheet. It could be useful for setting up variable state to use later, but also potentially hazardous to performance since loading the style sheets will still be a blocker to first render. There definitely should be support for functions that apply styles though, a la sass mixins.

To further intentionally blur the line between styles and scripts it should be easy to share your variables from scripts to styles or from styles to scripts. A good usecase for this I run into all the time is needing to share the color palette between scrips and styles without having to duplicate variables. There’s a lot of cases where your scripts want to dynamically color something based on state and should be able to access color palette variables, potentially manipulate them, and then apply them back as styles while still having them be strongly typed like all the other scripting variables.

TIP

Color manipulation in CSS has already come pretty far and you can do cool stuff like take one color in a variable and dynamically brighten and lower its opacity in one line:

oklch(from var(--color) calc(l + 0.1) c h / calc(alpha - 0.1))

Another big hole in CSS’ current capabilities is being able to define custom shaders (both geometry and vertex) for anything the browser draws. I found a couple decade old articles talking about this but apparently it never made it into the spec which is a shame. You can do some basic stuff with the predefined filter and backdrop-filter properties but it’s so limiting. You can make some truly wild yet performant visuals with shaders and I’d love to see web designers explore a whole new domain of possibilities. Think simulated material backgrounds that react to the cursor acting as light source. Think button borders that simulate pond ripples when you click them. These are silly examples but there’s really infinite possibilities and so many creative people out there that I’d love to see what they come up with. It’d also make web games much richer without having to resort to rendering everything in a canvas. All the shader effects would need to have fallback colors for accessibility and user preference reasons of course but I’d be leaving that setting on all day every day.

There’s a huge grab bag of other stuff I’d like to see improved like native shadow presets (with simulated light source), easier to understand and use layer stacking than z-index, a better thought out contrast color, anchor based positioning (put the top right of this box at the center of this other box), simplified naming (flexbox justify/align I’m looking at you), native nested selector support, etc, etc…

There’s a lot of innovation happening on the web in your style sheets right now! I want to keep that up and encourage sites to look unique, personal, and an artistic expression of their creators. This is still possible even with everyone using the same components for functionality under the hood.

Scripting#

To me it’s safe to say that the web has settled on components that rerender as a function of state for building applications. React really did revolutionize web development with that idea (or at least popularizing it). For those of use who remember the before times, it was grim having to manually update the DOM when your state changed or you were reacting to user events. The current situation is far from perfect though of course.

The first sticking point we’d have to decide on to make it better is the language. If we’re pushing the reset button, we better re-evaluate what the best choice really is.

I want a language that:

• Is strongly typed with a powerful type system like Typescript’s but with types available at runtime for reflection.
• Focuses on performance
• Is part of a strong ecosystem with lots of existing packages
• Has a comprehensive standard library so we don’t have to use all these different libraries to do the basics
• Has the ability to express refined types (arrays of known length, formatted strings like emails and UUIDs)
• Allows you to specify dependencies’ permissions where you opt into allowing disk/network/privileged API access
• Has robust dependency injection support
• Is terse yet expressive
• Skips the import nonsense and makes the compiler figure it out with namespaces
• Can compile to efficient code (both in size over the network and runtime) to save the browser parsing time

The closest language I know of to checks the boxes I’m looking for is C#, but it’s missing a few things. The good news is though that if this project were to ever actually get built you would need to create a language extensions to support returning the markdown language in the scripts like JSX/TSX files do. So you could also customize the language a little bit more to make it more purpose driven for this usecase which would also be a nice excuse to have the language be owned by the open source group instead of whatever corp or org the language originated from.

For actually adding interactivity the idea is that you can set the component attribute on a native component to customize its behavior or just create a new custom component which can render whatever it wants. So in the markup example above the section (component={someNamespace.SectionComponent}){ line is attaching the SectionComponent script to control the native section component. It would be able to hook into the native components’ state and user events to only change what it needs while leaving everything else to the native implementation. The someNamespace.myCoolComponent line is a custom component.

As mentioned above, I want the props (equivalent of HTML attributes) to be both strongly typed in what the component expects while also supporting any language built in type including arrays, objects and functions. There’s so much engineering that’s happened just to get over this limitation.

Two way binding a la svelte (and many others) is such a nice quality of life feature when you need it like for getting form component values without lots of cumbersome boilerplate.

Since all props are just in the DOM (whatever its evolution is), Server Side Rendering should now be easy. The backend can just render the application in whatever state is requested and send the document with all the props set up for the scripts to initialize with. Nothing should have to re-render when the scripts initialize until some user interaction (or network refresh) happens, and the browser can queue up user events for the scripts to process once they initialize so you never end up with those scenarios where a user clicks a button but nothing happens because the script isn’t ready yet.

Without further ado, here’s a very hypothetical example of what a component script might look like

// Similar to parameter binding in .Net minimal APIs, the ILogger comes from the DI container
class SectionComponent(Props props, ILogger logger){

  private views = 0;

  onButtonPress(){
    logger.Log("Button was pressed")
    // Just assign to variables and the compiler figures out rerenders
    views++;
  }

  render() {
    // Return markup syntax just like TSX
    return (
      section (...props){
        p {$"Welcome to my site! Views: {views}"}
        button(variant="secondary", onPress=onButtonPress) {
          "Press the button, Max"
        }
      }
    )
  }
}

Comprehensive Components & Standard Library#

A sizeable part of this whole idea is around having an awesome fully featured component library. It bugs me how inflexible and lacking native HTML elements are which means everyone ends up building their own which end up usually missing some functionality and makes everyone’s user experience inconsistent. As a perfect example of how terrible some native components are, let me remind you what the native multi-select component looks like:

I dare you to coach anyone non tech savvy over 50 years old how to use this.

As a grab bag, here’s my wish list of components that would be nice to stop building over and over and over again:

• Forms that actually handle real world usage with dependent state handling, dependent field validation, multi step workflows, and strongly typed values returned
• Form Inputs that handle icons, units, and formatting
• Number inputs that actually result in your script getting a number (just learned about valueAsNumber)
• Non sucky multi select, that includes variations in styling for tag style, count style, text style
• Combobox / async selection search
• Checkboxes with a variant to render as a toggle
• Tooltips
• Dialogs
• Carousels
• Drawers
• Toast messages
• Graphs & Charts
• Tabs
• Rich text editor
• Calendar
• Expandable Tree view
• Mosaic image layout
• Tables that can handle sorting, filtering, pagination, resizable columns, selections, inline editing, expandable rows, empty states, and the whole kitchen sink
• Standard layout components

Since the framework would be open source you would not only be able to fix bugs and add components everyone wants, but also dive deep and see the implementation of native components and how they’re composed from primitives when you need to customize them.

Another benefit of having the standard component library be so full featured (and therefore heavily used) is that accessibility tools (screen readers/voice input/AI) can build out specialized functionality for each of them and not have to worry as much about everyone’s special snowflake components that forgot about a11y. Keyboard users also rejoice.

Authentication#

Another easy layup for the evolution of the web is fixing the nightmare that is authentication as we know it today. How many logins do you have stored in your password manager? I have 636. And many of those sites have asinine password requirements that make them less secure and more annoying to use. Even more have already stored your credentials insecurely and been hacked. I, just like everyone else, am usually too lazy to go and update my password whenever they have a breach.

How about we have the browser manage your login(s) and then have APIs for the sites/apps to authenticate and get access to the profile info you want to share? Even 2FA could be managed by the browser login system so every site wouldn’t have to build (or buy) their own system. Each app could specify the maximum staleness of the authentication token to require you to log in again for critical operations.

Wouldn’t it be nice to log into pretty much everything at once?

Backend#

I’m not going to go into as much detail on the backend, first because this post is already way longer (and has taken way more time) than I expected, but also because backends are necessarily quite varied in what they need to do and the systems they need to talk to. I wouldn’t expect everyone’s backends to behave as similarly as my envisioned frontend, but there are a number of things that come to mind that would be nice to have…

Shared Resource Typing and Functionality#

The backend would be in the same language as the frontend. Obviously… It’s such a team efficiency boost to write everything in the same language so you don’t have to context switch your brain when hopping between frontend and backend, not to mention all the benefits of shared tooling and utilities. The web has already been marching in this direction with Node, Next.js and React server components of course but it’s worth it to restate how nice it is to be able to share code between the backend and the frontend (with a proper project structure set up to know what code is shared vs exclusive) especially for things like validation which need to happen on both sides anyways.

But this section is really about how great it would be if all projects (not just the ones that dedicate a ton of time to make this happen) had a tight integration between frontend and backend for synchronizing data organized into resources. Resources come in all shapes and sizes, backed by many different data stores, that result in many different capabilities the resource can provide. With that in mind it should be easy to define in your api per resource: if pagination is supported; which fields are sortable; which fields are immutable; which fields are only mutable at creation time or update time; and which fields are deprecated.

As that resource schema definition changes during development the client should be hooked up with automatic type generation to the changes so all the functions that interface with the API have the updated signatures and you can see type errors where changes need to be made to accommodate the API.

Data Synchronization#

The obvious next question though is how do you synchronize data between client and server. Do we use REST or GraphQL or gRPC or something else? Let me dodge that question for a sec and take a step back to think about things from first principles.

The data (resources) we want to store can always be represented as an array of objects, even if the array is just has a single item or the object has a single property. And those objects/resources often relate to each other so it makes sense to me that we should be leaning on a relational database to manage the data. On the backend there can be good reasons why you might choose not to use a relational database but they pretty much come down to data size and I/O performance. However, with the size of data clients should be dealing with I can’t see any reason not to use a relational data model for client application state that’s synchronized with the server (there would still be component level state in script variables).

So in the easiest case of a 100% offline app, state management would just look like updates to table rows. There of course already is IndexedDB available in browsers but apparently this is slow so I would like to see something more akin to sqlite wrapped in the browser with the table schema’s defined by the API resources plus any extra client-side only tables that are needed.

With that established, then the question becomes how do you seed the data on initial page load, and how do you keep the slice of data the client cares about (and is allowed to see) in sync with the backend. For initializing, we would need a new kind of resource request for an efficiently serialized and compressed database file that would be loaded (upserted by default) into the clients database as the page and scripts load. That database would be customized by the server to contain all the data the client needs across linked resources so no waterfall of network requests need to be made when a page loads. It would still need to be requested via URL so the contents could be customized (and cached) as needed per page, and there would also need to be a mechanism for the clients to only request it when their local database is empty or too stale.

For reading data (get one or get many of a resource) my ideal would be a request similar to how REST works today but with more strictly defined and enforced patterns for filtering, paging, metadata, and fetching related resources. Currently, REST is great for simplicity and ease of implementation but is way too lax in standardization which results in everyone implementing it differently. GraphQL is great for fetching related resources and having better standardization around paging and metadata but from my experience usually ends up as a headache to implement on the backend (securely and performantly) and the frontend ends up with an explosion of types as every component is trying to fetch things a little differently which is such a pain for shared utility methods. I think the pragmatic compromise is a protocol that has well defined patterns for fetching the slice of a resource you care about along with the related resources you want which the server can figure out the most efficient way to query and bundle up in a single request and response. The client could then upsert responses directly into the local database for components to re-render with. The client could also specify in its table schemas if it would like to receive automatic updates either via polling or a persistent web socket for push.

For data mutations originating on the client, it would look the same as in the offline app case where you would update the corresponding table row so the client can optimistically render, but then the changes would get batched by resource and uploaded to the server which would then respond with the full resource data including any server generated fields which the client would update its db with. The resource schema definition shared with the client would provide strong typing around which fields can be modified (or created with new records). There would also need to be strong error handling around failed or partial updates.

Isn’t this like…?#

All my dreaming might have sparked some associations to go off in your head with other technologies you’ve used that do a certain piece similarly. As far as I know and have been able to research there’s nothing that comes close to the full picture I’m laying out here. If there is, please let me know! With that said, here’s a couple of the technologies that come to mind as closest with my impression of what they’re missing.

Web components — have similarities with the custom elements with behavior I talk about in the scripting section, but obviously don’t address any of the other things.

Rails — The doctrine is an amusing read. I agree with most of the points like Optimize for programmer happiness and Convention over Configuration, but I don’t think they should come at the expensive of performance which lands you in #396th place out of 510 in performance benchmarks. Even if you were to take the performance of the #50 spot in comparison, you’d need 14 times more horsepower/servers/spend in comparison. While some of the philosophy is similar, my proposal here is also more broad in scope of course instead of what Rails can do is inherently limited to with the current web so it can’t have a shared syntax across server/styles/scripts/markup.

Blazor — I hadn’t really looked that closely at Blazor before writing/researching this post but there’s definitely some similarities. They are obviously still bound by the restrictions of the current stack (like having all string attributes) and also ship a ton of code to the browser which results in horrible frontend performance. Some of that could be improved by having the browser take on more responsibility but someone who knows more about it than I can weigh in on why they’re so heavy and slow. Even with those issues fixed, the most important part of this whole idea is that it needs to be a new open source standard not controlled by any one company trying to lock you in only to dump you when their objectives change.

Electron & alternatives — There’s a lot of different Electron alternatives out there that share some ideas with what I’ve laid out. But of course they’re all aimed at desktop apps, not a wholistic solution that encompasses everything from a static web document to an interactive web apps to a packaged offline desktop app. They’re not build for splitting the app into pages or chunks and require you to install the whole app. Plus, most of the ones that aren’t just wrappers on top of existing web tech look like they’d completely fall flat building complex UI’s and apps with themes. Here’s another comparison between them I found.

Will this ever get built?#

Probably not. But maybe! I certainly don’t have the time, do you?

There’s a thousand other things I’d change along the way if I were the one building this, even some of the initial ideas in this post I’m sure will fall apart when it comes to feeling the developer experience actually building an app in the new platform. The best ideas come from iteration experimentation and refinement.

I’m sure you don’t agree with all my ideas but I hope this gets you thinking about what your pie in the sky web dev stack would look like every time you have to re-invent the wheel.

Lemme know your ideas in the comments!