Custom Elements: A solution for soft-decoupled Drupal!

Let's decouple Drupal - Well, what exactly?

When it comes to decoupling, it turns out there are many options on how to decouple. Not only are there many technology choices in choosing the right frontend framework and APIs involved, but the questions become also more foundational: Which system should be in charge of handling different aspects of the application, e.g. routing, placing blocks or authentication?

So before moving on, we need to clarify what exactly we want to achieve:

Clarifying goals: Why to decouple?

For us, the main reasons are:

Independent frontend development

By separating the frontend from the backend, frontend development can happen completely independent of the backend. So frontend developers do not have to know Drupal development, but can focus solely on their part: the frontend. That way, it's much easier to find and onboard frontend developers.

Easier performance optimization

While efficiently lazy-loading CSS or JS assets of a page is quite hard to do with Drupal, modern frontend technologies like Nuxt.js handle this with breeze and simply lazy-load pre-built asset chunks as needed. Furthermore, the frontend can better optimize the page loading experience (and the assets needed per-page) since the frontend has the knowledge of what exactly is needed by which frontend component.

Re-usable frontend components

Re-usable frontend components - including asset dependency management - are a solved problem in the Javascript world and all the various Javascript frontend frameworks provide that. Additionally, tools like Storybook make it really easy to build up your own component library.

Modern frontend and UX

By building upon modern frontend tooling we can provide a more app-like UX to our users, handle client-side route changes and only reload and repaint what's needed when navigating pages. The web page can become and feel more app-like, or even be turned into a Progressive Web App that does more than provide an offline copy of the site!

Challenges faced

This sounds all good - but there are lots of challenges that must be well-considered:

Re-inventing the wheel

By throwing away Drupal's frontend, you also throw away all the existing integration Drupal provides with its own frontend. Besides the more obvious, like handling forms and logins, features like contextual edit links, previews or the layout builder just do not work out of the box anymore. So one has to develop alternative solutions for all those features - if needed. Depending on the solutions choosen, this can be a lot of work.

Server-Side-Rendering and Hosting

For delivering a decent user experience and being friendly to robots (Google) some sort of pre-rendering is needed on the server. So this needs extra thought and comes with new infrastructure requirements (Node.js) that, come at a cost, and must be considered when choosing where to host.

Cold cache performance

When the Drupal backend is connected via JSON API or GraphQL a first, uncached page load typically needs various kinds of data, quickly ending up in multiple API requests to the backend. Since all of those need to be resolved on a cold-cache page hit requests possibly end up really slow and lead to slow first page loads.

Coupling of the frontend to the backend

When the frontend talks to Drupal via its default JSON or GraphQL APIs, the frontend needs to know about Drupal's data structure. Field names, their types and entity relationships are no Drupal interna anymore, but become part of the contract between the backend and the frontend, limiting the possibilities to develop them separately. One could implement custom JSON endpoints or GraphQL Schema to mitigate and fine-tune that.

Our solution: Rendering into Custom Elements markup

We figured that in order to reach the mentioned goals, we can use a soft approach to decoupled Drupal in order to keep more of Drupal's features available. The basic idea is that Drupal keeps rendering pages by providing the main page content as custom elements (markup). Then, this simplified markup can be picked up by client-side frameworks like Nuxt.js to handle rendering the elements client-side.

An example of a custom element would be the following markup for a Twitter paragraph:

<pg-twitter src="https://twitter.com/bmann/status/1283090375742091264">
  <h3 slot="title">#Driesnote suggesting to ship an official component for React and Vue for managing menus</h3>
</pg-twitter>

Note that we use pg as an abbreviation for a paragraph here. Or a quote paragraph:

<pg-quote>
  <h1 slot="title">a quote from twitter...</h1>
  <p>#Driesnote suggesting to ship an official component for React and Vue for managing menus<br>Ship it in NPM as those devs expect.</p>
  <span slot="author">Boris Mann</span>
</pg-quote>

Generating this markup, is exactly what the Custom Elements module does. The module renders the data of all visible fields either as an attribute to the custom element tag, or as nested tag with a slot attribute.

As custom elements are part of Web components specification, web components provide a good fit for rendering the content client-side, but other client-side frameworks like Nuxt.js or Vue.js or React.js can pick up the data and render it as well - so there is plenty of choice for the best-suiting client-side solution.

So what about the page template?

Since the header and footer area is pretty static for most of the time on most of our sites, we figured they are best implemented by mostly static components in the frontend. Any dynamic parts, like meta tags or navigation elements can be complemented with data provided by (long-cached) API calls or even computed client-side (for example, the active menu item). If necessary, it's easy to re-render parts of it and client-side frameworks like Nuxt.js, Vue.js and React can handle updating only the necessary bits when navigating pages.    
While the dynamic per-page content is provided by the backend in a single API response per page, the header and footer can be controlled by the frontend. The frontend application takes the custom element content and renders it, next to taking care of updating essential page metadata like meta tags.

Drupal

Thus, for rendering individual pages, the frontend needs to fetch the main content - rendered as Custom Elements - as well as any page metadata that it needs for rendering the page from the backend. For that, we've implemented the Lupus Custom Elements Renderer module which turns Drupal into an API backend providing exactly that.

Summing up

By rendering the page shell and custom element content in a frontend framework of our choice, we achieve all of our goals stated. Moreover, the custom elements "format" provides a good way of decoupling the frontend of the backend, since any Drupalisms or backend complexities in the data model can easily by translated into a meaningful collection of custom elements. Those elements comprise a well-defined structure representing your site's content elements and transferring the necessary data to the frontend. In the frontend, the custom elements map nicely to Vue/React/Web/... components.

By keeping Drupal's page routing mechanism, we can keep using Drupal's path handling, including useful features like editor controlled path aliases or redirects. Since page responses are handled as whole by Drupal, we optimize and cache full-page responses and leverage Drupal's existing cache infrastructure and optimized cache tags for caching individual parts of the page.

Finally, the approach taken allows us to keep using some of Drupal's existing solutions like cookie-based authentication for user-specific page responses, the layout builder or even form processing.

Following up

Since there is so many details more to talk about, I'll be following up with further blog posts in the coming weeks, covering the following topics:

• Selecting a frontend framework. Server-Side-Rendering and hosting options
• Architecture variants, Authentication, Custom Elements markup & json formats
• A developer introduction: Creating components, adding Custom Element processors, the relationship to Drupal's render API, Custom routes and optimizing cache metadata.
• Handling blocks & layout builder, content previews, forms, caching & performance optimizations.

Finally, I'm going to talk more about this stack at the Drupalcon Europe 2020 in my session "Custom Elements: An alternate Render API for decoupled Drupal" - so mark your calendars!