Celestite: Reactive web apps with Crystal + Svelte

Crystal + Svelte = :zap:

Check out celestite on Github

Introduction

Celestite allows you to use the full power of Svelte reactive components in your Crystal web apps. It’s a drop-in replacement for your view layer – no more need for intermediate .ecr templates. With celestite, you write your backend server code in Crystal, your frontend client code in JavaScript & HTML, and everything works together seamlessly…and fast.

Pages are delivered to the client fully-rendered and SEO-friendly (like a traditional web app), but with the power and reactivity of Vue or Svelte taking over once the page has loaded.

This means you get the best of all worlds – clients see lightning fast page loads (since the initial payload is pure HTML/CSS) yet still get the benefit of rich dynamic JavaScript-enabled components the load without stutters or flickers.

And your job as a developer is easier: Instead of writing an intermediate view layer (typically .ecr or similar templates) which glues your server code & client code together, you write pure components once with all your server-side data seamlessly blended in. Awesome, right?

[Aside: Celestite (the mineral) is a UV-reactive crystal…see what I did there…that’s more commonly known as Celestine, but, hey, that’s a much more common project name, so here we are.]

Features

  • .svelte single-file component support
  • Server-side rendering w/ variables interpolated in Crystal and/or JavaScript
  • Client-side hydration for full-featured reactive pages
  • Plug-and-play support for Amber (other frameworks coming soon)

Background

Traditional web apps were mostly server-powered – a client (browser) would make a request, the server would do some processing, render the page, and send down HTML, CSS, and maybe a little JavaScript to spice things up. Pages loaded lightning fast and SEO was simple (crawlers saw basically the same thing as users).

With the advent of powerful JavaScript frameworks like React, Vue, and more recently, Svelte, much of the work shifted to the client. The server became a vessel for a client-loaded JavaScript bundle (and an API endpoint to communicate with that client) and your browser rendered most of what you’d see on the page via JavaScript. Web apps certainly became more powerful and dynamic (the notion of a Single Page App was born), but at the cost of higher initial load time, worse SEO (crawlers saw basically nothing since all the logic lived in JavaScript), and increased complexity (you now had to manage two view layers).

React, Vue, and Svelte all support Server Side Rendering support to mitigate this, but that support is baked solely into NodeJS (since it’s JavaScript all the way down.)

Crystal is awesome and a joy to use for server-side web apps. Svelte is awesome and a joy to use for client-side reactivity and UI composability.

Celestite now lets you bring those joys together. This is ideal for folks who believe the heavy lifting of page rendering should (mostly) be done by the server and yet still want the reactivity and dynamism of a powerful client-side framework.

THIS IS PREVIEW / EARLY ALPHA SOFTWARE

This is not much more than a proof-of-concept at the moment, but it does work! Standard warnings apply - it will likely break/crash in spectacular and ill-timed glory, so don’t poke it, feed it past midnight, or use it for anything mission-critical (yet).

Show Me The Money

Start with a regular ‘ol Amber controller:

# home_controller.cr

class HomeController < ApplicationController
  def index
    example_crystal_data = 1 + 1
    render("index.ecr")
  end
end

and .ecr template:

<!-- index.ecr -->

<div>
  <h1>Hello World!</h1>
  <p>This was rendered server-side (by crystal): <%= example_crystal_data %></p>
</div>

example_crystal_data is simple arithmetic here, but imagine it’s a database query in a more complex application). In this simple example, the basic .ecr template works perfectly.

If you want to add some client-side reactivity and, say, a Svelte component, things get more complex. You’ll have to split up your view logic between this .ecr file (which becomes rendered HTML) and the component (which doesn’t get rendered until the client finishes loading), pass in example_crystal_data via props, load it via a backchannel API call, or use some other series of intermediate glue steps.

Why not skip all of that and make the data available directly in the Svelte component? Replace the render call with our new celestite_render macro and move our server-side calculation into a special Hash:

# home_controller.cr - celestite powered

class HomeController < ApplicationController
  def index
    my_data = 1 + 1
    context = Celestite::Context{:my_data => my_data}
    celestite_render(context)
  end
end

Now instead of a server-rendered .ecr template + client-rendered Svelte component, you skip the .ecr step entirely and put all of your view code (including CSS) into a single Svelte component that’s both server- and client-rendered:

<!-- Index.svelte -->
<script>
  import { stores } from "@sapper/app";
  const { session } = stores();
</script>

<style>
  h1 {
    color: blue;
  }
</style>

<h1>Hello World!</h1>
<p>This was rendered server-side (by crystal): {$session.my_data}</p>

Et voila:

Ok that’s not so exciting. And it’s a bit more verbose. But there’s more to this, I promise! The key is that number 2 is not just a static ASCII character loaded via HTML – it’s actually a fully reactive element of a live Svelte component:

A quick peek at the raw HTML shows our server’s doing all the hard work (which is what servers were designed to do) and sending us a fully rendered page that, even if it were more complex, would load quickly and be SEO-friendly:

❯ curl localhost:3000

<!-- Trimmed for brevity -->

<html>
  <head>
    <style>
      h1.svelte-9noqbk {
        color: blue;
      }
    </style>
  </head>
  <body>
    <div id="celestite-app">
      <h1 class="svelte-9noqbk">Hello World!</h1>
      <p>This was rendered server-side (by crystal): 2</p>
    </div>
    <script>
      __SAPPER__ = { baseUrl: "", preloaded: [void 0, {}], session: { my_data: 2 } };
    </script>
    <script src="/client/4814d416e4a6675ff44d/main.js"></script>
  </body>
</html>

There’s a lot going on in there and we’ll get into more detail later, but in short: Celestite is taking the result of the server-side calculation (1 + 1), rendering the HTML, inserting a default layout, injecting the relevant CSS (scoped to this component), and compiling & linking to a client-side JS bundle.

Data Computation Options

Celestite gives you flexibility in where your data computation is done - on the server (in Crystal), on the server (in Node/JavaScript), or on the client (in JavaScript via Svelte). It totally depends on the needs of your application, and you can mix-and-match.

1) Server-side: Crystal

Our first example did the “work” (calculating 1 + 1) server-side, in crystal. Let’s look at the other options…

2) Server-side: JavaScript

Because celestite uses a live NodeJS process behind the scenes, you can choose to do computations in that context. This gives you access to the whole ecosystem of Node modules to use in your application as-needed.

Changing our Svelte component slightly:

<!-- Index.svelte -->

<script>
  import { stores } from "@sapper/app";
  const { session } = stores();

  let my_node_data = 2 + 2;
</script>

<style>
  h1 {
    color: blue;
  }
</style>

<h1>Hello World!</h1>
<p>This was rendered server-side (by crystal): {$session.my_data}</p>
<p>This was rendered server-side (by node): {my_node_data}</p>

Gives us, as expected:

And again, this is all being done server-side, the raw HTML tells the same story:

❯ curl localhost:3000

<!-- snip -->

<h1 class="svelte-9noqbk">Hello World!</h1>
<p>This was rendered server-side (by crystal): 2</p>
<p>This was rendered server-side (by node): 4</p>

<!-- snip -->

3) Client-side: JavaScript

Thus far we haven’t actually done any client-side DOM manipulation even though we’re using a client-side reactive framework. You can have celestite do pure (and only) client rendering if you want. We do this by taking advantage of the onMount() hook in Svelte, which is not called when doing server-side rendering.

Let’s modify our Svelte template once more:

<script>
  import { stores } from "@sapper/app";
  import { onMount } from "svelte";

  const { session } = stores();

  let my_node_data = 2 + 2;
  let my_client_data;

  onMount(() => {
    my_client_data = 4 + 4;
  });
</script>

<style>
  h1 {
    color: blue;
  }
</style>

<h1>Hello World!</h1>
<p>This was rendered server-side (by crystal): {$session.my_data}</p>
<p>This was rendered server-side (by node): {my_node_data}</p>
<p>This was rendered client-side (by Svelte): {my_client_data}</p>

And we see:

All looking good. Same with the raw HTML:

❯ curl localhost:3000

<!-- snip -->

<h1 class="svelte-9noqbk">Hello World!</h1>
<p>This was rendered server-side (by crystal): 2</p>
<p>This was rendered server-side (by node): 4</p>
<p>This was rendered client-side (by Svelte): undefined</p></div>

<!-- snip -->

Note our most recent addition is being rendered as undefined on the server because we’ve instructed our component only to perform the calculation when the component is mounted on a client.

Server rendering + Client-side hydration

Let’s make this more interesting: We want to actually have some fun, live, dynamic stuff happen on the client (which is why we’re using a framework Svelte in the first place, right?)

Celestite’s hybrid rendering model that we saw in the first example (which exposes the power of both server- and client-side rendering) is called client-side hydration, and it’s awesome.

Our fun, live, dynamic stuff is going to be a….clock. Ok fine I’ll come up with a better example later. But for now it’s simple and shows off the concept.

Our new Svelte component:

<script>
  import { onMount } from "svelte";
  let currentTime = new Date();
  function setCurrentTime() {
    currentTime = new Date();
  }
  onMount(() => {
    setInterval(setCurrentTime, 100);
  });
</script>

<p>The current time is {currentTime.toISOString()}</p>

When a request comes in, the server will render the component and send down the following (static) HTML, which represents a snapshot of the current time as of when the request came in:

❯ curl localhost:3000

<p>The current time is 2020-12-11T18:03:03.103Z</p>

In your browser, the javascript bundle will load and automatically understand that this is server-rendered Svelte code. It then “hydrates” the static element and makes it dynamic, as such:

Huzzah! Server-rendered initial view, reactive (hydrated) elements after load, and a rich dynamic (sort of) UI for the user, and all in a single piece of view code. Heaven.

How Celestite Works

Celestite launches and manages a local node process that runs a very lightweight HTTP server to perform server-side rendering. The celestite_render() macro simply makes an HTTP call to that node server with any Crystal-computed variables as parameters.

We use the Sapper framework behind the scenes. A lot of the heavy lifting for routing, etc. is handled elegantly by sapper’s file-based routing system (thank you Svelte/Sapper team!)

This is roughly the same architecture that AirBNB’s Hypernova uses, and has some of the same tradeoffs. While there’s definitely some overhead in making HTTP requests, it’s still extremely fast: A simple page render from Amber takes about 3ms on a 2015 MBP, including the back-and-forth over HTTP. This is an order of magnitude slower than a base case Amber render (approx. 300 microseconds). But it’s fast enough for now, though most certainly an area to explore for future optimization.

I’ve tried to comment the code extensively, but I’m also writing this up in more detail mostly as a high-level reference ahem and reminder to myself cough about exactly how the various internals work.

Project Status

My goal/philosophy is to release early, release often, and get as much user feedback as early in the process as possible, so even though the perfectionist in me would like to spend another 6 years improving this, by then it’ll be 2024 and who knows we might all be living underwater. No time like the present.

Contributions / Critique Wanted!

This has been a solo project of mine and I would love nothing more than to get feedback on the code / improvements / contributions. I’ve found by far the best way to learn and level-up development skills is to have others review code that you’ve wrestled with.

That is to say, don’t hold back. Report things that are broken, help improve some of the code, or even just fix some typos. Everyone (at all skill levels) is welcome.

Check out celestite on Github