/wakamoleguy

Hello, MDX! Hello, Islands! Goodbye, MDX!

For quite a while, I have been wanting to have this site support interactive widgets. Getting that to work was more involved than I thought. Here is an interactive counter. Click the button, number goes up!

Count: 0

The thing is, adding these "islands" of interactivity to an otherwise static site is still annoyingly difficult today. So I set off on a journey to explore how to add interactive content to this site while maintaining a simple authoring experience, a code base and pipelines that I can understand, and a minimal set of rock solid dependencies.

Granted, frameworks like Astro are built for this; I didn't have the appetite for a full site migration off of Next.js.

Enter MDX?

MDX is billed as "Markdown for the component era." It allows you to write Markdown files, sprinkle in imported components, and export metadata. This sounds great! If I have a Markdown file, I could write something like this:

---
title: 'Hello, MDX!'
---

import Counter from './components/Counter'

blah blah interesting prose

<Counter />

blah blah more interesting prose

With Next's app routing and the new-ish @next/mdx package, this seemed like a great opportunity to use a first-party framework package to introduce some nice new features to the blog. In practice, I ran into several issues.

Obstacle 1: Dynamic routes to MDX pages

My site is configured to render each post at a dynamic URL based on its ID. If you hit /p/[id], and a matching file exists, you get the page. If you enter some other unknown slug, you get a 404 error. This is important to me, as it allows me to create a new page by dropping in a single markdown file. (While you can't avoid markup, my goal is minimal overhead.)

It turns out that Turbopack can't handle this with MDX files. While the site is primarily statically built, that build still happens in two phases:

  1. Turbopack detects and bundles all routes.
  2. Next.js runs those bundles to create server-rendered static builds.

MDX support comes in only in the second phase, so the first phase fails. The workaround is to add a build step to convert the MDX files to TSX beforehand. That's doable, but it increases the complexity of the build process. The dev server also requires a separate file watcher to run that build step outside of next dev, introducing further complexity.

Instead of adding a build step, I sidestepped the issue by reading the MDX files from the filesystem and evaluating them semi-manually with @mdx-js/mdx and @mdx-js/mdx-react. (These files are trusted content written by yours truly, so evaluating them does not pose a security risk.) This ends up using React's renderToStaticMarkup under the hood. A similar process is also used for the index page's list of posts and the site's new RSS feed.

Obstacle 2: Excluding Islands from the RSS feed

As part of my desire to Publish on my Own Site, Syndicate Elsewhere (POSSE), I added an RSS feed to my site. RSS feeds use XML, which looks a lot like HTML but has a few different behaviors. One major limitation is that you can't use JavaScript and CSS. That means no interactive widgets in the RSS feed. That left me with two options: include only the descriptions of posts in the feed, or find a way to replace any interactive islands with fallback content. Since including the full text is best practice, that was my preference.

For a while, I explored whether we could have components that rendered their own fallback content. However, I hit a bigger obstacle: any attempt to import the island components failed. Islands are inherently client-side components. Take the Counter for example:

'use client'

import { useState } from 'react'
import Island from './island'

export default function Counter() {
  const [count, setCount] = useState(0)

  return (
    <p>
      Count: <strong>{count}</strong>
    </p>
    <button onClick={() => setCount((c) => c + 1)}>Increment</button>
  )
}

'use client' tells Next that this component has to be rendered on the client. It cannot be statically rendered on the server. And yet, renderToStaticMarkup is inherently a server-side rendering environment. Any attempt to import Counter will fail with a loud error message. Including <noscript> content doesn't matter if the server will never get to render it.

I still ended up using fallback content within the islands anyways, for when folks visit the live site with JavaScript disabled.

Obstacle 3: Rendering Islands from the Server

This same renderToStaticMarkup limitation actually affected the live pages, too. Because I was rendering MDX files manually in my dynamic route, even the live website was generating build-time errors when including client-side components. The solution to both issues: ditch the direct imports and maintain a mapping of components to use during evaluation.

Here's what the mapping code looks like today:

import { compiler } from 'markdown-to-jsx'
import { ComponentType, createElement } from 'react'
import Counter from './islands/counter'

type IslandName = 'Counter'
type IslandMap = Record<IslandName, ComponentType>

const liveIslands: IslandMap = { Counter }
const rssIslands: IslandMap = {
  Counter: () => (
    <p>
      <em>Interactive counter — view on the website</em>
    </p>
  ),
}

export default function NextMarkdown({
  children,
  rss = false,
}: {
  children: string
  rss?: boolean
}) {
  return (
    <>
      {compiler(children, {
        createElement,
        wrapper: null,
        overrides: rss ? rssIslands : liveIslands,
      })}
    </>
  )
}

This adds overhead when adding an island to a post. Instead of importing directly into my blog post file, I need to separately add the component to the mappings. I spent a long time looking for alternatives here. The only other viable approach was to use script tags to hand off islands using vanilla JS. That seemed significantly worse from a developer experience perspective, although it may come in handy for non-React islands in the future.

In the end, maintaining the maps is overhead that I can tolerate. After all, I don't use interactive islands very often, and when I do I need to define RSS fallback content anyways. Maintaining a live map and RSS map in the same file is not so bad.

What has MDX ever done for me?

At this point, everything was working. I could write posts in single files like I did before, and adding interactive widgets had only a small amount of overhead. The pipelines for the live posts, home page listing, and RSS feed were very similar and maintainable.

Then I realized MDX wasn't helping with any of it.

Without imports in the files, all I had was plain old Markdown with a worse metadata syntax!

export const metadata = {
title: 'Hello, MDX!'
}

blah blah interesting prose

<Counter />

blah blah more interesting prose

Rolling back to roll forward

In the end, I (asked Claude to) roll back all of the posts to their previous format. I kept the new pipelines and component maps, using markdown-to-jsx and graymatter to extract the content instead of @mdx-js/mdx. It's a bit anticlimactic, but in the end it's still fewer dependencies.

This was definitely an interesting exploration, and I learned a lot more detail about Next, Turbopack, and server-side component rendering. Having been in the Angular world at work for some time, it was a good refresher. It does leave me feeling like the ideal solution is still out there somewhere. (Or maybe, it's just Astro after all.)

Cheers!