Blog

Fetching Multi-Source Content

7/21/2020

One thing I absolutely need my website to do is to act as a portfolio for my programming work. To do that, I could just create separate project files and write all of the relevant information about them there, but... this seems to me like unnecessary duplication. Absolutely we need project files to at least tell my chosen web framework which projects I want to display, but beyond that a lot of the information I would display about them is already going to be documented elsewhere. It'll be in the project repository on GitHub, so... there's the problem to solve. How do I combine local information with project information sourced from GitHub?

First, GitHub has a nice and open API for fetching public information, including the files contained in a repository. So what jumps out to me as immediately valuable is the project's README.md. To fetch this, says the API documentation, a call to the API requiring no authorisation at the following address will do:

https://api.github.com/repos/:owner/:repo/readme

Replace :owner and :repo with their corresponding values for the sourced repository.

So I'm gonna go ahead and create a local JSON file representing this website's project information:

{
  "title": "thombruce.com",
  "description": "Personal website, blog and portfolio for Thom.",
  "github": "thombruce/thombruce.com"
}

And you can see there, I've put that :owner/:repo information into a single variable, github. If this were all I needed, I could just now call it from my page template with project.github, but I want to use that repo address to instead fetch the readme content that will populate my page.

So, first, I need to fetch the local file which will describe where to find my repo. With Nuxt Content, that goes a little something like this:

const article = await $content('projects', params.slug)
  .fetch()

This article constant will contain the variables I've written into the JSON file above. I then use axios to fetch the README.md content. Axios is established as a plugin tailored specifically to GitHub API calls in my Nuxt project like so:

// plugins/axios.js
export default function({ $axios }, inject) {
  // Create a custom axios instance
  const github = $axios.create({
    baseURL: 'https://api.github.com/repos',
    headers: {
      common: {
        Accept: 'application/json'
      }
    }
  })

  // Inject to context as $github
  inject('github', github)
}

And I can then use the $github app variable to call the API with my article constant like so:

// pages/projects/_slug.vue
const readme = await $github.get(article.github + '/readme').then((res) => {
  return Buffer.from(res.data.content, res.data.encoding).toString()
})

The content is base64 encoded by default, hence the Buffer.from(...).toString() usage to decode it.

This gives me a readme variable I can now display on my page.

We're most of the way there! But that readme content is still unrendered markdown. I need to convert it to HTML.

Since Nuxt Content supports markdown files, I took a look to see how the nuxt-content module works to convert markdown to HTML. Under the hood, it uses remark and remark-rehype to handle markdown conversion. To avoid introducing additional dependencies, I want to use the same process but I'll still need to add these as explicit dependencies to my project and figure out how I'm going to use them.

One option would be a custom component, much like the nuxt-content module behaves as mentioned above. Another option is a custom filter. Alternatively, I could modify it in place as the Buffer usage above does. The component approach seems to be the most immediately preferable, so I'll go with that: a component named something like... markdown-content or md-content. We'll go explicit and longform; markdown-content it is.

So, I've created a new file called "MarkdownContent.vue" with enough configuration to get me back into the same shape I'm currently in (it shows the markdown, but still hasn't processed it):

<template>
  <div>
    {{ markdown }}
  </div>
</template>

<script>
export default {
  props: ['markdown']
}
</script>

The documentation for remark and rehype doesn't seem particularly friendly, but I think I've narrowed the minimum dependencies for this specific purpose down to:

var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var format = require('rehype-format')
var html = require('rehype-stringify')

So I add these using Yarn:

yarn add unified remark-parse remark-rehype rehype-format rehype-stringify

And the resultant component I've made is this:

<template>
  <div v-html="html"></div>
</template>

<script>
var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var format = require('rehype-format')
var html = require('rehype-stringify')

export default {
  props: ['markdown'],
  data() {
    return {
      html: ''
    }
  },
  created() {
    unified()
      .use(markdown)
      .use(remark2rehype)
      .use(format)
      .use(html)
      .process(this.markdown, (err, file) => {
        if (err) {
          return reject(err)
        }
        this.html = file.contents
      })
  }
}
</script>

A little bit involved, but note the use of v-html to render the finalised HTML. The HTML itself is produced from the input markdown prop, which is processed as soon as the component is created. This created() function, then sets the html var used by the template.

The result is... not pretty. But it is functionally exactly what I've asked for. With time, I'll probably tweak this to support more markdown options and... one thing that's clearly missing is highlight.js, which will pretty up any code segments. But it's fine for now. It works, and when it works, we commit! Will modify iteratively from there.