How I'll Write a Novel

10/23/2020

Journal

So with a plan to actually write in place, and knowing that perfect is my enemy - I should just do, just do enough to get something done - it's time I thought about organising a novel or short story project... without actually getting bogged down with perfection.

There are some fundamentals I do need to consider...

  1. Where does my story text live? Is it separated into drafts?
  2. Where do character bios and location descriptions live? Where do I describe my setting?
  3. What about my outline? Is that a separate document?
  4. Do I need a place for additional notes?
  5. Oh, wait, what file format even? This probably should've been point 1.

The File Format

This is an easy one... just about. Back in the day, when I was writing fiction in secondary school, I would use Microsoft Word. Later, as I got into scriptwriting, I used Celtx and Final Draft to write screenplays, and more recently I've used Highland 2 to convert some of those old screenplays from either format into a format called Fountain. The problem with Word, Celtx and Final Draft, see, is... I believe they all store files in a binary format. This isn't an issue for most people but given they're all proprietary, it isn't easy to convert between the formats, or to parse them to HTML or to share them with others. PDF, the standard for shareable rich documents suffers similarly. I want to be able to convert to HTML so that I can manage the presentation of them online, and I want to be able to convert to an eBook format.

Highland 2 supports novel and manuscript formats as well as screenplays, and unlike screenplays it saves these as Markdown files (actually, Highland saves all three in a proprietary .highland format, but this is actually just a compressed zip file by another name and contains a .textbundle folder with the text and supporting document information).

Markdown is the ideal format, but whether I write my stories in Highland 2, I'm not sure yet. I'm also pretty fond of a writing tool in Beta called Typora.

Whereas the focus in Highland is the document itself, and provides an overview of its structure, Typora... well, it has a similar overview panel, but it also has a multiple-file side panel which would make it easier to jump between research, notes and the document itself. However... Highland 2 also has a bunch of other features like Sprint mode, character analysis (though this may not work for manuscripts, so far as I know), and a nifty {{INCLUDE}} feature that can import chapters into the one document from separate files. Might have to explore using that, though I expect it would make my conversion to HTML a little trickier.

So, I'll start out writing in Highland 2 and saving as a .highland document. I can easily rename, unzip and extract the textbundle from this for access to the actual Markdown document which will be displayed on my website.

Hmm, scratch all of that. Highland's great, but if I can't version control it properly, I don't want to use it... See the next section for my realisation of this problem.

Draft 1, 2, 3...

Now, here's an issue. A compressed file, like a .zip or .highland file, is not simply a folder with raw text documents, even if that is what the uncompressed version is.

I know for a fact that I want to use Git version control to manage drafts, and Git does not natively support binary format files like that. This means that either...

  1. I'm going to have to reconsider my choice of the .highland format above
  2. I should use Git LFS (Large File Storage) to manage the documents

...but there is a potential issue with that.

I believe the way that Git LFS works is to save a new copy of the large file whenever it is changed, meaning multiple copies saved on whatever git server I choose to host this. This usually isn't a problem, as I use Git LFS to store things like images a lot... these just aren't changed all that frequently.

But a novel will be updated, committed and changed very frequently, meaning I could easily exceed the storage limit entirely unnecessarily.

Yes, a compressed file does mean less storage space is being used for the file itself but... that gain could be offset by the fact that it needs to be stored multiple times over for drafting and version control, compared to a text document that would only ever have the latest version stored (git uses pointers to describe document changes, rather than store full copies of a document).

So, we'll use Git, but now I'm thinking... do we use .highland?

No. No, and in fact I think I would rather just use .fountain (rather than a compressed .highland file containing a .fountain document) for screenplays too. Unfortunately, Highland 2 seems to force the use of the .highland format... but for good reason; it supports all of the additional meta info that be stored along with a script or novel. It does allow the user to "export" to .fountain or Markdown, but... well, it's not gonna make saving a document swift and easy.

Typora only recognises a subset of the Markdown that Highland 2 does - for example, a = denoted synopsis can be added in Highland 2 that isn't supported in Typora. Typora, really, is a simple Markdown editor... not a screenplay or manuscript or novel editor...

...but it does save to Markdown.

When in doubt... VSCode, the reigning champ of text editors for coding, actually has support via extensions for a bunch of fancy things like previewing Markdown or Fountain files, syntax highlighting, presenting an outline.

VSCode. I'll just start with VSCode and commit text formats like Markdown and Fountain straight into Git version control.

Directory Structure

Okay, so... easy start is Markdown documents checked into Git version control, written in VSCode. This won't be a permanent setup, but because Markdown is a super clear, super open format, it will be very easy to switch between technologies on the fly should I choose to. A markdown document, in fact, can also have metadata at the top which adds additional context; something like:

---
title: This is a Title
date: 2020-10-23

---

Start markdown body...

The two sets of three dashes allow for me to add metadata in YAML format to the Markdown document. Other forms of metadata markup exist, but this is the most common and - I think - the most ideal.

Fountain actually uses the MultiMarkdown metadata format, which looks like this:

Title: My Screenplay
Date: 2020-10-23

INT. BAR - DAY

And so the screenplay began.

...this is a lot harder to parse if the data is particularly complex. I know, because I'm currently working on a parser for just that.

Anyway... So some metadata can go in the head of the document: the story title, the date, whether it belongs to a series of works, what number in that series... which means the document itself can describe some additional relationships, but one doesn't necessarily want to overcrowd that header, and some additional information might want to be stored elsewhere.

For instance, if I wanted to write a several page scene description for a location to get a feel for it... this would be a lot to put into a metadata header, but it may have value to store it alongside the work in a... "Notes" folder or... "Locations", "Info" or... docs/?

docs/ is a common folder in software development for storing the documentation for the project it lives in. It's typically a set of files that describe how to use the software, what features it may have... It's sort of a wiki for the software.

I'm thinking it makes a lot of sense to borrow this tradition for a novel, and to have a docs/ folder that contains character information, location descriptions and... so on. It doesn't have to be a strict list of things, but if it describes the story in some way without being part of it... it goes in docs/.

Now, I already have my screenplays under version control and available to read on this website. I actually put them all under the one version controlled folder, "screenplays", but I... need to make some changes. For one, they're still saved in .highland format, which I've now realised I'm not satisfied with and for two... I believe each project should have its own folder, its own git repository. Of course, questions can be raised about whether a series should be grouped together... Let's take my webseries screenplays for Last Thursday as an example. There are six episodes. Should they all live under a last-thursday folder, or should there be multiple project folders - one for each episode - such as last-thursday-1-genesis, last-thursday-2-schordingers-cat, etc. The question is: Is the series the project... or is the episode? I believe... it depends.

A series of shorts belonging to the same overarching story... It makes sense to house these in the one project. An anthology series, same deal. I'm thinking... the season is the project. So "Season 1" of a series is one project, and "Season 2" is another. A series of novel-length novels, or a series of feature length films, those should likely be individual projects.

With that in mind, we're looking at something kind of like this...

my-novel
├── README.md
├── docs
│   ├── characters
│   │   └── john-doe.md
│   └── locations
│       └── london.md
└── src
    ├── chapter-1.md
    ├── chapter-2.md
    └── chapter-3.md

...and I believe separating the text into chapters makes sense. It allows periodic publishing, more clarity when changes happen... and each chapter can have its own metadata, perhaps even saying, "these characters appear in this chapter, and these locations".

You'll see above the inclusion of a README.md, a fairly standard inclusion in software projects - particularly in open source - that offers a project overview... so a good place for a synopsis.

The src/ folder is... It's a common folder for software projects that holds the source files of an application - meaning the stuff that developers actually wrote which makes the program work. I'm borrowing it for lack of a better common term for, say... story anthologies, TV series, feature length screenplays, and novels. Just... house their actual documents under src/ so it's clear where the... uncompiled, actual written versions live. A dist/ folder may be generated later too, for instance to... store the ebook format or PDFs, whatever non-source format I may need to produce.

There's no specific outline document in the example above... Perhaps a docs/outline.md file makes sense? Or maybe an outline folder with multiple documents that can be reordered, though... well, I have some ideas how that might work but... best avoided for now. Mustn't get distracted by programming a fancy UI to manage it.

It's a start.

Project Outlines

Novel Project Outline

my-novel
├── README.md
├── docs
│   ├── characters
│   │   └── john-doe.md
│   └── locations
│       └── london.md
└── src
    ├── chapter-1.md
    ├── chapter-2.md
    └── chapter-3.md

Feature Length Screenplay Project Outline

my-screenplay
├── README.md
├── docs
│   ├── characters
│   │   └── john-doe.md
│   └── locations
│       └── london.md
└── src
    └── my-screenplay.fountain

TV Series Project Outline

my-series
├── README.md
├── docs
│   ├── characters
│   │   └── john-doe.md
│   └── locations
│       └── london.md
└── src
    ├── episode-1.fountain
    ├── episode-2.fountain
    └── episode-3.fountain

Story Anthology Project Outline

my-anthology
├── README.md
├── docs
│   ├── characters
│   │   └── john-doe.md
│   └── locations
│       └── london.md
└── src
    ├── first-story.md
    ├── second-story.md
    └── third-story.md