Could frontmatter parsing be extracted into a plugin?

[Ajedi32]

I just found out about Metalsmith while looking for alternatives to Jekyll that don't require me to put frontmatter in my markup files and so far my reaction has been... wow. You managed to capture the essence of a static site generator, and distill it down into ~345 lines of code. Seriously. I haven't been this impressed with the sheer simplicity of a project's architecture since I looked at the internals of Rack. Maybe my opinion will change once I actually try to start using Metalsmith, but for now I have to say... this looks pretty awesome.

Anyway, back to the issue at hand. While reading Metalsmith's source code, I noticed that the code which reads the YAML frontmatter into the file metadata smells suspiciously like a plugin. It takes the file data in, mutates it (adding metadata, truncating the contents), and returns.

Would you be okay with a PR which extracts that code to a plugin? Obviously it would have to be enabled by default, as there are likely plugins in the wild which rely on that metadata being present, but it still seems like a worthwhile refactoring. Also, is there anything special I should be aware of when writing such a PR? Thanks.

[Ajedi32]

Question: Is the read function considered part of Metalsmith's public API? Does it need to continue returning files with the frontmatter already parsed in order to avoid breaking backwards compatibility?

[Ajedi32]

Okay, so assuming read and readFile are part of the public API (they're documented, and not marked as private, so I guess that's a safe assumption) here's my current plan:

Frontmatter parsing will be extracted into a plugin called metalsmith-frontmatter, which Metalsmith will depend on. This plugin will expose not only the metalsmith plugin itself, but also a function for extracting frontmatter from individual file objects. This function will be used directly by the readFile method in metalsmith. This way, the current behavior of the read and readFile methods will not change, allowing this change to be incorporated into the next minor or patch release of metalsmith.

Next, I propose that Metalsmith deprecate automatic frontmatter parsing as the next major release (3.0) approaches. So in Metalsmith 2.1 (or whatever the newest version prior to 3.0 ends up being) Metalsmith would print warning messages for anyone who doesn't explicitly turn off automatic frontmatter parsing. So anyone who wants to use frontmatter parsing without the warning message would have to write something along the lines of Metalsmith(__dirname).frontmatter(false).use(frontmatter()).

In Metalsmith 3.0 frontmatter(false) would print a warning saying that frontmatter parsing is disabled by default and that setting is deprecated, and frontmatter(true) would throw an error saying to use(frontmatter()) instead. In 4.0 the frontmatter setting would be removed entirely.

[RobLoach]

Metalsmith uses gray-matter as its front-matter parser. It parses the front-matter when reading in the files.

It could be brought out into its own plugin. Definitely for a future major version, however.

[Ajedi32]

Well, it's been extracted:

https://github.com/Ajedi32/metalsmith-frontmatter

Now I just have to send a PR which makes metalsmith use that library. Thoughts?

[woodyrew]

Why does the automatic parsing of front matter need to be turned off? Is it causing issues?

[Ajedi32]

Well the goal here isn't really to turn off frontmatter parsing per-se, though that would be a side effect of completely extracting it from metalsmith, which I think is a desirable thing to do for a number of reasons.

One reason is increased modularity; this system makes it easier to swap in a different system for gathering file metadata if you so choose. This is also in keeping with Metalsmith's philosophy that "everything is a plugin".

Second is maintainability; this change simplifies the metalsmith core further, and decouples frontmatter parsing from the core, making future changes to both systems easier.

Third is simplicity; making frontmatter parsing "just another plugin" simplifies Metalsmith's API, eliminating the need to treat enabling and disabling the feature differently from any other plugin.

Basically, I just think having frontmatter parsing as a plugin is a better architecture overall.

[sethlopezme]

I would have to agree with @Ajedi32. I thought it was weird that metalsmith included frontmatter parsing, and it would make more sense for it to be a plugin like everything else.

[ianstormtaylor]

I debated it in the beginning, and even debated not turning it on by default. But the more I thought about it, it seems like 99% of cases need the front matter to be parsed, so I decided that even though it could be represented as a plugin, it wasn't worth the overhead to do it; there is overhead (I think), in understanding and code size, compared to just inlining it. (Although, that's very debatable.)

That said, I definitely appreciate the push for simplifying everything :)

[Ajedi32]

@ianstormtaylor

@dominicbarnes and I were actually just discussing that very topic over on Metalsmith's Slack channel. (I noticed you haven't joined yet. Here's the signup link if you're interested: http://metalsmith-slack.herokuapp.com/)

Anyway, my counterpoint to that argument (which I sort of touched on in my prior discussions on Slack) is that it's not really "frontmatter parsing" that is needed in those 99% of cases you mentioned. All that's really needed is some mechanism for adding metadata to files. Frontmatter parsing is one way you can do that, but certainly not the only way.

In fact, I plan to create an alternate plugin for assigning metadata to files in the near future for use in my own site. I'll link it here once it's done so you can see an example of what I mean.

[Ajedi32]

Oh! One other advantage I just thought of: having frontmatter parsing in a plugin makes it composable. For example, you could use it with metalsmith-branch to turn on frontmatter parsing for only a subset of files, or you could place it after metalsmith-s3 in the plugin list so that it parses frontmatter in downloaded files.

[Ajedi32]

Just discovered yet another advantage to having frontmatter parsing in a plugin. Using a plugin provides a clean API for passing options to the underlying frontmatter parsing library. For example, you can change the delimiter used to separate frontmatter from the rest of the file, or set TOML as the default frontmatter format.

I've updated metalsmith-matters to include this functionality.

[Ajedi32]

As promised, here is the plugin I wrote which provides an alternative to adding metadata via frontmatter: metalsmith-metafiles.

A Metalsmith plugin to read file metadata from separate files (as an alternative to frontmatter).

For example, when using this plugin you could add metadata to a file named index.html by creating a file named index.html.meta.json and putting your metadata in that file.

Personally, I prefer this method over frontmatter, as frontmatter can cause problems with linters, syntax highlighters, and other tools.

Anyway, hopefully this helps demonstrate that frontmatter parsing is not an essential part of Metalsmith. It can be replaced.