jxson / front-matter

Extract YAML front matter from strings

front-matter

Extract meta data (front-matter) from documents.

This modules does not do any IO (file loading or reading), only extracting and parsing front matter from strings.

This concept that was originally introduced to me through the jekyll blogging system and is pretty useful where you want to be able to easily add meta-data to content without the need for a database. YAML is extracted from the the top of a file between matching separators of "---" or "= yaml =". It will also extract YAML between a separator and "...".

Install

With npm do:

npm install front-matter

Example

So you have a file example.md:

---
title: Just hack'n
description: Nothing to see here
---

This is some text about some stuff that happened sometime ago

NOTE: As of [email protected] valid front matter is considered to have the starting separator on the first line.

Then you can do this:

var fs = require('fs')
  , fm = require('front-matter')

fs.readFile('./example.md', 'utf8', function(err, data){
  if (err) throw err

  var content = fm(data)

  console.log(content)
})

And end up with an object like this:

{
    attributes: {
        title: 'Just hack\'n',
        description: 'Nothing to see here'
    },
    body: 'This is some text about some stuff that happened sometime ago',
    bodyBegin: 6,
    frontmatter: 'title: Just hack\'n\ndescription: Nothing to see here'
}

Methods

var fm = require('front-matter')

fm(string, { allowUnsafe: false })

Return a content object with two properties:

• content.attributes contains the extracted yaml attributes in json form
• content.body contains the string contents below the yaml separators
• content.bodyBegin contains the line number the body contents begins at
• content.frontmatter contains the original yaml string contents

NOTE: By default fm() uses ys-yaml's safeLoad unless you set allowUnsafe in the options object to true.

fm.test(string)

Check if a string contains a front matter header of "---" or "= yaml =". Primarily used internally but is useful outside of the module.

Returns true or false

    fm.test(string) #=> true || false

Contributing

front-matter is an OPEN Source Project so please help out by reporting bugs or forking and opening pull requests when possible.

All code is linted/formatted using standard style, any non-conforming code can be automatically formatted using the the fmt make task: make fmt.

Maintainers

• Adrian Gimenez
• Jason Campbell - @jxson

Contributors

This module is awesome because of all the folks who submitted pull requests:

• Jordan Santell - @jsantell
• Jean-Philippe Monette - @jpmonette
• Kai Davenport
• Marc-André Arseneault - @im_arsnl
• Bret Comnes - http://bret.io
• Shinnosuke Watanabe
• Matt Dickens
• Rod Knowlton
• Rich DeLauder
• Sean Lang - http://slang.cx
• Benjamin Tan - https://d10.github.io/
• Kenneth Lim - https://kenlimmj.com
• Cameron Moy
• Fernando Montoya - https://montogeek.com
• Martin Heidegger
• Leo Liang
• Kasper Pedersen

CHANGELOG

3.0.0

• CI only tests Node versions >= v6 (drops v4, and v5)

LICENSE (MIT)

Copyright (c) Jason Campbell ("Author")

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

front-matter's Issues

Do not convert entities like "."

I am using entities like somename.site in my markdown so it does not get detected by auto linking.
However, when I send this markdown through front-matter the entities seem to be resolved and end up as somename.site in the body.
Could we add an option to not auto resolve entities?

Spaces after yaml document start cause parse failure

YAML spec allows spaces after the ---

The regex used by this package prevents that.

Tests

Use js-yaml over yaml-js

https://github.com/nodeca/js-yaml is more maintained and faster.

Frontmatter date parsing oddities

Greetings,
I have a markdown file that has a date frontmatter field that expects the date in YYYY-MM-DD format. When I read the data out using front-matter (for instance, using the date '2021-04-01'), I get the date with a bunch of other crap in there. It seems to be actually parsing the date and adjusting for timezone, so in the previous instance I get 'Wed Mar 31 2021 19:00:00 GMT-0500 (Central Daylight Time)'. Is there a way to tell it not to try and infer the type without changing the format (which I can't do without breaking a lot of other stuff)? I really just need the raw text value.

Generating front matter from object

Are you interested in adding the ability to generate a front matter string in the opposite direction? IE Give the following object:

var frontmatter = { attributes: { title: 'Just hack\'n'
  , description: 'Nothing to see here'
  }
, body: 'This is some content'
}

var generated = fm.generate(frontmatter)

Where generated would then be a string containing:

---
title: Just hack'n
description: Nothing to see here

---

This is some text about some stuff that happened sometime ago

Reverse generating a file

Just curious if there are any plans to support the reverse action; which is to take a object with "attributes" and "body" and return a string to be written to a file?

Or if you would be open to accepting a PR with that functionality.

Merging with dsfm

@jxson I've written dsfm for parsing strings containing front-matter of whatever type.

Looking through your issues, I see that you've been planning to implement custom parsers and stream support. dsfm has both of these, and the API is mostly borrowed from d3's csv/tsv parser which is the best that I've been able to find for a module that allows you to implement custom parsers, but also has defaults parsers (yaml and json in this case).

Also, there is some performance improvement; see this gist, those are benchmark results from here and here. In either case clone the repo, npm install and npm run benchmark to test them yourself.

I wonder how you'd feel about maybe merging the two modules, given that you have the best name on npm and that I don't feel the need to maintain a seperate module if yours will eventually implement the same API? maybe into v3.0.0 of front-matter (but I could also extend the API of front-matter 2.x.x).

If your up for this, let me know and I'll start a branch on your repo and start making pull requests.

Cheers

FEATURE REQUEST rebundle modified front-matter object into markdown

i was trying modify date from attribute.

console.log(`File: ${file}, Creation Date: ${creationDate}`);
	const read = fs.readFileSync(path.join(process.cwd(), file), {
		encoding: "utf-8",
	});
	try {
		const parse = fm(read);
		parse.attributes.date = creationDate;
		console.log(parse);
	} catch (error) {
		console.error(`fail parse ${file}`);
	}

the main problem is, how to rebundle into markdown front-matter format ?

---
title: xxxx
any other attributes
---

body string

add the function please

Front matter Encoding < and > to &lt; and &gt;

Im sending the following bits thru to a html document using a gulp process to merge several files to turn it into a razor template
--- layout: default model: "@inherits RazorEngine.Templating.TemplateBase<BookingConfirmationEM>" name: comformation generic subject: Booking Confirmation ---

then putting it in at the top like this

{{model}}

but it is coming out the other end encouded as & lt ; and & gt ; (minus the spaces)
is there a way to stop this from happening?

Add v4 Changelogs

Please add changelogs and actions to be taken to migrate from v3 to v4

Custom separator and parser.

Since the name of this package is very generic, it would be nice to be able to specify a separators and parsers. For example, parse front-matter as CSON when a *** separator is used:

***
title: 'Hello'
greatDocumentaries: [
    'earthlings.com'
    'forksoverknives.com'
    'cowspiracy.com'
]
***

# Hello world!

v5

• Remove Yaml parsing, front-matter should only extract and allow users to choose their own parsing methods
• Refactor testing
• Setup a better CI/CQ (github actions?)
• Verify TS support is correct
• Update readme
• Cleanup issue and PR backlog

Typescript types are broken.

The types exported are broken I think... Once you put the test() method on FM it breaks and TS can't compile due to "has no call signatures".

You might want to just export two functions in a namespace or a class.

You should also export FrontMatterResult so that it's easily usable as in interface in someone else's code.

bodyBegin behaves unexpectedly

Hey there, I am using this to find the space between the front matter and markdown content so that I can insert an image after the front matter and before everything else.

The bodyBegin attribute seemed to be exactly what I was looking for, but it doesn't behave like I thought it would.

In documents with front matter and a heading (h1, h2, etc.), the bodyBegin line is the line after the heading. I thought that was an intentional choice until I tried without a heading. The bodyBegin line is then the line after the first paragraph of content.

For now, I am just subtracting the amount needed each time (1 or 2 lines). Is this how it's intended to be used though?

This attribute is really useful and hard to find in similar libraries. Yours was the only one I could find. Any help is much appreciated, thanks!

yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead

Hi! Thank you for this amazing package. Maybe I doing something wrong here, or maybe this is a new bug because of versions, but reporting it here for y'all :-)

Was following the Remix tutorial here and my application blew up with the following error, originating from front-matter.

Expected behaviour

Should parse frontmatter

Actual behaviour

Blows up with an error.

Error: Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default.
    at /Users/monarchwadia/Sandbox/2022-02-21-remix-run/action/node_modules/js-yaml/index.js:10:11
    at parse (/Users/monarchwadia/Sandbox/2022-02-21-remix-run/action/node_modules/front-matter/index.js:64:20)
    at extractor (/Users/monarchwadia/Sandbox/2022-02-21-remix-run/action/node_modules/front-matter/index.js:26:12)
    at /Users/monarchwadia/Sandbox/2022-02-21-remix-run/action/app/post.ts:22:30
    at async Promise.all (index 1)
    at Object.callRouteLoader (/Users/monarchwadia/Sandbox/2022-02-21-remix-run/action/node_modules/@remix-run/server-runtime/data.js:73:14)

Relevant files

Here are the contents of /Users/monarchwadia/Sandbox/2022-02-21-remix-run/action/app/post.ts:22:30 :

3      import parseFrontMatter from "front-matter";

// ...

22    const { attributes } = parseFrontMatter<Frontmatter>(file.toString());

My md files are the same ones from the tutorial.

What I've tried

I tried allowUnsafe: true in the options object, and that worked as expected, probably because it uses yaml.load() under the hood.

But I'm not sure if this is wise. allowUnsafe is scary :-)

Docs

Reserved characters/list requiring quoting?

We have run into some edges with the parsing:

• colons in values, including colon at the end of the line. YAML says that colons should be legal in values, but not if they have ": " colon space. I guess ":" at the end of the line is failing because of an overly broad regex. But now that I know those case, I can trap for it.
• a field value cannot start with a "#" but can exist elsewhere in the value
  Both of these edges are fine if you quote them, so I am pre-parsing the data before passing the data to the parser.

But It makes me wonder what other edge cases there are that require quoting and we're just not aware of yet. Is there a definitive list somewhere or can you point me to where in the code I can look to try to create the list?

Why only YAML support ?

Frontmatter can be written in YAML, TOML, JSON and more
Mostly all frontmatter packages including this one only support YAML.

Are there any plans on extending this package ?

Only interpret "..." at a newline?

examples/dots-ending.md shows the frontmatter ending with ...

That's great and all, but some of our YAML values end with three dots. Shouldn't the regex be bound to the start of a line?

Multiline blocks of text in front-matter

Does this accept large blocks of text?

For example:

description: |
   This is a really long description so
   it needs two lines

I couldn't get the above example to work. Is there another way to do it? Couldn't find an example in docs or tests. Thanks

Regex Match Error when parsing '---'

Problem

Attempting to extract front matter from a string containing only --- throws the error "Cannot read property 'length' of null". Encountered this because I was creating a markdown editor that accepts YAML headers, but when I started typing the opening line ---, kittens died :(

The problem appears to be with the following function:

function parse(string) {
  var match = regex.exec(string)
  var yaml = match[match.length - 1].replace(/^\s+|\s+$/g, '')
  var attributes = parser.load(yaml) || {}
  var body = string.replace(match[0], '')

  return { attributes: attributes, body: body }
}

regex.exec returns null if nothing in the requested match fails, but the next line assumes that match is defined by calling match.length.

Steps to Reproduce

See here for a MWE.

Proposed Fix

Guard the regex match with a nullity check defaulting to the empty string:

var match = regex.exec(string) || '';

Serializer too

I'd love to be able to do fm.serialize(attributes, body).

This should work:

const metadata = {foo: "bar"}
const payload = "Hi there!"
const combined = fm.serialize(metadata, payload)
const parsed = fm(combined);
assert(JSON.stringify(parsed.attributes) === JSON.stringify(metadata))
assert(parsed.body === payload)

Dead project?

Hey!

front-matter seems to receive a very small amount of updates, but it still seems to be the nicest front matter parser on NPM. Any plans of future maintenance, or would you like someone to fork it or take over maintenance?

Best regards,
Malcolm

Method for testing strings for front-matter?

Are you interested in a PR that would add a method that provided a truthy test on a string to check if it contains front-matter? IE `fm.test('string') // Returns true or false``

Is the yaml front-matter readonly?

Is this a readonly parse? If I want to change the yaml and save, what should I do?

Passing URLS through front matter

Has anyone considered / is it possible to pass HTML through front matter? I need to allow the person writing the front-matter variables to dynamically include URLS in their sentences.

Create a Security Policy

Hi team, I've noticed that there is not a security policy for this project. GitHub recommends that projects have a Security Policy (SECURITY.md). This is a simple document that explains how the project wishes to receive and handle responsible disclosure of potential vulnerabilities.

There are a few ways to receive such disclosures:

• have an email or website available to receive such reports; and/or
• use GitHub's private vulnerability reporting feature.

If you're interested in GitHub's feature, it must be activated for the repository by:

1. Open the repo's settings
2. Click on Code security & analysis
3. Click "Enable" for "Private vulnerability reporting (Beta)"

If you activate that, I can send a PR suggesting a Security Policy afterwards. Thanks!

test() doesn't check that frontmatter is at the front

You describe frontmatter in the way that I understand it: it must come at the top of a file.

However, test() doesn't check that the YAML block comes at the start of the passed string:

front-matter/index.js

Line 79 in 71f4ea0

return regex.test(string)

because regex isn't defined as being at the start of the string:

front-matter/index.js

Lines 4 to 11 in 71f4ea0

	var pattern = '^(' +
	optionalByteOrderMark +
	'(= yaml =|---)' +
	'$([\\s\\S]*?)' +
	'^(?:\\2|\\.\\.\\.)\\s*' +
	'$' +
	(platform === 'win32' ? '\\r?' : '') +
	'(?:\\n)?)'

Please either change test() to respect this constraint, or allow an extra parameter to be passed that defines whether this constraint is respected or not.

Very large size in browser bundle

When bundled for browsers, this modules adds over 170 KB in bundle size via js-yaml (38 KB) and its dependency esprima (130 KB). You can verify the estimated size on bundlephobia.

I noticed that this isn't limited to front-matter. Other packages like gray-matter and yaml-front-matter are in the same 170+ KB ballpark size. However, it's worth mentioning this in the readme (looks like it's one of the todos in #37) because the added size makes this package unsuitable for browsers.

Using a different/custom parser may solve this issue.

How to Preserve break lines

How can I preserve break lines in a long string?

---
title: Test
description: My description with multiple
lines
---

I've this error:
YAMLException: can not read a block mapping entry; a multiline key may not be an implicit key at line 4, column 13

I tried using | character but don't work

---
title: Test
description: |
My description with multiple
lines
---

Then I've tried wrap the text in single quotes, the error has gone but the break lines aren't preserved.

---
title: Test
description: 'My description with multiple
lines'
---

I apreciate your response, thanks!

Breaks with horizontal lines if there's no metadata

Something like below will break front-matter. It thinks the content between the horizontal lines is metadata.

# hello world

this is a markdown document

---

some text here

---

foo

This can be avoided by declaring an empty metadata, but maybe a proper fix is to only look for metadata in the beginning of the file.

---

---

etc

Add support for TOML

https://github.com/mojombo/toml

Uncaught SyntaxError: Unexpected token ... ，run error in Hybrid Mobile App

options = options instanceof Object ? { ...defaultOptions, ...options } : defaultOptions

run error in Hybrid Mobile App，build by webpack，
develop and web browser run ok ,

Streaming Support

Would be cool if there were an method like this:

fs.createReadStream('file.html')
   .pipe(frontMatter.stream(function(attributes, stream) {
        console.log(attributes)
        stream.pipe(fs.createWriteStream('withoutyml.html'))
   })

yaml.safeLoad()

yaml.load() is insecure and dangerous. (first google result)
We should be using yaml.safeLoad().

The types don't represent the module correctly

While front-matter exports the front-matter method using module.exports = fm the type-declaration states export default fm.
This causes TypeScript to not work correctly with the "front-matter" module.

I'll open up an issue to fix this asap.

systems using front-matter?

Is there a overview of systems using the front matter format?

Beside jeykll I found:

• metalsmith
• middleman

empty front matter makes attributes key undefined

Please add Yarn support ..

I tried yarn add front-matter --dev and it shows

[4/4] Building fresh packages...

warning "[email protected]" has no binaries
Done in 3.57s.

V3 Punchlist

Here is a meta list of items for v3, in no particular order. This is mostly a brain dump and some notes from chatting with @axdg

• Refactor the tests so they work in browser and add a SauceLabs badge #6
• Add benchmarks and add results to the README
• Refactor internal implementation to resemble axdg/dsfm (Faster Regex usage, etc.)
• Steaming support #24
• Custom parser support (maybe the only built it one is JSON, everything else should use this API to keep down dependency sizes?) #22
• Deprecate v2
• Clean up README
  • Add information about the size of the module (including dependencies) for browser users
  • Add better information for collaborators and maintainers about development workflow (testing, code reviews, publishing, etc.)

Attributes is empty

When I tried to get contents of a yml file, attributes is empty, but body have contents

Browser tests

Doesn't work when the .md file is not in the same directory as the code

When I run front-matter it only works if the file is in the same directory as the code. If I have all my .md files in a sub directory it does not parse the file correctly. Any ideas of why this is happening?

Bug when missing newline after delimiter

If the source has only front-matter and missing a newline after the ending delimiter things break.