Is it possible to add a new file type and modify the associated front matter about denote

[protesilaos]

Thanks for a useful and powerful package.

You are welcome!

I am wondering if it is possible to add a new file type?

In principles, yes. We just need to know as much about it as possible in order to make the right choices.

Specifically, .rmd and / or .qmd, which use polymode. These file types are based on markdown but have embedded code-blocks that are executed by, in my main case, R. These file types use the markdown-yaml style front matter

So the only difference compared to the Markdown+YAML we support now is the file type extension? The front matter can otherwise be the same?

but I might want to add another field, such as bibliography.

It probably is possible and is documented in the manual: https://protesilaos.com/emacs/denote#h:7f918854-5ed4-4139-821f-8ee9ba06ad15. Please take a look. I can always help you if you have any questions.

Is this something I can add in my own config? Please let me know if you need any more information about this.

The .{rmd,qmd} extensions require changes on our end. I think this should be fairly easy to do.

[atanasj]

Thanks @protesilaos.

So the only difference compared to the Markdown+YAML we support now is the file type extension? The front matter can otherwise be the same?

Basically, yes. There are other options that can be added, including the bibliography section, however there are other options related to what type of file you want to convert the .rmd/.qmd file to (e.g., html, pdf, etc). But if this can be man.

I was able to add to the front matter following the manual too, thanks for that.

[protesilaos]

Just to note that we are having a discussion with @jeanphilippegg about refining the code we have for file type extensions. Your request will be granted, though it is better we do it the right way so that the code can scale to match our evolving needs.

[atanasj]

Thanks for keeping me in the loop.

[aa-parky]

Good morning Prot!

First, thank you for a fantastic package.

I have a similar question. I was reading through the documentation as was looking for a way to create a new file type.

I am using denote for my notes in plain text and also for writing notes to a gemini capsule.
gemini protocol

The gemtext files are plain text. At the moment I rsync my denote created txt files, then run a bash script to convert these to gmi extensions. This workflow has many drawbacks. Some of them can be solved by writing the transferred items to a file then using that as an exclude list for rsync. Obviously it also means having two sets of files.

I tried an experiment, after creating a note from denote, I convert the txt extension to gmi of the source file. Using denote-open-or-create on this gmi file I appear to get all the same functionality from denote regardless of the extension. Is this an acceptable solution until any possible changes to denote are implemented? Or am I potentially creating a future problem?

The extensions can readily be switched back with a simple bash script.

Many thanks again.

[protesilaos]

Hello @aa-parky!

We have the infrastructure in place we have not yet made it easy-to-access. The reason is that it is not polished and is an advanced use-case.

Here is a demo of how you can register any file type and associate relevant functions with it:

(defvar my-denote-gemtext-front-matter
  "title:      %s
date:       %s
tags:       %s
identifier: %s
---------------------------\n\n"
  "Sample of gemtext front matter (copy of `denote-text-front-matter').
It is passed to `format' with arguments TITLE, DATE, KEYWORDS,
ID.  Advanced users are advised to consult Info node `(denote)
Change the front matter format'.")

(add-to-list 'denote-file-types
             '(gemtext
               :extension ".gmi"
               :front-matter my-denote-gemtext-front-matter
               :title-key-regexp "^title\\s-*:"
               :title-value-function identity
               :title-value-reverse-function denote-trim-whitespace
               :keywords-key-regexp "^tags\\s-*:"
               :keywords-value-function denote-format-keywords-for-text-front-matter
               :keywords-value-reverse-function denote-extract-keywords-from-front-matter))

(setq denote-file-type 'gemtext)

The doc string of denote-file-types covers the technicalities.

---

Can you show me a sample gemtext file that you would like to maintain? I suppose it also applies to how links are formatted, right? Maybe it is something we can add directly to denote.el or, at the very least, I can write a detailed guide in the manual on how to implement support for it.

[aa-parky]

@protesilaos Your posted example works perfectly for my requirements. Many many thanks!

"gemtext" markup is incredibly simple. It comprises of only the following;

• Text (best written leaving "long lines" on)

• Links

`=> https://example.com A nice website
=> gopher:// example.com Gopher hole
=> gemini://example.com a Gemini Capsule'

• Headings

# Heading
## Sub-heading
### Sub-sub-heading

• Lists
  * List item

• Blockquotes
  > Written as a simple long line

• Preformatted text
  ``` starting and ending with three ```

That's it.

I am using yasnippets in conjunction with denote to take care of the links etc.

As an example below is a copy paste from a denote note;

`title: yasnippet headers and footers
date: 2022-10-23
tags: denote emacs
identifier: 20221023T120238

2022-October-23

yasnippet headers & footers

Related to:

[[denote:20221021T084413][MidJourney workflows]]

=> gemini://tilde-pi.org//Users/aapark/Documents/denote/tilde-pi/20221021T084413--midjourney-workflows__emacs_midjourney.txt MidJourney workflows

---

Background

Building on from my MidJourney workflows I have created a few more snippets.
These snippets allow me to;

• Generate headers and footers
• Link to denote files and convert to gemini links
• I also created another function that strips the local path and replaces the gemini path

`

I use a normal keybind to insert a denote link
(global-set-key (kbd "C-c C-l") 'denote-link)

I use a yasnippet trigger <tl to insert the link to be readable by a gemini server

=> gemini://tilde-pi.org/${1:$$(unless yas-modified-p (completing-read "File: " 'read-file-name-internal))} ${2:LinkName}
I run the following to remove the local path

(defun gemini-strip-local-filenames() (interactive) (goto-char (point-min)) (while (re-search-forward "\\/Users/aapark/Documents/denote/[a-z]*\\/" nil t) (replace-match"") (forward-line 1))

That is pretty much the workflow. I know it could be done more elegantly but it works :)

I run a few other functions to fix line spacing etc. But with your code above I no longer have to regex for txt as links contain the gmi extension already.

All in all it works really well for me and I adore denote.el !

[protesilaos]

Just to note that I am making several changes to the code so that the denote-file-types can fully support an arbitrary file type. This will greatly expand the potential of Denote. More to follow.

[protesilaos]

I started a thread on the mailing list to announce that this feature is probably complete: https://lists.sr.ht/~protesilaos/denote/%3C87a65deryt.fsf%40protesilaos.com%3E

@aa-parky The following uses the latest commit in the git repo. It registers the gemtext file type and demonstrates how to specify its link formats. Please read the relevant documentation. I remain at your disposal for any questions, suggestions, etc.

;; DEMO to add gemtext file type
(defvar my-denote-gmi-link-format "=> denote:%s %s")

(defvar my-denote-gmi-link-in-context-regexp
  (concat "=> " "denote:" "\\(?1:" denote-id-regexp "\\)" " .*"))

(setq denote-file-types
      '((gemtext
         :extension ".gmi"
         :date-function denote-date-iso-8601
         :front-matter denote-text-front-matter
         :title-key-regexp "^title\\s-*:"
         :title-value-function identity
         :title-value-reverse-function denote-trim-whitespace
         :keywords-key-regexp "^tags\\s-*:"
         :keywords-value-function denote-format-keywords-for-text-front-matter
         :keywords-value-reverse-function denote-extract-keywords-from-front-matter
         :link my-denote-gmi-link-format
         :link-in-context-regexp my-denote-gmi-link-in-context-regexp)
        (org
         :extension ".org"
         :date-function denote-date-org-timestamp
         :front-matter denote-org-front-matter
         :title-key-regexp "^#\\+title\\s-*:"
         :title-value-function identity
         :title-value-reverse-function denote-trim-whitespace
         :keywords-key-regexp "^#\\+filetags\\s-*:"
         :keywords-value-function denote-format-keywords-for-org-front-matter
         :keywords-value-reverse-function denote-extract-keywords-from-front-matter
         :link denote-org-link-format
         :link-in-context-regexp denote-org-link-in-context-regexp)
        (markdown-yaml
         :extension ".md"
         :date-function denote-date-rfc3339
         :front-matter denote-yaml-front-matter
         :title-key-regexp "^title\\s-*:"
         :title-value-function denote-surround-with-quotes
         :title-value-reverse-function denote-trim-whitespace-then-quotes
         :keywords-key-regexp "^tags\\s-*:"
         :keywords-value-function denote-format-keywords-for-md-front-matter
         :keywords-value-reverse-function denote-extract-keywords-from-front-matter
         :link denote-md-link-format
         :link-in-context-regexp denote-md-link-in-context-regexp)
        (markdown-toml
         :extension ".md"
         :date-function denote-date-rfc3339
         :front-matter denote-toml-front-matter
         :title-key-regexp "^title\\s-*="
         :title-value-function denote-surround-with-quotes
         :title-value-reverse-function denote-trim-whitespace-then-quotes
         :keywords-key-regexp "^tags\\s-*="
         :keywords-value-function denote-format-keywords-for-md-front-matter
         :keywords-value-reverse-function denote-extract-keywords-from-front-matter
         :link denote-md-link-format
         :link-in-context-regexp denote-md-link-in-context-regexp)
        (text
         :extension ".txt"
         :date-function denote-date-iso-8601
         :front-matter denote-text-front-matter
         :title-key-regexp "^title\\s-*:"
         :title-value-function identity
         :title-value-reverse-function denote-trim-whitespace
         :keywords-key-regexp "^tags\\s-*:"
         :keywords-value-function denote-format-keywords-for-text-front-matter
         :keywords-value-reverse-function denote-extract-keywords-from-front-matter
         :link denote-org-link-format
         :link-in-context-regexp denote-org-link-in-context-regexp)))

[protesilaos]

With the release of Denote version 1.2.0 this issue can finally be closed. Check the release notes for denote-file-types: https://protesilaos.com/codelog/2022-12-16-denote-1-2-0/.