hashobject / perun Goto Github PK

Might require splitting the render if half so hiccup can be modified. Or could maybe work on existing html. (#30).

But first we need highlighting lib usable from Boot. There doesn't seem to be any maintained Java implementations. Jygments looks the most useful but it is not released on maven.

Instead we could use a pod with Jython and use Pygments (a python lib). Example https://github.com/tailrecursion/boot-hoplon/blob/master/src/tailrecursion/boot_hoplon/pygments.clj.

I may just be doing something wrong, my page built with static does not hot reload while other changes with collection or render reload fine.

You can reproduce in this repo: https://github.com/sooheon/sooheon.com/blob/master/src/site/layout.clj#L68

Currently when using this together with watch task with 20 posts recompilation takes maybe 1.5 seconds. This could probably be made much faster by only parsing the changed md files.

Idea:

• Instead of writing metadata to a file, assoc it to fileset metadata
• Store metadata as a map of md file -> metadata
• Markdown task would check for changed files and update those in the metadata

I think the draft task does not provide a lot of value and could be easily emulated by using (set-env! :resource-paths ...) and the likes. Less code to maintain and users learn something about boot which might be useful to explore their own ideas etc.

@martinklepsch and @Deraen let me know what do you think are the most important tasks to work on.

I can see following potential tasks (add more if yours are different):

• add new tasks/plugins
• improve existing plugins/tasks (E.x. I think I still have bug where sitemap ignore generated pages from the collections plugin)
• improve/Rewrite Readme
• improve error handling in Perun. E.x. validate symbol options

Java's String.replaceAll (used here) treats '$' signs as regex group match references. Most inputs containing '$' lead to IllegalArgumentExceptions (for example, injecting cljs optimized with the Google Closure Compiler).

Each post should have canonical-url property. This can be calculated as "domain"+"post url"

Currently perun related metadata is stored in :metadata in the vars metadata. Inception.
It seems like a good idea to me to use a more descriptive perun-specific key to avoid collisions with other tasks and just overall be clearer about what's in there.

Simple suggestion: :io.perun.

All files located in folders defined as :resources will be candidates for the perun tasks. Is there a way to specify a dedicated location/fileset that will be available to perun tasks only?

The render and collection task implementations could be split into render-hiccup, collection-hiccup and write-html tasks I think.

A use case would be to do some transformations to hiccup in between:

(comp (render-hiccup :rendered 'foobar/post)
      (highlight)
      (write-html :out "public/post/foobar.html"))

render and collection could still work as is, they would just combine two other tasks:

(deftask render [...]
  (comp (render-hiccup :rendered renderer) (write-html :out ...)))

I could implement this when I have need for this. For now I'm using Highlight.js.

How can I trigger the browse window to reload whenever the boot watch task reruns?

Hi, I would like to suggest a little change to the current logo on the GitHub front page.

Currently the pinkish and blueish triangles seem misaligned: the lower left edge of the pink triangle passes close to the left edge of the blue internal triangle. It seems a bit off.

You could possibly fix it by:

• aligning the edges more precisely,
• making the triangles non-transparent.

Just a suggestion. Thanks.

RSS doesn't contain the complete post content so it's quite useless, I guess Atom XML is more common these days. Luckily it's trivial to write the XML using data.xml even in the project, but I guess this is so common that we should provide an existing task for this:

(ns blog.views.atom
  (:require [clojure.data.xml :as xml]
            [blog.dates :refer [datestr]]
            [blog.views.common :as common]))

(defn render
  [{:keys [author base-url site-title]}
   posts]
  (xml/emit-str
    (xml/sexp-as-element
      [:feed {:xmlns "http://www.w3.org/2005/Atom"}
       [:title site-title]
       [:link {:href (str base-url "atom.xml") :rel "self"}]
       [:link {:href base-url}]
       ; [:updated "fixme"]
       [:id base-url]
       [:author
        [:name (:name author)]
        [:email (:email author)]]
       (for [{:keys [canonical-url content name date-published]} (take 10 posts)]
         [:entry
          [:title name]
          [:link canonical-url]
          ; (if date-updated [:updated date-updated])
          [:content {:type "html"} (str content)]
          ])])))

Both are used but it's probably best to stay consistent

Probably a good idea to establish some sort of namespacing for metadata i.e. perun/markdown can only write to :perun/markdown key in metadata.

When the site-title is supplied as task option the atom-paths function loses it the moment it uses grouped-paths: https://github.com/hashobject/perun/blob/master/src/io/perun.clj#L1022

Merging in (dissoc options :grouper :sortby :comparator) here: https://github.com/hashobject/perun/blob/master/src/io/perun.clj#L794-L796 should fix the issue but I'm not sure if thats the right solution.

... and decide how to pronunce "Perun".

https://en.wikipedia.org/wiki/International_Phonetic_Alphabet

Metadata in perun should be keyed by the path relative to the fileset instead of filenames. Plain filenames can be overriden by files with the same name.

I'm also thinking of redundantly adding the path to the metadata map which would easily allow filtering and grouping by path without having to deal with MapEntries all the time.

I would like to suggest that the readme and perun.io site ought to link to the Getting Started guide.

In fact perhaps the site itself would benefit from some concise instructions?

Depending on which phase this is done, might require splitting rending task (#30) or something. Best candidates are probably to hyphenate hiccup data or html.

I'm working on converting python hyphenation implementation to Clojure and once that is done it should be quite easy to implement a task for this.

Waiting for theJohnnyBrown/endophile#8

I've added this in a fork over here. I wanted this so I could use Pandoc instead of Endophile for markdown parsing. Would you be interested in supporting this ?

;; argument is an edn vector of strings
(markdown :cmd ["pandoc"])

Thanks for the library by the way, really cool idea.

I think it is time :)

I'm thinking about creating tags plugin like this (https://github.com/totocaster/metalsmith-tags).
@martinklepsch, @Deraen do you need tag pages for your blogs, or it's not important right now?

@martinklepsch I think we can extract your pagination task(https://github.com/martinklepsch/martinklepsch.org/blob/master/build.boot) and put it into perun. What do you think?

E.x. https://github.com/hashobject/perun.io/blob/master/build.boot#L32 this should work without creation of fake index.md

By default, the permalink task only works if the slug ask was run previously. This caught me off-guard.

Possible solutions:

• call slug if metadata key not already present?
• throw an exception at runtime if slog is not present
• warn at runtime if slug is not present
• add documentation

I use global metadata to set the title and url of the site so it would be very useful if I could use that for feeds also. As the global metadata schema is not defined this is not very straightforward.

Alternatives:

1. Define some global metadata keys which will be used by built-in tasks if set
2. Add options to the tasks to set which keys to use

Inspired by https://github.com/Jack000/Expose/blob/master/expose.sh I'd like to add image-meta plugin that will attach following metadata to the images:

1. several common colors in the image
2. width
3. height
4. filetype

Looks like one instance was missed when the name binding was changed to title. PR incoming.

See http://jekyllrb.com/docs/frontmatter/

I'm trying to get a perun set up. I got to point where it doesn't throw any errors when I run boot build, but I does not write any files to public, while it claims that it does.

[slug] - added slugs to 2 files
[ttr] - added TTR to 2 files
[word-count] - added word-count to 2 files
[permalink] - added permalinks to 2 files
[canonical-url] - added canonical urls to 2 files
[build-date] - added date-build to 2 files
[gravatar] - added gravatar to 2 files
[render] - rendered 2 pages
[collection] - rendered collection index.html
[inject-scripts] - injected 0 scripts into 3 HTML files
[sitemap] - generated sitemap and saved to public/sitemap.xml

My build.boot file is

https://gitlab.com/200ok/200ok.gitlab.io/blob/master/build.boot

I get this error when I use the example code in the README:

 ❯ boot dev                                                                                                                                                                                    [12:26:52]
clojure.lang.ExceptionInfo: Unable to resolve var: s3-sync in this context
    data: {:file
           "/var/folders/8j/27sz_2rj0kd1b3v9y4n1_1t00000gn/T/boot.user3794536342121430912.clj",
           :line 19}
java.lang.RuntimeException: Unable to resolve var: s3-sync in this context

I checked the version on the s3-sync and it seems to match. Is this a weird Maven thing on my machine?

I have a renderer in app.static that eventually gets some records as arguments. The records are defined in namespace app.records. app.static transitively requires app.records.

When the renderer is called with it's arguments (the records) the arguments seem to be deserialized before the renderer-fn is called. Because of this I get an exception like this one:

java.lang.ClassNotFoundException: app.records.Track

I solved the issue manually now by running the following before the task is executed:

(boot.pod/require-in @perun/render-pod 'app.static) ; transitively requires app.records

but not having to do that would be nice. For some reason the require-in docs state "Avoid this function." but I'm not sure why. Maybe @micha or @alandipert can weigh in on that.

I want to have plugin where I can specify some global settings that would be accessible to all templates.
E.x. I'm thinking that base_url, site_name, copyright_dates are repetitive and can be used on any page.
Right now there os no way of specifying such options globally for all tasks.

I'm thinking creating super simple task called meta. Some thing like this:

(comp (meta {:name "Hashobject Blog" :base_url "http://blog.hashobject.com"})
          (markdown)
           (draft)
          ...)

Thank this meta can be merge with every file meta, or it can be accessible as (:global-metadata (meta fileset)).

I'm just discovering perun and I noticed it's not included in The Clojure Toolbox (http://www.clojure-toolbox.com/). It definitely should be!

I have now idea how to submit it to James Reeves though. Maybe a PR to https://github.com/weavejester/clojure-toolbox.com?

Vianney

If I change e.g. post or collection render function I have to restart boot to see the changes. I can think of two alternatives to make this work with watch task so that the changes are picked up instantly.

1. A task witch uses clojure.tools.namespaces refresh
2. render and collection tasks are instead run inside a new pod each time

I would probably prefer the second option. It should be quite fast when used with pod pool.

I'm currently doing something like this:

(p/collection :renderer 'org.martinklepsch.blog/error-page :page "error.html")

but dropping any of the passed arguments, i.e. generating a truly static html file:

(defn error-page [_ _]
  (base ...))

While using the collection task works perfectly well in this case it might make sense to provide a general task for this usecase. Potentially task options could be used to pass arguments to the render-fn.

I'm working on an Asciidoc parser using Asciidoctor. The Ruby commands are run in boot-jruby. As it is being used for documentation, the intention is to use the diagram back-end for generating PlantUML and Graphviz images to be included in the final doc. Generating pdf versions on the side would be another option, but in my opinion this goes beyond the goals of Perun.

I have a couple of points worth discussing:

1. One of our current documents has too many characters to be handled in the eval statement when handling pods (concerning the render stage currently). I'm considering a file-based solution for HTML inclusion. So the Asciidoctor plugin would generate a (partial) HTML file and adds the referred filename in say a :include key in the metadata. This allows pages to refer to this file, rather than load the value from the :content key. This slurp could also be handled within the Asciidoctor plugin to end up with a similar behavior as say the Markdown plugin.
2. Asciidoc offers several built-in attributes to handle metadata. I plan on parsing the Asciidoc file to extract these attributes and include these into the metadata to be reference.
3. Asciidoc offers the ability to evaluate the file for different inline inclusion macros. My use-case uses these ifdef statements to include code examples for different languages. We intend to generate the document for these different attributes. This evaluation could be done by parsing all ifdef commands and just evaluating all options. A more conservative way would be to either include the attributes in the plugin call, and/or announcing the desired evaluation attributes in a commented asciidoc file header.

http://wiki.languagetool.org/java-api#toc3
I think this can be very useful for the blogs.

Use case:

Our editors want to set, for example, the rel or target attribute for links while using markdown syntax. Inspired by Jekyll I extended the markdown syntax for our frontend editor like so:

[foo](bar.html){:rel "nofollow" :blank? true :title "foo bar"}

The tag is handled (replaced with final html) via re-seq/read-string before the frontend markdown parser does it's magic for the preview. Now I would like to be able to pass a :pre-parse-fn to the perun markdown task to do the same for the backend part.

Took a stab at the feature but as the markdown parsing happens in it's own pod this seems impossible?
Ideally the passed function would run after https://github.com/hashobject/perun/blob/master/src/io/perun/markdown.clj#L97

I could probably create a separate task that does the pre processing before the markdown task but would like to hear your opinions first. Happy to create a PR if this is somehow feasible.

Would you accept a PR removing +defaults+ looks like dead code and in my opinion task option defaults should probably be documented on a per task basis.

I just noticed that Pegdown has been EOL'd as of five days ago (see README: https://github.com/sirthias/pegdown), so I think a replacement is probably worth it for Perun. I've been toying with the idea of adding Pandoc support, and there are a few other mentions of this in other issues, but I'm not sure if Pandoc can support all of the same features that Pegdown does. Pegdown's deprecation notice recommends switching to flexmark-java, which is apparently a drop-in replacement.

I can integrate one or both of these, but I'd like to get some opinions on what you'd like to see, whether Pandoc, flexmark-java, some other option, or possibly multiple integrations. Let me know what you think.

See http://blog.hashobject.com/sitemap.xml.

Collection plugin should register output file in the metadata so that downstream plugins (sitemap, rss, atom) can include this file.

Probably due to recent refactorings, the draft task is currently broken: it doesn't actually cause the drafts not to be rendered (or shown in collections).

Here's a version that works for me:

(deftask remove-draft
  []
  (with-pre-wrap fileset
    (let [files (filter identity (pm/get-meta fileset))
          updated-files (->> files
                             (map #(cond-> % (:draft %) (dissoc :content :include-atom :include-rss))))]
      (pm/set-meta fileset updated-files))))

Instead of looking for metadata like date_published be default it seems more idiomatic to look for date-published. As far as I understand both are valid YAML keys.

Happy to create a PR if wanted.

reported on Slack by @jjttjj

I don't believe this would be very easy to fix in master right now, but it will be easy after a future PR. I just wanted to leave a note about it, so I don't forget.