magnars / optimus

A Ring middleware for frontend performance optimization.

Optimus

A Ring middleware for frontend performance optimization.

It serves your static assets:

• in production: as optimized bundles
• in development: as unchanged, individual files

In other words: Develop with ease. Optimize in production.

Features

Depending on how you use it, Optimus:

• concatenates your JavaScript and CSS files into bundles.
• adds cache-busters to your static asset URLs
• adds far future Expires headers
• minifies your JavaScript with UglifyJS 2
• minifies your CSS with clean-css
• inlines CSS imports while preserving media queries

You might also be interested in:

• optimus-autoprefixer - an asset middleware which uses autoprefixer to add vendor prefixes to CSS files.
• optimus-angular - which comes with a custom asset loader that prepopulates the Angular.JS template cache. It also has Optimus asset middleware that prepares Angular.JS code for minification.
• optimus-less - which adds a custom asset loader for LESS files.
• optimus-sass - which adds a custom asset loader for Sass/SCSS files.
• optimus-coffeescript - which adds a custom asset loader for CoffeeScript files.
• optimus-jsx - which adds a custom asset loader for React JSX files.
• optimus-img-transform - an asset middleware to transform your images' size, quality and rendering methods.

Install

Add [optimus "2023.11.21"] to :dependencies in your project.clj.

This project no longer uses Semantic Versioning. Instead we're aiming to never break the API. Feel free to check out the change log.

There were breaking changes in 0.16, 0.17, 0.19 and 2022-02-13. If you're upgrading, you might want to read more about them.

Usage

Let's look at an example:

(ns my-app.example
  (:require [optimus.prime :as optimus]
            [optimus.assets :as assets] ;; 1
            [optimus.optimizations :as optimizations] ;; 2
            [optimus.strategies :as strategies])) ;; 3

(defn get-assets [] ;; 4
  (concat ;; 5
   (assets/load-bundle "public" ;; 6
                       "styles.css" ;; 7
                       ["/styles/reset.css" ;; 8
                        "/styles/main.css"]) ;; 9
   (assets/load-bundles "public" ;; 10
                        {"lib.js" ["/scripts/ext/angular.js"
                                   #"/scripts/ext/.+\.js$"] ;; 11
                         "app.js" ["/scripts/controllers.js"
                                   "/scripts/directives.js"]})
   (assets/load-assets "public" ;; 12
                       ["/images/logo.png"
                        "/images/photo.jpg"])
   [{:path "/init.js" ;; 13
     :contents (str "var contextPath = " (:context-path env))
     :bundle "app.js"}]))

(-> app
    (optimus/wrap ;; 14
     get-assets ;; 15
     (if (= :dev (:env config)) ;; 16
       optimizations/none ;; 17
       optimizations/all) ;; 18
     (if (= :dev (:env config)) ;; 19
       strategies/serve-live-assets ;; 20
       strategies/serve-frozen-assets)) ;; 21
    (ring.middleware.content-type/wrap-content-type) ;; 22
    (ring.middleware.not-modified/wrap-not-modified)) ;; 23

1. Assets are scripts, stylesheets, images, fonts and other static resources your webapp uses.

2. You can mix and match optimizations.

3. You can choose different strategies for how you want to serve your assets.

4. Declare how to get your assets in a function.

5. It returns a list of assets.

6. The helpers in optimus.assets load files from a given directory on the classpath (normally in the resources directory). So in this case, the files are loaded from resources/public/.

7. The name of this bundle is styles.css.

8. It takes a list of paths. These paths double as URLs to the assets, and paths to the files in the public directory.

9. The contents are concatenated together in the order specified in the bundle.

10. You can declare several bundles at once with load-bundles.

11. You can use regexen to find multiple files without specifying each individually. Make sure you're specific enough to avoid including weird things out of other jars on the class path.

    Notice that angular.js is included first, even tho it is included by the regex. This way you can make sure dependencies are loaded before their dependents.

12. You can add individual assets that aren't part of a bundle, but should be optimized and served through Optimus. This is useful to add cache busters and far future Expires headers to images served straight from your HTML.

    If you use the optimus.assets helpers, you don't have to list images and fonts referenced in your CSS files - those are added along with the stylesheet.

13. Assets don't have to be files on disk. This example creates an asset on the path /init.js that is bundled along with the app.js bundle.

14. Add optimus/wrap as a Ring middleware.

15. Pass in the function that loads all your assets.

16. Pass in the function that optimizes your assets. You can choose from those in optimus.optimizations, or write your own asset transformation functions.

17. optimizations/none does nothing and returns your assets unharmed.

18. When you use optimizations/all you get everything that Optimus provides. But you can easily exchange this for a function that executes only the transformations that you need.

19. Pass in your chosen strategy. Set up properly with environment variables of some kind.

20. In development you want the assets to be served live. No need to restart the app just to see changes or new files.

21. In production you want the assets to be frozen. They're loaded and optimized when the application starts.

    Take note: You're free to serve optimized, live assets. It'll be a little slow, but what if your javascript doesn't minify well? How do you reproduce it? It's damn annoying having to restart the server for each change. Here's a way that optimizes just like production, but still serves fresh changes without restarts.

22. Since Ring comes with content type middleware, Optimus doesn't worry about it. Just make sure to put it after Optimus.

23. The same goes for responding with 304 Not Modified. Since Optimus adds Last-Modified headers, Ring handles the rest.

Using the new URLs

Since we're rewriting URLs to include cache busters, we need to access them through Optimus.

Notice that we use map, since there is likely more than one URL in development mode.

(ns my-app.view
  (:require [optimus.link :as link]))

(defn my-page
  [request]
  (hiccup.core/html
   [:html
    [:head
     (map (fn [url] [:link {:rel "stylesheet" :href url}])
          (link/bundle-paths request ["styles.css"]))]
    [:body
     (map (fn [url] [:script {:src url}])
          (link/bundle-paths request ["lib.js" "app.js"]))]]))

There's even some sugar available:

(defn my-page
  [request]
  (hiccup.core/html
   [:html
    [:head
     (optimus.html/link-to-css-bundles request ["styles.css"])]
    [:body
     (optimus.html/link-to-js-bundles request ["lib.js" "app.js"])]]))

These link-to-*-bundles will return a string of HTML that includes several script/link tags in development, and a single tag in production.

Specifying the optimizations

If you want to mix and match optimizations, here's how you do that:

(defn my-optimize [assets options]
  (-> assets
      (optimizations/minify-js-assets options)
      (optimizations/minify-css-assets options)
      (optimizations/inline-css-imports)
      (optimizations/concatenate-bundles)
      (optimizations/add-cache-busted-expires-headers)
      (optimizations/add-last-modified-headers)))

(-> app
    (optimus/wrap
     get-assets
     (if (= :dev (:env config))
       optimizations/none
       my-optimize)
     the-strategy))

Just remember that you should always add cache busters after concatenating bundles.

Adding your own asset transformation functions is fair game too. In fact, it's encouraged. Let's say you need to serve all assets from a Content Delivery Network ...

Yeah, we are using a Content Delivery Network. How does that work?

To serve the files from a different host, add a :base-url to the assets:

(defn add-cdn-base-url-to-assets [assets]
  (map #(assoc % :base-url "http://cdn.example.com") assets))

(defn my-optimize [assets options]
  (-> assets
      (optimizations/all options)
      (add-cdn-base-url-to-assets)))

This supposes that your CDN will pull assets from your app server on cache misses. If you need to push files to the CDN, you also need to save them to disk. Like this:

(defn export-assets []
  (-> (get-assets)
      (my-optimize (get-optimisation-options))
      (optimus.export/save-assets "./cdn-export/")))

You can even add an alias to your project.clj:

:aliases {"export-assets" ["run" "-m" "my-app.example/export-assets"]}

And run lein export-assets from the command line. Handy.

Hey, I'm serving my app from a sub-directory, how can I avoid referencing it everywhere?

Locally, your app is known as http://localhost:3000/, but sometimes in production it must share the limelight with others like it. Maybe it'll go live at http://limelight.com/myapp/. Wouldn't it be nice if you could do that without adding the extra folder and referencing it everywhere?

To serve the files from a directory/context path add a :context-path to the assets:

(defn add-context-path-to-assets [assets]
  (map #(assoc % :context-path "/myapp/") assets))

(defn my-optimize [assets options]
  (-> assets
      (add-context-path-to-assets)
      (optimizations/all options)))

Now your links to your assets (including those in CSS files) will reference assets with the context path + the file path.

(Note that you need to add context paths before optimizing assets, otherwise absolute references in CSS files will not include the context path).

Those are a whole lot of files being exported.

Yeah, two reasons for that:

• Optimus supports linking to individual assets even after they're bundled. If you don't want that, remove the :bundled assets.

• Optimus supports linking to assets by their original URL. If there are no external apps that need to link to your assets, remove the :outdated assets.

Like this:

(defn export-assets []
  (as-> (get-assets) assets
        (optimizations/all assets options)
        (remove :bundled assets)
        (remove :outdated assets)
        (optimus.export/save-assets assets "./cdn-export/")))

Serving bundles under a custom URL prefix

optimizations/concatenate-bundles generates a bundled asset with the "/bundle" prefix. If you need to serve assets with a different prefix, provide the :bundle-url-prefix config option to either optimizations/all or optimizations/concatenate-bundles:

(-> app
    (optimus/wrap
     get-assets
     (optimizations/all {:bundle-url-prefix "/assets/bundles"})
     the-strategy))

So how does all this work in development mode?

The paths are used unchanged. So given this example:

(-> app
    (optimus/wrap
     #(assets/load-bundle "public" "app.js"
                          ["/app/some.js"
                           "/app/cool.js"
                           "/app/code.js"])
     optimizations/none
     strategies/serve-live-assets))

When you call

(optimus.link/bundle-paths request ["app.js"])

it returns

["/app/some.js"
 "/app/cool.js"
 "/app/code.js"]

And those are served from resources/public/, or more specifically on eg. public/app/some.js on the classpath.

What about production mode?

When you use the serve-frozen-assets strategy, all the contents for each bundle is read at startup. And with optimizations/all, the URLs are generated from the hash of the contents and the identifier of the bundle.

So when you call (link/bundle-paths request ["app.js"]), it now returns:

["/bundles/d131dd02c5e6/app.js"]

and the middleware handles this URL by returning the concatenated file contents in the order given by the bundle.

What if the contents have changed?

All the contents are read at startup, and then never checked again. To read in new contents, the app has to be restarted.

No, I mean, what if someone requests an old version of app.js?

With a different hash? Yeah, then they get a 404. In production, you should serve the files through Nginx or Varnish to avoid this problem while doing rolling restarts of app servers.

Why not just ignore the hash and return the current contents?

Because then the user might be visiting an old app server with a new URL, and suddenly she is caching stale contents. Or worse, your Nginx or Varnish cache picks up on it and is now serving out old shit in a new wrapping. Not cool.

This of course depends on how your machines are set up, and how you do your rolling restarts, but it's a source of bugs that are hard to track down.

What if I need to share static files with someone else?

Well, they have no way of knowing the cache buster hash, of course. Luckily the files are still available on their original URLs.

When you're serving optimized assets, the bundles are also available. For instance: /bundles/d131dd02c5e6/app.js can also be accessed on /bundles/app.js.

Please note: You have to make extra sure these URLs are not served with far future expires headers, or you'll be in trouble when updating.

How do I handle cache busters on images?

CSS files that reference images are rewritten so that they point to cache busting URLs.

If you're using static images in your HTML, then you'll add a list of these files with optimus.assets/load-assets like point 11 in the big example.

And then grab the cache buster URL like so:

(link/file-path request "/images/logo.png")

You can also add a fallback image, if the given one doesn't exists.

(link/file-path request (str "/images/members/" (:id member) ".png")
                :fallback "/images/anonymous.png")

Sam Ritchie has written this HTML transformation using Enlive that rewrites all your image tags with no extra work. That is pretty cool!

Can I tweak how Optimus behaves?

There are some options to be tuned, but if you're planning on doing major things there's nothing wrong with writing your own strategies or optimizations. A pull request is welcome too.

Now, for the options. You pass them to the wrapper after the strategy:

(-> app
    (optimus/wrap
     get-assets optimize the-strategy
     {:cache-live-assets 2000
      :uglify-js {:mangle-names true
                  :transpile-es6? false}
      :clean-css {:level 2}}))

Values in this example are all defaults, so it's just a verbose noop.

• :cache-live-assets - Assets can be costly to fetch, especially if you're looking up lots of different regexen on the class path. Considering that this has to be done for every request in development mode, it can take its toll on the load times.

  Tune this parameter to change for how many milliseconds the live assets should be frozen. false disables the caching.

:uglify-js

• :mangle-names - When minifying JavaScript, local variable names are changed to be just one letter. This reduces file size, but disrupts some libraries that use clever reflection tricks - like Angular.JS. Set to false to keep local variable names intact.

• :transpile-es6? - UglifyJS does not support the new syntax in ES6. Set this to true to let Babel transpile ES6 code into ES5 before minification.

:clean-css

These options are passed straight to clean-css. Please see the clean-css documentation for available options.

In earlier versions of Optimus, this was a curated set of options. These old options will still work (we're trying not to break your stuff), but it is probably a good idea to take a look at all the available settings in clean-css.

Automatic compilation when assets source files change

Sometimes in development mode serving live assets may be too slow even with caching (for example if you're looking up lots of regexen on the class path). In this case you can use serve-live-assets-autorefresh strategy. This strategy will watch for changes in assets source files and recompile assets in the background whenever it's needed. Compiled assets are then cached until the next change in the source files. By default it assumes that assets sources are located in resources directory, but it can be customized with :assets-dirs config option.

(-> app
    (optimus/wrap
     get-assets optimize serve-live-assets-autorefresh
     {:assets-dirs ["resources/public"]}))

What are these assets anyway? They seem magical to me.

Luckily they're just data. The most basic operation of Optimus is serving assets from a list, with this minimal structure:

[{:path :contents}]

It serves the :contents if the request :uri matches :path.

In addition to :path and :contents, the asset map may contain:

• :bundle - the name of the bundle this asset is part of.
• :headers - headers to be served along with the asset.
• :original-path - the path before any changes was made, like cache-busters.
• :outdated - the asset won't be linked to, but is available when referenced directly.
• :base-url - prepended to the path when linking.
• :last-modified - when the asset was last modified, in milliseconds since epoch.

There's also the case that some assets may be binary. Some of them might be large. Instead of keeping those :contents in memory, they have a :resource instead, which is a URL to be served as a stream.

Built on top of that is a bunch of operations that either help you:

• Load assets to put in the list: optimus.assets
• Optimize the assets in the list somehow: optimus.optimizations
• Decide how you want to serve the assets: optimus.strategies
• Link to the assets: optimus.link

If you want to know more, the tests are a good place to start reading. They go in to all the details of how Optimus works and even has some commentary on reasoning and reasons.

Are there any working examples to look at?

Take a look at these:

• The emacsrocks.com and parens-of-the-dead.com websites both use Optimus to optimize their assets. Source here and here.

• August Lilleaas wrote a blog about using React on the server with JDK8. In the example code, he uses Optimus for frontend optimization.

• The sinonjs.org site is a static website written with Stasis, and it too uses Optimus for frontend optimization. Here's the sinon-docs repo.

Are you using Optimus in an open-source project? Please do let me know, and I'll add it to the list.

Why not split Optimus into a bunch of middlewares?

I set out to create a suite of middlewares for frontend optimization. The first was Catenate, which concerned itself with concatenation into bundles. So I certainly agree with your point. You'd be hard pressed to think otherwise in the Clojure community, I think, with its focus on "decomplecting". The reason I gave up on that idea is two-fold:

• Assets aren't first class in the Ring middleware stack.
• The different optimizations are not orthogonal.

I'll try to elaborate.

First class assets

Let's look at two examples:

• When you bundle files together, your HTML has to reference either the bundle URL (in prod) or all the individual files (in dev). There has to be some sort of lookup from the bundle ID to a list of URLs, and this is dependent on your asset-serving strategy.

• When you add cache-busters to URLs, you need some sort of lookup from the original URL to the cache-busted URL, so you can link to them with a known name.

In other words, both the bundle middleware and the cache-busting middleware either needs to own the list of assets, or it needs to rest on a first class asset concept in the stack.

Now add the ability to serve WebP images to browsers that support it. Not only do you have to change the image URLs, but you also have to serve a different set of CSS to use these new images. So this middleware would have to know which CSS files reference which files, and rewrite them.

All of these could be fixed with a well-thought out Asset concept in the Ring middleware stack. Which is what Optimus is an attempt at. It adds a list of assets to the request, with enough information for the linking functions to figure out which versions of which files to link.

Orthogonality

Different transformations aren't orthogonal. Some examples:

• You can't add cache-busters first, and then bundle assets together, since you wouldn't get cache buster URLs on your bundles.

• If you minify first, then bundle, you'll get suboptimal minification results in production. If you bundle first, then minify, you won't know which file is to blame for errors in development.

• You should never add far-future expires headers unless the asset has a cache-buster URL.

So ordering matters. You can't just throw in another middleware, you have to order it just so-and-so. I started writing documentation for this in Catenate. It would say "If you're also using cache-busting middleware, make sure to place it after Catenate." After writing a few of those sentences, I came to the conclusion that they were not entirely separate things. Since they're so dependent on each other, they should live together.

There's also the case of when to optimize. In production you want to optimize once - either as a build step, or when starting the application. In development you don't want any optimization (unless you're debugging), but you still need to create the list of assets so you're able to link to it. This is something all the optimization middlewares would have to tackle on their own - basically each layer freezing their optimized assets on server start, and all but the last one doing so in vain.

Optimus' solution

Optimus solves this by creating a separate middleware stack for optimizations, that work on assets (not requests), and that can be done at different times by different asset-serving strategies.

So yes, the optimizations have been split into several middlewares. But not middlewares for the Ring stack. They are Asset-specific middlewares.

Optimus is open for extension

Even tho Optimus itself doesn't do transpiling, building a transpiler to use with Optimus is pretty nice. I created optimus-less as an example implementation.

You create a custom asset loader and plug it into Optimus' load-asset multimethod. Let :original-url be the original "styles.less", so the linking features can find it, replace the :contents with the compiled CSS, and serve it under the :path "styles.css".

It's about 20 lines of code, including requires. And adding support for more transpilers require no changes to Optimus itself.

Can I switch JavaScript engines?

Yes.

Optimus relies on the javax.script (JSR-223) API to load a JS engine from the classpath. Optimus only adds GraalJS to the classpath, as an explicit dependency, to ensure there is a sensible default.

So it is possible to swap out GraalJS for any other javax.script-compatible JS engine that is available on the classpath. Simply add your alternative JS engine as a dependency using your project's build system. (Note: if doing so, you may wish to add Optimus' GraalJS dependencies to its :exclusions.)

Once another JS engine is made available to javax.script, Optimus can be instructed to prefer it using the environ setting :optimus-js-engines.

This can be declared in various possible ways:

• Using Leiningen: add lein-environ as a plugin. Then add :env {:optimus-js-engines "my-engine-name"} to project.clj (or other source that gets merged into the project map).
• Using Boot: add boot-environ as a dependency and invoke its environ task, passing :env {:optimus-js-engines "my-engine-name"}.
• Using Java system properties, without dependencies, pass -Doptimus.js.engines=my-engine-name.
• Using the shell's environment, ensure the variable OPTIMUS_JS_ENGINES=my-engine-name is set for the Java process running Optimus.

The value my-engine-name can be a single preferred engine, or a comma-separated list of engine names, e.g. nashorn,rhino, ordered left-to-right from most-preferred to least-preferred. Only the first, most-preferred available engine gets loaded and used.

Example: Rhino

To use Rhino, first add a dependency on a JSR-223 API for Rhino such as [cat.inspiracio/rhino-js-engine "1.7.10"] (this uses bunkenberg/rhino-js-engine).

Then if using Leiningen, add {:env {:optimus-js-engines "rhino"}} to project.clj. (You may want to add this to a profile, e.g. if switching between engines.)

Alternatively, define a Java system property in your run context, such as -Doptimus.js.engines=rhino. Or set the shell environment variable OPTIMUS_JS_ENGINES=rhino for the execution context.

Example: Nashorn

Warning: Nashorn is slow and deprecated beyond Java 9.

Using a suitable JDK that ships with Nashorn, just declare:

• :env {:optimus-js-engines "nashorn"} in Leiningen with the lein-environ plugin, or
• :env {:optimus-js-engines "nashorn"} in Boot with the environ task from boot-environ, or
• -Doptimus.js.engines=nashorn as a Java system property, or
• export OPTIMUS_JS_ENGINES=nashorn in the shell context.

What about V8 or other engines?

In past Optimus versions before we switched to javax.script, V8 was loaded using clj-v8, but clj-v8 does not expose a javax.script API. At the time of writing, there is no maintained V8 binding for general-purpose use from Java, let alone a maintained javax.script binding for V8.

If such a project should emerge, it should be possible to add it to your dependencies and declare its short name in :optimus-js-engines to load it, without any changes to Optimus itself.

Likewise, for any other JS engine that implements javax.script interfaces.

Change log

There were breaking changes in 0.16, 0.17 and 0.19. If you're upgrading, you might want to read more about them.

From 2023.10.13 to 2023.11.21

• Update clean-css to 5.3.2

  This includes exposing all the clean-css options. The legacy option format is still supported, but switching to the new one is recommended. Read more

• Heuristic for detecting already minified assets reduced.

  This used to be 5000 chars on a single line, now reduced to 1000 chars.
  This due to smaller clojurescript projects not needing 5000 chars in
  total. We're hoping people don't write 1000 chars wide lines manually.
  

• Support directly using optimized path in link/file-path

  In case you already have the optimized path for some reason, this will no
  longer trip you up.
  

From 2023-10-03 to 2023.10.13

• Add option for multiple :assets-dirs (used by live reload)

  The old :assets-dir is still supported for backwards compatibility, but no longer documented.

From 2023-02-08 to 2023-10-03

• Use nextjournal/beholder instead of juxt/dirwatch for a faster live assets autorefresh strategy. (Christian Johansen)

From 2022-02-13 to 2023-02-08

• Added option to transpile JS from ES6 to ES5 with Babel before running UglifyJS. (the-exodus)
• Bumped java.data dependency

From 0.20.2 to 2022-02-13

• Pluggable JS engine via JSR223. Execute JS optimizations w/ GraalJS, Nashorn or Rhino (radhika reddy)

  This means that the dependency on clj-v8 and its V8 binaries are gone. This allows us to:

• Add support for Windows (August Lilleaas)

This is what we have been waiting for to release the big 1.0, except that we no longer belive in semantic versioning - or to be more precise: We aim to never again break the API, removing the need.

Also:

• removed clj-time and Joda
• bumped dependency versions

From 0.20.0 to 0.20.2

• Respect context-path in export (Christian Johansen)
• Allow for spaces in url( ... ) in CSS

From 0.19.0 to 0.20

• Use a fixed version of clj-v8 that does not crash when starting two processes using it at the same time.
• Fixed an issue with files referenced in CSS files when using :base-url. (Luke Snape)
• Added support for custom :bundle-url-prefix (Christian Johansen)
• Added support for :context-path per asset (Christian Johansen)

From 0.18.5 to 0.19

• We have updated to Clojure 1.8.0 and now require JDK 1.7 or higher.
• Fixed loading issues with serve-live-assets-autorefresh

From 0.18 to 0.18.5

• Like CSS before, JS-files with a single line over 5000 characters are now considered already minified, and skipped.

• Avoid memory leak via clj-v8 contexts (Allen Rohner)

• We now designate specific versions of our JavaScript dependencies, avoiding random breakage when these packages change.

From 0.17 to 0.18

• A new strategy serve-live-assets-autorefresh is available. It watches for changes in assets source files and recompiles assets in the background whenever it's needed.
• CSS files now adopt the newest modification time of its imports
• Now doesn't fail when given outdated MicroSoft behavior CSS.

From 0.16 to 0.17

• The optimus.hiccup namespace is removed in favor of optimus.html

  The old optimus.hiccup/link-to-css-bundles created hiccup data structures, which is a mistake. It's specific to one rendering method, and Hiccup works just as well (or better) with strings.

  If you were using optimus.hiccup, just replace it will optimus.html and all is well.

From 0.15 to 0.16

• Optimus now uses clean-css instead of CSSO for minification.

  CSSO was abandoned along with quite a few bugs. clean-css is faster, has fewer bugs, a comprehensive test suite, and is under active development.

  See the new customization options for clean-css

• Passing options to optimus.prime/wrap now uses a regular map

  It used to take syntactic sugar varargs, but this actually makes it harder to use in practice. You now just pass in a map of options like a normal person. :)

• Options are now grouped

  The old {:mangle-js-names true} is now {:uglify-js {:mangle-names true}}. In the same fashion, the new options for clean-css is bundled under {:clean-css {...}}

• CSS files with a single line over 5000 characters is considered already minified, and skipped. This avoid issues with huge bootstrap.css files and its ilk.

From 0.14 to 0.15

• Add cache-busting hash to end of path instead of beginning. (Francis Avila)

  This is so the root paths are still stable and predictable. For example, if a site keeps static files under /static, cache-busted files will still be under /static.

From 0.13 to 0.14

• Support for @imports in CSS
• Binary assets now has a :resource value instead of a :get-stream function
• Removed inline clj-v8 now that official repo supports bundling in uberjars
• Include both Cache-Control and Expires headers again, for Chrome's sake
• Improved documentation and bugfixes

From 0.12 to 0.13

• Add Last-Modified headers
• Remove Cache-Control headers (superflous when serving Expires)
• Create extension point for asset loading
• Bugfixes

From 0.11 to 0.12

• Move Angular.JS features into its own project.

From 0.10 to 0.11

• Add support for :base-path on assets for CDNs.
• Add exporting of assets to disk. Also for CDNs.

From 0.9 to 0.10

• Split strategies and optimizations so they can vary independently.

Contribute

Yes, please do. And add tests for your feature or fix, or I'll certainly break it later.

Installing dependencies

You need npm installed to fetch the JavaScript dependencies. The actual fetching is automated however.

Running the tests

lein midje will run all tests.

lein midje namespace.* will run only tests beginning with "namespace.".

lein midje :autotest will run all the tests indefinitely. It sets up a watcher on the code files. If they change, only the relevant tests will be run again.

lein with-profile +rhino midje [...] will fetch and load Rhino dependencies, and execute JS code in tests with it.

lein with-profile +nashorn midje [...] will use Nashorn to execute JS code in tests, only on JDKs which ship with it.

Contributors

• Christian Johansen added CSS and JS minification, and more.
• Shaharz fixed a bug with external URLs in CSS-files.
• Francis Avila improved placement of cache busters in URLs.
• Anton Onyshchenko added the serve-live-assets-autorefresh strategy.
• Allen Rohner fixed a memory leak.
• Luke Snape fixed an issue with files referenced in CSS files when using :base-url.
• radhika reddy introduced JSR223, removing our dependency on V8 binaries.
• August Lilleaas added support for Windows.

Thanks!

License

Copyright © Magnar Sveen, since 2013

Distributed under the Eclipse Public License, the same as Clojure.

optimus's Issues

clojure.core.cache

Heads up, I was running into:

#<CompilerException java.lang.RuntimeException: No such var: clojure.core.cache/through, compiling:(clojure/core/memoize.clj:52:3)>

Until I added:

[org.clojure/core.cache "0.6.4"]

... to my project.clj.

Optimus isn't properly rewriting URLs inside minified CSS

I'm trying to understand the extent of Optimus's rewriting of URLs inside of CSS files. Looking at the tests, it looks like optimus only rewrites relative URLs to become absolute URLs. I'd like Optimus to rewrite these URLs to be proper cache busters, served via my CDN (through a base-url).

The docs seem to imply that this SHOULD be happening:

CSS files that reference images are rewritten so that they point to cache busting URLs.

In my code, I'm finding that the css bundle properly finds assets, but Optimus is not rewriting the image URLs to be proper cache busting URLs.

I'm creating the bundle like this:

(defn css-bundle []
  (assets/load-bundle "public" "guru.css"
                      ["/css/paddleguru.css"]))

And using it like this:

(defn base-styles [req]
  (concat [jquery-css twitter-bootstrap-css]
          (link/bundle-paths req ["guru.css"])))

The two referenced images in the CSS are output without cache busters, as /img/logo.png.

What a frazzled bug report. Am I doing something obviously wrong, or am I misunderstanding the scope?

Thanks!

paths starting with /

why must paths start with a slash ?

Setting up npm modules

An issue and a thought regarding installing npm dependencies for the first time.

The repository doesn't include a node_modules/ directory or package.json, and build-js-sources.sh doesn't create either of them. So the default behaviour of npm install is to use a non-cwd root, which breaks things.

It could help to either commit some node_modules/.stub file so checkouts get the directory automatically, or just to have build-js-sources.sh mkdir it.

Alternatively, do you think it would be better to let a plugin such as lein-npm handle it? This would allow npm dependencies to be declared explicitly in project.clj, too.

`realize-regex-paths` matches paths outside of the given public directory

With this directory structure:

resources/js/bootstrap.js
resources/js/google-analytics.js
resources/js/jquery-1.11.1.js
resources/static/gemmy/gemmy.js
resources/static/gemmy/phaser.js

I had this line of code:

(optimus.assets/load-bundle "js" "all.js" [#".*\.js"])

Which I expected to match the three files inside the js directory. This is what was actually matched:

(optimus.assets.creation/realize-regex-paths "js" #".*\.js")
#=> ("/bootstrap.js" 
     "/google-analytics.js"
     "/jquery-1.11.1.js"
     "Users/tom/proj/tomdalling.github.io/resources/static/gemmy/gemmy.js"
     "Users/tom/proj/tomdalling.github.io/resources/static/gemmy/phaser.js")

Which later on causes a FileNotFound exception for the path Users/tom/proj/tomdalling.github.io/resources/static/gemmy/gemmy.js.

I traced the problem back to this line:

optimus/src/optimus/assets/creation.clj

Line 87 in 512b2ed

(filter (fn [#^String p] (.contains p public-dir)))

The realize-regex-paths function doesn't test whether a path is within the public-dir directory, it just checks whether the public-dir string exists anywhere within the path. In my case, I was trying to match a directory called "js", but it also matched all javascript files with the file extension "js".

Help with building a babel.js middleware

Hi!
This is not really a bug but more of a question, apologies up front if there is a better place for this sort of thing and I will take my question there.

I want to use ES6 features in my javascript files, but I need to support IE11 so I want to make a middleware for Babel to complie my ES6 to ES5. (https://babeljs.io/docs/en/next/babel-standalone.html).

Is there a preffered example of a JS middleware that I can look at to try and get started? I tried to look through some of the other middlewares listed in the readme but they all seem to use different JS engines etc - is there a newest and best one that you'd reccomend?

(Also, thanks a lot for this awesome library - without it I don't think it would've been possible to build a customer-facing website like the one of my company!)

V8 usage causes memory leaks

My app is using https://github.com/stuartsierra/component. As is expected w/ component, I create a new system for each test. Recently, tests started failing on CircleCI because my app was using more than 4GB of RAM. As part of my system creation, I create a full optimus middleware, with all optimizations enabled.

I was clearly leaking memory, and it wasn't part of the java heap. I ran the leak down, and it is caused by optimus leaking v8 context handles in optimus.optimizations.minify

As proof, run ps -o rss <pid of your clojure repl>

then run:

(dotimes [i 1000] (v8.core/cleanup-context (v8.core/create-context)))

observe essentially no increase in memory usage. Then run:

(dotimes [i 1000] (v8.core/create-context))

And note massive increase in memory usage. (500MB -> 1500MB for me). Note that every usage of v8/create-context in minify.clj fails to call v8.core/cleanup-context. If you repeatedly load assets, or minify a large number of files, this will cause enormous memory usage.

"Maximum call stack size exceeded" during Javascript minification

Not sure what info I can provide, but this is the file that uses Optimus:

(ns fynder.exporter
  (:require [clojure.java.io :as io]
            [clojure.string :refer [replace]]
            [net.cgrand.enlive-html :as enlive :refer [at do-> content html-resource attr= prepend append html emit* transform-content]]
            [fynder.env :refer [config]]
            [optimus.assets :as assets]
            [optimus-less.core]
            [optimus.export]
            [optimus.html :refer [link-to-css-bundles link-to-js-bundles]]
            [optimus.optimizations :as optimizations]
            [stasis.core :as stasis]))

(defn js-src
  "Generate a JavaScript src reference."
  [src]
  (html [:script {:type "text/javascript"
                  :src src}]))

(defn js-tag
  "Generate a JavaScript tag that runs the given (JavaScript) code."
  [string]
  (html [:script {:type "text/javascript"} string]))

(defn loader-tag
  [page-name]
  (js-tag (format "fynder.interop.call_on_load(fynder_%s.main.main);" page-name)))

(defn rewrite-index-page
  "Rewrites the main.html page for use in production.

  This is a single-page app, and that page should be targetted at
  development. It's this function's job to do any writing necessary to
  make it production-ready."
  [{:keys [uri page-config]
    :as context}]
  (->> (at (html-resource "public/main.html")

           ;; Remove anything tagged for development with a data-info='dev' attribute.
           [(attr= :data-info "dev")]
           nil

           ;; Remap the Google Analytics tracking code
           [(attr= :data-info "ga")]
           (transform-content #(replace % #"XXXX" (:tracking-code (page-config uri))))

           ;; Remap the Heap tracking code
           [(attr= :data-info "heap")]
           (transform-content #(replace % #"XXHEAPXX" (or (:admin-heap-tracking-code (page-config uri)) "")))

           ;; ;; Add in the compiled CSS.
           [:head]
           (prepend
            (link-to-css-bundles context (:css-files (page-config uri))))

           ;; Add in the compiled ClojureScript.
           [:span#scripts-above-ie8]
           (do->
            (append (link-to-js-bundles context (:js-files (page-config uri))))
            (append (loader-tag (:page-name (page-config uri))))))

       (emit*)
       (apply str)))

(defn rewrite-loader-page
  [{:keys [uri page-config]
    :as context}]
  (->> (at (html-resource "public/loader.html")
           ;; ;; Add in the compiled CSS.
           [:head]
           (append
            (html (link-to-css-bundles context (:css-files (page-config uri)))))

           ;; Add in the compiled ClojureScript.
           [:body]
           (append
            (html
              (link-to-js-bundles context (:js-files (page-config uri))))))

       (emit*)
       (apply str)))


(defn get-assets []
  ;; TODO If we were being super-classy, we could pick the JS targets up out of the project.clj file.
  (concat (assets/load-assets  "public" [#"\.png$"
                                         #"\.jpg$"])
          (assets/load-bundles "public" {"admin.css"       ["/style/admin.less"]
                                         "trainer.css"     ["/style/trainer.less"]
                                         "mobile.css"      ["/style/mobile.less"]
                                         "sweatybetty.css" ["/style/sweatybetty.less"]
                                         "loader.css"      ["/style/loader.less"]})

          (assets/load-bundles "public"   {"preamble.js" ["/datatransfer.js"
                                                          "/vendor/es5-shim.js"
                                                          "/vendor/es5-sham.js"
                                                          "/vendor/socket.io-1.2.1.js"
                                                          "/vendor/moment.min.js"
                                                          "/vendor/fastclick.js"
                                                          "/vendor/markdown.min.js"
                                                          "/vendor/xss.js"
                                                          "/vendor/jquery-1.10.2.min.js"
                                                          "/vendor/bootstrap.min.js"
                                                          "/vendor/bootstrap-select.min.js"
                                                          "/vendor/daterangepicker.js"
                                                          "/vendor/colorpicker/js/bootstrap-colorpicker.js"
                                                          "/vendor/countdown.js"
                                                          "/vendor/papaparse.min.js"]})

          (assets/load-bundles "public" {"sweatybetty-extras.js" ["/vendor/base32.js"]})

          (assets/load-bundles "assets/js/main"   {"cljs.js"         ["/cljs.js"]
                                                   "admin.js"        ["/admin.js"]
                                                   "trainer.js"      ["/trainer.js"]
                                                   "mobile.js"       ["/mobile.js"]
                                                   "sweatybetty.js"  ["/sweatybetty.js"]})
          (assets/load-bundles "assets/js/loader" {"fynder-loader.js" ["/fynder-loader.js"]})))

(def app-configs
  {"/admin.html" {:page-name "admin"
                  :css-files ["admin.css"]
                  :tracking-code (config :admin-tracking-code)
                  :admin-heap-tracking-code (config :admin-heap-tracking-code)
                  :js-files ["preamble.js" "cljs.js" "admin.js"]}

   "/trainer.html" {:page-name "trainer"
                    :css-files ["trainer.css"]
                    :tracking-code (config :trainer-tracking-code)
                    :js-files ["preamble.js" "cljs.js" "trainer.js"]}

   "/mobile.html" {:page-name "mobile"
                   :css-files ["mobile.css"]
                   :tracking-code (config :widget-tracking-code)
                   :js-files ["preamble.js" "cljs.js" "mobile.js"]}

   "/sweatybetty.html" {:page-name "sweatybetty"
                        :css-files ["sweatybetty.css"]
                        :tracking-code (config :widget-tracking-code)
                        :js-files ["preamble.js" "cljs.js" "sweatybetty-extras.js" "sweatybetty.js"]}})

(def loader-config
  {"/loader.html" {:page-name "loader"
                   :css-files ["loader.css"]
                   :tracking-code ""
                   :js-files ["fynder-loader.js"]}})

(defn my-optimize [assets options]
  (-> assets
      (optimizations/minify-css-assets options)
      (optimizations/inline-css-imports)
      (optimizations/concatenate-bundles)
      (optimizations/add-cache-busted-expires-headers)
      (optimizations/add-last-modified-headers)))

(defn -main
  []
  (let [app-pages (zipmap (map first app-configs)
                          (repeat rewrite-index-page))

        loader {"/loader.html" rewrite-loader-page}

        pages (merge loader app-pages)

        assets (optimizations/all (get-assets) {})

        target-dir "target/site"
        config {:page-config (merge loader-config app-configs)
                :optimus-assets assets}]
    (optimus.export/save-assets assets target-dir)
    (stasis/export-pages pages target-dir config)))

And this is the stack trace:

Exception in thread "main" java.lang.Exception: Exception in /fynder-loader.js: Maximum call stack size exceeded (line undefined, col undefined), compiling:(/tmp/form-init1279519951643400366.clj:1:73)
    at clojure.lang.Compiler.load(Compiler.java:7249)
    at clojure.lang.Compiler.loadFile(Compiler.java:7175)
    at clojure.main$load_script.invoke(main.clj:275)
    at clojure.main$init_opt.invoke(main.clj:280)
    at clojure.main$initialize.invoke(main.clj:308)
    at clojure.main$null_opt.invoke(main.clj:343)
    at clojure.main$main.doInvoke(main.clj:421)
    at clojure.lang.RestFn.invoke(RestFn.java:421)
    at clojure.lang.Var.invoke(Var.java:383)
    at clojure.lang.AFn.applyToHelper(AFn.java:156)
    at clojure.lang.Var.applyTo(Var.java:700)
    at clojure.main.main(main.java:37)
Caused by: java.lang.Exception: Exception in /fynder-loader.js: Maximum call stack size exceeded (line undefined, col undefined)
    at optimus.optimizations.minify$throw_v8_exception.invoke(minify.clj:15)
    at optimus.optimizations.minify$run_script_with_error_handling.invoke(minify.clj:50)
    at optimus.optimizations.minify$minify_js.invoke(minify.clj:61)
    at optimus.optimizations.minify$minify_js_asset$fn__1850.invoke(minify.clj:67)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invoke(core.clj:630)
    at clojure.core$update_in.doInvoke(core.clj:5919)
    at clojure.lang.RestFn.invoke(RestFn.java:445)
    at optimus.optimizations.minify$minify_js_asset.invoke(minify.clj:67)
    at optimus.optimizations.minify$minify_js_assets$fn__1854.invoke(minify.clj:74)
    at clojure.core$map$fn__4550.invoke(core.clj:2622)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4125.invoke(core.clj:135)
    at clojure.core$map$fn__4550.invoke(core.clj:2614)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4125.invoke(core.clj:135)
    at clojure.core$map$fn__4550.invoke(core.clj:2614)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.next(RT.java:674)
    at clojure.core$next__4109.invoke(core.clj:64)
    at clojure.core.protocols$fn__6517.invoke(protocols.clj:151)
    at clojure.core.protocols$fn__6474$G__6469__6483.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
    at clojure.core.protocols$fn__6500.invoke(protocols.clj:81)
    at clojure.core.protocols$fn__6448$G__6443__6461.invoke(protocols.clj:13)
    at clojure.core$reduce.invoke(core.clj:6515)
    at clojure.core$group_by.invoke(core.clj:6857)
    at optimus.optimizations.concatenate_bundles$concatenate_bundles.invoke(concatenate_bundles.clj:27)
    at optimus.optimizations$all.invoke(optimizations.clj:20)
    at fynder.exporter$_main.invoke(exporter.clj:163)
    at clojure.lang.Var.invoke(Var.java:375)
    at user$eval5.invoke(form-init1279519951643400366.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:6792)
    at clojure.lang.Compiler.eval(Compiler.java:6782)
    at clojure.lang.Compiler.load(Compiler.java:7237)
    ... 11 more

Needs to handle data URLs in CSS

Now it tries to fetch it as a resource, which it isn't.

magnars optimus:- Does not support windows environment

I am trying to use this lib in my project but geting this exception: (tried on both windows xp / 7)

Exception in thread "main" java.lang.Exception: Unsupported OS/archetype: Windows XP x86, compiling:(core.clj:36:1)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3463)
at clojure.lang.Compiler.compile1(Compiler.java:7153)
at clojure.lang.Compiler.compile(Compiler.java:7219)
at clojure.lang.RT.compile(RT.java:398)
at clojure.lang.RT.load(RT.java:438)
at clojure.lang.RT.load(RT.java:411)

Customise caching for serving live assets

I would like to tweak the way assets are cached by optimus.strategies/serve-live-assets.

It would be nice to have an ability to pass through the options a function that would wrap get-optimized-assets defined in optimus.strategies/serve-live-assets and implement it's own caching.

The reason I need this is that for me serving an uncached assets is slow (page loading takes up to 5 seconds, probably because of several regular expressions in assets definition). TTL caching is not an option for me, because I wan't to see changes instantly when I edit the files. So I decided to implement a strategy with directory watching and cache invalidation when files in directory are changed. So far it's working fine for me, but I had to copy-paste all private functions from optimus.strategies.

Serving static assets gets "weird" when adding compojure in front of stasis/optimus

(defroutes app
  "This is the function we pass to Ring. It will be called with a
   request map for every request."
  (GET "/client-type-endpoints" [] (render-file "templates/client-type-endpoints.html" {:somedata "data"}))
  (route/resources "/")
  (route/not-found (-> get-pages
      (stasis/serve-pages)
      wrap-exceptions
      (optimus/wrap get-assets optimize
                    (if freeze-assets? serve-frozen-assets serve-live-assets))
                    )))

This is a snippet from my current code. We're using stasis and optimus to serve static files like markdown. However, I was in a need to add new routes to this project with more than just static content.

I added compojure and selmer because I suck at clojure, and while it works I now receive a very weird 404 on the files stasis/optimus is supposed to serve.

It renders everything perfectly except I get 404 on all static files.

An image:

As you can see, the compiled static files all gets 404 even the initial request even if it renders the correct html (which is weird indeed).

But if I do a request to the resource requested by the browser, it renders perfectly well:

The problem started when I added the compojure dependency because I wanted more simple routing, however the only way I found to send all request that didn't fit a specific route where the route/not-found function and my guess is that's what making the problem occur.

My question is if you know a simple way to keep all existing stasis/optimus code and how to add a new route that can render a specific page? If it could be done with the compojure dep it would be awesome. This might not be stasis related exactly (I think), but since I am a noob at clojure I am not sure.

Examples using selmer?

Wondering if there are any examples using optimus asset loaders in a luminus/selmer project? We are keen to try optimus but struggling to wire everything up!

Uberjar is unhappy with spaces in its absolute path

When an Optimus+Ring uberjar is run using java -jar foo-standalone.jar, it can fail to start if the jar is under a directory path that includes spaces in the name.

The jar-file-last-modified function in assets/creation.clj assumes that a URL's getPath is a file path; in fact, it returns a URL-encoded file path.

So given "/a/directory/with/a space/foo-standalone.jar", the ZipFile constructor attempts to open "/a/directory/with/a%20space/foo-standalone.jar" and fails.

(This function can be patched to decode using java.net.URLDecoder, but with a caveat that plus signs should be handled specially by percent-encoding as %2b and then decoding again... because Java.)

I'm opening this as an issue and not a PR because xsc's PR #20 appears to avoid this bug by loading directly from a URL with io/resource. Consider this a vote in favour, but I can submit a PR if #20 doesn't get merged.

Issue with :base-url that has a context path

When you're serving assets from a CDN, you can add a :base-url to each asset. This URL will be added to links to the assets. Like this:

{:path "/scripts/code.js"
 :base-url "http://cdn.example.com"}

When you call (optimus.link/file-path request "/scripts/code.js") you get http://cdn.example.com/scripts/code.js.

The problem arises in CSS files. They too link to assets, but unless you explicitly update their :contents, they won't use the :base-url. This isn't a problem in the example above. Since they're using absolute URLs, they'll use the base URL as long as they're served from there.

However, if the base URL contains a context path: http://cdn.example.com/mysite - then the absolute paths in CSS fail.

In that case, we either need to update the contents of all CSS files with the base URL. Or we could use relative paths instead.

optimus.assets/load-bundle can't find file in uberjar

At dev time with lein ring server, this works fine:

(ns app.assets
  (:require [optimus.assets :as assets]))

(defn get-assets []
  (concat
    (assets/load-bundle "." "main.js" ["/js/main.js"])))

But after

$ lein uberjar
Compiling app.assets
Created app/target/uberjar+uberjar/app-0.1.0-SNAPSHOT.jar
Created app/target/uberjar/app-0.1.0-SNAPSHOT-standalone.jar
$ java -jar target/uberjar/app-0.1.0-SNAPSHOT-standalone.jar

At runtime, I get this exception:

java.io.FileNotFoundException: /js/main.js
        at optimus.assets.creation$existing_resource.invoke(creation.clj:25)
        at optimus.assets.creation$load_text_asset.invoke(creation.clj:60)
        at optimus.assets$fn__185.invoke(assets.clj:5)
        at clojure.lang.MultiFn.invoke(MultiFn.java:231)
        at optimus.assets.creation$load_asset_and_refs.invoke(creation.clj:71)
        at optimus.assets.creation$load_assets$fn__149.invoke(creation.clj:99)

The file is in resources:

$ ls resources/js
main.js

And the file does appear to be in the root of the jar:

$ jar tf target/uberjar/app-0.1.0-SNAPSHOT-standalone.jar | grep 'js\/main\.js'
js/main.js

What am I missing?

Issue locating assets in jar file

Optimus is working nicely in our dev environment, but in packages builds we see exceptions like the one below. I've verified, in this case that the jar does contain the indicated file a the expected path.

Exception in thread "main" java.lang.ExceptionInInitializerError
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:344)
    at clojure.lang.RT.loadClassForName(RT.java:2093)
    at clojure.lang.RT.load(RT.java:430)
    at clojure.lang.RT.load(RT.java:411)
    at clojure.core$load$fn__5066.invoke(core.clj:5641)
    at clojure.core$load.doInvoke(core.clj:5640)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5446)
    at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
    at clojure.core$load_lib.doInvoke(core.clj:5485)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$load_libs.doInvoke(core.clj:5524)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:626)
    at clojure.core$require.doInvoke(core.clj:5607)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at ring.server.leiningen$load_var.invoke(leiningen.clj:7)
    at ring.server.leiningen$serve.invoke(leiningen.clj:14)
    at clojure.lang.Var.invoke(Var.java:379)
    at gropius.handler.main$_main.invoke(main.clj:1)
    at clojure.lang.AFn.applyToHelper(AFn.java:152)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at gropius.handler.main.main(Unknown Source)
Caused by: java.lang.IllegalArgumentException: Not a file: jar:file:/Users/andy/code/kontor/gropius/target/gropius.jar!/public/sass/admin/admin.scss
    at clojure.java.io$fn__8588.invoke(io.clj:63)
    at clojure.java.io$fn__8572$G__8556__8577.invoke(io.clj:35)
    at clojure.java.io$file.invoke(io.clj:414)
    at optimus_sass.core$load_sass_asset.invoke(core.clj:18)
    at optimus_sass.core$fn__9064$fn__9065.invoke(core.clj:27)
    at clojure.lang.MultiFn.invoke(MultiFn.java:231)
    at optimus.assets.creation$load_asset_and_refs.invoke(creation.clj:71)
    at optimus.assets.creation$load_assets$fn__3613.invoke(creation.clj:99)
    at clojure.core$map$fn__4245.invoke(core.clj:2557)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:484)
    at clojure.core$seq.invoke(core.clj:133)
    at clojure.core$apply.invoke(core.clj:624)
    at clojure.core$mapcat.doInvoke(core.clj:2586)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at optimus.assets.creation$load_assets.invoke(creation.clj:98)
    at optimus.assets.creation$load_bundle.invoke(creation.clj:104)
    at optimus.assets.creation$load_bundles$fn__3625.invoke(creation.clj:113)
    at clojure.core$map$fn__4245.invoke(core.clj:2559)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:484)
    at clojure.core$seq.invoke(core.clj:133)
    at clojure.core$apply.invoke(core.clj:624)
    at clojure.core$mapcat.doInvoke(core.clj:2586)
    at clojure.lang.RestFn.invoke(RestFn.java:423)
    at optimus.assets.creation$load_bundles.invoke(creation.clj:112)
    at gropius.middleware$get_assets.invoke(middleware.clj:21)
    at optimus.strategies$serve_frozen_assets.invoke(strategies.clj:40)
    at optimus.prime$wrap.doInvoke(prime.clj:4)
    at clojure.lang.RestFn.invoke(RestFn.java:470)
    at gropius.middleware$fn__9403.invoke(middleware.clj:34)
    at noir.util.middleware$wrap_middleware.invoke(middleware.clj:138)
    at noir.util.middleware$app_handler.doInvoke(middleware.clj:184)
    at clojure.lang.RestFn.invoke(RestFn.java:713)
    at gropius.handler__init.load(Unknown Source)
    at gropius.handler__init.<clinit>(Unknown Source)
    ... 25 more

CSS minifier does not handle data uri

Trying to run minifier on CSS files that contain url with data uri fails. Since it tries to retrieve resource in file system:

CompilerException java.io.FileNotFoundException:     /css/jqueryui/data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7,

How to specifiy optimizations on a per-bundle basis.

In the main example you provide three asset bundles in get-assets which are then piped through a single optimization function. What is the correct way to direct specific bundles to different optimization functions? (i.e. post images to one, while theme images to another. In order to use different transform-images settings)

Paths to assets used in Sass are not rewritten correctly

Assets like images that are used in the sass files such as background-image: url('../img/this-asset.png');do not load properly when a contextual path is in use. For example if my site renders two versions of its content at http://my-example-site.com and at http://my-example-site.com/preview/ the former would get the assets correctly but the site with the preview will not. A 404 is generated when attempting to retrieve this-asset.png
In the compiled CSS the path is rewritten to /img/this-asset-png as in background-image: url('/img/this-asset.png');
For the non-contextual instance this OK but for the /preview site it is not

Loading assets in subdirectory

How to deploy ClojureScript optimized with Optimus to Heroku?

I am working on a Clojure and ClojureScript app that is deployed to Heroku. Before using Optimus, ClojureScript was compiled on deployment by hooking lein-cljsbuild into Leiningen. Once I added Optimus, that wouldn't work anymore because Leiningen compiles the Clojure code before the ClojureScript code, which means that when this code is compiled...

(def application (-> #'app-routes
                     (optimus/wrap  get-assets
                                    ...)))

the compiled ClojureScript files (.js files) don't exist yet.

To work around this I removed the lein-cljsbuild hook from project.clj and configured Heroku using the following commands to explicitly compile ClojureScript before compiling Clojure (got the idea from this tip)

heroku config:add BUILD_CONFIG_WHITELIST=BUILD_COMMAND
heroku config:add BUILD_COMMAND="lein with-profile production do cljsbuild once, compile :all"

Now the app compiles successfully but it fails to start...

2015-01-09T18:35:13.533073+00:00 heroku[web.1]: Starting process with command `java $JVM_OPTS -cp target/myapp-standalone.jar clojure.main -m myapp.server`
2015-01-09T18:35:14.147782+00:00 app[web.1]: Picked up JAVA_TOOL_OPTIONS: -Xmx384m  -Djava.rmi.server.useCodebaseOnly=true
2015-01-09T18:35:14.254493+00:00 app[web.1]: Error: Could not find or load main class clojure.main

I understand that this failure may not be related to Optimus, but at least I would like to share and know about others' experience deploying ClojureScript optimized with Optimus. What's your approach when deploying ClojureScript optimized with Optimus? Do you know if there's a way to keep using the lein-cljsbuild hook? (I don't like the BUILD_COMMAND approach, it seems like a fragile hack, and in fact it doesn't work for my app, at least as-is.)

Can't bundle CSS containing behavior:url(#id)

Try to bundle https://github.com/mapbox/mapbox.js/ 's css, which contains:

.foo { behavior:url(#default#VML); } [1]

When trying to load the bundle, you'll get an exception like:

Caused by: java.io.FileNotFoundException: /home/ubuntu/rasterize/resources/public/bower/mapbox.js (Is a directory)
 at java.io.FileInputStream.open0 (FileInputStream.java:-2)
    java.io.FileInputStream.open (FileInputStream.java:195)
    java.io.FileInputStream.<init> (FileInputStream.java:138)
    clojure.java.io/fn (io.clj:238)
    clojure.java.io$fn__9102$G__9095__9109.invoke (io.clj:69)
    clojure.java.io/fn (io.clj:165)
    clojure.java.io$fn__9115$G__9091__9122.invoke (io.clj:69)
    clojure.java.io$reader.doInvoke (io.clj:102)
    clojure.lang.RestFn.invoke (RestFn.java:410)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.RestFn.applyTo (RestFn.java:132)
    clojure.core$apply.invoke (core.clj:632)
    clojure.core$slurp.doInvoke (core.clj:6653)
    clojure.lang.RestFn.invoke (RestFn.java:410)
    optimus.optimizations.add_cache_busted_expires_headers$get_contents.invoke (add_cache_busted_expires_headers.clj:18)
    optimus.optimizations.add_cache_busted_expires_headers$add_cache_busted_expires_header.invoke (add_cache_busted_expires_headers.clj:23)
    optimus.optimizations.add_cache_busted_expires_headers$add_cache_busted_expires_headers_in_order.invoke (add_cache_busted_expires_headers.clj:68)
    optimus.optimizations.add_cache_busted_expires_headers$add_cache_busted_expires_headers.invoke (add_cache_busted_expires_headers.clj:74)
    rasterize.http.assets$full_optimizations.invoke (assets.clj:30)
    optimus.strategies$serve_frozen_assets.invoke (strategies.clj:40)
    optimus.prime$wrap.doInvoke (prime.clj:4)
    clojure.lang.RestFn.invoke (RestFn.java:470)
    rasterize.http.assets$wrap_assets.invoke (assets.clj:38)
    rasterize.http.app.WebApp.start (app.clj:94)

It looks like what's happening is that optimus.assets.load-css first converts url(#default#VML) to an absolute url: "/bower/mapbox.js/#default#VML", and then calls remove-url-appendages which leaves "/bower/mapbox.js/"as an asset path. Code downstream tries to slurp, and throws the 'is a directory' exception.

[1] - This appears to be an IE CSS extension: http://stackoverflow.com/questions/26339276/what-is-behavior-url-property-in-css

Boot and Optimus (clj-v8) cause `java.lang.UnsatisfiedLinkError`

When using Optimus with Boot, I get this error:

clojure.lang.Compiler$CompilerException: java.lang.UnsatisfiedLinkError: Native Library /private/var/folders/t2/kbjbdrwd5dzf_hh9gjfnz_b00000gn/T/libv8.dylib.clj-v8 already loaded in another classloader, compiling:(v8/core.clj:36:1)
         java.lang.UnsatisfiedLinkError: Native Library /private/var/folders/t2/kbjbdrwd5dzf_hh9gjfnz_b00000gn/T/libv8.dylib.clj-v8 already loaded in another classloader

How close are we to moving from clj-v8 to Nashorn? Is this fixable before the move?

/CC @micha @alandipert @pbiggar

Can't minimize bootstrap theme with external resource

Hello!

I downloaded theme "Superhero" from http://bootswatch.com and Optimus thrown an exception on external resource in css file (bootstrap.css):

@import url("//fonts.googleapis.com/css?family=Lato:300,400,700");

If you remove this line it will be compiled.

PS: It is not very major issue for me, but maybe for somebody..

ClojureScript module

Would it make sense to have an optimus-clojurescript module or is there too much overlap with lein-cljsbuild?

CSS minifier fails with DOS line endings

undefined:13: SyntaxError: Unexpected token ILLEGAL
        var compressed = compressor.compress(srcToCSSP('@font-face {
\n  font-
                                                       ^^^^^^^^^^^^^

Content Encoding

Quick question: I have some precompressed (gzipped) javascript assets I'd like to serve via Optimus, however, they need "Content-Encoding: gzip" attached as a response header.

Any suggestion for how to accomplish that?

Uglify removes seemingly unused global vars

hmm have you ever encountered js minifier used by optimus to remove vars defined outside functions ?

var appletId;
var appletRef;
var s = "asasasdasd";
function appletReady(){
    alert(s);
    var d = "Ddd";
    alert(d);
    var tokens = "mytokens";
    alert(tokens);
}

Maybe they're optimized away because uglify thinks we're in a node script, and it needs to be told that it's in a browser.

Use nashorn in java 8 instead of clj-v8?

Would be nice to avoid the trouble with binaries, and also get support on windows.

Character encoding in minified js

Having the string with symbols such as: ĄČĘĖĮĮŠŲŪŪ replaces them with question marks when compiled. Though this does not seem to be the problem when trying to compile JS with such string in http://closure-compiler.appspot.com/home Closure compiler in appspot replaces them with the equivalent unicode character codes.

Source maps

Any suggestions for delivering a source map? At the end of the bundle.js file there is

//# sourceMappingURL=bundle.js.map

But, of course, that file can't be found at /token/bundle.js.map. Any ideas?

Get rid of request dependency for asset url generation

First of all: thanks for the great library!

Now the problem: currently it's not very convenient to carry the request hash down via the function calls chain to the function that calls link/file-path.

What is the reason for storing :optimus-assets in request? Can we store them in an atom maybe and thus get rid of request dependency? I could implement that change if you are fine with this idea.

clean-css upgrade request

thanks for writing optimus!
It would be nice if the clean-css dependency could be upgraded to the latest version, as it contains some bugfixes regarding attribute merging etc.

FileNotFoundException

Sorry to pester you for a second time in a week. I suspect there's something wrong in my configuration:

java.io.FileNotFoundException: /teachbanzai/fonts/FiraSans-Extralight.eot
 at optimus.assets.creation$existing_resource.invokeStatic (creation.clj:25)
    optimus.assets.creation$existing_resource.invoke (creation.clj:24)
    optimus.assets.creation$create_binary_asset.invokeStatic (creation.clj:54)
    optimus.assets.creation$create_binary_asset.invoke (creation.clj:52)
    optimus.assets.creation$eval1967$fn__1968.invoke (creation.clj:68)
    clojure.lang.MultiFn.invoke (MultiFn.java:233)
    optimus.assets.creation$load_asset_and_refs.invokeStatic (creation.clj:71)
    optimus.assets.creation$load_asset_and_refs.invoke (creation.clj:70)
    optimus.assets.creation$load_asset_and_refs$fn__1972.invoke (creation.clj:72)

The error is thrown in the build process, and it happens when I incorporate a CSS file as a bundle:

(load-bundles "public" {"style.css" ["/teachbanzai/css/style.css"]})

The font file is indeed there, and when style.css is used without optimus, the font file is found normally. Any thoughts?

Example project

Could you please add an example project? It would be very useful for beginners.

Regex paths are not checked in dependencies

This means that you must specify the exact path when referencing assets in jar dependencies, which makes using things like WebJars awkward.

Cache-Control: immutable ?

https://bitsup.blogspot.no/2016/05/cache-control-immutable.html

Maybe this would be a nice feature. Should it be opt-in?

Failing last-modified test?

This is a vanilla checkout at 119725b.

$ lein midje

FAIL "Files in JARs report the correct last-modified time." at (assets_test.clj:34)
    Expected: [["/blank.gif" 1384517932000]]
      Actual: (["/blank.gif" 1384474732000])
FAILURE: 1 check failed.  (But 1881 succeeded.)
Subprocess failed

I'm not sure what the magic expected timestamp represents and where it comes from. Have I missed some preliminary step? Any ideas?

Possible error in documentation

From the documentation around lein export-assets

(defn export-assets []
  (-> (get-assets)
      (my-optimize options)
      (optimus.export/save-assets "./cdn-export/")))

Seems erroneous because options is not defined in this context.

optimus.strategies/serve-live-assets-autorefresh triggers NullPointerException

Attempting to call optimus.prime/wrap with optimus.strategies/serve-live-assets-autorefresh is triggering a NullPointerException for me. It looks like it's having trouble resolving 'watch-dir on this line: https://github.com/magnars/optimus/blob/master/src/optimus/strategies.clj#L52

> (optimus.homeless/jdk-version)
1.8
> (clojure-version)
"1.7.0"

Strange cache-busting behavior for `.woff2` files linked from CSS

I'm having an issue with the cache busting feature <-> font assets. With optimizations on I can see Optimus rewriting the CSS to include cache-busting prefixes, but it's still serving one of the assets w/o the prefix, which results in a 404 when the browser later attempts to fetch it. Weirdly, this appears to be only happening for one resource, a .woff2. I created a (somewhat) minimal sample repo to assist with diagnosis if that helps.

Cache-busting url replacements ignore :base-url

My ultimate goal is to get the cache-busting part of the path to not be the root, because that arrangement makes it difficult to do path filtering on web servers and such. (E.g., you can't rely on all static assets being in a common directory, e.g., /static, after cache-busting.)

I was trying to work around this issue using :base-url, but add-cache-busted-expires-header doesn't work like link/full-path and ignores the :base-url.

This issue is closely connected with #9, but more specific because it only concerns assets rewritten after cache-busting optimizations.

I'm thinking any of these solutions might be better in add-cache-busted-expires-header:

1. Include :base-url as in link/full-path, i.e. "<base-url>/<cache-buster><original-path>"
2. Allow injection of a function to determine the shape of the cache-busted url. Inputs to the function are asset and sha1-hash, output is new path.