Coder Social home page Coder Social logo

mincer's Introduction

Mincer - assets processor

Build Status NPM version

NOTE: This project is now under limited maintenance.

JavaScript port of Sprockets (v2.10.0). It features same declarative dependency management (with exactly same language) for CSS and JavaScript and preprocessor pipeline. Mincer allows you to write assets in the languages like: CoffeeScript, LESS, Stylus and others. Moreover mincer has advanced built-in features, not available in sprockets:

  • sourcemaps support
  • macros support (nice alternative to EJS)

See Sprockets, Mincer API Documentation and Mincer examples for more details.

Supported engines are described in Wiki. If you wish to add new engine support - read tutorial. Also you can search existing extensions in npm.

Notice on upgrade to 1.4.x

If your project is using autoprefixer or csswring processors:

  • upgrade csswring to >=4.x and autoprefixer to >= 6.x
    • if you used legacy autoprefixer-core package, replace it with autoprefixer
  • include postcss >= 4.1

Installation

Install Mincer from npm registry:

$ npm install mincer

Using Mincer from CLI

To use Mincer from CLI, you will need to install it globally:

$ npm install mincer -g

Usage is really simple (see mincer -h for details):

$ mincer --include assets/javascripts \
         --include assets/stylesheets \
         --output public/assets \
         application.js application.css

If you are using mincer CLI often, you would probably want to "preset" some of the options/arguments for your project. Just create .mincerrc file and put argument you want in it. For example:

--include assets/javascripts --include assets/stylesheets --output public/assets

Understanding the Mincer Environment

You'll need an instance of the Mincer.Environment class to access and serve assets from your application.

The Environment has methods for retrieving and serving assets, manipulating the load path, and registering processors. It is also used by Mincer.Server which can be mounted directly as request event handler of http.Server or as connect middleware.

The Load Path

The load paths is an ordered list of directories that Mincer uses to search for assets.

In the simplest case, a Mincers environment's load path will consist of a single directory containing your application's asset source files. When mounted, server will serve assets from this directory as if they were static files in your public root.

The power of the load path is that it lets you organize your source files into multiple directories -- even directories that live outside your application -- and combine those directories into a single virtual filesystem. That means you can easily bundle JavaScript, CSS and images into a library and import them into your application.

Manipulating the Load Path

To add a directory to your environment's load path, use the appendPath and prependPath methods. Directories at the beginning of the load path have precedence over subsequent directories.

environment = new Mincer.Environment();
environment.appendPath('app/assets/javascripts');
environment.appendPath('lib/assets/javascripts');
environment.appendPath('vendor/assets/jquery');

In general, you should append to the path by default and reserve prepending for cases where you need to override existing assets.

Accessing Assets

Once you've set up your environment's load path, you can mount the environment as a server and request assets via HTTP. You can also access assets programmatically from within your application.

Logical Paths

Assets in Mincer are always referenced by their logical path.

The logical path is the path of the asset source file relative to its containing directory in the load path. For example, if your load path contains the directory app/assets/javascripts:

Asset source file Logical path
app/assets/javascripts/application.js application.js
app/assets/javascripts/models/project.js models/project.js

In this way, all directories in the load path are merged to create a virtual filesystem whose entries are logical paths.

Serving Assets Over HTTP

When you mount an environment, all of its assets are accessible as logical paths underneath the mount point. For example, if you mount your environment at /assets and request the URL /assets/application.js, Mincer will search your load path for the file named application.js and serve it.

var connect = require('connect');
var Mincer  = require('mincer');

var environment = new Mincer.Environment();
environment.appendPath('app/assets/javascripts');
environment.appendPath('app/assets/stylesheets');

var app = connect();
app.use('/assets', Mincer.createServer(environment));
app.use(function (req, res) {
  // your application here...
});

Accessing Assets Programmatically

You can use the findAsset method to retrieve an asset from a Mincers environment. Pass it a logical path and you'll get a BundledAsset instance back.

Call toString on the resulting asset to access its contents, length to get its length in bytes, mtime to query its last-modified time, and pathname to get its full path on the filesystem.

var asset = environment.findAsset('application.js');

asset.toString(); // resulting contents
asset.length;     // length in bytes
asset.mtime;      // last modified time
asset.pathname;   // full path on the filesystem

Using Engines

Asset source files can be written in another language, like Stylus or CoffeeScript, and automatically compiled to CSS or JavaScript by Mincer. Compilers for these languages are called engines.

Engines are specified by additional extensions on the asset source filename. For example, a CSS file written in Stylus might have the name layout.css.styl, while a JavaScript file written in CoffeeScript might have the name dialog.js.coffee.

Styling with Stylus

Stylus is a revolutionary new language, providing an efficient, dynamic, and expressive way to generate CSS. Supporting both an indented syntax and regular CSS style.

If the stylus Node module is available to your application, you can use Stylus to write CSS assets in Mincer. Use the extension .css.styl.

Styling with LESS

LESS extends CSS with dynamic behavior such as variables, mixins, operations and functions.

If the less Node module is available to your application, you can use LESS to write CSS assets in Mincer. Use the extension .css.less.

Styling with Sass

Sass is an extension of CSS3, adding nested rules, variables, mixins, selector inheritance, and more.

If the node-sass Node module is available to your application, you can use Sass to write CSS assets in Mincer. Use the extension .css.sass or .css.scss.

Scripting with CoffeeScript

CoffeeScript is a language that compiles to the "good parts" of JavaScript, featuring a cleaner syntax with array comprehensions, classes, and function binding.

If the coffee-script Node module is available to your application, you can use CoffeeScript to write JavaScript assets in Mincer. Use the extension .js.coffee.

JavaScript Templating with Jade

Mincer supports JavaScript templates for client-side rendering of strings or markup. JavaScript templates have the special format extension .jst and are compiled to JavaScript functions.

When loaded, a JavaScript template function can be accessed by its logical path as a property on the global JST object. Invoke a template function to render the template as a string. The resulting string can then be inserted into the DOM.

// templates/hello.jst.jade
div Hello, #{ name }!
// application.js
//= require templates/hello
$("#hello").html(JST["templates/hello"]({ name: "Sam" }));

Mincer supports one template languages: Jade.

If jade Node module is available to your application, you can use Jade templates in Mincer. Jade templates have the extension .jst.jade. To use compiled templates you will need to require Jade runtime before calling renderer functions.

Invoking JavaScript with EJS

Note see macros description for more convenient alternative.

Mincer provides an EJS engine for preprocessing assets using embedded JavaScript code. Append .ejs to a CSS or JavaScript asset's filename to enable the EJS engine.

You will need ejs Node module available to your application.

Note: Mincer processes multiple engine extensions in order from right to left, so you can use multiple engines with a single asset. For example, to have a CoffeeScript asset that is first preprocessed with EJS, use the extension .js.coffee.ejs.

JavaScript code embedded in an asset is evaluated in the context of a Mincer.Context instance for the given asset. Common uses for EJS include:

  • embedding another asset as a Base64-encoded data: URI with the asset_data_uri helper
  • inserting the URL to another asset, such as with the asset_path helper (you must register your own helper for this purpose, but it's dead simple).
  • embedding other application resources, such as a localized string database, in a JavaScript asset via JSON
  • embedding version constants loaded from another file

Using helpers

Mincer provides an easy way to add your own helpers for engines:

environment.registerHelper('version', function () {
  var path = require('path');
  return require( path.join(__dirname, '/package.json') ).version;
});

Now, you can call that helper with EJS like this:

var APP = window.APP = {version: '<%= version() %>'};

NOTICE Helpers currently work for EJS and Stylus only. So to use them with Less you will need to add EJS engine as well:

// file: foobar.less.ejs
.btn {
  background: url('<%= asset_path('bg.png') %>');
}

Macros

This feature is designed as simple alternative to EJS, that does not requires additional extention and does not break language syntax. When enabled, any '$$ expression $$' or "$$ expression $$" pattern will be replaced with evaluated expression value. In expression you can write JS code and use registered helpers. Macros are off by default. You should enable those for particular extensions:

Mincer.MacroProcessor.configure(['.js', '.css']);

Managing and Bundling Dependencies

You can create asset bundlesย -- ordered concatenations of asset source files -- by specifying dependencies in a special comment syntax at the top of each source file.

Mincer reads these comments, called directives, and processes them to recursively build a dependency graph. When you request an asset with dependencies, the dependencies will be included in order at the top of the file.

The Directive Processor

Mincer runs the directive processor on each CSS and JavaScript source file. The directive processor scans for comment lines beginning with = in comment blocks at the top of the file.

//= require jquery
//= require jquery-ui
//= require backbone
//= require_tree .

The first word immediately following = specifies the directive name. Any words following the directive name are treated as arguments. Arguments may be placed in single or double quotes if they contain spaces, similar to commands in the Unix shell.

Note: Non-directive comment lines will be preserved in the final asset, but directive comments are stripped after processing. Mincer will not look for directives in comment blocks that occur after the first line of code.

Supported Comment Types

The directive processor understands comment blocks in three formats:

/* Multi-line comment blocks (CSS, Stylus, JavaScript)
 *= require foo
 */

// Single-line comment blocks (Stylus, JavaScript)
//= require foo

# Single-line comment blocks (CoffeeScript)
#= require foo

Mincer Directives

You can use the following directives to declare dependencies in asset source files.

For directives that take a path argument, you may specify either a logical path or a relative path. Relative paths begin with ./ and reference files relative to the location of the current file.

The require Directive

require path inserts the contents of the asset source file specified by path. If the file is required multiple times, it will appear in the bundle only once.

The include Directive

include path works like require, but inserts the contents of the specified source file even if it has already been included or required.

The require_directory Directive

require_directory path requires all source files of the same format in the directory specified by path. Files are required in alphabetical order.

The require_tree Directive

require_tree path works like require_directory, but operates recursively to require all files in all subdirectories of the directory specified by path.

The require_self Directive

require_self tells Mincer to insert the body of the current source file before any subsequent require or include directives.

The depend_on Directive

depend_on path declares a dependency on the given path without including it in the bundle. This is useful when you need to expire an asset's cache in response to a change in another file.

The stub Directive

stub path allows dependency to be excluded from the asset bundle. The path must be a valid asset and may or may not already be part of the bundle. Once stubbed, it is blacklisted and can't be brought back by any other require.

Credits

Great thanks to Sam Stephenson and Joshua Peek for the Sprockets, the most awesome and powerfull web assets processor I ever used, and which became a great source of inspiration (and model of almost all logic behind Mincer). Special thanks to Joshua for his assistance in hacking into Sprockets sources.

Author

Aleksey V Zapparov (follow @zapparov on twitter).

License

Copyright (c) 2012 Vitaly Puzrin

Released under the MIT license. See LICENSE for details.

mincer's People

Contributors

abravalheri avatar btd avatar cgc avatar codynguyen avatar coridyn avatar grindars avatar inukshuk avatar ixti avatar jeremyruppel avatar js-kyle avatar katyo avatar kramerc avatar mjonuschat avatar moddular avatar naxmefy avatar panta avatar rhyzx avatar rodrigoscna avatar ronen avatar rosswilson avatar siboulet avatar silvenon avatar slm-linus avatar spect88 avatar tikotzky avatar tomchentw avatar tricknotes avatar twhy avatar utkarshkukreti avatar vjpr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mincer's Issues

Feature: CoffeeScript Source Map Support

http://ryanflorence.com/2012/coffeescript-source-maps/

Only a few simple changes needed:

  1. Add a new PostProcessor for application/javascript
# stdlib
path = require "path"

# internal
Template = require "mincer/lib/mincer/template"

#//////////////////////////////////////////////////////////////////////////////

# Class constructor
SourceMapComments = module.exports = SourceMapComments = ->
  Template.apply this, arguments

require("util").inherits SourceMapComments, Template

# Process data
SourceMapComments::evaluate = (context, locals, callback) ->
  pathname = path.join(context.rootPath, context.logicalPath)
  @data = "\n\n//@ sourceMappingURL=#{pathname}.map\n\n" + @data
  callback null, @data
@env.registerPostProcessor 'application/javascript', require('./SourceMapComments')
  1. Serve .map files compiled with CoffeeScriptRedux. I'm digging into this now to see how it integrates with Mincer.

Generating source maps involves the following code:

csAst = coffee.parse input
jsAst = coffee.compile csAst
sourceMap = coffee.sourceMap jsAst

Should be as easy as using an engine to handle the .map extension.

Better error message when callback is not passed to Asset#compile

At the moment if you forget to pass a callback you see undefined is not a function.

I use this method in a few places where I do not need a callback. Although the user should probably be checking for errors.

Either way a better error message or optional callback would be nice.

Less.JS helpers (locals)

Styles renderer receives all registered Environment helpers as locals. It would be great to have same with less. Right now problem can be solved with help of EJS:

/** file: app.less.ejs **/
.btn {
  background: url('<%= asset_path("buttons/dark-bg.png") %>');
}

See Also:

Returns nothing: Environment.prototype.findAsset(logical_path)

Example:

function findAssetPaths (logical_path) {
  console.log(environment.__assets__); // log

  var asset = environment.findAsset(logical_path), // BUG
      paths = [];

  if (!asset) {
    return null;
  }

  if ('production' !== process.env.NODE_ENV && asset.isCompiled) {
    asset.toArray().forEach(function (dep) {
      paths.push('/assets/' + dep.logical_path + '?body=1');
    });
  } else {
    paths.push('/assets/' + asset.digestPath);
  }

  return paths;
}

Outputs:

{ '/Users/xxx/Development/projects/example/client/scripts/app.js0': {},
  '/Users/xxx/Development/projects/example/client/scripts/app.js1': {},
  'app.js1': {},
  '/Users/xxx/Development/projects/example/client/styles/app.css.styl0': {},
  '/Users/xxx/Development/projects/example/client/styles/app.css.styl1': {},
  'app.css1': {} }

...but endpoint works:

environment.precompile(['app.js', 'app.css'], function (err) {
    console.log(arguments)
});

Outputs:

{ '0': null,
  '1': 
   { files: 
      { 'app-834f4be8b490d996e796173b74c47669.js': [Object],
        'app-e5dde260db087c9bfdbefe636f1dd767.css': [Object] },
     assets: 
      { 'app.js': 'app-834f4be8b490d996e796173b74c47669.js',
        'app.css': 'app-e5dde260db087c9bfdbefe636f1dd767.css' } } }

CoffeeEngine with options {bare: false} only works for the first file

In https://github.com/nodeca/mincer/blob/master/lib/mincer/engines/coffee_engine.js#L85 options seems to be passed by reference, and modified by the CoffeeScript compiler.

The second time this bit of code runs, options is full of CoffeeScript compiler temporary variables including {bare: false}, but it doesn't compile bare.

The fix for me is cloning options before passing it to CoffeeScript via _.clone(options).


Might be something to do with the non-conventional this.require('coffee-script') perhaps?

Typo: examples/server.js

On 83 and 86 line in current revision

Now:
if ('production' !== process.env.NODE_ENV && asset.isCompiled) {
asset.toArray().forEach(function (dep) {
paths.push('/assets/' + rewrite_extension(dep.logicalPath) + '?body=1');
});
} else {
paths.push('/assets/' + rewrite_extension(asset.digestPath));
}

As I think, must be:
if ('production' !== process.env.NODE_ENV && asset.isCompiled) {
asset.toArray().forEach(function (dep) {
paths.push('/assets/' + rewrite_extension(dep.logicalPath, ext) + '?body=1');
});
} else {
paths.push('/assets/' + rewrite_extension(asset.digestPath, ext));
}

Sorry for the lack of diff

Replace Types with Collections

We are using Types for Hash at the moment. This should be replaced with collections/map.

Here's example of usage:

// ordinary hash:
var hash1 = new Map();

// hash with auto-create on get, similar to Ruby's:
//   Hash.new  { |h,k| h[k] = [] }
var hash2 = new Map([], null, null, function (k) {
  this.set(k, []);
  return this.get(k);
});

`defaultMimeType` for HamlCoffeeTemplate should be `application/javascript`

Removing prop(HamlCoffeeEngine, 'defaultMimeType', 'application/javascript'); means that templates must use the extension .js.hamlc instead of just .hamlc.

To resolve this in my code I simply added:

 prop = require('mincer/lib/mincer/common').prop
 prop Mincer.HamlCoffeeEngine, 'defaultMimeType', 'application/javascript'
 @env.registerEngine '.hamlc', Mincer.HamlCoffeeEngine

However, HamlCoffeeEngine always compiles to .js so the defaultMimeType should be application/javascript.

With this setting it is still possible to chain .hamlc in other extension chains too (i.e. .html.hamlc but I'm not sure why you would ever want to)

I think adding the defaultMimeType is more flexible.

One consideration though is haml-coffee can also return text/html. I use a separate HamlCoffeeStatic engine for this with the defaultMimeType as .html, but having this functionality in one engine might work.

40af3e6

require_tree not working properly on Windows

Hey,

I've narrowed down a bug's approximate location within the code as to why require_tree is flaky on Windows.

base.js around line 203 it does some checks and it's doing it's little recursive lookup in Base.prototype.resolve();

It would appear that as long as an "index.js" does not exist within ALL sub directories then the require_tree can't find the files.

But if an index.js exists, it works fine.

This is ONLY windows issue, my Mac works fine.

That being said...
Why are you using Base.prototype.resolve here instead of using the similar traversing process that is found within the trail/index of Hike? did you need custom logic that wasn't in Hike?

Thanks for all you do!

Base#precompile doesn't throw error when file not found

In examples/environment.js, add environment.appendPath('assets'); above all the load paths.

Then run node server.js.

The precompile environment.precompile(['app.js', 'app.css'] (line 42) will fail silently. There will be no error, only a 2nd argument containing { files: {}, assets: {} }

This seems to be caused by a bug beginning on mincer/base.js#374. this.eachLogicalPath's iterator method is never called. This is because app.js's logical path now becomes js/app.js because of the modified load paths. I'm a little confused as to how the resolution process for files works and why this causes a silent fail instead of asset not found.

precompile should return an error if paths.length < files.length after running line 374. Better yet, an error should be thrown specifying which files were unable to be resolved.

The documentation is a little confusing too. It asks that you specify a list of files, but these are actually logicalPaths which are specified.

Support for JS templating (JST)

Sprockets have support of EJS and Eco as JS templating engines:

Sprockets supports JavaScript templates for client-side rendering of strings or markup. JavaScript templates have the special format extension .jst and are compiled to JavaScript functions.

When loaded, a JavaScript template function can be accessed by its logical path as a property on the global JST object.

See JavaScript Templating with EJS and Eco

Better support for Bower

In Bower, I find that I have to write #= require underscore/underscore. Can we have it so that if you require a directory, if it contains a file by the same name, that will be required. So that I can just write #= require underscore, without every library having to add an index.js file.

Alternatively, if we require a directory, it checks the directory for a component.json (Bower's package.json equivalent) and looks for the main property which is an array of the main files from this library. This would be ideal.

Allow assets to be served as separate files

A helper to replicate the Rails Asset Pipeline functionality that allows you to serve the dependencies of an asset as separate files.

js('application')

renders to:

<script src="a.js" />
<script src="b.js" />

Something like this:

assetsDir = '/assets/'
exports.helper = ->
  css: (route) ->
    "<link rel='stylesheet' href='#{assetsDir}#{route}.css' type='text/css'></link>"
  js: (route) ->
    if expandTag
      routes = []
      env.findAsset(route, bundle: true).compile (err, asset) ->
        throw err if err
        asset.dependencies.forEach (dep) ->
          routes.push dep.logicalPath
      ("<script src='#{assetsDir}#{r}'></script>" for r in routes).join '\n'
    else
      "<script src='#{assetsDir}#{route}.js'></script>"

I'm having some problems though:

  • The above approach results in each file being served including a concatenation of its dependencies. I just want the compiled file without its dependencies.

Add option to Base#precompile to only build dependency graph

When using the Mincer server, I like to have an HTML tag generated for each dependency. The problem is that templating engines are synchronous and asset.compile is asynchronous. This means that on first request, the templating engine is unable to render the template with each dependency as a separate tag because Asset#toArray() returns an error.

Workarounds

  • Precompile on app load.
    • Problem: slows down server start.
  • Serve up a single html tag when !asset.isCompiled, and then on next refresh use expanded tags.
    • Problem: Messy, and it breaks breakpoints.

Solutions

  1. Provide an option in the Base#precompile method to only build dependency graphs for assets without actually compiling files.
    • Allow Asset#toArray() to return array of dependencies if this precompile options has been passed in.
  2. Provide a synchronous method on Asset to resolve dependencies.

This would allow server to start quickly.


Seems the only way to do it would be to add an option to Asset#compile to prevent compilation which then flows through when traversing dep graph in processed.js#resolve_dependencies.

Stylus `import` helper which calls `context.dependOn` to ensure changes to imported files are detected

I want the following helper:

import: (arg) ->
  context.dependOn(arg.string)
  return "@import '#{arg.string}'"

This could just be added to stylus_engine.js.

Is there any easy way to access the context argument of the Template#evaluate method from a helper defined by env.registerHelper? Then I could just create my own helper without modifying stylus_engine.

I think Rails allows you to use the native Stylus @import and has a DirectivePreprocessor which looks for imports.

So I think the best solution is to create a CSSDirectivePreprocessor or StylusDirectivePreprocessor which looks for @import and runs Context#dependOn.

feature request: support for haml-coffee/jade > handlebars > jst

i was able to get .jst.jade to work, but the jade compiler is really annoying. especially since the jade runtime still has an open issue for a bug whenever its used on the client-side that prevents it from working, plus adds unnecessary bloat and its not very fast on the client-side. most people are using precompiled js templates like hogan or handlebars these days. latest handlebars benchmarks much faster than hogan, so i'd like to use it in combination with haml or jade. the idea being to include mustache markup inside my jade templates, have jade compiled to html, then the html with mustache still in it precompiled with handlebars to a .hbs javascript function registered with JST.

Caching differences between Mincer and Sprockets

Okay, so I got two versions of my app up.
One running Ruby Sprockets and the other running Node Mincer.

Both of them use a similar "server" style distribution for my development process and both of them have different issues preventing me from moving forward on this type of implementation.

My issues with Sprockets currently is that when using LESS @imports, it can't properly find my files EVER, no matter what I type as the path to the file in the @import lol...

The issue currently with Mincer is that it's kind of slow when calling http://localhost:3000/assets/application.js when I request application.js after it has been modified.

Also, the main reason for this "issue" is that in Sprockets, if I change a file that is a directive in application.js, and then reload http://localhost/assets/application.js, the change appears within the file.

If I do this with Mincer, the application.js request uses the Not Modified version of the file...?

Confirmed that process.env.NODE_ENV is saying "development" in server.js

For some reason, it's still compressing my files and caching instead of loading my files in each time a request is made.
In Sprockets, this a really fast process and doesn't cache the file when I request it. No delay in refreshing all of it's directives...?

Getting an error on server example... :-\

Error: Failed to process '/Users/rountrjf/Sites/ce-platform/app/assets/javascripts/application.js'
    at beautify_err (/Users/rountrjf/Sites/ce-platform/node_modules/mincer/lib/mincer/processor.js:57:13)
    at Processor.evaluate (/Users/rountrjf/Sites/ce-platform/node_modules/mincer/lib/mincer/processor.js:75:16)
    at Function.environment.jsCompressor (/Users/rountrjf/Sites/ce-platform/environment.js:81:5)
    at Processor.evaluate (/Users/rountrjf/Sites/ce-platform/node_modules/mincer/lib/mincer/processor.js:73:20)
    at Context.evaluate (/Users/rountrjf/Sites/ce-platform/node_modules/mincer/lib/mincer/context.js:320:14)
    at async.forEachSeries.iterate (/Users/rountrjf/Sites/ce-platform/node_modules/async/lib/async.js:108:13)
    at Object.async.forEachSeries (/Users/rountrjf/Sites/ce-platform/node_modules/async/lib/async.js:124:9)
    at Context.evaluate (/Users/rountrjf/Sites/ce-platform/node_modules/mincer/lib/mincer/context.js:315:9)
    at BundledAsset.compile (/Users/rountrjf/Sites/ce-platform/node_modules/mincer/lib/mincer/assets/bundled.js:86:15)
    at async.series.results (/Users/rountrjf/Sites/ce-platform/node_modules/async/lib/async.js:486:21)

Problem with windows... directives not found?

I have a simple application.css that says

/*
=require ./file
*/

and the server is returning blank when visiting /assets/application.css
but when I go to /assets/file.css it renders properly.

I assume it's a file path issue, any way to turn on logging / debugging of file paths?

Manifest#compile fails when encountering invalid symbolically linked directories

https://github.com/nodeca/mincer/blob/master/lib/mincer/base.js#L403

self.stat(pathname) returns null if it encounters an invalid symbolically linked directory and then when isDirectory() is called on it, it throws the following message:

TypeError: Cannot call method 'isDirectory' of null
    at /Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/base.js:404:29
    at Array.forEach (native)
    at Environment.eachEntry (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/base.js:398:22)
    at /Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/base.js:424:10
    at Array.forEach (native)
    at Environment.eachFile (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/base.js:423:14)
    at Environment.eachLogicalPath (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/base.js:498:8)
    at Manifest.compile (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/manifest.js:191:20)

Pretty rare occurrence though. Not really a priority.

Add cache to some engines

Some engines should be cacheable, to provide significant boost.

  • LESS (with dependencies) - it really sucks, bootstrap compilation speed is veeeery slooow.
  • CoffeScript - that should be slow, if you have lot of files
  • Stylus - not critical, but easy to implement

Currently exists, but sould be rewritten to use unified cache interface:

  • CSS optimizer
  • JS minifier

NOTES

  1. Cache should depends on (file, engine) and invalidated by dependencies list.
  2. Since some files can be auto-generated (like in fontello/nodeca), is't good idea to rely on file CRC, instead of path/mdate. Though, that's quite rate, so we can use CRC as fallback, when path/mdate does not match.

Hmm... Cannot find module "haml-coffee"

I think I'm being a n00b here...
I've placed 'haml-coffee' in my package.json and have ran npm install and it installed haml-coffee into my node_modules folder.

But when I go to require("haml-coffee") it says it can't find the module?

Any tips?

Avoid assets recompilation in production

Uglification requires (lots of) time and resources. It would be nice to have "original" digests (with dependencies graph) in manifest file, so we could skip full recompilation of assets that were not changed.

Maximum call stack size exceeded

I've been stuck on this bug for a while.

I've started getting the following error:

Served asset /application.js?body=1 - 500 Error compiling asset: Maximum call stack size exceeded (503ms)

Refreshing the browser and the issue is resolved.

Here is a stack trace from the Context#compile method.

The file that is failing is always the last file which I have required.

The strange thing though is that removing a few require statements get rid of the error. Removing any require statement doesn't fix it though.

There is nothing wrong with these files. Renaming the files doesn't fix it. And it's not the contents of the files.

I'm really stumped.

    at Context.evaluate (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/context.js:323:17)
    at SafetyColons.SafetyColons.evaluate [as evaluate] (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/processors/safety_colons.js:51:5)
    at Context.evaluate (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/context.js:320:14)
    at async.forEachSeries.iterate (/Users/Vaughan/dev/sidekick/node/node_modules/async/lib/async.js:108:13)
    at async.forEachSeries.iterate (/Users/Vaughan/dev/sidekick/node/node_modules/async/lib/async.js:119:25)
    at Context.evaluate (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/context.js:326:7)
    at CoffeeEngine.CoffeeEngine.evaluate [as evaluate] (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/engines/coffee_engine.js:77:5)
    at Context.evaluate (/Users/Vaughan/dev/sidekick/node/node_modules/mincer/lib/mincer/context.js:320:14)
    at async.forEachSeries.iterate (/Users/Vaughan/dev/sidekick/node/node_modules/async/lib/async.js:108:13)
    at async.forEachSeries.iterate (/Users/Vaughan/dev/sidekick/node/node_modules/async/lib/async.js:119:25)
---------------------------------------------
    at new Server (https.js:38:10)
    at Object.exports.createServer [as createServer] (https.js:48:10)
    at module.exports (/Users/Vaughan/dev/sidekick/node/src/app.coffee:152:33)
    at Object.<anonymous> (/Users/Vaughan/dev/sidekick/node/app.js:2:21)
    at Module.Module._compile [as _compile] (module.js:449:26)
    at Object.Module._extensions..js [as .js] (module.js:467:10)
    at Module.Module.load [as load] (module.js:356:32)
    at Function.Module._load [as _load] (module.js:312:12)

UPDATE:

Occurs during SafetyColons processor. Removing the processor fixes it. However, commenting out the code in the SafetyColons processor doesn't fix it. Strange.

NOTE: I have a very large project. I am using the longjohn module to print long stack traces.


Just added a couple more files and the error came back. Must be related to the number of files.

Possible to use server for non-node project during development?

So my developers have local dev environments built with a different server language.
I'd love it if we could take advantage of the Mincer library / server and allow for more clean and rapid development of front-end css / js with the benefits of the dependency directives.

We wouldn't run this server in production (obviously) but it would be a nice fit (i think) for our development flow.

Unless you can recommend a better solution, I'm interested in entertaining how this might play out.

Essentially, while we're developing, we would point all of our <script> / <link> tags to http://localhost:3000/assets/<whatever>
and we would expect that it would either give a bundle in return, or possibly a list of <script> / <link> tags for each individual found dependency?

For instance, I could write a function in my server language to parse a file the server makes of all dependencies and create script / link tags for my server language based on the list of dependencies in the manifest?

Am I making any sense?

Please advise!

relative paths do not work

traced it to

https://github.com/nodeca/mincer/blob/master/lib/mincer/processors/directive_processor.js#L109

and

https://github.com/nodeca/mincer/blob/master/lib/mincer/context.js#L322

and

https://github.com/nodeca/mincer/blob/master/lib/mincer/context.js#L240

which looks like it should be './' == pathname.substr(0,2)

the path i was trying to resolve was ./../../views/shared/users/dashboard which is a .js.coffee file. i was using #= require from within ./app/assets/coffee/application.js.coffee.

also if i try to require an absolute path, it treats it as a relative path. like #= require /var/www/blah...

require_self causes error

TypeError: Cannot read property '0' of undefined
    at [object Object].resolve (/home/ixti/Projects/fontello/node_modules/mincer/lib/mincer/context.js:151:23)
    at [object Object].requireAsset (/home/ixti/Projects/fontello/node_modules/mincer/lib/mincer/context.js:229:19)
    at Object.require_self (/home/ixti/Projects/fontello/node_modules/mincer/lib/mincer/processors/directive_processor.js:121:18)
    at /home/ixti/Projects/fontello/node_modules/mincer/lib/mincer/processors/directive_processor.js:333:30
    at /home/ixti/Projects/fontello/node_modules/mincer/node_modules/async/lib/async.js:194:13
    at /home/ixti/Projects/fontello/node_modules/mincer/node_modules/async/lib/async.js:118:13
    at /home/ixti/Projects/fontello/node_modules/mincer/node_modules/async/lib/async.js:134:9
    at /home/ixti/Projects/fontello/node_modules/mincer/node_modules/async/lib/async.js:193:9
    at Object.mapSeries (/home/ixti/Projects/fontello/node_modules/mincer/node_modules/async/lib/async.js:183:23)
    at DirectiveProcessor.processDirectives (/home/ixti/Projects/fontello/node_modules/mincer/lib/mincer/processors/directive_processor.js:330:9)

Manifest#compile needs to return which file failed compilation

manifest.js:202 needs to return the logical path of the asset it was compiling when it failed otherwise it's very tough to find what caused it.

Just a log message with the logicalPath would suffice.

E.g. When my haml-coffee templates failed I received this message:

SyntaxError: unmatched OUTDENT on line 20

TypeError: Cannot redefine property: gzipped

This message shows up sporadically (a Higgs-Bugson if you will) when accessing a particular asset from the Mincer server.

Before https://github.com/nodeca/mincer/blob/master/lib/mincer/server.js#L146 I added console.log(undefined === asset.gzipped || undefined === asset.deflated) which evaluates to false before the error.

Seeing as there is the following check before running the set_compressed_cache method:

if (is_compressable(asset) && (undefined === asset.gzipped || undefined === asset.deflated)) {
  ...
  # set_compressed_cache

It appears to be a race condition somewhere.

It also appears to happen for files which are being served multiple times, but not always.

Here is the error:

TypeError: Cannot redefine property: gzipped
    at Function.defineProperty (native)
    at module.exports.prop (xxx/node_modules/mincer/lib/mincer/common.js:25:10)
    at code (xxx/node_modules/mincer/lib/mincer/server.js:150:7)
    at Gzip.onEnd (zlib.js:155:5)
    at Gzip.EventEmitter.emit [as emit] (events.js:64:17)
    at zlib.js:318:10
    at Gzip.callback (zlib.js:404:13)
---------------------------------------------
    at zlibBuffer (zlib.js:160:10)
    at exports.gzip (zlib.js:98:3)
    at set_compressed_cache (/xxx/node_modules/mincer/lib/mincer/server.js:146:3)
    at async.apply (/xxx/node_modules/async/lib/async.js:532:23)
    at async.series.results (/xxx/node_modules/async/lib/async.js:486:21)
    at _asyncMap (/xxx/node_modules/async/lib/async.js:185:13)
    at async.forEachSeries.iterate (/xxx/node_modules/async/lib/async.js:108:13)
    at async.forEachSeries (/xxx/node_modules/async/lib/async.js:124:9)

Here is a trace from running the following code before the method that is erroring.

  console.log('')
  console.log(asset.pathname)
  console.log(name)
  console.log(undefined === asset[name])
  prop(asset, name, buffer);
[redacted]/node/views/extension/logger.html.haml
gzipped
true

[redacted]/node/views/extension/logger.html.haml
deflated
true
null
Served asset /extension/logger.html - 200 OK (100ms)
null
Served asset /extension/util/Exports.js?body=1 - 200 OK (4ms)

[redacted]/extension/chrome/javascripts/extension/util/Url.coffee
gzipped
true

[redacted]/extension/chrome/javascripts/extension/util/Helpers.coffee
gzipped
true

[redacted]/node/app/assets/javascripts/vendor/jquery-ui-1.8.16.custom.min.js
gzipped
true

[redacted]/extension/chrome/javascripts/extension/util/Url.coffee
deflated
true
null
Served asset /extension/util/Url.js?body=1 - 200 OK (28ms)

[redacted]/extension/chrome/javascripts/extension/util/Helpers.coffee
deflated
true
null
Served asset /extension/util/Helpers.js?body=1 - 200 OK (22ms)

[redacted]/node/app/assets/javascripts/vendor/jquery-ui-1.8.16.custom.min.js
deflated
true
null
Served asset /vendor/jquery-ui-1.8.16.custom.min.js?body=1 - 200 OK (71ms)

[redacted]/extension/chrome/javascripts/extension/util/Services.coffee
gzipped
true

[redacted]/extension/chrome/javascripts/extension/util/PubSubReq.coffee
gzipped
true

[redacted]/extension/chrome/javascripts/extension/util/Services.coffee
deflated
true
null
Served asset /extension/util/Services.js?body=1 - 200 OK (60ms)

[redacted]/extension/chrome/javascripts/extension/util/PubSubReq.coffee
deflated
true
null
Served asset /extension/util/PubSubReq.js?body=1 - 200 OK (54ms)

[redacted]/extension/chrome/javascripts/extension/logger.coffee
gzipped
true
Served asset /extension/util/Helpers.js?body=1 - 200 OK (0ms)
Served asset /extension/util/Exports.js?body=1 - 200 OK (0ms)
Served asset /vendor/jquery-ui-1.8.16.custom.min.js?body=1 - 200 OK (0ms)

[redacted]/extension/chrome/javascripts/extension/logger.coffee
deflated
true
null
Served asset /extension/logger.js?body=1 - 200 OK (64ms)
Served asset /extension/util/PubSubReq.js?body=1 - 200 OK (0ms)
Served asset /extension/util/Url.js?body=1 - 200 OK (0ms)

[redacted]/node/app/assets/javascripts/vendor/jquery-1.7.2.js
gzipped
true
Served asset /extension/logger.js?body=1 - 200 OK (3ms)
Served asset /extension/util/Services.js?body=1 - 200 OK (0ms)

[redacted]/node/app/assets/javascripts/vendor/jquery-1.7.2.js
gzipped
false

# ERROR HERE

Add custom helpers example

Add sample, how to use custom helpers (asset_path, asset_include, ...). Consider how to improve documentation.

Engines as separate repos

It would be nice to have mincer just be the pipeline and have all of the engines as separate repos. They would have their own dependency on the 3rd party engine, instead of the app declaring it. For example:

//package.json
{
  "name": "mincer-stylus",
  "dependencies": {
    "stylus": "~0.32.0"
  }
}

And in my app's package.json:

{
  "name": "my-app",
  "dependencies": {
    "mincer": "~0.4.3",
    "mincer-stylus": "~0.32.0"
  }
}

My app ends up including an extra dependency either way but I think it's better to have it be the engine rather than the 3rd party preprocessor so it will have the correct version of the preprocessor in the engine. If there is an upgrade in the stylus api, for example, and I just declare stylus as a dependency in my app the engine will break since it expects the old api.

What are your thoughts?

Recommendations on creating custom engines.

I'm trying to create an engine to automatically compile handlebars templates, and not getting very far. What are the requirements for creating an engine and making it be used?

Add gzip & cache support

  • serve ALL from memory
  • add compression
  • add proper Vary for compressed data
  • don't forget about HEAD support

Add CLI

Make CLI, the same as Sprockets has.

Major bug making the lib useless

In development environment, using Mincer.createServer(environment), the goal of this lib is to check the mtimes of assets and recompile them if necessary, right?

It's currently not happening. There are 2 bugs regarding this.

First bug

This bug marks all the dependency files as stale even if there are no changes in that file

https://github.com/nodeca/mincer/blob/master/lib/mincer/assets/asset.js#L136 should be

if (dep.mtime >= Date.parse(stat.mtime)) {
  return true;
}

Second bug

Take a modified file. After checking that mtime on the file is indeed new, the code tries to calculate the digest and check if it's different. You can see it in this line. https://github.com/nodeca/mincer/blob/master/lib/mincer/assets/asset.js#L142

What's actually happening is that environment.getFileDigest doesn't seem to return the new digest and instead returns the old one which in return marks the file as fresh which is wrong.

I tried to debug, by doing console.log('hello') just after this line https://github.com/nodeca/mincer/blob/master/lib/mincer/base.js#L244. What I found that the program never prints hello which tells me that Base.prototype.getFileDigest doesn't run.

Please fix these 2 bugs as soon as possible and publish a new version.

Thanks

Mind helping with an issue I'm having with Mincer?

Hello, I'm certain it's not an issue... just my noob self.

I'm now building a nodejs web app using Express.

I used some boilerplate code I found on Github.

However, that all works fine...

ISSUE (1)
I used the examples server.js / environment.js files you provide but I'm getting an error in my Jade template that says:

ReferenceError: /Users/rountrjf/Sites/ojects/app/views/includes/head.jade:23
    21|   meta(property='fb:admins', content='1037213945')
    22| 
  > 23|   !=stylesheet('application.css')
    24|   
    25|   style
    26|       body {

stylesheet is not defined

Once I refresh again, it loads... almost as if "Stylesheet" doesn't exist after the first load, but second load is fine.

ISSUE (2)
Also, I'm doing //= require bootstrap
in my Application.css... which in tern is @import 'variables'; etc.
Whenever I change one of the internal bootstrap files, it doesn't recompile upon refreshing the page?..?

Is mincer not watching the @imported files? is there a trick to this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.