ladjs / express-cdn Goto Github PK

:cloud: Node module for delivering optimized, minified, mangled, gzipped assets with Express and Amazon's CDN (S3/CloudFront)

License: MIT License

JavaScript 91.38% HTML 8.62%

express-cdn's People

Contributors

Stargazers

Watchers

Forkers

amumu-dev emostar everblaze andrewdeandrade joshuagross grydstedt ninjablocks urbancups timshadel cjroebuck tracend jaketrent jeresig saas786 brandingbrand numcracker realfilling alexloi epd danielsantiago leostera mateusz- bbm1hao dservy nvdnkpr gkorland netfluence crushlovely samihaq patrickjs maximecony aninfeel agurha sars philippsimon avocado-yali romanminkin hendrikswan dex8 zhangchiqing alihalabyah katt cloudxtreme send2moran dcard mattthered adamrights philipheinser anthony-legible aaronroberson sincraianul guidekick andrewho83 johanlaidlaw wzx8118 postcard auzolano mattjstar georgezhang wujcheng nickdarnell vincentkwok govindmalviya calebsevern pairin snyk-community bitponics postster baugarten wangyx0055 jmgsquared214 evertonteotonio fernandogot chuyennm pascualy chrishughes startmatter vishak-kavalur bhanditz sidikriders dexternatan vanessa-bell danishack cortfr pradumio trilogysoftwareinc galatimus marshadshaheen agentmishra ksarpotdar

express-cdn's Issues

Express 3.0.0 compatibility

What needs to be done to make this express 3.0.0 compatible.

Two immediate bugs I encountered.

(1) The check to see if the app object is of typeof object now fails since express no longer inherits from http.Server and is now of typeof function instead of typeof object
ref: https://github.com/niftylettuce/express-cdn/blob/master/lib/main.js#L363

(2) dynamicHelpers was dropper in version 3.0 and replaced with app.locals
ref: http://expressjs.com/api.html#app.locals
ref: https://github.com/visionmedia/express/wiki/Migrating-from-2.x-to-3.x

To get CDN to work I had to use app.locals({ CDN: CDN() }); instead of app.dynamicHelpers({ CDN: CDN });

app is getting crashed in production mode

Can you please check this gist for express-cdn configurations we are using.

In development mode, app seems working with express-cdn, we can see the timestamped querystring with every stylesheet/js.

As soon as we set production mode and deploy it, app process exited with code 0. As of yet we couldn't debug this issue as we are not getting any log even though we configured winston correctly in there. Our code is hosted on heroku. We are using express latest version. We already uploaded CDN data on aws s3 and configured cloudfront for the cdn subdomain of our site.

Support for Handlebars.js

I'm wondering what the recommended method of integration into handlebars.js is. The examples outlined in the README.md only outline implementations in Jade and EJS, but my site uses Handlebars templates. I see that on this line: https://github.com/niftylettuce/express-cdn/blob/master/lib/main.js#L514, you reference only two valid extensions - does this mean there isn't support for any templating language other than these two? If so, this should be made explicit in the README.md

How to use with express 4.x

I'm trying to get this up and running but don't see how I can use app.dynamichelpers with express 4.x?

Working with html + swig?

Is this library able to work with html + swig instead of jade or ejs?

I had this code:

{% for cssFile in cssFiles %}<link rel="stylesheet" href="{{cssFile}}">
{% endfor %}

where cssFiles it is the list of Css Files stored as String in a config file.

I've change it to this:

    {% for cssFile in cssFiles %}
      <script type="text/javascript">
          var cssFileVal = "{{cssFile}}";
                  {{ CDN(cssFileVal) }}
      </script>
    {% endfor %}

But the code in {{}} is not pre processed, so I end having this:

{ "assets": cssFileVal }
                 ^
SyntaxError: Unexpected token c

I will really appreciate some help to make this work. Thanks

CSS minification breaks if background image is base encoded

my css looks like this:

.bl-widget {
        width: 172px;
        height: 27px;
        border: 1px solid #c0c0c0;
        border-radius: 4px;
        margin-bottom: 10px;
        background: #f9f9f9 url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAABRCAIAAACc+VspAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAyRpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMy1jMDExIDY2LjE0NTY2MSwgMjAxMi8wMi8wNi0xNDo1NjoyNyAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENTNiAoTWFjaW50b3NoKSIgeG1wTU06SW5zdGFuY2VJRD0ieG1wLmlpZDo2MzEzQkUyNDhCRkExMUUyQjdBRTg1RUM3MzhCN0Y0QSIgeG1wTU06RG9jdW1lbnRJRD0ieG1wLmRpZDo2MzEzQkUyNThCRkExMUUyQjdBRTg1RUM3MzhCN0Y0QSI+IDx4bXBNTTpEZXJpdmVkRnJvbSBzdFJlZjppbnN0YW5jZUlEPSJ4bXAuaWlkOjRBMjg0NDZGOEJGQTExRTJCN0FFODVFQzczOEI3RjRBIiBzdFJlZjpkb2N1bWVudElEPSJ4bXAuZGlkOjRBMjg0NDcwOEJGQTExRTJCN0FFODVFQzczOEI3RjRBIi8+IDwvcmRmOkRlc2NyaXB0aW9uPiA8L3JkZjpSREY+IDwveDp4bXBtZXRhPiA8P3hwYWNrZXQgZW5kPSJyIj8+nVl/8QAAADlJREFUeNpi+P//PxMIMzAwgDGMTQ0a2VwgZsAihqIHlxtIVQfCv3//RuEzMjKCMYxNDRrqJ4AAAwBFHUVtdoNJaQAAAABJRU5ErkJggg==) repeat-x 0 0;
      }

and I get this error in production:

TypeError: Cannot call method 'replace' of undefined
    at /home/ec2-user/bloglovin-widget/node_modules/express-cdn/lib/main.js:188:16
    at Request._callback (/home/ec2-user/bloglovin-widget/node_modules/express-cdn/lib/main.js:330:5)
    at Request.self.callback (/home/ec2-user/bloglovin-widget/node_modules/express-cdn/node_modules/request/index.js:142:22)
    at Request.EventEmitter.emit (events.js:98:17)
    at Request.<anonymous> (/home/ec2-user/bloglovin-widget/node_modules/express-cdn/node_modules/request/index.js:856:14)
    at Request.EventEmitter.emit (events.js:117:20)
    at IncomingMessage.<anonymous> (/home/ec2-user/bloglovin-widget/node_modules/express-cdn/node_modules/request/index.js:808:12)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:895:16
    at process._tickCallback (node.js:415:13)

Image optimization not working on nodejitsu

Nick, I've switched to 0.1.0 with the compiled bins and I started getting some nasty errors on my app. Here is the log:

[04/19 12:01:26 GMT-0300] Unsupported platform: sunos x64
[04/19 12:01:26 GMT-0300] Unsupported platform: sunos x64
[04/19 12:01:26 GMT-0300] info: [object Object]
[04/19 12:01:26 GMT-0300] child_process.js:782
[04/19 12:01:26 GMT-0300]   var r = this._handle.spawn(options);
[04/19 12:01:26 GMT-0300]                        ^
[04/19 12:01:26 GMT-0300] TypeError: Bad argument
[04/19 12:01:26 GMT-0300]     at ChildProcess.spawn (child_process.js:782:24)
[04/19 12:01:26 GMT-0300]     at exports.spawn (child_process.js:618:9)
[04/19 12:01:26 GMT-0300]     at compile (/opt/haibu/apps/realfilling/FarmShares
QA/package/node_modules/express-cdn/lib/main.js:233:23)
[04/19 12:01:26 GMT-0300]     at checkStringIfModified (/opt/haibu/apps/realfill
ing/FarmSharesQA/package/node_modules/express-cdn/lib/main.js:392:91)
[04/19 12:01:26 GMT-0300]     at ClientRequest.onResponse (/opt/haibu/apps/realf
illing/FarmSharesQA/package/node_modules/express-cdn/node_modules/knox/lib/clien
t.js:49:7)
[04/19 12:01:26 GMT-0300]     at ClientRequest.EventEmitter.emit (events.js:96:1
7)
[04/19 12:01:26 GMT-0300]     at HTTPParser.parserOnIncomingClient [as onIncomin
g] (http.js:1582:7)
[04/19 12:01:26 GMT-0300]     at HTTPParser.parserOnHeadersComplete [as onHeader
sComplete] (http.js:111:23)
[04/19 12:01:26 GMT-0300]     at CleartextStream.socketOnData [as ondata] (http.
js:1485:20)
[04/19 12:01:26 GMT-0300]     at CleartextStream.CryptoStream._push (tls.js:544:
27)
[04/19 12:01:26 GMT-0300] info: [object Object]
[04/19 12:01:28 GMT-0300] info: All dependencies loaded, preparing to configure
qa environment.
[04/19 12:01:29 GMT-0300] info: Database connected.
[04/19 12:01:30 GMT-0300] info: FarmShares server listening on port: 80
[04/19 12:01:30 GMT-0300] Unsupported platform: sunos x64
[04/19 12:01:31 GMT-0300] info: [object Object]
[04/19 12:01:31 GMT-0300] child_process.js:782
[04/19 12:01:31 GMT-0300]   var r = this._handle.spawn(options);
[04/19 12:01:31 GMT-0300]                        ^

CDN: Error: connect ECONNREFUSED (socket.io related?)

I am running into issues connecting to my S3 account, and cannot figure out what the junk is going on. I am assuming it is a socket.io issue, but I cannot figure out how to work around it.

CDN: Error: connect ECONNREFUSED

I have quadruple checked my credentials. Any clue?

I looked through the issue 12, but they seem to be unrelated. Created a gist for you. Let me know if you need to see anything more...

Error with bin files

I keep getting this error, any ideas?:

[04/25 06:24:59 GMT-0300] child_process.js:782
[04/25 06:24:59 GMT-0300] var r = this.handle.spawn(options);
[04/25 06:24:59 GMT-0300] ^
[04/25 06:24:59 GMT-0300] TypeError: Bad argument
[04/25 06:24:59 GMT-0300] at ChildProcess.spawn (child_process.js:782:24)
[04/25 06:24:59 GMT-0300] at exports.spawn (child_process.js:618:9)
[04/25 06:24:59 GMT-0300] at compile (/opt/haibu/apps/realfilling/FarmSharesStaging/package/node_modules/expres
s-cdn/lib/main.js:267:24)
[04/25 06:24:59 GMT-0300] at checkStringIfModified (/opt/haibu/apps/realfilling/FarmSharesStaging/package/node
modules/express-cdn/lib/main.js:397:92)
[04/25 06:24:59 GMT-0300] at ClientRequest.onResponse (/opt/haibu/apps/realfilling/FarmSharesStaging/package/no
de_modules/express-cdn/node_modules/knox/lib/client.js:49:7)
[04/25 06:24:59 GMT-0300] at ClientRequest.EventEmitter.emit (events.js:96:17)
[04/25 06:24:59 GMT-0300] at HTTPParser.parserOnIncomingClient as onIncoming
[04/25 06:24:59 GMT-0300] at HTTPParser.parserOnHeadersComplete as onHeadersComplete
[04/25 06:24:59 GMT-0300] at CleartextStream.socketOnData as ondata
[04/25 06:24:59 GMT-0300] at CleartextStream.CryptoStream._push (tls.js:544:27)

Add Google Cloud Storage support

I read #70 but it would be useful to add support for Google Cloud Storage. What would it take to add another provider? I ask because there is now gcloud-node.

Looking for contributors (are you interested?)

Interested in helping out here? Have a fork? Just ping me and I can get you collaborator status and maintainer on NPM.

Update ReadMe to Express 4.0 documentation

https://github.com/strongloop/express/wiki/Migrating-from-3.x-to-4.x

Express is now distributing 4.0 and some of the methods are different, eg:
no more app.configure
change from app = express.createserver() to app = express()

URLs for RequireJS

I'm using RequireJS on my app, and I'm now into the task of setting the paths configuration to my modules. I have a setup like this:

paths : {
  // requirejs
  text: '/javascripts/lib/require/text',
  async: '/javascripts/lib/require/async',
  font: '/javascripts/lib/require/font',
  goog: '/javascripts/lib/require/goog',
  image: '/javascripts/lib/require/image',
  json: '/javascripts/lib/require/json',
  noext: '/javascripts/lib/require/noext',
  mdown: '/javascripts/lib/require/mdown',
  propertyParser : '/javascripts/lib/require/propertyParser',
  markdownConverter : '/javascripts/lib/Markdown.Converter',
  // knockout
  knockout: '/javascripts/lib/knockout/knockout-2.2.1',
  'knockout.mapping': '/javascripts/lib/knockout/knockout.mapping',
  'knockout.validation': '/javascripts/lib/knockout/knockout.validation',
  'knockout.unapplyBindings': '/javascripts/lib/knockout/knockout.unapplyBindings',
  'knockout.jqueryui': '/javascripts/lib/knockout/knockout.jqueryui',
  // jquery
  'jquery-ui': '/javascripts/lib/jquery/jquery-ui-1.10.2.custom',
  'jquery.fileupload': '/javascripts/lib/jquery/jquery.fileupload',
  'jquery.fileupload-fp': '/javascripts/lib/jquery/jquery.fileupload-fp',
  'jquery.iframe-transport': '/javascripts/lib/jquery/jquery.iframe-transport',
  // app
  baseVM : '/javascripts/admin/baseVM',
  crudBaseVM : '/javascripts/admin/crudBaseVM',
  geolocationVM : '/javascripts/admin/geolocationVM',
  farmsVM : '/javascripts/admin/farmsVM',
  sharesVM : '/javascripts/admin/sharesVM',
  farmsAddInit : '/javascripts/admin/farmsAddInit'
}

I would need to get URLs to my resources in that format to be able to configure requirejs for the specified modules, right now using CDN('path/to/script.js', { raw : true }) returns a path but with a trailing ".js?v=xxxxxxxxx" and a carriage return, so the path look like this:

paths : {
  // requirejs
  text: 'http://cdn.mydomain.com/javascripts/lib/require/text.js?v=2394564923658
  ',
  async: '/javascripts/lib/require/async',
  ...
}

This brakes my code completly, I thought about adding a new parameter, such as raw, called amd, that when set to true would return the path without the trailing part that's not working. I offer to add support and functionality to this myself.

Let me know what you think or if I'm doing things wrong or how would you do it, I really need express-cdn to be able to define paths as strings for requirejs.

Cheers!

CDN: Empty Results

Hi there guys, here is the code I'm using to instantiate and configure express-cdn:

var app = express();
// some more code here
app.configure( /*...*/ );

// Set the CDN options
var options = {
    publicDir  : path.join(__dirname, 'public')
  , viewsDir   : path.join(__dirname, 'views')
  , domain     : 'cdn.farmshares.com'
  , bucket     : 'farmshares'
  , key        : 'removed'
  , secret     : 'removed'
  , hostname   : 'localhost'
  , port       : 443
  , ssl        : true
  , production : true
  , logger     : winston.info
};

// Initialize the CDN magic
var CDN = require('express-cdn')(app, options);

// Add the view helper
app.locals({ CDN: CDN() });

The public and views folders all have contents, meaning my assets are in place in the public folder (under stylesheets, images and javascripts) and all my jade templates are organized in subfolders below the views folder.

I'm using the latest express-cdn version (*) available on npm with express "3.x.x", running on node 0.8.14.

The problem raises when I run the app I get this error message:

/Users/leostera/Repositories/farmshares/node_modules/express-cdn/lib/main.js:32
  throw new Error('CDN: ' + msg);
        ^
Error: CDN: empty results
    at throwError (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/lib/main.js:32:9)
    at Walker.CDN (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/lib/main.js:470:12)
    at Walker.EventEmitter.emit (events.js:93:17)
    at Walker._wNext (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:212:8)
    at Walker._wNext (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:210:19)
    at Walker._wNext (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:210:19)
    at Walker._wNext (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:210:19)
    at Walker._wOnEmitDone (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:109:8)
    at nextWhenReady (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/node-type-emitter.js:69:29)
    at Object.emitPluralEvents [as emitNodeTypeGroups] (/Users/leostera/Repositories/farmshares/node_modules/express-cdn/node_modules/walk/node-type-emitter.js:77:5)

But, if I disable production mode in the CDN everything works just fine (regardless of the SSL/port options).

Any ideas? This is a follow up from a twitter chat:

@niftylettuce hi nick, any ideas of why would express-cdn throw an Empty results error? The code suggests I have no assets but I do! Thanks!
— Leandro Ostera (@leostera) December 10, 2012

ECONNREFUSED on Nodejitsu

when I try to setup express-cdn with my nodejitsu account I keep getting ECONNREFUSED error when starting the app.

My code and output error are in this gist : https://gist.github.com/Nevtep/b2dde38348eb1d655ef2

Using the CDN helper to CDN() dynamically generated paths.

I have the following jade code:

#slideshow
  ul#carousel.elastislide-list
    for image in farm.gallery
      li(data-preview="/farms/#{farm.slug}/gallery/#{image}")
        a(href="#")
          != CDN("/farms/#{farm.slug}/gallery/#{image}",{ alt: "#{image}"})

That worked like a charm before introducing the CDN, and now collapses in the following error message:

undefined:1
{ "assets": "/farms/#"attributes": {farm.slug}/gallery/#{image}",{ alt: "#{ima
                      ^
SyntaxError: Unexpected token a
    at Object.parse (native)
    at Walker.CDN (/home/ubuntu/farmshares/node_modules/express-cdn/lib/main.js:457:27)
    at Walker.EventEmitter.emit (events.js:93:17)
    at Walker._wNext (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:212:8)
    at Walker._wNext (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:210:19)
    at Walker._wNext (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:210:19)
    at Walker._wNext (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:210:19)
    at Walker._wOnEmitDone (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/walk.js:109:8)
    at nextWhenReady (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/node-type-emitter.js:69:29)
    at Object.emitPluralEvents [as emitNodeTypeGroups] (/home/ubuntu/farmshares/node_modules/express-cdn/node_modules/walk/node-type-emitter.js:77:5)

I even tried to save the full path as a string in a variable:

#slideshow
  ul#carousel.elastislide-list
    for image in farm.gallery
      - var cdn_path = "/farms/#{farm.slug}/gallery/#{image}"
      li(data-preview="/farms/#{farm.slug}/gallery/#{image}")
        a(href="#")
          != CDN(#{cdn_path},{ alt: "#{image}"})

But that gives me the following error:

undefined:1
{ "assets": #"attributes": {cdn_path},{ alt: "#{image}"} }
            ^
SyntaxError: Unexpected token #
    at Object.parse (native)
    at Walker.CDN (/home/ubuntu/farmshares/node_modules/express-cdn/lib/main.js:457:27)

I'm obviously doing some wrong here. Whether it is wrong to pass #{cdn_path} (using just the variable name didn't work) or generating the string using jade, there should be an obvious way to do this.

Any ideas? Thanks

Mistake in example code?

if (semver.lt(express.version, '3.0.0')) {
  app.locals({ CDN: CDN() });
} else {
  app.dynamicHelpers({ CDN: CDN });
}

Shouldn't it be semver.gte? I think dynamicHelpers are from older Express

Add documentation for logger (e.g. loggly/winston example)

We should add docs in for pull request #6.

Issue on Heroku

I am facing the following issue while using express-cdn on Heroku. I believe it has something to do with the libjpeg library. Works fine on local machine using both the production and development modes.

app web.1 - - { task: 'express-cdn',
app web.1 - -   message: '"img/inspired/bg2.1349451313000.jpeg" was not found on S3 or was modified recently' }
app web.1 - - /app/node_modules/express-cdn/lib/main.js:32
app web.1 - -   throw new Error('CDN: ' + msg);
app web.1 - -     at Socket.emit (events.js:67:17)
app web.1 - -     at Pipe.onread (net.js:367:14)
app web.1 - -         ^
app web.1 - - Error: CDN: execvp(): No such file or directory at Socket.<anonymous> (/app/node_modules/express-cdn/lib/main.js:235:11)

support for multiple viewdirs

in my sample project there is a layout directory in the root with the views dir. In your current config it wont upload any assets that are included in those jade files.

S3 not uploading in production mode

I followed the example to set up S3 uploading however nothing seems to happen. In development mode everything works as expected, however in production mode my URLs are translated to the cloudfront url but not uploaded (according to AWS logs), there is no output in my node console either. Am I doing something wrong or is this a bug?

var options = { 
  publicDir: '/Users/james/Sites/sops-loco/app/public',
  viewsDir: '/Users/james/Sites/sops-loco/app/views',
  domain: 'd2fzxitfbadxdo.cloudfront.net',
  bucket: 'sops-cdn',
  key: 'omitted',
  secret: 'omitted',
  hostname: '127.0.0.1',
  port: 8080,
  ssl: false,
  production: true 
};

var CDN = require('express-cdn')(express, options);
express.dynamicHelpers({ CDN: CDN });

Probable n00b question: install errors and failure to upload

Following some emergency fixes for heartbleed, I'm having some problems I'm guessing have to do with a newer express-cdn version. When I build the package for our (hitherto working) code, I get several errors to this effect:

npm WARN unmet dependency /home/ubuntu/sumazi/front/node_modules/express-cdn/node_modules/optipng-bin/node_modules/bin-wrapper/node_modules/bin-check requires executable@'^0.1.0' but will load
npm WARN unmet dependency /home/ubuntu/sumazi/front/node_modules/express-cdn/node_modules/optipng-bin/node_modules/bin-wrapper/node_modules/executable,
npm WARN unmet dependency which is version 0.1.1
npm WARN unmet dependency /home/ubuntu/sumazi/front/node_modules/express-cdn/node_modules/optipng-bin/node_modules/bin-wrapper/node_modules/bin-check requires win-spawn@'^2.0.0' but will load
npm WARN unmet dependency /home/ubuntu/sumazi/front/node_modules/express-cdn/node_modules/optipng-bin/node_modules/win-spawn,
npm WARN unmet dependency which is version 2.0.0
npm WARN unmet dependency /home/ubuntu/sumazi/front/node_modules/express-cdn/node_modules/optipng-bin/node_modules/bin-wrapper/node_modules/download/node_modules/decompress requires mkdirp@'^0.3.5' but will load
npm WARN unmet dependency /home/ubuntu/sumazi/front/node_modules/express-cdn/node_modules/optipng-bin/node_modules/bin-wrapper/node_modules/download/node_modules/mkdirp

Followed by a bunch of these:

 task: 'express-cdn',
  message: '"scripts/dashboard.models.js" was not found on S3 or was modified recently' }
{ task: 'express-cdn',
  message: '"scripts/dashboard.views.js" was not found on S3 or was modified recently' }
{ task: 'express-cdn',
  message: '"scripts/jquery.textsearch.js" was not found on S3 or was modified recently' }
{ task: 'express-cdn',

and then a bunch of these

unsuccessful upload of script "scripts/jquery.ui.js" to S3
unsuccessful upload of script "scripts/less.js" to S3
unsuccessful upload of script "scripts/jquery.waypoints.js" to S3
unsuccessful upload of script "scripts/jquery.validity.js" to S3
unsuccessful upload of script "scripts/sumazi.validation.js" to S3
unsuccessful upload of script "scripts/bootstrap-switch.js" to S3

It appears clear that I'm not able to get to the S3 bucket for some reason. The secret and key are correct.

This is inherited code that I don't know very well; guidance would be very helpful.

jpegtran exited with code 2

I'm having an issue while trying to deploy on heroku. Locally it's fine (with the 'production' option set to both true and false). When the app starts on Heroku, I get the following:

2014-02-07T05:54:31.980572+00:00 app[web.1]: { task: 'express-cdn', message: 'jpegtran exited with code 2' }
2014-02-07T05:54:31.981818+00:00 app[web.1]: 
2014-02-07T05:54:31.982421+00:00 app[web.1]: /app/node_modules/express-cdn/lib/main.js:33
2014-02-07T05:54:31.982739+00:00 app[web.1]:   throw new Error('CDN: ' + msg);
2014-02-07T05:54:31.982935+00:00 app[web.1]:         ^
2014-02-07T05:54:31.986097+00:00 app[web.1]: Error: CDN: /app/node_modules/express-cdn/node_modules/jpegtran-bin/vendor/jpegtran: 1: Syntax error: "(" unexpected

Any thoughts?

Support for Dust.js

In the pipeline? Thoughts?

No cache busting in CSS scraper?

Might have misunderstood this, but shouldn't the CSS image scraper replace the resource with the cache busted image?

return attribute+':url('+url+')';

CDN: Error: connect ECONNREFUSED (socket.io related?)

I am running into issues connecting to my S3 account, and cannot figure out what the junk is going on. I am assuming it is a socket.io issue, but I cannot figure out how to work around it.

CDN: Error: connect ECONNREFUSED

I have quadruple checked my credentials. Any clue?

I looked through the issue 12, but they seem to be unrelated. Created a gist for you. Let me know if you need to see anything more...

Error: CDN: missing option "undefined"

I am wrong some were on this can some one please help me

/home/anand/node_modules/express-cdn/lib/main.js:33
throw new Error('CDN: ' + msg);
^
Error: CDN: missing option "undefined"
at /home/anand/node_modules/express-cdn/lib/main.js:33:9
at /home/anand/node_modules/express-cdn/lib/main.js:380:7
at Array.forEach (native)
at /home/anand/node_modules/express-cdn/lib/main.js:378:12
at Object. (/home/anand/Desktop/anandof86.me/app.js:23:32)
at Module._compile (module.js:446:26)
at Object..js (module.js:464:10)
at Module.load (module.js:353:31)
at Function._load (module.js:311:12)
at Array.0 (module.js:484:10)

Usage without S3

Is there anyway to use this module without uploads to S3? I'd like all the minification and optimization without uploading to S3 and instead just cache the results in memory and serve them on subsequent requests.

Purpose of CDN provider-specific modules?

Hi,

Could you explain the purpose of having CDN provider-specific add-on modules listed here?

It looks like those modules don't do anything and that you can use the normal express-cdn module without specifically using CloudFront. e.g. see this MaxCDN blog about using express-cdn.

Just curious if there was intent to provider deeper integration with various CDN providers? If so, what would that look like?

Thanks

CSS background-image is not pointing to CDN endpoint in production mode

On
https://github.com/niftylettuce/express-cdn/blob/master/lib/main.js#L188

String.replace returns a new string and hence the above line is effectively a NOOP
Furthermore the replace function is returning -- attribute+':url('+url+')'; -- irrespective of options.production.

The same issue will affect the font etc. too

Getting error 'CDN is not a function' in jade in production mode

Hi,
Was stuck with getting error while starting nodejs server, using in jade file as:

script(src=CDN("config.js"))
script(src=CDN("js/login.js"))

<<<

configured as:

var CDN = require('express-cdn')(app, options); (options as defined properly as mention in README)
app.locals.CDN = CDN();

with expressjs version as: 4.13.3v

Please tell if you need more.

Blows up for CSS

Express server listening on port 5000 in production mode
"jquery.cookie.js+underscore.min.js+angular.min.js+app.js+controllers.js+directives.js+filters.js+services.10731100894000.js" not modified and is already stored on S3
"bootstrap.min.css+app.2682145981000.css" was not found on S3 or was modified recently

/Volumes/dev/hatchet/node_modules/express-cdn/lib/main.js:33
throw new Error('CDN: ' + msg);
^
Error: CDN: Error: connect ECONNREFUSED
at /Volumes/dev/hatchet/node_modules/express-cdn/lib/main.js:33:9
at /Volumes/dev/hatchet/node_modules/express-cdn/lib/main.js:139:14
at /Volumes/dev/hatchet/node_modules/async/lib/async.js:190:13
at /Volumes/dev/hatchet/node_modules/async/lib/async.js:88:21
at /Volumes/dev/hatchet/node_modules/async/lib/async.js:187:17
at Request._callback (/Volumes/dev/hatchet/node_modules/express-cdn/lib/main.js:245:5)
at /Volumes/dev/hatchet/node_modules/express-cdn/node_modules/request/main.js:120:22
at Request. (native)
at Request.emit (events.js:67:17)
at ClientRequest. (/Volumes/dev/hatchet/node_modules/express-cdn/node_modules/request/main.js:223:10)
9 Jul 17:19:52 - [nodemon] app crashed - waiting for file changes before starting...

Files not versioning correctly

I'm seeing old versions of my js assets served from cloudfront and files in S3 don't have timestamps on their names anymore. S3 has the latest version thou cloudfront delivers old ones.

Any idea what could I be doing wrong? this is my config:

cdnOptions = {
publicDir: path.join(__dirname, 'public')
, viewsDir: path.join(__dirname, 'lib')
, domain: process.env.CDNHOST
, bucket: process.env.S3BUCKET
, key: process.env.AWSKEY
, secret: process.env.AWSSECRET
, hostname: 'staging.farmshares.com'
, port: 80
, ssl: false
, production: false
, logger: winston.info
};

Support require.js?

Would you consider (optionally) supporting require.js and putting all the scripts in a require.config object, in production mode?

Defining the "paths" seems straightforward. I'd assume that the "shim" and "deps" can be part of the individual script's options...

if it's a welcomed feature, I'd be happy to send a pull request that addresses it.

Cheers

uglify leaves code pretty

uglify doesnt seem to uglify my code. In production its exactly the same code as in dev. Not sure if anyone else has come across this?

I'm using browserify to stitch together my script, thought it might be because I had enabled sourceMaps or something..but turning that off doesn't have any effect.

Bottleneck while starting multiple instances of Node

Hi,
I'm using Phusion Passenger to deploy an Express App. When using passenger with 8 instances of node, express-cdn checks 8 times for updated resources in cdn. This ends up choking & timeouts in cdn requests.

Is there a way to configure to check for resource and update them only once ?

support hash for cache busting instead of timestamp

Using timestamp means that assets are only cached between deploys.

error when use CDN() with css

/Users/philip/node/vrvz/node_modules/express-cdn/lib/main.js:203
          minify = minify.replace(/(?:background\-image|background|content|bor
                          ^
TypeError: undefined is not a function
    at /Users/philip/node/vrvz/node_modules/express-cdn/lib/main.js:203:27
    at fs.js:334:14
    at FSReqWrap.oncomplete (fs.js:95:15)

404 response on amazon leads to failure

In my setup with aws region eu-west-1, line 147 in main.js (if (err) throwError(err);) propagates a 404 response as an error if the assets dont exist in my bucket.

Uncaught error in attributes when not string

Reproduce like so:
This works:
!=CDN('/images/brand/logo-hi-def.png', {'class': 'sh-logo'})

This fails without a clear error message:
!=CDN('/images/brand/logo-hi-def.png', {class: 'sh-logo'})

I guess this should still fail, I'll add checking if the attribute is a string and fail with a readable error message cause this specific use case might happen a lot (simple typo).

No putGzipBuffer function exists in knox

I am getting the below error It is trying to call putGzipBuffer on a knox client, but no such function exists. My google foo was unable to find a node project with putGzipBuffer as a function.

"javascripts/application.1340806904000.js" was not found on S3 or was modified recently
ERROR:

Message: Object #<Client> has no method 'putGzipBuffer'

Stacktrace:
====================
TypeError: Object #<Client> has no method 'putGzipBuffer'
    at /home/vagrant/proproofing/node_modules/express-cdn/lib/main.js:157:14
    at Gzip.onEnd (zlib.js:171:5)
    at Gzip.emit (events.js:64:17)
    at zlib.js:358:10
    at Zlib.callback (zlib.js:448:13)

express-cdn doesn't work properly with less-middleware

I can see in the features list that express-cdn should work with less but I can't find any examples/docs that says how to do so.

Should I use another lib instead of less-middleware?

Multi cdn

First, congratulations for the application!

Is it possible to use multi cdn at the same time?

Like a round Robin
ex.: CloudFront cloudflare +

Is it possible?

Thank you!

Command to upload on build time instead of run time

I'd like to avoid the "warming up" period when assets are being uploaded to S3.

Is there a way to invoke the upload on build time, so I can then just switch to another version of the app on production?

Stylus files not being parsed

I keep getting this error when I remove de css files from the stylesheet folder, this are generated from .styl files.

this is the error:

fs.js:524
  return binding.stat(pathModule._makeLong(path));
             ^
Error: ENOENT, no such file or directory 'C:\Users\tcorellano\Dev\farmshares\public\stylesheets\app\buttons.css'
    at Object.fs.statSync (fs.js:524:18)
    at processAssets (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\lib\main.js:368:25)
at Walker.CDN (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\lib\main.js:469:29)
at Walker.EventEmitter.emit (events.js:93:17)
at Walker._wNext (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\node_modules\walk\walk.js:212:8)
at Walker._wNext (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\node_modules\walk\walk.js:210:19)
at Walker._wNext (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\node_modules\walk\walk.js:210:19)
at Walker._wNext (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\node_modules\walk\walk.js:210:19)
at Walker._wOnEmitDone (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\node_modules\walk\walk.js:109:8)
at nextWhenReady (C:\Users\tcorellano\Dev\farmshares\node_modules\express-cdn\node_modules\walk\node-type-emitter.js:69:29)

this is my code:

app.configure(function(){
  app.set('port', port);

  app.use(function(req, res, next) {
      req.i18n = new i18n({
        locales: ['es', 'en'],
        request: req
      });

      ['__','__n','getLocale','isPreferredLocale'].forEach(function(method){
        res.locals[method] = req.i18n[method].bind(req.i18n);
      });

      next();
});

  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');

  app.use(express.favicon());
  app.use(express.logger('dev'));
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(express.cookieParser());
  app.use(express.session({
      secret: "thatisthewayitshouldbeitismydestiny",
      store: new MongoStore({
        url: mongoose.dbUrl
      })
    }));
  app.use(everyauth.middleware());
  app.use(app.router);

  app.use(require('stylus').middleware(__dirname + '/public'));
  app.use(express.static(path.join(__dirname, 'public')));
});

app.configure('development', function(){
  app.use(express.errorHandler());
});

app.configure('qa', function(){
  app.use(express.errorHandler());
});

/**
  * CDN Config
  */
// Set the CDN options
var options = {
    publicDir  : path.join(__dirname, 'public')
  , viewsDir   : path.join(__dirname, 'lib')
  , domain     : 'cdn-prod.farmshares.com'
  , bucket     : 'farmshares-live'
  , key        : 'AKIAJFEJZNQ2Q753OQ3A'
  , secret     : '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'
  , hostname   : 'localhost'
  , port       : 3000
  , ssl        : false
  , production : true
  , logger     : winston.info
};

// Initialize the CDN magic
var CDN = require('express-cdn')(app, options);
// Add the view helper
app.locals({ CDN: CDN() });

Any idea what I'm doing wrong?

Timestamps not being synched on Nodejitsu

Hi Nick, I'm having this problem when deploying my app on nodejitsu. The assets get deployed to S3, but the timestamps are not correct. this are the ones that I see on my page:

on S3 the numbers are sligthly different. this is my log file:

[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:28 GMT-0300] info: [object Object]
[04/16 08:23:29 GMT-0300] info: [object Object]
[04/16 08:23:29 GMT-0300] info: [object Object]
[04/16 08:23:29 GMT-0300] info: [object Object]
[04/16 08:23:29 GMT-0300] info: [object Object]
[04/16 08:23:30 GMT-0300] info: [object Object]
[04/16 08:23:30 GMT-0300] info: [object Object]
[04/16 08:23:30 GMT-0300] GET /stylesheets/app/site.css 200 1578ms - 4.18kb
[04/16 08:23:30 GMT-0300] GET /stylesheets/app/stitches.css 200 1582ms - 573b
[04/16 08:23:30 GMT-0300] GET /stylesheets/app/fonts.css 200 1576ms - 450b
[04/16 08:23:30 GMT-0300] GET /stylesheets/libs/bootstrap.css 200 1570ms - 100.9kb
[04/16 08:23:30 GMT-0300] info: [object Object]
[04/16 08:23:30 GMT-0300] info: [object Object]
[04/16 08:23:30 GMT-0300] GET /stylesheets/app/footer.css 200 1588ms - 1.77kb
[04/16 08:23:30 GMT-0300] info: [object Object]
[04/16 08:23:30 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: successfully uploaded image "images/skype.1366015416000.png" to 
S3
[04/16 08:23:31 GMT-0300] info: successfully uploaded image "images/fb.1366015416000.png" to S3
[04/16 08:23:31 GMT-0300] info: [object Object]
[04/16 08:23:31 GMT-0300] info: successfully uploaded image "images/mail.1366015416000.png" to S
3
[04/16 08:23:32 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/twitter.1366015416000.png" t
o S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/pin.1366015416000.png" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/youtube.1366015416000.png" t
o S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] GET /stylesheets/app/slider.css 200 1361ms - 2.23kb
[04/16 08:23:33 GMT-0300] GET /stylesheets/app/navbar.css 200 1366ms - 1.47kb
[04/16 08:23:33 GMT-0300] GET /stylesheets/app/ribbon.css 200 1427ms - 1.71kb
[04/16 08:23:33 GMT-0300] GET /stylesheets/app/admin.css 200 1482ms - 167b
[04/16 08:23:33 GMT-0300] GET /stylesheets/app/newsletter.css 200 1419ms - 305b
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] GET /stylesheets/vendor/jquery.fileupload-ui.css 200 429ms - 1.49kb
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/glyphicons-halflings.png" to
 S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/glyphicons-halflings-white.p
ng" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/bg.png" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "front/Meet.1366015416000.jpg" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/cl.1366015416000.png" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] GET /stylesheets/app/farm.css 200 440ms - 12.72kb
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "front/Learn.1366015416000.jpg" to S
3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/us.1366015416000.png" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "front/Eat.1366015416000.jpg" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/footer-bg.png" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/nav.png" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/progressbar.gif" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/navbar-bg.png" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/loading.gif" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/loading.gif" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/truck.png" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/truck.png" to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/guarantee.1366015416000.png"
 to S3
[04/16 08:23:33 GMT-0300] info: successfully uploaded image "images/shares/base.png" to S3
[04/16 08:23:33 GMT-0300] info: [object Object]

Everything seems ok but the css files have smaller timestamps (eg S3/site.1366111178000.css vs. app/site.1366111409000.css)

I'm replacing filenames manualy to work around but it's a pain, any ideas why I'm getting this?

Add-on support for CDN/storage modules (see Readme section)

See this section of the Readme for insight:

https://github.com/niftylettuce/express-cdn#add-on-modules-coming-soon

I'm wondering how you guys think we should do this?

Fingerprinting: MD5 digest vs modified time

Currently, express-cdn uses modified time as its cache invalidation strategy. Rails 2.x used to use this approach and switched to an MD5 digest approach for reasons outlined here:
http://edgeguides.rubyonrails.org/asset_pipeline.html#what-is-fingerprinting-and-why-should-i-care

Since I am using less stylesheets, I ran into problems with the mtime approach because every the css file would be recompiled with a new mtime even though the file contents remain identical to previous compiles.

PS Google has a good document describing caching strategies here:
https://developers.google.com/speed/docs/best-practices/caching

Is it possible to insert 'apple-touch-icon' links?

I want to generate something like this through Jade:

link(rel="apple-touch-icon", sizes="57x57", href="/icons/apple-touch-icon-57x57.png")

I tried

!= CDN("/icons/apple-touch-icon-57x57.png", { rel:"apple-touch-icon", sizes:"57x57" })

but that generates an img tag.

load assets outside of views dir

would be nice to be able to use this for assets inside node.js code i.e :

app.use(function(req, res, next) {
  res.send(CDN('/path/to/asset'));
});

Is there any way to achieve this ?

Thanks !

ladjs / express-cdn Goto Github PK

express-cdn's People

Contributors

Stargazers

Watchers

Forkers

express-cdn's Issues

CDN: Error: connect ECONNREFUSED

CDN: Error: connect ECONNREFUSED

Recommend Projects

Recommend Topics

Recommend Org