dustyburwell / connect-file-cache Goto Github PK

TL;DR: Detect already-gzipped content stream or remove compression feature altogether now that there is connect.compress() available.

I'm using this module through connect-assets. I don't have a working test case for this yet but please bear with me as I've confirmed this to be reproducible and can point you to problemetic lines if you want.

I'm using connect-assets in conjunction with connect.compress() The problem is that connect.compress() will compresses all compatible content wether or not it is already compressed. So if me or another user, configures express or connect like this:

app.use(express.compress())
app.use(require('connect-assets')())

This will results in all assets files response being double-gzipped. (because connect-file-cache will write already gzipped-data from its cache to the response which is already piped through the gzip stream setup by express.compress().

See here: https://github.com/TrevorBurnham/connect-file-cache/blob/master/lib/cache.js#L99

I have to do something like this: curl 0.0.0.0:7000/js/site.js -H "Accept-Encoding: gzip" | gunzip | gunzip for the content to render.

Although, this can be worked around by switching the app.use order. I believe this should be fixed because more people will now use the vanilla express.compress() call and the order of middleware placement really shouldn't have effects on how files are being compressed or not.

This can get confusing because it will often shows up when people are re-arranging the middlewares app.use calls and suddenly getting a bad response from all the asets without an idea of what's going on.

Hello,

I have app.js like this:

express = require("express")
connectCache = require("connect-file-cache")

app = express.createServer()

cache = connectCache({
  src: "javascripts",
  basePath: "/js"
});

app.use(cache.middleware);

app.get("/", function (req, res) {
  res.end("hello\n");
});

app.listen(3000);

And the content of javascripts/script.js is

alert("Hello world!");

I expect the application to send files from "javascripts" directory on requests to "/js/...", meaning GET /js/script.js should respond with javascripts/script.js. The documentation of connect-file-cache says that this is possible (routePrefix: Data will be served from this path (defaults to /)), but:

$ curl localhost:3000/js/script.js
Cannot GET /js/script.js
$ curl localhost:3000/script.js
alert("Hello world!"); # this is javascripts/script.js
$ curl localhost:3000
curl: (52) Empty reply from server
# the app finished with Error: EISDIR, illegal operation on a directory

I looked at src/cache.coffee and @options.routePrefix is used only in comment and the constructor and seems to be ignored otherwise, causing the above behavior.

Is this a bug or a feature?

Thank you,
Jan Špaček

P.S.: express and connect-file-cache are installed via npm, node is v0.6.12.

Currently, files are stored in buffers, which are only accessible from one process. This creates problems when using something like cluster to run multiple Node processes on a server; see connect-assets issue 32. For that reason, connect-file-cache should have an option to store files in Redis instead.

Note that this will require a major API change, because get/set operations will no longer necessarily be synchronous.

Since node 0.6.x there's a zlib binding.

The currently used custom zlib binding doesn't work anymore with 0.6.

I'm working with @adunkman on maintenance of connect-assets. In the spirit of this comment assaf/zombie#377 (comment) I'm totally willing to take over maintenance of the connect-file-cache module.

Hi,

I'm running into an issue running Zombie.js on my app, which uses
Express.js > connect-assets > connect-file-cache

After some investigation, it seems Zombie is getting gzipped Javascript, which is does not handle. So I ran the requests through Charles, and this is what I get. It seems that connect-file-cache returns gzipped content in cases where I don't expected it to.

Looking at the source, it seems this line is true regardless of the value of accept-encoding
if cacheHash.gzippedData and req.headers['accept-encoding']?.indexOf /gzip/

This is the small test case I investigated:

Basic request

curl http://localhost:3000/js/script.js --proxy localhost:8888

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/javascript
Last-Modified: Wed, 03 Oct 2012 06:02:00 GMT
Content-Length: 2188
Date: Wed, 03 Oct 2012 06:06:10 GMT
Connection: keep-alive

Asking for a gzip response

curl http://localhost:3000/js/script.js --proxy localhost:8888 --header 'accept-encoding: gzip'

[same except]
Content-Encoding: gzip
Content-Length: 675

Asking for an untouched response

curl http://localhost:3000/js/script.js --proxy localhost:8888 --header 'accept-encoding: identity'

[same except]
Content-Encoding: gzip
Content-Length: 675

Any encoding?

curl http://localhost:3000/js/script.js --proxy localhost:8888 --header 'accept-encoding: foobar'

[same except]
Content-Encoding: gzip
Content-Length: 675

cache.js line 80. The in memory mtime includes milliseconds; whereas the client's header string does not making this comparison almost always presume that the file has been modified even if it hasn't.