Coder Social home page Coder Social logo

gulp-cache's Introduction

gulp-cache

NPM version Node version Dependencies status Build status Coverage status

A temp file based caching proxy task for gulp.

Install

npm i -D gulp-cache
# or
yarn add -D gulp-cache

Usage

import gulp from 'gulp';
import favicons from 'gulp-favicons';
import srcset from 'gulp-srcset';
import cache from 'gulp-cache';

gulp.task('favicon', () =>
    gulp.src('src/favicon.svg')
        .pipe(cache(
            // Target plugin, the output of which will be cached.
            favicons(faviconsConfig),
            // Options for `gulp-cache` plugin.
            {
                // Bucket to store favicons in cache.
                name: 'favicons'
            }
        ))
        .pipe(gulp.dest('./favicons'))
);

gulp.task('images', () =>
    gulp.src('src/**/*.{jpg,png,svg}')
        .pipe(cache(
            // Target plugin, the output of which will be cached.
            srcset(srcsetRules),
            // Options for `gulp-cache` plugin.
            {
                // Bucket to store images in cache.
                name: 'images'
            }
        ))
        .pipe(gulp.dest('./images'))
);
Complex usage example
import fs from 'fs';
import gulp from 'gulp';
import jshint from 'gulp-jshint';
import cache from 'gulp-cache';

const jsHintVersion = '2.4.1';
const jshintOptions = fs.readFileSync('.jshintrc');

function makeHashKey(file) {
    // Key off the file contents, jshint version and options
    return `${file.contents.toString('utf8')}${jshintVersion}${jshintOptions}`;
}

gulp.task('lint', () =>
    gulp.src('src/**/*.js')
        .pipe(cache(
            // Target plugin, the output of which will be cached.
            jshint('.jshintrc'),
            // Options for `gulp-cache` plugin.
            {
                key: makeHashKey,
                // What on the result indicates it was successful
                success(jshintedFile) {
                    return jshintedFile.jshint.success;
                },
                // What to store as the result of the successful action
                value(jshintedFile) {
                    // Will be extended onto the file object on a cache hit next time task is ran
                    return {
                        jshint: jshintedFile.jshint
                    };
                }
            }
        ))
        .pipe(jshint.reporter('default'))
});

API

cache(pluginToCache [, options])

pluginToCache

Target plugin, the output of which will be cached.

options

Options for gulp-cache plugin.

options.fileCache

[Optional] Where to store the cache objects

options.name

[Optional] The name of the bucket which stores the cached objects

  • Defaults to default
options.key

[Optional] What to use to determine the uniqueness of an input file for this task.

  • Can return a string or a Promise that resolves to a string.

  • The result of this method is converted to a unique MD5 hash automatically; no need to do this yourself.

  • Defaults to file.contents if a Buffer, or undefined if a Stream.

options.success

[Optional] How to determine if the resulting file was successful.

  • Must return a truthy value that is used to determine whether to cache the result of the task. Promise is supported.

  • Defaults to true, so any task results will be cached.

options.value

[Optional] What to store as the cached result of the task.

  • Can be a function that returns an Object or a Promise that resolves to an Object.

  • Can also be set to a string that will be picked of the task result file.

  • The result of this method is run through JSON.stringify and stored in a temp file for later retrieval.

  • Defaults to 'contents' which will grab the resulting file.contents and store them as a string.

Clearing the cache

If you find yourself needing to clear the cache, there is a handy dandy cache.clearAll() method:

import cache from 'gulp-cache';

gulp.task('clear', () =>
    cache.clearAll()
);

You can then run it with gulp clear.

One-to-many caching

To support one-to-many caching in Your Gulp-plugin, you should:

  • Use clone method, to save _cachedKey property:
const outputFile1 = inputFile.clone({ contents: false });
const outputFile2 = inputFile.clone({ contents: false });

outputFile1.contents = new Buffer(...);
outputFile2.contents = new Buffer(...);

const outputFiles = [
    outputFile1,
    outputFile2,
    ...
];
  • Or, do it manually:
const outputFiles = [
    new Vinyl({..., _cachedKey: inputFile._cachedKey}),
    new Vinyl({..., _cachedKey: inputFile._cachedKey}),
    ...
];

License

The MIT License (MIT)

Copyright (c) 2014 - present Jacob Gable

gulp-cache's People

Contributors

dangreen avatar dependabot[bot] avatar fidian avatar janb87 avatar jgable avatar jklmli avatar kilianc avatar puggan avatar shaodahong avatar shinnn avatar sinedied avatar stevemao avatar zbennett10 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gulp-cache's Issues

Exclude list of files from being cached

Hey there,

maybe that's already possible but I'm searching for a possibility to exclude certain files from being cached.
The setup right now looks like src -> babel -> cache. My problem is that I have a babel plugin to inline an imported yaml file as js object. When this file changes I want to rebuild the bundle but as the importing file didn't changed it is still cached with the old value.

An option like exclude(file) { return file.path.inludes('file.js'); } would be cool

What do you think?

Best
FragSalat

0.4.3 breaks cache behavior for plugins which change the file extension

I'm using gulp cache in combination with gulp-less.

After the latest release (0.4.3) my build broke.

What happens if my file is in the cache:
0.4.2 Build styles.less => outputs styles.css
0.4.3 Build styles.less => outputs styles.less

var gulp = require('gulp');
var less = require('gulp-less');
var cache = require('gulp-cache');

gulp.src('**/*.less')
                .pipe(cache(less(), {
                    key: function (file) {
                        return [file.contents.toString('utf8'), file.path, 'less'].join('');
                    },
                    success: function (file) {
                        console.log('Compiled ' + file.path);
                        return true;
                    }
                }))
                .pipe(gulp.dest('.'));

Memory leak when running task

☁  assets [master] ⚡ gulp images
[gulp] Using file /Users/donaldallen/Sites/somesite.ca/app/assets/gulpfile.js
[gulp] Working directory changed to /Users/donaldallen/Sites/somesite.ca/app/assets
[gulp] Running 'images'...
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
    at Stream.EventEmitter.addListener (events.js:160:15)
    at _.extend._runProxiedTask (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/lib/TaskProxy.js:134:19)
    at _.extend._runProxiedTaskAndCache (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/lib/TaskProxy.js:107:21)
    at /Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/lib/TaskProxy.js:26:25
    at tryCatch1 (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/util.js:64:19)
    at Promise$_callHandler [as _callHandler] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:708:13)
    at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:724:18)
    at Promise$_settlePromiseAt [as _settlePromiseAt] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:896:14)
    at Promise$_fulfillPromises [as _fulfillPromises] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:1041:14)
    at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/async.js:64:12)
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
    at Stream.EventEmitter.addListener (events.js:160:15)
    at Stream.EventEmitter.once (events.js:185:8)
    at _.extend._runProxiedTask (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/lib/TaskProxy.js:137:19)
    at _.extend._runProxiedTaskAndCache (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/lib/TaskProxy.js:107:21)
    at /Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/lib/TaskProxy.js:26:25
    at tryCatch1 (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/util.js:64:19)
    at Promise$_callHandler [as _callHandler] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:708:13)
    at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:724:18)
    at Promise$_settlePromiseAt [as _settlePromiseAt] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:896:14)
    at Promise$_fulfillPromises [as _fulfillPromises] (/Users/donaldallen/Sites/somesite.ca/app/assets/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:1041:14)

And here is my task:

gulp.task('images', function() {
    return gulp.src(['img/**/*.png', 'img/**/*.jpg', 'img/**.*.gif'])
        .pipe(cache(image()))
        .pipe(gulp.dest('../../public/assets/public/img'));
});

So the errors above show up, then there's about a 15 second pause, and the task continues. In this case, my images start compressing and moving. It feels slower than it should be.

Fails under Node 5.0

Trying to run the latest release of gulp-cache from NPM under Node 5.0 currently fails with:

Unhandled rejection TypeError: this.task.setMaxListeners is not a function

suggestion: api change / alternative

so right now I have something like:

gulp.task('inline:html', function () {
  return gulp.src('dist/**/*.html')
    .pipe(plugins.cache(plugins.inlineSource({
      svgAsImage: true,
      rootpath: process.cwd() + config.root
    })))
    .pipe(gulp.dest('dist'))
})

gulp.task('minify:html', function () {
  return gulp.src('dist/**/*.html')
    .pipe(plugins.cache(plugins.minifyHtml({
      conditionals: true,
      quotes: true
    })))
    .pipe(gulp.dest('dist/'))
})

which in general is kinda ugly (plugins.cache(plugins.minifyHtml({) and it becomes even worse because you need to address the fact that those two tasks are using the same cache because running on the same files.

gulp.task('inline:html', function () {
  return gulp.src('dist/**/*.html')
    .pipe(plugins.cache.start('inline'))
    .pipe(plugins.inlineSource({
      svgAsImage: true,
      rootpath: process.cwd() + config.root
    }))
    .pipe(plugins.cache.end('inline'))
    .pipe(gulp.dest('dist'))
})

gulp.task('minify:html', function () {
  return gulp.src('dist/**/*.html')
    .pipe(plugins.cache.start('minify'))
    .pipe(plugins.minifyHtml({
      conditionals: true,
      quotes: true
    }))
    .pipe(plugins.cache.end('minify'))
    .pipe(gulp.dest('dist/'))
})

With some magic we could control the internal flow, have better syntax, also you cloud potentially .pipe multiple times between start/end block.

N.B. really opinionated suggestion, feel free to tell me to STFU. 🍻

gulp cache v0.5.0 doesn't work with gulp-imagemin

var gulp     = require('gulp'),
    imagemin = require('gulp-imagemin'),
    cache    = require('gulp-cache');

gulp.task('imagemin', function() {
	return gulp.src('app/img/**/*')
		.pipe(cache(imagemin()))
		.pipe(gulp.dest('dist/img')); 
});

When I run command "gulp imagemin" in terminal, it starts task, but command ends not finishing task
gulp-cahce-error
My package.json devDependencies part looks like this:
package-json
However, if I use gulp-cahce version v0.4.6, it works fine.
Here source code to download:
gulp-cahce-error.zip

Caching task doesn't seem to finish when there are no new files

Output without cache:

$ gulp build --no-cache
[gulp] Using gulpfile ~/Projects/gulp-starter/gulpfile.js
[gulp] Starting 'templates'...
[gulp] Starting 'styles'...
[gulp] Starting 'fonts:clean'...
[gulp] Starting 'html:clean'...
[gulp] Starting 'images:clean'...
[gulp] Finished 'images:clean' after 34 ms
[gulp] Starting 'images'...
[gulp] Finished 'html:clean' after 63 ms
[gulp] Starting 'html'...
[gulp] Finished 'fonts:clean' after 77 ms
[gulp] Starting 'fonts'...
[gulp] Finished 'templates' after 196 ms
[gulp] Finished 'styles' after 2.82 s
[gulp] Finished 'html' after 2.76 s
[gulp] Finished 'fonts' after 2.75 s
[gulp] gulp-imagemin: ✔ pipboy.jpg (already optimized)
[gulp] Finished 'images' after 2.81 s
[gulp] Starting 'build'...
[gulp] Finished 'build' after 14 μs

Output with cache:

$ gulp build
[gulp] Using gulpfile ~/Projects/gulp-starter/gulpfile.js
[gulp] Starting 'templates'...
[gulp] Starting 'styles'...
[gulp] Starting 'fonts:clean'...
[gulp] Starting 'html:clean'...
[gulp] Starting 'images:clean'...
[gulp] Finished 'images:clean' after 29 ms
[gulp] Starting 'images'...
[gulp] Finished 'html:clean' after 58 ms
[gulp] Starting 'html'...
[gulp] Finished 'fonts:clean' after 72 ms
[gulp] Starting 'fonts'...
[gulp] Finished 'templates' after 214 ms
[gulp] Finished 'styles' after 2.77 s
[gulp] Finished 'html' after 2.71 s
[gulp] Finished 'fonts' after 2.7 s

Caching part in the gulpfile.js file:

// Images
gulp.task('images:clean', function(){
  return gulp.src(Config.paths.dist.images + '/**/*', { read: false })
    .pipe(gulpif(!Config.cache, clean()));
});
gulp.task('images', ['images:clean'], function(){
  return gulp.src(Config.paths.app.images + '/**/*')
    .pipe(gulpif(Config.cache, cache(imagemin(Config.imagemin))))
    .pipe(gulpif(!Config.cache, imagemin(Config.imagemin)))
    .pipe(gulp.dest(Config.paths.dist.images + '/'));
});

You can find the whole gulpfile.js file here:
https://github.com/3bola/gulp-starter/blob/master/gulpfile.js

Does not preserve whole object from the stream

It sure would be nice if I could do something like this. Assume the standard require() calls are there.

combine = require('stream-combiner');
// LESS files get converted then written to a build directory for
// live viewing.
processLess = combine(less(), rename(function (path) {
        path.dirname = "build/" + path.dirname;
    }), gulp.dest("./"));
gulp.src("lib/**/*.less")
    .pipe(cache(processLess))
    .pipe(cssmin())
    .pipe(concat("www/styles.css"))
    .pipe(gulp.dest("./"));

It looks like the filename gets dropped by the default handle for "value". When I write .pipe(cache(processLess, {value: function (file) { return file; }})) (or something very similar) it works as expected. After sending the files through code similar to what's above, I need them to get processed by cssmin, but that plugin only converts files that have a ".css" extension. So, when the file does not change and it is pulled from the cache then the production version doesn't have 100% minified code.

I generate live CSS from LESS files but only wish to convert them when they are not in the cache. These are written out to "build/" for easier debugging with a special version of the application that links directly to the unminified files.

Perhaps you could save more properties from the varnish object? Or allow for some syntax that would let the user specify multiple properties that you're picking off the object?

Homedir as cache path on centOS

Hi there,

First of all thank you creating and maintaining this package.

This morning I'm trying to setup a remote development machine (centOS) and use the current users homedir with os.homedir() (docs).

Versions installed:

$ node -v
v6.9.1

$ gulp -v
CLI version 1.2.2.
Local version 3.9.1

# npm -v
3.10.8

You can take a look at my gulpfile.js right here.

The part I'm interested in is how do I setup gulp-cache to use the users homedir as mentioned in Issue #32?

As you can see in my gulpfile.js from line 239 to 246 I'm trying to setup a Cache instance and use the provided methods to setup my desired result. At line 36 I've required the os API provided by Node.

// Line 36
const os = require('os');
// Line 239 to 246
gulp.task('clear', function (done) {
    $.cache.Cache({
        tmpDir: os.homedir(),
        cacheDirName: "gulp-cache"
    });

    return $.cache.clearAll(done);
});

When I console.log(os.homedir()) it returns /home/tdwit, so I know it returns the right string. When I run my default task gulp, it returns the following:

[08:29:36] Using gulpfile ~/projects/gulp/gulpfile.js
[08:29:36] Starting 'clean'...
[08:29:36] Starting 'clear'...
[08:29:36] Finished 'clean' after 22 ms
[08:29:36] 'clear' errored after 19 ms
[08:29:36] Error in plugin 'gulp-cache'
Message:
    Problem clearing the cache: EACCES: permission denied, rmdir '/tmp/gulp-cache/default'

Any help is greatly appreciated, thanks in advance.

Should be able to clear the cache

Brought up in #9.

Add two new methods to exported cache module; clear and clearAll.

gulp.task('clearCache', function() {
  // Still pass the files to clear cache for
  gulp.src('./lib/*.js')
    .pipe(cache.clear());

  // Or, just call this for everything
  cache.clearAll();
});

clear will only clear cache entries for the passed in files, clearAll will clear all cache entries.

Cannot work with streams files

In many cases file path + mtime is enough to take from cache.
Why does the plugin force to read the file? That counters performance.

gulp.src('**/*.js', {read: false}) should be cacheable, why not?

file.path + file.mime -> cache?
  -> yes -> get content and all needed from cache
  -> no -> populate content and process it

Images had returned from the cache are corrupted

Launch gulpCacheTest for the first time:

> gulp images
[gulp] Using file C:\gulpCacheTest\Gulpfile.js
[gulp] Working directory changed to C:\gulpCacheTest
[gulp] Running 'images'...
[gulp] gulp-imagemin: ? hackers-movie-poster.jpg (saved 1.3 kB)
[gulp] gulp-imagemin: ? movie-night_t_nv.jpg (saved 70.4 kB)
[gulp] gulp-imagemin: ? scotch_egg.jpeg (saved 3.2 kB)
[gulp] gulp-imagemin: ? exitbuddy.png (saved 3.3 kB)
[gulp] gulp-imagemin: ? slimer1.jpg (saved 11.6 kB)
[gulp] gulp-imagemin: ? road-house-patrick-swayze-31226564-2560-1851.jpg (saved 31.7 kB)
[gulp] Finished 'images' in 4.57 s

Windows Photo Viewer shows all these pictures well
But if it launch images task for the second time:

> gulp images
[gulp] Using file C:\gulpCacheTest\Gulpfile.js
[gulp] Working directory changed to C:\gulpCacheTest
[gulp] Running 'images'...
[gulp] Finished 'images' in 377 ms

Windows Photo Viewer can't open this picture because the file appears to be damaged or corrupted
P.S.: OS Win 7 x64, node v0.10.28, gulp-cache v0.1.3 (on v0.1.9 the same result)

\node_modules\gulp-cache\lib\index.js:60 SyntaxError: Unexpected identifier

Hey guys I'm running the same project on my mac and my PC. My colleague added gulp-cache to it recently (on his mac), and since then I can't work from my PC anymore because of an issue with gulp-cache, it works just fine on my mac as well as on my colleague's mac.

Here's the version we're using : "gulp-cache": "^1.1.3"

I've tried removing it and reinstalling it, and also removing my node_modules dir and reinstalling everything, same issue.

$ gulp --env=dev

C:\wamp\www\GamingSchool\symfony\node_modules\gulp-cache\lib\index.js:60
async _processFileAsync(inputFile, signals = new EventEmitter()) {
^^^^^^^^^^^^^^^^^
SyntaxError: Unexpected identifier
at Object.exports.runInThisContext (vm.js:76:16)
at Module._compile (module.js:542:28)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object. (C:\wamp\www\GamingSchool\symfony\gulpfile.js:18:13)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)

Question: single file vs wildcard

I'm struggling with something that worked in previous versions and is no longer working:
(modified slightly for clarity)

gulp.src("./dist/**/*.js")
    .pipe(
       cache(
        map(function (file, cb) {
          console.log("Mapping: " + file.basename);
          cb(null, file);
        })
      )
    );

If there are 3 files in the dist folder with the .js extension the mapper is not called if the file hasn't changed. If I modify the code to only get a single file (there is an example in the Readme that does this) it seems like it doesn't cache or thinks the file has changed even if it hasn't. Is there anything super obvious that I'm doing wrong?

I created this simplified example and even if test.js doesn't change it still calls my map function.

(FYI, I need the map function to make a call to a more complicated scenario, so "remove map" isn't really going to cut it, although if that's the culprit I'm open but doesn't seem to be since as I said the above example does work)

gulp.src("dist/test.js")
    .pipe(
      cache(
        map(function (file, cb) {
          console.log("Mapping: " + file.basename);
          cb(null, file);
        })
      )
    );

How to fix ReferenceError: cache is not defined error ?

Hi,
I would to clear my browser cache with gulp and browser-sync but I get this error when my task is running:
ReferenceError: cache is not defined
My gulpfile contain:

var basePaths = {
    project:  './',
    projectsourcemap:  '../',
    src: './sass/**/*.scss', // fichiers scss à surveiller
};
gulp.task('clearCache', function() {
  // Or, just call this for everything
  cache.clearAll();
});

gulp.task('default', ['browser-sync'], function(){
  gulp.watch(basePaths.src, ['sasscompil']);
  gulp.watch(basePaths.src, ['clearCache']);
});

Thanks for help

Renamed file remains cached under its original name

To reproduce:

  1. Make two jpgs, tetsuo.jpg and kaneda.jpg.
  2. Run through cache(imageMin());
  3. tetsuo.jpg and kaneda.jpg are output to the dest directory.
  4. Rename kaneda.jpg to kanedaaaaaaaa.jpg
  5. Run the cached imageMin task.
  6. No file named kanedaaaaaaaa.jpg will be output.
  7. Delete the destination directory
  8. Run the cached imageMin task.
  9. tetsuo.jpg and kaneda.jpg (not kanedaaaaaaaa.jpg) are output to the dest directory.

After clearing the cache, kanedaaaaaaaa.jpg is output to the dest directory. Maybe the plugin isn't looking at file names, only contents?

Cleaning the cache

HI, first of all thank you for this awesome plugin. I am using generator-gulp-angular and when doing gulp serve:dist things go wrong. I tried removing the usage of gulp-cache and then everything is ok. I am guessing the cache got corrupted, so is returning wrong values. Is there any way to clean the cache?
Documentation is very sparse and I had hard time finding where is cache stored and how to clean it.
Thank you!

Difference from gulp-cached unclear

Hi,

I'm new to gulp, and I accidentally installed this plugin when I was truing to install gulp-cached (just forgot the d). In investigating this project, I was a little confused to find that it seems pretty similar.

Is the difference that this is file based caching instead of in memory based caching?

Perhaps it may make sense to rename the project to alleviate confusion for others in the future? If the difference is that it's file based caching, then maybe gulp-file-cache would be a more clear distinction between the two?

Ditch bluebird

I'd suggest to ditch bluebird in the next major release. Promises are native.

self.task.setMaxListeners is not a function

Running into the same error as #45.

gulp.task 'css:test', ->
  gulp.src "front/src/css/site/front/lib/about.less"
    .pipe cache(new cache.Cache { cacheDirName: 'custom-cache' }, {})
    .pipe gulp.dest util.paths.less.out

Broken with latest @babel/runtime v7.3.0

With latest @babel/runtime v7.3.0,

TypeError: Cannot convert undefined or null to object
at Function.getOwnPropertyDescriptors ()
at _objectSpread (/Users/huocp/bw/bcx-broker-au-frontend/node_modules/@babel/runtime/helpers/objectSpread.js:19:46)
at plugin (/Users/huocp/bw/bcx-broker-au-frontend/node_modules/gulp-cache/lib/index.js:491:19)
at buildJs (/Users/huocp/bw/bcx-broker-au-frontend/gulpfile.js:102:17)
at build (/Users/huocp/bw/bcx-broker-au-frontend/gulpfile.js:128:5)

Caused by this:

gulp-cache/src/index.js

Lines 75 to 78 in 37039dc

const options = {
...plugin.defaultOptions,
...task.cacheable,
...inputOptions

The task.cacheable is undefined, causing @babel/runtime to fail.

This seems caused by the compiled code lib/index.js is based on old babel version (presets, plugins?).

Two possible fixes:

  1. (tested) fix the dep of @babel/runtime in package.json to "@babel/runtime": "~7.2.0", pinning it to 7.2.x.
  2. (not tested yet) update all deps, and publish recompiled file.

gulp-cache with gulp-imagemin

Hi!

I'm using gulp-cache with other modules with success, but don't with gulp-imagemin... I'm using:

gulp.src(srcImg + '/**/*.{gif,ico,jpg,png,svg}', { base: srcImg })
    .pipe(cache(imagemin({
        interlaced: true,
        pngquant: true,
        progressive: true
    })))
    .pipe(gulp.dest(distImg));

But always I ran the task all the images are treated by gulp-imagemin.

There's something it's possible to do or gulp-cache it's not supposed to do something like that?

Great plugin and best regards!

A way to filter files and pipe multiple tasks after? (like gulp-cached)

I have a pretty heavy and lengthy stream that I'd like to cache/filter before it starts, but it's not clear how to do it with gulp-cache. Your example only seems to cache a single task, but mine is something like this:

gulp.task('extract-frames', function(){
  return gulp.src('files/*.mov')
    .pipe(exec('ffmpeg …'))
    .pipe(exec('ffmpeg …'))
    .pipe(exec('ffmpeg …'))
    .pipe(exec('ffmpeg …'))
    .pipe(exec('ffmpeg …'))
    .pipe(exec('ffmpeg …'))
    .pipe(exec('ffmpeg …'));
});

I've seen that most gulp plugins just filter the stream once, rather than receive a task as a parameter. gulp-cached is an example:

gulp.task('lint', function(){
  return gulp.src('files/*.js')
    .pipe(cached('linting'))
    .pipe(jshint())
    .pipe(jshint.reporter())
});

This lets me pipe as many tasks as I want after the filter. Can this be done with gulp-cache?

Support many-to-1 / many-to-many tasks

gulp-cache currently assumes the wrapped task is 1-to-1: it only passes 1 file to the task at a time, and only expects 1 file back. This means it doesn't work with tasks like gulp-concat, gulp-closure-compiler etc which need to operate on multiple files at once.

I’ve got a PR in the works, but thought I’d preface it with some discussion.

The implementation in my PR will pass all input files to TaskProxy(), which will base the cache key on all of these, pass all into the proxied task, and cache all files returned by the task. This will handle 1-to-1, 1-to-many, many-to-1 and many-to-many tasks all correctly.

However, it’s hard to do this without introducing a breaking change: the customizable key, value and success functions are currently only passed 1 file at a time, so it’s likely consumers’ implementations will only expect 1 file — and for many-to-many tasks all 3 of these will need to accept multiple files.

So, my questions:

  • Is there a reason it wasn’t done like this in the first place?
  • If not, and we agree this change is desirable: Is this a good case for a breaking change?

The library’s not at 1.0.0 yet, so it might be hard to communicate this change.

I could avoid a breaking change by making this new behaviour optional, e.g. requiring manyToMany: true to be set in the options argument, but this could be awkward to document… the README would have to say something like this, which isn’t the clearest:

key

[Optional] A function to determine the uniqueness of an input file or set of files for this task.

  • If manyToMany is true, will be passed an array of files as the first argument; otherwise will be passed a single file.
  • Can return a string or a promise that resolves to a string. Optionally, can accept a callback parameter for idiomatic node style asynchronous operations.
  • The result of this method is converted to a unique MD5 hash automatically; no need to do this yourself.
  • Defaults to a concatenation of the current version of gulp-concat plus the contents of each input file which is a Buffer.

Thoughts?

Must pass a task to default cache.proxy

I'm trying to follow the example from the readme to cache image files and only compress changed/new ones.

I'm getting the error: Must pass a task to default cache.proxy

gulp.task('imagemin', function() {

    gulp.src('src/images/**/*')
        .pipe(cache.proxy(imagemin(), {
            key: makeHashKey,
            success: function (imageFile) {
                return imageFile;
            },
            value: function (imageFile) {
                return {
                  imagemin: imageFile.imagemin
                };
            }
        }))
        .pipe(imagemin())
        .pipe(gulp.dest('dist/assets/img'));

});

function makeHashKey(file) {
    return file.contents.toString('utf8');
}

I'm not fully sure what I'm doing so I'm probably missing something. I tried using the example directly and I got the same error.

Any help appreciated,
Cheers!

Missing some file in my dist folder when using imagemin

Hello,

I use a rather classic gulp / cache / imagemin workflow:

gulp.task('cordova:resources', function() {
  return gulp.src('resources/**')
    .pipe($.cache($.imagemin({
      optimizationLevel: 3,
      progressive: true,
      interlaced: true
    })))
    .pipe(gulp.dest(path.join(conf.paths.tmp, 'resources')))
    .pipe($.size());
});

But with this, some file are not copied to the dest folder?

When removing the $.cache(), everything works well.
If I use their file name directly instead instead of a glob, it also works (after clearing the cache)...

Any ideas?

Bad dockumentation of Options.

Got the same error as #45 and #49.

It looks like they both missread the dockumentation, on how to use the options.
So I guess I don't get the dockumentation either.

Could you please add an exemple no how to use the options.

Most other plugins use options as an optional parameter, like:
.pipe(plug.cache({name: 'less'}, plug.less()))

But i guess thats not the case in your plugin.

How to handle error?

I'm using this plugin with gulp-coffee and gulp.watch to do incremental build.
When there is an error thrown by coffee plugin, it seems to crash gulp.watch.
I tried to use gulp-plumber but didn't help.
What's the correct way of handling error?

stream = gulp.src bundles[filename]
  .pipe plumber()
  .pipe coffeeFilter
  .pipe cache coffee()
  .pipe coffeeFilter.restore()
  .pipe concat filename

No path specified Error while using gulp cache with WebFundamentalsKit

Gulp v: 3.8.7 and gulp-cache (map-stream): 0.1.0

Here is the task in gulpjs file

// Optimize Images
gulp.task('images', function () {
  return gulp.src('app/images/**/*')
    .pipe($.cache($.imagemin({
      progressive: true,
      interlaced: true
    })))
    .pipe(gulp.dest('dist/images'))
    .pipe($.size({title: 'images'}));
});

and when I try using gulp images and this is the error I see.

[01:00:05] Starting 'images'...
Possibly unhandled Error: No path specified! Can not get relative.
    at File.Object.defineProperty.get (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/node_modules/vinyl/index.js:150:27)
    at DestroyableTransform.saveFile [as _transform] (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/lib/dest/index.js:40:48)
    at DestroyableTransform.Transform._read (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at DestroyableTransform.Transform._write (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
    at doWrite (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)
    at writeOrBuffer (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:227:5)
    at DestroyableTransform.Writable.write (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:194:11)
    at Stream.ondata (stream.js:51:26)
    at Stream.EventEmitter.emit (events.js:95:17)
    at queueData (/Users/rayabhagis/projects/nerdtogeeks.com/node_modules/gulp-cache/node_modules/map-stream/index.js:43:21)

But I can evade this one by just removing

.pipe($.cache($.imagemin({
      progressive: true,
      interlaced: true
    })))

part from the gulp task

How to clear cache?

So I randomly had an issue when using the cache on images, where the gulp-images plugin would spit out the same image for all images, and after some debugging, this seemed to be related to gulp-cache. When I take caching out it works fine, but if I put the cache in, I get all the same image.

Anyways, I am guessing that clearing out this cache would solve it. Any suggestions on how to remove the cached items so image minification can go back to normal?

Flushing cache?

I'm using gulp-cache for caching images that I minify using gulp-imagemin.

Works fine and lovely in my original directory. But when I copy the directory to start work on a new project, I get this error message:

Possibly unhandled Error: No path specified! Can not get relative.

My guess is that there's an absolute path to cacheDirName but I can't find reference to it anywhere. Is there a way to flush the cache? Or specify relative cacheDirName?

Full error message:

[gulp] Running 'minimage'...
Possibly unhandled Error: No path specified! Can not get relative.
    at File.Object.defineProperty.get (/Users/growdigital/Sites/brochureware/htdocs/wp-content/themes/projectx/node_modules/gulp/node_modules/vinyl-fs/node_modules/vinyl/index.js:120:27)
    at Transform.stream._transform (/Users/growdigital/Sites/brochureware/htdocs/wp-content/themes/projectx/node_modules/gulp-rename/index.js:20:34)
    at Transform._read (_stream_transform.js:179:10)
    at Transform._write (_stream_transform.js:167:12)
    at doWrite (_stream_writable.js:226:10)
    at writeOrBuffer (_stream_writable.js:216:5)
    at Transform.Writable.write (_stream_writable.js:183:11)
    at Stream.ondata (stream.js:51:26)
    at Stream.EventEmitter.emit (events.js:95:17)
    at queueData (/Users/growdigital/Sites/brochureware/htdocs/wp-content/themes/projectx/node_modules/gulp-cache/node_modules/map-stream/index.js:43:21)

defaultKey function fails when directories are passed

I'm encountering an exception when trying to start my gulp project with gulp-cache-0.2.7. The gulpfile.js works with gulp-cache-0.2.4.

You can see the exception I'm encountering in this gist.

Please let me know if you need more information. I'm happy to give you any info you need, it's just that I'm not particularly familiar with what's useful for debugging gulp scripts.

How to specify cache path?

Docs say:

fileCache

[Optional] Where to store the cache objects

Defaults to new Cache({ cacheDirName: 'gulp-cache' })

Where am I supposed to get the Cache constructor from? what does that look like?

Default settings merge files with same content

The default settings look buggy.

Imagine you have 1.js and 2.js with same content.
The plugin will make the same cache for them.

And will restore them in the single file also (twice 1.js in the stream instead of both). Just checked.

Usage with jade causes some files to not be compiled

I'm using gulp-cache with gulp-jade and for some reason it causes the occasional file to not be created within jade.

The app I'm working on is getting quite large and this only started to happen recently. Even resetting the cache doesn't make the file get created!!!
I've tried not using a key function but that was no help, however If I remove $.cache() it works properly

My gulp task is as such

gulp.task('jade', function() {
  return gulp.src('app/**/*.jade')
    .on('error', shallowError)
    .pipe($.cache($.jade({pretty: true, doctype:'html'}), {key: createKey}))
    .pipe(gulp.dest('.tmp'))
})
var createKey = function(file) {
  return [file.contents.toString('utf8'), file.stat.mtime, file.stat.size].join('')
}

Possibly unhandled Error: path should be string

I'm getting the following error:

Possibly unhandled Error: path should be string
    at File.Object.defineProperty.set (/Users/benjaminrh/appem/node_modules/gulp/node_modules/vinyl-fs/node_modules/vinyl/index.js:166:41)
    at Function.assign (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/lodash-node/modern/objects/assign.js:63:21)
    at /Users/benjaminrh/appem/node_modules/gulp-cache/lib/TaskProxy.js:20:19
    at tryCatch1 (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/util.js:64:19)
    at Promise$_callHandler [as _callHandler] (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:708:13)
    at Promise$_settlePromiseFromHandler [as _settlePromiseFromHandler] (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:724:18)
    at Promise$_settlePromiseAt [as _settlePromiseAt] (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:896:14)
    at Promise$_fulfillPromises [as _fulfillPromises] (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/promise.js:1041:14)
    at Async$_consumeFunctionBuffer [as _consumeFunctionBuffer] (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/async.js:64:12)
    at Async$consumeFunctionBuffer (/Users/benjaminrh/appem/node_modules/gulp-cache/node_modules/bluebird/js/main/async.js:37:14)

Cache invalidation

Hi I have been trying to find a way to invalidate a cache key on file deletion
the issue is,

Let us assume there are three css files I am minifying and then concatenating
As of now, I am doing it this way

Gulp task - styleTask:

gulp.src(css/**/*.css)
.pipe(cache(cleanCss()))
.pipe(concat('app.css')
.pipe(gulp.dest(dist))

Then I watch the css folder with
gulp.watch('css/**/*',['styleTask'])
Everything works perfect.

However, on deletion of a file, I need to delete the corresponding cache entries too.

Is that a possibility via configuration?

else,

I can manually delete them, If I have the cache keys handy.

The plugin creates multiple cache entries per file whenever it is modified,

would it be possible to return the cache key by exposing API methods?

I can then just clear those entries instead of clearing the whole cache

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.