Coder Social home page Coder Social logo

gulp-progeny's Introduction

gulp-progeny Build Status

A dependency-resolving plugin for Gulp. It grabs all files related to one modification to building system. Make Gulp Fast Again

##Introduction Gulp provides incremental building. However it is agnostic about the dependencies among files. Say, if child.jade depends on parent.jade, then whenever parent changes child should be recompiled as well. Existing gulp plugins do not support this. Or one could fall back on edit-wait-10s-view loop.

gulp-progeny aims to solve this. If parent.jade is edited and passed to gulp-progeny, all files that recursively depends on that file will be collected by progeny and are passed to succesive building stream.

##What does gulp-progeny do This plugin brings the agility of brunch to gulp world. It provides generic dependency detection to various file types. progeny parses files by grepping specific import statements, and builds dependency trees for building tasks. So it just use Regular Expression to extract dependency information. This simple solution is fast and dirty, but also working.

##Usage gulp-progeny out of box supports jade, less, sass and stylus. To exploit the power of gulp-progeny, use gulp-cached in tandem.

var cache = require('gulp-cached');
var progeny = require('gulp-progeny');
var stylus = require('gulp-stylus');

gulp.task('style', function(){
  return gulp.src('*.styl')
    .pipe(cache('style'))
    .pipe(progeny())
    .pipe(stylus())
});

gulp.task('watch', function(){
  gulp.watch('*.styl', ['style']);
});

cached will pass all files to progeny in the first run, which enables dependency tree building, just pass changed files later for incremental building.

Advanced configuration

As gulp-progeny includes progeny in its core, please see their configuration docs for the options that can be used with this plugin.

Limitation

gulp-progeny, by the virtue of its design, has following limitations.

  1. Filenames must be static in source code. Otherwise regexp fails to work.

  2. Sass/Scss filenames should not contain apostrophes. This limitation is due toSass's multiple import feature. Matching filenames with '" is far beyond the reach of regular expressions with only one match, which is the format used in this plug in.

Limitations above are almost ineluctable for this simple plugin. To keep API and extension simple and generic, some features are doomed to be dropped. This plugin does not fill the chasm between a single one Regexp and full-bloom parsers.

gulp-progeny's People

Contributors

diagramatics avatar herringtondarkholme avatar joeycozza avatar ln-e avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

gulp-progeny's Issues

Error when deleting cache[child] in pushFileRecursive

Hi!

I get this error at a moment I cannot define, when using progeny in my Less watchers:

[...]/node_modules/gulp-progeny/dest/index.js:57
      results.push(delete cache[child]);
                                ^
RangeError: Maximum call stack size exceeded

I looked a bit to the code, and it seems the variable child is not defined anywhere in the code. Maybe you meant childPath ?

Sync reading files

I've just started looking code.
I spotted that a lot of synchronous code for reading/checking files fs.readFileSync, fs.existsSync.
Is it necessary? Can you give short explanation?

Usage with gulp-pug and includes

I've spent countless hours trying to set up some sort of an incremental build system with gulp-changed, gulp-jade-inheritance etc., but always scrapped it in the end as it. just. did. not. work. This plugin seems very promising – I've managed to plug it into my workflow and make it work, thanks!

Can you please provide an example on how to use it to compile .pug containing includes/mixins? The classic scenario of compiling a page if you change the linked file.

index.pug
  - mixins/
    - _helper.pug

index.pug containing include mixins/_helper.pug.

I want index.pug to compile to index.html regardless if I edit it directly or change something in _helper.pug. Naturally I don't want _helper.pug to result in _helper.html.

The regex for stylus doesn't work

For example, here's a stylus test case:

@import a
@import b

a
  color red

Regex like /^\s*(?:@import|@require)\s+['"]?([^'"]+)['"]?/ will simply match all the content.

Add license

My company needs license information before an external package can be used. Can you please add license information to the package

internal depCache gets overridden when using merge-stream

I have a project that uses Bower to bring in 3rd party dependencies. I created a gulp task to grab all the stylus files from the Bower components and my apps assets and run them through glup-progeny and stylus. Because the folder structure is different, I used merge-stream to run the same stream on two different sources and dests.

However, when gulp-progeny is called from two different streams, the internal depCache only lists the assets from the second stream and the first stream is never listed. So any assets in the first stream do not have their dependencies listed and so editing a dependency does not update the parent.

Here is an example folder structure

./
├ assets
│  ├ foo.styl
│  ├ bar.styl
│  └ concat.styl 
│
└ bower_components
   └ project
     ├ faz.styl
     ├ baz.styl
     └ concatz.styl 

Where assets/concat.styl imports foo.styl and bar.styl, and bower_components/project/concatz.styl imports faz.styl and baz.styl.

Here is an example gulpscript

var gulp = require('gulp');
var stylus = require('gulp-stylus');
var cache = require('gulp-cached');
var progeny = require('gulp-progeny');
var merge = require('merge-stream');

function buildStylus(files, config, dest) {
  return gulp.src(files, config)
    .pipe(cache('stylus'))
    .pipe(progeny())
    .pipe(stylus())
    .pipe(gulp.dest(dest));
}

gulp.task('stylus', function() {
  var assetsStylus = buildStylus(
    'assets/**/*.styl'
    {base: 'assets', nodir: true},
    'dist'
  );

  var componentsStylus = buildStylus(
    'bower_components/**/*.styl'
    {base: './', nodir: true},
    'dist'
  );

  return merge(assetsStylus, componentsStylus);
});

gulp.task('watch', function(){
  gulp.watch('*.styl', ['stylus']);
});

Editing foo.styl or bar.styl never cause assets/concat.styl to update, but editing faz.styl or baz.styl do cause bower_components/project/concatz.styl to update.

dependency graph not updated after first file read

It appears that the dependency graph doesn't change to respect added/removed/changed import statements after the file has been read the first time. Given:

parent.test:

(That is, an empty file.)

child.test:

import parent.test

gulpfile.js:

var gulp = require('gulp');
var progeny = require('gulp-progeny');
var cache = require('gulp-cached');
var debug = require('gulp-debug');

gulp.task('test', function() {
  return gulp.src('*.test')
    .pipe(cache('test'))
    .pipe(progeny({
      regexp: new RegExp('import (.*)'),
      debug: true
    }))
    .pipe(debug());
});

gulp.task('watch', [ 'test' ], function() {
  return gulp.watch('*', [ 'test' ]);
});

(Caches files and just prints out the names.)

I get the following behavior (annotations with my actions inline):

$ gulp watch

### Run for the first time -- dependency tree looks right.

Using gulpfile ~/Documents/Programming/temp/gulpfile.js
Starting 'test'...
gulp-debug: child.test
DEP child.test
    |--parent.test
gulp-debug: parent.test
DEP parent.test
    |  NO-DEP
gulp-debug: 2 items
Finished 'test' after 37 ms
Starting 'watch'...
Finished 'watch' after 13 ms

### Trivially modify child.test (e.g. add a newline). Behaves as expected.

Starting 'test'...
gulp-debug: child.test
DEP child.test
    |--parent.test
gulp-debug: 1 items
Finished 'test' after 100 ms

### Trivially modify parent.test. Behaves as expected.

Starting 'test'...
gulp-debug: parent.test
DEP parent.test
    |  NO-DEP
gulp-debug: child.test
gulp-debug: 2 items
Finished 'test' after 5.88 ms

### Remove dependency that child.test has on parent.test. Appears to behave as expected, including updated dependency. 

Starting 'test'...
gulp-debug: child.test
DEP child.test
    |  NO-DEP
gulp-debug: 1 items
Finished 'test' after 4.87 ms

### Trivially modify parent.test again. This should not pick up child.test, but it does.

Starting 'test'...
gulp-debug: parent.test
DEP parent.test
    |  NO-DEP
gulp-debug: child.test
gulp-debug: 2 items
Finished 'test' after 8.32 ms

contribute back to progeny

I see you've overwritten a lot of the default settings per filetype here. If your changes improve upon the settings that are built in to the progeny module, why not open PRs to get them incorporated there?

Plus it would be great if someone would tackle the problem of multiple children defined in a single import statement. Should be very possible, just not something I've been able to spend time on so far.

Should use Vinyl objects instead of reading from the file system

I've just been trying this out and was surprised to find that this actually reads every file on its own instead of using the Vinyl objects from the stream. Is there a specific reason for that? After all, you can expect to get the contents from the stream and do not need to repeat the read process, do you?

Example for .less files please

Looks like what we're after. Do you have a sample gulpfile for using progeny to handle

  1. incremental building of .css from .less sources
  2. minimization of .js
  3. gzipping
  4. uglification

Exposing cache data

Hi, thank you for the great tool! I really enjoy using it :)

What do you think about exposing cache data like gulp-cached?
I'm tackling a large project and want the initial build to be faster. My idea is re-using the cache of the previous gulp process with (re-)storing the in-memory caches as a file. This approach works well for gulp-cached, I want to apply this for gulp-progeny too. 🙂

Update description

Currently, the project description is "Makes gulp blazing fast".
I feel that it's a missed opportunity to tie into Drumpf culture and be "Make Gulp Fast Again"?

I've made a small image to inspire this change.
america

Error on included Pug partials

Everything is ok on root files but an error occurred on included partials:

2017-05-02 15:16 gulp[2466] (FSEvents.framework) FSEventStreamFlushSync(): failed assertion '(SInt64)last_id > 0LL'
gulpfile

const $ = gulpLoadPlugins()
const bs = browsersync.create()
gulp.task('views', () => {
	bs.notify('Compiling HTML…', 999999)
    return gulp.src(['app/*.pug', 'app/views/**/*.pug'])
        .pipe($.if(global.isWatching, $.cached('pug')))
	.pipe($.progeny())
        .pipe($.pug({ pretty: true }))
        .pipe(gulp.dest('.tmp'))
		.pipe(bs.reload({ stream: true }))
})

gulp.task('serve', () => {
	global.isWatching = true
	runSequence(['clean'], ['views', 'styles', 'scripts'], () => {
		
		gulp.watch(
			[
				'app/**/*.pug',
				'app/includes/**/*',
			],
			['views']
		)
		
	})
})

The same as with gulp-pug-inheritance pure180/gulp-pug-inheritance#5

Maybe something wrong with my gilpfile?

Reverse Progeny

Any thoughts on a reverse method?
When a child is updated, the parent(s) are recompiled?

Deep dependency detection deficiency

Quoted from @seansfkelley

I just accidentally reproed this while checking out the fix for #5. This seems to happen when the grandparent is added after the task starts; so starting with the same repro described in #5:

### Start repro same was #5 starts.

Starting 'test'...
gulp-debug: child.test
DEP child.test
    |--parent.test
gulp-debug: parent.test
DEP parent.test
    |  NO-DEP
gulp-debug: 2 items
Finished 'test' after 30 ms
Starting 'watch'...
Finished 'watch' after 11 ms

### Add empty grandparent.test file.

Starting 'test'...
gulp-debug: grandparent.test
DEP grandparent.test
    |  NO-DEP
gulp-debug: 1 items
Finished 'test' after 8.75 ms

### Add 'import grandparent.test' line in parent.test.

Starting 'test'...
gulp-debug: parent.test
DEP parent.test
    |--grandparent.test
gulp-debug: child.test
gulp-debug: 2 items
Finished 'test' after 86 ms

### Touch grandparent file (e.g. add a newline) -- does not recompile child.test.

Starting 'test'...
gulp-debug: grandparent.test
DEP grandparent.test
    |  NO-DEP
gulp-debug: parent.test
gulp-debug: 2 items
Finished 'test' after 4.98 ms

Where grandparent.test is just an empty file. If you start the task while all three levels of files exist, it works as expected:

### Start task with all three files in place.

Starting 'test'...
gulp-debug: child.test
DEP child.test
    |--parent.test
    |--grandparent.test
gulp-debug: grandparent.test
DEP grandparent.test
    |  NO-DEP
gulp-debug: parent.test
DEP parent.test
    |--grandparent.test
gulp-debug: 3 items
Finished 'test' after 43 ms
Starting 'watch'...
Finished 'watch' after 12 ms

### Modify grandparent.test -- everything is recompiled, as expected.

Starting 'test'...
gulp-debug: grandparent.test
DEP grandparent.test
    |  NO-DEP
gulp-debug: child.test
gulp-debug: parent.test
gulp-debug: 3 items
Finished 'test' after 9.49 ms

gulp-stylus -> gulp-progeny fails with progeny attempting to parse non-existant .css files

After reinstalling the entire node_module dependency tree in an app (and most likely installing new versions of everything) gulp-progeny started throwing ENOENT (file not found) errors.

The issue seems to be that gulp-progeny attempts to parse the path that is emitted by gulp-stylus (with .styl replaced by .css) instead of the original file path. The following code throws the error for me:

gulp.src(['styles/**/*.styl'], {cwd: 'styles'})
  .pipe(gulpProgeny())
  .pipe(gulp.dest(destDir));

Temporarily switching the file extensions back to .styl using gulp-rename seems to work however:

gulp.src(['styles/**/*.styl'], {cwd: 'styles'})
  .pipe(gulpRename({ extname: '.styl' }))
  .pipe(gulpProgeny())
  .pipe(gulpRename({ extname: '.css' }))
  .pipe(gulp.dest(destDir));

Gulp versions:

Support for progeny's altPaths

Based on es128/progeny, I don't think we have support for altPaths yet unfortunately.

Which brings me to another question: is it a possibility if we just use es128/progeny for the parser, use their API then fit in Gulp's structure instead?

Issue with many levels of dependencies

I'm currently having an issue where updating a grand-grand-grand parent fil does not trigger the correct changes. My structure right now looks like this, where -> is an import statement:

richtext-custom-class.less -> "./Mixins/_mixins.less" -> "../../../../../someotherfile.less" -> "../../../../../somethirdfile.less" -> "./Mixins/TheStyleImChanging.less". In this case, richtext-custom-class does not update properly, or at all, when changing TheStyleImChanging. The documentation doesn't make it clear whether or not this is a limitation.

Consider dependency parser as separate components

Leave it up to user to determine how dependencies are parsed,
maybe used like:

var stylDepParser = function(text){/*...*/}
gulp.src('*.styl')
  .pipe(progeny({
    depParser: stylDepParser
  }))

then the core should be more simple, and with no limitation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.