Comments (7)
@vicchi unfortunately, I don't use nor know anything about how Grunt works. I am wondering if Grunt is somehow hijacking the STDIO streams so STDERR doesn't output to the console. What is this.files
object look like in this context?
from ogr2ogr.
Grunt should be just plain Node.js. The fact that my standalone code works fine, makes me suspect that this isn't a Grunt problem per se, but a synchronous vs. asynchronous problem.
I could test this via Grunt's async()
support, but to do that, I'd need either a way of getting a final, task completed, callback back from ogr2ogr
or a way of working out how to deal with the data that ogr2ogr.exec
passes to its callback so I can get my final destination files written (and zipped if appropriate).
What I think is happening is that Grunt is assuming that ogr2ogr
runs synchronously and isn't waiting for the (internal) callbacks that ogr2ogr
uses to process the input and output files, so effectively Grunt finishes up whilst ogr2ogr
is still doing "stuff". If that makes sense.
Any suggestions for how to synchronise with ogr2ogr
actually completing?
Also, the contents of this.files
are
[
{
"src": [],
"dest": "data/dest/geojson/ne_110m_admin_0_sovereignty.geojson",
"orig": {
"src": [
"data/src/ne_110m_admin_0_sovereignty.shp"
],
"dest": "data/dest/geojson/ne_110m_admin_0_sovereignty.geojson"
}
}
]
... but I don't think this is a file specification problem.
from ogr2ogr.
I could test this via Grunt's async() support, but to do that, I'd need either a way of getting a final, task completed, callback back from ogr2ogr or a way of working out how to deal with the data that ogr2ogr.exec passes to its callback so I can get my final destination files written (and zipped if appropriate).
Streams returned from calling .stream()
from ogr2ogr will emit a close
event when all the processing is finished that you can listen for.
from ogr2ogr.
I could test this via Grunt's async() support, but to do that, I'd need either a way of getting a final, task completed, callback back from ogr2ogr or a way of working out how to deal with the data that ogr2ogr.exec passes to its callback so I can get my final destination files written (and zipped if appropriate).
Streams returned from calling .stream() from ogr2ogr will emit a close event when all the processing is finished that you can listen for.
Thanks. I'll give that a try. I'm offline travelling for a few weeks now but I'll give it a go when I'm back and online.
from ogr2ogr.
Progress. Had a spare few minutes before getting on a plane. Forcing Grunt to wait for the completion of ogr2ogr2
via the stream's close
event works perfectly. If you're interested, the key is using this.async()
and then calling the return value in the close
event handler. See below.
'use strict';
module.exports = function(grunt) {
var ogr2ogr = require('ogr2ogr');
var fs = require('fs');
var path = require('path');
// Please see the Grunt documentation for more information regarding task
// creation: http://gruntjs.com/creating-tasks
grunt.registerMultiTask('ogr2ogr', 'ogr2ogr wrapper task for Grunt', function() {
grunt.log.debug(JSON.stringify(this.files, null, 2));
// Merge task-specific and/or target-specific options with these defaults.
var options = this.options({
format: 'GeoJSON',
// skipFailures: true,
// projection: 'EPSG:4326',
// options: null
// timeout: 15000
});
var self = this;
// Iterate over all specified file groups.
this.files.forEach(function(f) {
var src = f.src.filter(function(filepath) {
if (!grunt.file.exists(filepath)) {
grunt.log.warn('Source file "' + filepath + '" not found.');
return false;
}
else {
return true;
}
}).map(function(filepath, i) {
grunt.file.mkdir(path.dirname(f.dest));
var done = self.async();
var stream = ogr2ogr(filepath).format(options.format).stream();
stream.on('close', function() {
done();
});
stream.pipe(fs.createWriteStream(f.dest));
});
});
});
};
I've loaded the output GeoJSON and Shapefile up in QGIS and all looks good. One final and unrelated question.
$ file ne_110m_admin_0_sovereignty.zip
ne_110m_admin_0_sovereignty.zip: Zip archive data, at least v2.0 to extract
$ unzip ne_110m_admin_0_sovereignty.zip
Archive: ne_110m_admin_0_sovereignty.zip
inflating: OGRGeoJSON.dbf
inflating: OGRGeoJSON.prj
inflating: OGRGeoJSON.shp
inflating: OGRGeoJSON.shx
Is there any way to specify the name of the (compressed) shapefile (as opposed to the resultant ZIP archive) to be something other than OGRGeoJSON
?
from ogr2ogr.
@vicchi for your last question you can change the name of the output file by using the -nln
(ie. new layer name) flag
for instance:
var shapefile = ogr2ogr('/path/to/spatial/file.geojson')
.format('ESRI Shapefile')
.options(['-nln', 'output_name'])
.skipfailures()
.stream()
shapefile.pipe(fs.createWriteStream('/shapefile.zip'))
from ogr2ogr.
Closing due to age, reopen if issue persists.
from ogr2ogr.
Related Issues (20)
- stream() from geojson HOT 7
- shz file saved as a directory on linux, breaking stream processing HOT 4
- converting
- converting to geopakacge not working propaly HOT 1
- . HOT 2
- Currently can't provide custom args when specifying a custom command HOT 6
- How to write -update statement ? HOT 2
- how to use on angular project? HOT 2
- OGR_XLSX_HEADERS HOT 1
- precision? HOT 5
- geoJSON features order HOT 2
- Uncaught errors when opening a bad CSV file HOT 4
- Load CSV to Postgres very slow HOT 5
- change http method HOT 3
- ogr2ogr is not a function HOT 1
- How to export postegresql table to .shp using org2org? HOT 3
- More examples of how to write data HOT 6
- Unexpected behavior with double quotes in PG connection string HOT 2
- Unknown option name '-spat' HOT 2
- Result.cmd is missing a space between command and args
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ogr2ogr.