claviz / bellboy Goto Github PK
View Code? Open in Web Editor NEWHighly performant JavaScript data stream ETL engine.
License: MIT License
Highly performant JavaScript data stream ETL engine.
License: MIT License
It would be fantastic being able to pass null
/ undefined
to yield in recordGenerator
to be able to skip a row
I'm interested in contributing a PR to this project, but I see many test failures, even when running in Docker. Are the tests running for others?
Below code is not storing data in database from excel file
const processor = new bellboy.ExcelProcessor({
path: directoryPath,
hasHeader: true,
});
const destination = new bellboy.PostgresDestination({
connection: {
user: "postgres",
password: "123456",
server: "localhost",
database: "ITA_projects",
},
table: "ita",
upsertConstraints:"inserts",
recordGenerator: async function* (record) {
//console.log(record);
yield {
...record.raw.obj,
status: "done",
};
},
});
//console.log("show destination==>", destination);
const job = new bellboy.Job(processor, [destination])
job.on("endProcessingStream", async (file) => {
const filePath = path.join(directoryPath, file);
const newFilePath = path.join(
__dirname,
"..",
"..",
"..",
"destination",
file
);
await fs.renameSync(filePath, newFilePath);
});
await job.run();
@jansivans It seems that the column names are being forced as case-sensitive. In my case, I am using MSSQLProcessor and PostgresDestination and was getting "XYZ" table or column does not exist on the pg side. When I used lowercase table name and edited PostgresDestination.prototype.loadBatch to convert "key" toLowerCase() that seemed to fix my issue, but I think this might be a broader bug so wanted to flag here. Could you please take pass at that?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.