Comments (4)
Hello @hugovandernotte I added this to the team backlog but as you said at the moment it won't be prioritized. I don't think it has any workaround for now :(
from airbyte.
Hello @hugovandernotte I added this to the team backlog but as you said at the moment it won't be prioritized. I don't think it has any workaround for now :(
Hey @marcosmarxm,
Thanks for your answer!
Could you at least tell me what happened to the link that was posted by your team in the discuss.airbyte.io? Here is the link https://github.com/airbytehq/airbyte/issues/14624 but it seems it was moved somewhere else
I think that a quick fix on you side would be to write per batch of 1k rows (there are 5 columns so we should be below what is acceptable in term of bind variables number)
from airbyte.
@hugovandernotte you can try doing the change locally and build the connector to your use case. You need to use the airbyte-ci
for that. As I said, this won't get fix for now unfortunately.
from airbyte.
I think I might have found a workaround @marcosmarxm, but I would need your advice on how to proceed :)
I am running incremental run and at every run I only have a fund hundreds of rows to insert (which should work, from what i tested). The issue is coming from the initial load. I think that I can do this initial load myself and then ask airbyte to start the incremental append from that.
However, I don't know, how I could "fake" airbyte for it to consider the initial load was done by itself and that it only has the new data to insert, is there a doc somewhere in which I could found how to do that?
from airbyte.
Related Issues (20)
- [source-prestashop] API Error 500 on cart_rules Endpoint Due to Method Signature Mismatch in CartRule.php
- Error: couldn't find key DATABASE_USER in Secret airbyte-airbyte-secrets HOT 3
- [source-file] Error reading zipped CSV file with latin-1 encoding HOT 1
- Depoly Failed HOT 1
- airbyte-temporal failed to connect to RDS when rds.force_ssl enabled in RDS HOT 2
- Airbyte CDK: add logic for checkpointing state in the end of sync to Checkpoint Reader if records are not ordered by cursor field
- [source-posthog] attributes `properties` and `uuid` not extracted from API HOT 2
- Scheduler with Cron don't trigger when needed HOT 1
- [destination-bigquery] You must upgrade your platform version to use this connector version (latest version already) HOT 3
- Helm Chart AppVersion Update HOT 1
- Deploy Airbyte on Kubernetes using Helm Failed HOT 1
- [source-mongo] data loading only first time HOT 1
- Live tests: wire through a connector certification & connector type check to control which tests to run
- [EPIC] Live tests Automation
- Bump langchain and langchain_core in airbyte-cdk
- [GCS Destination] State code: SignatureDoesNotMatch; Message: Access denied
- [postgres-source]: Performance regression
- [cdk, connectors] Error translation framework prototype
- [EPIC] Error translation framework
- [Blocker] Airbyte sync jobs are failing with docker repo throttle error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airbyte.