Comments (10)
Regarding events it would be good to see what is used by other db's with kafka or eg debezium. I have a call with the latter on Monday
from neo4j-streams.
You mean for async execution? I actually want to move to Kotlin co-routines.
from neo4j-streams.
I made this, If you think that is a good starting point, I can reimplement it with co-routines. Sorry but I didn't know them until now!
from neo4j-streams.
I like it but we might want to make it configurable.
Also we need to be careful esp. with data that has been deleted (changed) if you access it outside of the current tx you will either not see it anymore (if the tx has been commited) or not see it yet (if data has just been created).
So the read from the graph will have to happen in the current tx only the publication could be async.
Also after discussing with Omar, I was thinking about changing the published record from being a simple data record to something more like an event, with { meta, audit, operation, previous, current }
at least as an option.
from neo4j-streams.
Great! Just a question because I want to better understand how the trigger management works. Consider this scenario:
- Some data is created;
- The Kafka module broadcast the data to consumers in the beforeCommit method;
- Another Trigger for some reason, maybe one that provide a graph data model validation, rolled back the data; (or even the database crashes)
Is this a possibile scenario?
I think that the switch to the event is better!
from neo4j-streams.
ping :)
from neo4j-streams.
Hey sorry.
We can create the events in beforeCommit but then create a marker that we return and check it it aftercommit/afterrollback and in case of rollback put a negating/cancellation event on the stream
from neo4j-streams.
Hi @jexp ,
I checked the co-routines and seems they are an experimental feature.
Maybe at this point it is better to use the runnables?
from neo4j-streams.
Co-Routines have been along for a long time and should be quite stable.
So I would try to use those.
from neo4j-streams.
Perfect!
from neo4j-streams.
Related Issues (20)
- Cud format is in beta ? HOT 2
- neo4j streams cdc for neo4j 5 HOT 2
- Upgrade kafka-avro-serializer dependency
- Add Kafka-connect sink connector 'neo4j.cypher.dropUnwind' property
- [Question] Connect to Neo4J via SSO
- Struct can not be created from an array with optional elements HOT 2
- Structuring nested object causes org.apache.kafka.connect.errors.DataException
- Error while starting Kafka-neo4j connect using distributed properties | Error [ java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics] HOT 3
- LAST_COMMITTED from configuration should use interval with fallback value
- ( Impact on BC/DR - Data loss ) - High priority, Nodes with PointValue property cannot be successfully sink to Neo4j HOT 3
- ( Impact on BC/DR - Data loss ) - High priority, Nodes with PointValue property cannot be successfully sink to Neo4j With Neo4j Streams Plugin HOT 2
- Issue with `$value` parameter in Neo4j Sink Connector with Confluent Cloud Kafka HOT 2
- How to get an extra dependency to Open Source version of Apache Kafka.
- Missing nodes in Neo4j with "Neo4j Kafka Connector Sink" HOT 1
- CUD Strategy - Ids in Relations
- Nested timestamps cause a DataException to be thrown by kafka sink connector HOT 2
- ( Impact on BC/DR - Data loss ) - High priority, Nodes with field value is 2D Point will throw exception and cannot be successfully sink to Neo4j HOT 1
- Kafka Connect Source connector problems with AVRO serialization HOT 1
- org.apache.kafka.connect.errors.SchemaBuilderException Error with Neo4jSource connector 5.0.3+ but not with 5.0.2
- Objects inside an array coming from a source connector have their types set to string in the generated Schema HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from neo4j-streams.