Comments (6)
@itsgrimetime I'm politely asking for an update. Has this problem been addressed?
from emr-dynamodb-connector.
I also encountered similar trouble. I got NULL
on parsing a List property of Binary data as array<binary>
.
According to AWS EMR release guide, Hive array can be mapped to "number set (NS), string set (SS), or binary set (BS)". Does it mean I cannot use List properties to import to Hive?
from emr-dynamodb-connector.
@sebastien-de-saint-florent @astj Thanks for bringing this to our attention, and we apologize for the latency in response. Just so you know, we are tracking this issue internally and hope to look at it in the near future. I'll be sure to update you here when we have more information.
from emr-dynamodb-connector.
I was facing the same problem while trying to use the connector with spark. I noticed that same information can be obtained by calling the getL api instead of the getSS api on AttributeValue
from emr-dynamodb-connector.
@sebastien-de-saint-florent @naataaniitsosie @astj @0xCafeDude sorry for taking so long to get to this. I've finally gotten some time to investigate and found that the connector assumes a List type is actually a StringSet, and that assumption gets propagated and causes the DynamoDBObjectInspector to receive the SS attribute (which is null) instead of the L attribute. I'm hoping to have a fix for this posted soon. Thanks again for your patience.
from emr-dynamodb-connector.
Oh - actually, I just realized #71 takes care of this, and adds additional types. I'll review that and see if we can't get it closed out.
from emr-dynamodb-connector.
Related Issues (20)
- Conditional write support
- How does EMR connect to other Hadoop hdfs
- DynamodbImport - Support ignore empty lines
- Upgrade aws-java-sdk.version to latest aws java sdk version
- getAttributes of DynamoDBRecordReaderContext seems not in use
- Limited operator support in DynamoDBFilterPushdown for table hash key HOT 9
- Read throughput of table is not getting set correctly in 4.16.0 version
- DynamnoDBExport support for query criteria HOT 1
- Support for arbitrary precision number attributes
- Is this connector only for EMR? HOT 1
- Regarding pushdown filter predicate HOT 3
- How to configure DynamoDBExport READ_THROUGHPUT For On-Demand Billing Mode
- Deploy to maven central HOT 1
- Configured throughput and percentage ignored HOT 1
- mvn install / mvn package fails HOT 3
- Use of temporary credentials via env variables
- Hadoop scan with start end row example? (Row key filter)
- Use actual number of input splits when calculating target throughput per task
- Does this connector work with Spark 3.3 and EMR 6.11 HOT 1
- 5.x releases may not have been published as of March 2024 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from emr-dynamodb-connector.