Comments (3)
Hey @gadaldo,
In this case, the error json-avro-convert is throwing is correct because nested2 is not defined in your sample json and the schema provides no default for it. Avro should only accept this datum in either of those cases and I've verified it does in both with this code:
(using a default value)
def 'should convert nested nullable records'() {
given:
def schema = '''
{
"type": "record",
"name": "Root",
"fields": [
{
"name": "field1",
"type": [
"long",
"null"
]
},
{
"name": "nestedRecord",
"type": [
{
"type": "record",
"namespace": "root",
"name": "NestedRecord",
"fields": [
{
"name": "nested1",
"type": [
"long",
"null"
]
},
{
"name": "nested2",
"type": [
"long",
"null"
], "default": 42
}
]
},
"null"
]
}
]
}
'''
def json = '''
{
"field1" : 10999859003,
"nestedRecord":
{
"nested1" : 123321321, "nested2":42
}
}
'''
when:
def result = converter.convertToJson(converter.convertToAvro(json.bytes, schema), schema)
then:
toMap(result) == toMap(json)
}```
**(using a provided value)**
```groovy
def 'should convert nested nullable records2'() {
given:
def schema = '''
{
"type": "record",
"name": "Root",
"fields": [
{
"name": "field1",
"type": [
"long",
"null"
]
},
{
"name": "nestedRecord",
"type": [
{
"type": "record",
"namespace": "root",
"name": "NestedRecord",
"fields": [
{
"name": "nested1",
"type": [
"long",
"null"
]
},
{
"name": "nested2",
"type": [
"long",
"null"
]
}
]
},
"null"
]
}
]
}
'''
def json = '''
{
"field1" : 10999859003,
"nestedRecord":
{
"nested1" : 123321321, "nested2":43
}
}
'''
when:
def result = converter.convertToJson(converter.convertToAvro(json.bytes, schema), schema)
then:
toMap(result) == toMap(json)
}```
from json-avro-converter.
At level 0 of the tree it does, the problem is just when the algorithm goes in recursion.
Anyway, I created my own version because I needed it, that JSON comes from TableRow object when reading from Bigquery (with BigQueryIO) and I have to transform in AVRO. It'a a feature that Google does behind the scene but they don't want to expose the API seen they do a further middle transformation to proto as documented here.
So based on this algorithm I created mine but I don't know if you can close the issue.
Thank you anyway
from json-avro-converter.
is this issue got fixed? Please let me know how to resolve it
from json-avro-converter.
Related Issues (20)
- Dockerfile
- Failed to convert JSON to Avro HOT 2
- Support schema type BYTES HOT 1
- Feature request: support JSONL
- importing schemas with custom avro types
- Converting classes that have avro fields to json HOT 5
- IllegalAccessError on latest avro HOT 2
- not able to cast string to int HOT 2
- Update to Avro v1.9.1 to address downstream library vulnerability
- Convert LogicalType BigDecimal to Json HOT 1
- Next releases
- Validator is not published to maven central
- Problem decoding union of map: Failed to convert JSON to Avro HOT 2
- issue when schema contains some logical-type
- Ability to serialize to confluent avro format
- 0.2.11 version not allowing extra attributes in request json HOT 2
- Problem decoding union of INT or STRING : Failed to validate JSON to Avro
- Can not deserialize instance of java.util.LinkedHashMap out of START_ARRAY with JsonAvroConverter
- getting error tech.allegro.schema.json2avro.converter.AvroConversionException: Failed to convert JSON to Avro Path in schema: --> payload__id HOT 1
- Replacement Project
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from json-avro-converter.