criccomini / twister Goto Github PK

Issue Description:
The open source project Twister currently lacks a convenient way to map Kafka Connect structs to a key-value data structure. This issue proposes adding a Map facade to Twister, similar to AvroWrapper and ProtoWrapper, to simplify Kafka Connect struct mapping.

Issue Description:

Proposal:
Introduce a ParquetReader in Twister to enable the mapping of Parquet bytes to Java Maps, similar to AvroReader and ProtoReader. This reader will provide a convenient way to read Parquet data and extract it as key-value pairs stored in Maps.

Expected Behavior:
The ParquetReader should allow developers to read Parquet bytes and map the data to Java Maps. Each Parquet row will be represented as a Map object, where column names are used as keys and corresponding values are populated.

Benefits:

1. Simplified Parquet data processing.
2. Seamless integration with existing Java data structures.
3. Enhanced performance and efficiency.

Implementation Considerations:
Utilize existing Parquet libraries, such as Apache Parquet, to handle low-level parsing and decoding. Support various data types defined in the Parquet schema and handle nullable fields appropriately.

Contributor Resources:
Refer to the Twister project's contribution guidelines for instructions on setting up the development environment and submitting a pull request.

Environment:

• Twister version: [Specify the version or commit hash]
• Operating System: [Specify the OS]
• Java version: [Specify the Java version]
• Additional environment details: [Provide any relevant details about the environment]

Issue Description:
The open source project Twister currently lacks a convenient way to map JDBC result sets to a key-value data structure. This issue proposes adding a Map facade to Twister, similar to AvroWrapper and ProtoWrapper, to simplify JDBC row mapping.

Proposal:
Introduce a Map facade in Twister to enable developers to easily map JDBC result sets to a Map representation. This will provide a consistent interface for handling different data formats and improve developer productivity.

Expected Behavior:
The Map facade should allow developers to map JDBC result sets to a list of Maps, where each row's columns are represented as key-value pairs.

Benefits:

1. Simplified JDBC result set mapping.
2. Consistency across data formats.
3. Improved developer productivity.

Implementation Considerations:
Utilize existing JDBC APIs, such as ResultSetMetaData, to retrieve column names and values for each row. Handle various data types supported by JDBC and ensure compatibility with popular JDBC drivers.