The onetl from mobiletelesystems

mongodb docs restructured 1

onetl/onetl/connection/db_connection/mongodb/options.py

Line 99 in 6849b46

    
                   `supported by connector <https://www.mongodb.com/docs/spark-connector/current/configuration/read>`_,

it seems that mongodb docs restructured and right now link https://www.mongodb.com/docs/spark-connector/current/configuration/read redirects to https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-read-config/

pytest: command not found

I followed the instructions provided in the README file to run the tests. When attempting to execute docker-compose run --rm onetl ./run_tests.sh, I receive the following error message: "/app/pytest_runner.sh: line 15: pytest: command not found"

I also noticed that running the command docker-compose run --rm onetl bash -c 'pip freeze | wc -l' outputs 0. This indicates that there are no pip packages installed within the container.

However, if I remove the volume specified in the docker-compose.yml file, the pip packages become visible within the container, but the onetl files are no longer accessible.

I would appreciate any assistance or guidance in resolving this issue. Thank you!

Прошу добавить поддержку Aerospike

mogodb docs restructured 4

onetl/onetl/connection/db_connection/mongodb/options.py

Line 133 in 6849b46

    
                   `supported by connector <https://www.mongodb.com/docs/spark-connector/current/configuration/read>`_,

same as with read/pipeline options: mongodb docs restructured and now old link (https://www.mongodb.com/docs/spark-connector/current/configuration/read) redirects to https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-write-config/

monogdb docs restructured 2

onetl/onetl/connection/db_connection/mongodb/options.py

Line 133 in 6849b46

    
                   `supported by connector <https://www.mongodb.com/docs/spark-connector/current/configuration/read>`_,

it seems that mongodb docs restructured and right now link https://www.mongodb.com/docs/spark-connector/current/configuration/read redirects to https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-read-config/

new link for the j connector docs

onetl/onetl/connection/db_connection/mysql/connection.py

Line 42 in 0d04466

    
               (`official MySQL JDBC driver <https://dev.mysql.com/downloads/connector/j/8.0.html>`_).

link need to be changed to https://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html

recreate_entire_table || replace_entire_table - 2?

onetl/onetl/connection/db_connection/hive/options.py

Line 236 in d2536b6

    
                   Used **only** while **creating new table**, or in case of ``if_exists=recreate_entire_table``

it seems that recreate_entire_table need to be replaced with replace_entire_table

Any plans for Apache Hudi Data Source

for a most latest microsoft doc version link needs to be changed

onetl/onetl/connection/db_connection/mssql/connection.py

Line 97 in 11dec83

    
                   <https://docs.microsoft.com/en-us/sql/connect/jdbc/setting-the-connection-properties?view=sql-server-ver16#properties>`_

to https://learn.microsoft.com/en-us/sql/connect/jdbc/setting-the-connection-properties#properties

recreate_entire_table || replace_entire_table?

onetl/onetl/connection/db_connection/hive/options.py

Line 209 in d2536b6

    
                   Used **only** while **creating new table**, or in case of ``if_exists=recreate_entire_table``

it seems that recreate_entire_table need to be replaced with replace_entire_table

extra options for oracle connection source

onetl/onetl/connection/db_connection/oracle/connection.py

Line 147 in 11dec83

    
                   <https://docs.oracle.com/cd/E11882_01/appdev.112/e13995/oracle/jdbc/OracleDriver.html>`_

it seems that one of this pages is better candidate for extra connection properties: https://docs.oracle.com/en/database/oracle/oracle-database/23/jajdb/oracle/jdbc/OracleConnection.html
https://docs.oracle.com/en/database/oracle/oracle-database/23/jajdb/oracle/jdbc/OracleDriver.html

override KafkaReadOptions

onetl/onetl/connection/db_connection/kafka/options.py

Line 74 in 6849b46

    
                   are populated from connection attributes, and cannot be set in ``KafkaReadOptions`` class and be overridden

it seems that
'and cannot be overridden by the user in KafkaReadOptions class to avoid issues.'
will be more clear than
'and cannot be set in KafkaReadOptions class and be overridden by the user to avoid issues.'

mogodb docs restructured 3

onetl/onetl/connection/db_connection/mongodb/options.py

Line 99 in 6849b46

    
                   `supported by connector <https://www.mongodb.com/docs/spark-connector/current/configuration/read>`_,

same as with read/pipeline options: mongodb docs restructured and now old link (https://www.mongodb.com/docs/spark-connector/current/configuration/read) redirects to https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-write-config/

here we need a link change

onetl/onetl/connection/db_connection/mysql/connection.py

Line 96 in 0d04466

    
                   <https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-configuration-properties.html>`_

mysql j connector docs structure changed, article address need to be updated to https://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html

mobiletelesystems / onetl Goto Github PK

onetl's People

Contributors

Stargazers

Watchers

Forkers

onetl's Issues

Recommend Projects

Recommend Topics

Recommend Org