ilri / odktools Goto Github PK

Currently select_one_external are ignored. These must be implemented in the same way as select_one

The new MySQL client and library has --ssl-mode=PREFERRED (https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1872541/comments/25) .

Currently, the toolset does not work in Ubuntu 20.4 when connecting to 5.7.X servers.

Currently it is possible to edit the schema using the XML files however it is not possible to add new indexes to the schema. This can only be done by editing the SQL file but such edits will disappear with a new run of createFromMySQL

Current schema files does not have the metadata information for fields and tables. This is required for CreateFromXML to also generate the metadata

In ODKToMySQL If a table with multiple selects gets separated the multi select tables still link to the original table and not to the separated tables

If a ' is used then the comment could break the SQL. " must be used

select_one and select_multiple can dynamically pull the choice lists from CSVs using the search() expression. Such cases are currently ignored but must be handled in the same way as simple select_one and select_multiple.

At the moment JSONToMySQL does not insert into the multi select tables

or_other should add other to the lookup table and add a new _other field to the table

Since the separation file is edited by the user it should include the description of each separation to be included as the table labels.

There seems to be a bug with this new odk tool. When I try to run ODKToMYSQL as shown below:
cadoyo@ILRI16648:~/Data/Projects/GHGENandi$ ~/Data/Tools/odktools/ODKToMySQL/odktomysql -v q6seca_hhid -t geninfo -x ~/Data/Projects/GHGENandi/Nandi_FinHH/XLS050_JO.xlsx

Well I know, its a memory problem but I cant just seem to find where the bug is.

Collins

In some XLSX if the choice label is empty ODKToMySQL crashes with a Segmentation fault

Currently, ODK to MySQL generate insert scripts from ODK options but it is only when such scripts are executed that one is able to determine if an option has repeated codes. The tool should check for those errors and report about them.

A series of utilities are needed when dealing with multiple versions of an ODK. For example if one has 3 versions of a ODK with differences in terms of variables and repeats one should have one final database accommodating the 3 versions.

Certain error messages like separation file and languages should be displayed in a machine-readable format like XML to allow better integration with third-party tools and web services.

The UUID-triggers.sql file is only created by ODKToMySQL however when an schema is modified by editing its XML, the output of createFromXML should recreate the UUID-triggers.sql file since such editing may add or remove tables.

There is an issue while processing data with a multi-select column where the related field in the multi-select table does not get added in the create.sql file. Everything works fine until you get to the processing of the JSON into mysql where yo get the error about the missing field in a table.

Attached are the xlsx, separation file xml and a few JSON files for the testing.

NOTE: Github refused to upload .zip file so I renamed the attached file to .png Download it and rename to .zip to be able to open the files

There seems to be a library missing in this: I get this error when I try to build.

/home/cadoyo/Data/Tools/odktools/3rdparty/QtXlsxWriter/src/xlsx/xlsxzipwriter.cpp:27: error: private/qzipwriter_p.h: No such file or directory
#include <private/qzipwriter_p.h>
^

Collins

If pulldata() is used in calculated fields the pulled options are ignored and a lookup table is no created

In some versions of Excel particularly using UTF8 the separator between select and the list name is a 00A0 (hex C2A0) character and not a space (Hex 20). Patch code to allow 00A0

ODKToMySQL - Allow Multiselect as primary keys.

Currently compareCreateXML compare two XML schemata for incremental changes and generates one combined XML. This is fine, but also it should generate the necessary updates for schemata that already has data.

The export to STATA and SPSS is generating erroneous line sizes when data is null

The external class in JavaScript can disable a field from being inserted however the insert process ignore this.

When the main table is a repeat the tool does not import the sub-tables of the repeat.

We need to rollback 44d5325. This because XMLToJSON can be used without a database created by ODKToMySQL thus it cannot depend on a manifest file.

MySQL support field and table descriptions. This should be included in the SQL

The output files particularly the SQL files need to include the input parameters that generated such files. This is important for trace-ability

The current code of JSONToMySQL assumes that the keys of a parent pass to a child however the key of a parent could be different of the child. For example:

Table A (Key A+B)
Table B: Child of A (Key C), (A+B are referenced fields to table A)
Table C: Child of B (Key C+D) --> Here fields A+B does not exists however JSONToMySQL assumes that they are.

Is common to use selected-at() in ODK and this should be handled as a lookup.

It might be better to move this project from ilri/odktools to ilri/rmg-odktools. Most projects in the organisation are now following this naming convention.