bernhardschlegel / sql-dumper Goto Github PK

Hi,

Can you please let us know, with how many millions rows you have tested this cmd to export? I need to export 3 Million data, how much time it will take to export using the SQL-dumper?

Thanks,
Chandru

Problem

sqldump.etl.Query replacing ; with empty string while loading queries from file. (see line 60)

This will case the problem for some case for example below query

with temp as ( SELECT 'value1;value2;value3' text FROM  dual )
select regexp_substr(t.text, '[^;]+', 1, levels.column_value) as text from temp t,
	table(cast(multiset(select level from dual connect by level <= length (regexp_replace(t.text, '[^;]+')) + 1) as sys.OdciNumberList)) levels

Possible solution:

1. Should not replace the ; (ask in documentation/readme to not put semi-colon in file)
2. Remove only the last ; from query (in the end of file)

Does not work with large tables.

19.01.2018 14:08:07 [INF]: converting data to CSV (one dot corresponds with 50k lines).......Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Arrays.java:2694)
at java.lang.String.(String.java:203)
at java.lang.StringBuilder.toString(StringBuilder.java:405)
at sqldump.etl.Dump.toFile(Dump.java:93)
at sqldump.etl.Query.executeQuery(Query.java:221)
at sqldump.etl.Query.getDataFromSQLFile(Query.java:62)
at sqldump.SQLDump.run(SQLDump.java:89)
at sqldump.Main.main(Main.java:7)