Comments (2)
[user@host programs]$ /path/to/pgloader/bin/pgloader --client-min-messages=notice database.load
2024-02-22T10:43:40.001437+01:00 NOTICE Starting pgloader, log system is ready.
2024-02-22T10:43:40.007689+01:00 LOG pgloader version "3.6.af8c3c1"
[...]
2024-02-23T02:34:37.652045+01:00 LOG report summary reset
table name errors read imported bytes total time read write
---------------------------------- --------- --------- --------- --------- -------------- --------- ---------
fetch meta data 0 211 211 0.494s
Create Schemas 0 0 0 0.001s
Create SQL Types 0 0 0 0.000s
Create tables 0 100 100 0.360s
Set Table OIDs 0 50 50 0.002s
---------------------------------- --------- --------- --------- --------- -------------- --------- ---------
[...]
---------------------------------- --------- --------- --------- --------- -------------- --------- ---------
COPY Threads Completion 0 8 8 14h37m57.750s
Create Indexes 0 111 111 4h47m9.918s
Index Build Completion 0 111 111 48m26.715s
Reset Sequences 0 27 27 0.870s
Primary Keys 0 49 49 1.044s
Create Foreign Keys 0 50 50 24m11.139s
Create Triggers 0 0 0 0.000s
Install Comments 0 0 0 0.000s
after load 0 23 23 0.364s
---------------------------------- --------- --------- --------- --------- -------------- --------- ---------
Total import time ✓ 4449186294 4449186294 227.9 GB 20h37m47.805s
Output from the virtual server at my customer.
Duration based on the log lines = ~ 15.5 hours
Duration reported by pgLoader = 20.5 hours
from pgloader.
We are having the same issues. On our end it seems that the "Create Indexes" part makes no sense. As an example, this is a summary which only migrates the schema. Real time it took was 14 sec:
table name errors rows bytes total time
---------------------- --------- --------- --------- --------------
fetch meta data 0 3405 0.293s
Create Schemas 0 0 0.437s
Create SQL Types 0 0 0.004s
Create tables 0 1062 3.530s
Set Table OIDs 0 531 0.047s
---------------------- --------- --------- --------- --------------
---------------------- --------- --------- --------- --------------
Index Build Completion 0 1906 8.285s
Create Indexes 0 1906 1m38.173s
Reset Sequences 0 310 0.144s
Primary Keys 0 521 0.282s
Create Foreign Keys 0 968 0.785s
Create Triggers 0 0 0.000s
Install Comments 0 664 0.274s
after load 0 2 0.013s
---------------------- --------- --------- --------- --------------
Total import time ✓ 0 1m47.957s
from pgloader.
Related Issues (20)
- no tables are found in geopackage/sqlite file HOT 1
- Issue with pgloader migration from MySQL MariaDB to Supabase HOT 2
- MySQL conversion syntax error at or near "\" HOT 3
- SQLite - primary keys not transfered, unique index creation fails, reset sequences fail
- sqlite to postgresql (Windows + Docker) HOT 1
- Casting errors when using Redshift as a data source
- Option "rows per range" not working in PostgreSQL to PostgreSQL migration?
- MSSQL Materialize view is downloading all views instead of filtering & downloading only the ones passed in config. HOT 2
- Cannot import SQLite array columns for various types into PostgreSQL (`TEXT[]`, `NUMERIC[]`, `BYTE[]`)
- Postgres - pgloader - Column names are converted to lower case in PostgreSQL by default HOT 1
- postgres -> postgres include/exclude logic not working correctly HOT 1
- Pgloader 3.6.2 - "on error resume next" functionality doesn't work for Pgloader.
- An error occured when sync mysql tables to the postgre on the data type 'set' into 'test_str_8[]'
- mysql8.0, miss on-update-current-timestamp trigger when the column has default value
- Common Lisp trivial-utf-8 error: Invalid byte at start of character: 0xFC HOT 2
- DB-CONNECTION-ERROR: Failed to connect to pgsql
- ENUM Being Renamed
- MATERIALIZE VIEWS does not work for MySQL HOT 2
- How to migrate stored procedure in MySQL
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pgloader.