Coder Social home page Coder Social logo

Comments (4)

ahouska avatar ahouska commented on August 27, 2024 1

The ability to change table definition was already proposed, see

#15
and
#18

pg_squeeze relies on the fact that definition of the table it's currently processing does not change - it checks the system catalog several times and aborts if any related catalog change is detected. If pg_squeeze performed catalog changes itself, it'd become significantly more complex.

As for the partitioning, it'd perhaps be doable (although not trivial), however size and rate of DMLs of the original (not-yet-partitioned) table might be a problem. I've seen at least one case where pg_squeeze could not finish table processing until the host ran out of disk space. The point is that pg_squeeze needs to prevent WAL archiving until the phase called "initial load" is done. However this initial load can take hours, and if enough data changes are performed, the amount of WAL files accumulated can become too high.

In other words, pg_squeeze might encounter problems when processing huge tables, but tables that users want to make partitioned are supposed to be huge.

from pg_squeeze.

ahouska avatar ahouska commented on August 27, 2024 1

Perhaps this blog post can be useful if you want to turn non-partitioned table into partitioned one without noticeable outage: https://blog.hagander.net/repartitioning-with-logical-replication-in-postgresql-13-246/

from pg_squeeze.

kmoppel avatar kmoppel commented on August 27, 2024

+1 for the idea. Probably user still needs to create the the table beforehand already though with desired partitioning in place. @ahouska do you think it would be doable?

from pg_squeeze.

jobinau avatar jobinau commented on August 27, 2024

Yes Agree, @kmoppel , User should be allowed to create a table as he/she wants beforehand with partitioning or whats soever. That opens up different possibilities and use-cases.
Another use case is for this feature is Online schema changes. For example, I want to convert a column with "NUMERIC" datatype to "INT"

from pg_squeeze.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.