apsaradb / polardb-for-postgresql Goto Github PK
View Code? Open in Web Editor NEWA cloud-native database based on PostgreSQL developed by Alibaba Cloud.
Home Page: https://apsaradb.github.io/PolarDB-for-PostgreSQL/zh/
License: Apache License 2.0
A cloud-native database based on PostgreSQL developed by Alibaba Cloud.
Home Page: https://apsaradb.github.io/PolarDB-for-PostgreSQL/zh/
License: Apache License 2.0
There is a configure error : header file <Python.h> is required for Python.
But if run " ./configure --prefix=/home/postgres/polardb/polardbhome ", it's ok.
I tried to install python, it didn't work.
Industry: General
Function Expectations: statement-level timeout hints, for example
/*+ Set(statement_timeout "100ms") */ select * from abc where info like '%abcde%';
What problem is solved: avalanche prevention, specifying an upper limit on the execution time of some particular SQL
Alternative: Due to implementation issues, pg_hint_plan is currently unable to set a timeout on statement_timeout.
like the difference between polardb and pg....
Industry: Internet
Expectations: PG_BASEBACKUP is capable of multi-threaded backups as well as backups in standby libraries
What problems to solve: First of all, multi-threaded backup can play a very good role in speeding up large databases, which can effectively reduce the burden of the primary database and reduce the occurrence of exceptions by performing backups from the standby database
Expected function rating: there must be use
Database pain point: now with more than 2T instances, single-threaded backups can take several hours and can only be done in the primary database, greatly increasing the risk of the primary database. After checking the current mainstream backup tools (Barman, bgbackreset), they are all based on pg_basebackup for development and upgrade. The principle is basically the same, need to execute pg_start_backup and pg_stop_backup, these two commands can only be executed in the main database.
Rotate Table support, by specifying the table size upper limit, the number of records upper limit, LILO, FIFO and other configable data aging algorithm, automatically aging data.
Index partition (not partition table index)
I'm presuming that you may want to add the copyright and year as part of the LICENSE file, line 190.
I want to build an HA cluster with three physical servers. Is the configuration correct?DDB1 dDB2 DDB3 is the host map configured to /etc/hosts.
polardb_paxos.txt
Update | delete the skip locked, improve batch concurrent operations (the current need to select for update the skip locked, then update. How many interactions are needed)
Logical replication supports DDLs
If you open-source a project, then ensure that everyone is able to participate.
English is spoken by everyone in the open-source communities.
Either accept this fact and ensure that every communication is in English AND NOT in Chinese.
如果你開源一個項目,那麼確保每個人都能參與。
開源社區中的每個人都說英語。
要么接受這個事實,並確保每一次交流都是用英語而不是中文,要么把你的項目移到不同的地方。
Global index of partitioned table
File level migration capability, support DATAFILE + WAL incremental migration, improve migration speed.
Products currently supported on the market:
postgrespro: pg_transport
aws: pg_transport
Materialized view based on mv log, At present, MV does not support MV Log. To refresh MV requires a join comparison between MV Log and the base table, which may cost a lot when refreshing
Multi-tenancy function, time based dynamic configuration quota.
Storage quota function (table level, user level, schema level, database level)
Resource isolation (session level, user level, DB level, schema level)
for exp:
08:00-23:00 for oltp resource management configure.
23:00-08:00 for OLAP resource management configure.
Industry: General
Expectations: Configurable connection life cycles, such as a connection that lives for a few minutes and is released immediately when it is idle (actively disconnecting from the client)
In some PaaS services, the longer a connection lives, the more PG metadata accessed, the more local memory in the session (will not be freed), and the more connections will cause OOM. Global Cache can also solve this kind of problem.
SQL syntax support delete | update limit, is suitable for batch operation
Undo table access method engine, reduce bloat
The Global Cache, which has a large number of objects in a large number of partitioned tables or PaaS user scenarios, currently PG uses the session-level Catalog Cache, which can result in a large amount of object metadata memory footprint, resulting in a large amount of wasted memory.
postgresql社区不缺分布式集群部署机制。
if can support helm to install k8s, we can fast experience PolarDB
SQL syntax: Support to bypass the shared buffer SELECT and INSERT. Advantage: Bursting large table queries does not affect hot data in the shared buffer
Wal log writing in parallel, currently a large amount of data import, there may be two performance bottlenecks, datalbock extend exclusive lock, or Wal insert exclusive lock
Block level compression
Online DDL support, some DDL need rewrite table, support PG_REPACK 2-stage File Replace capability, reduce the lock table time of such DDL operations.
Does PolarDB support failover slot such as Aliyun RDS for PostgreSQL?
I am trying to compile a fresh checkout on Ubuntu 20.10 and get this error while linking:
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `ENGINE_cleanup'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `SSL_get_ex_new_index'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `SSLv23_method'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `ERR_free_strings'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `SSL_load_error_strings'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `SSL_library_init'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `OPENSSL_add_all_algorithms_noconf'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `EVP_cleanup'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `CRYPTO_set_id_callback'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `CRYPTO_set_locking_callback'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `CRYPTO_num_locks'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `CRYPTO_cleanup_all_ex_data'
/usr/bin/ld: ../../src/backend/polar_dma/lib//libpolar_consensus_wrapper.so: undefined reference to `CRYPTO_mem_leaks_fp'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:71: postgres] Error 1
make[2]: Leaving directory '/home/ajung/src/PolarDB-for-PostgreSQL/src/backend'
make[1]: *** [Makefile:42: all-backend-recurse] Error 2
make[1]: Leaving directory '/home/ajung/src/PolarDB-for-PostgreSQL/src'
make: *** [GNUmakefile:11: all-src-recurse] Error 2
环境:centos 7.6.1810
问题:编译安装执行 ./onekey.sh all
时,报错如下:
checking which semaphore API to use... unnamed POSIX
checking for /dev/urandom... yes
checking which random number source to use... /dev/urandom
checking Python.h usability... no
checking Python.h presence... no
checking for Python.h... no
configure: error: header file <Python.h> is required for Python
Index scan sample method, corresponding to table sample
在拉取了最新代码,执行./build.sh进行编译,然而并不能正常编译通过,我不知道你们是怎么编译通过的。
gcc version 10.2.0
cmake version 3.19.6
细节请看:
https://asciinema.org/a/GUXq3STFScbiUDTPy4avaMODk
类似
/root/PolarDB-for-PostgreSQL/src/backend/polar_dma/libconsensus/dependency/easy/src/thread/easy_uthread.c:266:17: error: ‘SIG_BLOCK’ undeclared (first use in this function)
这种,应该是你们编码错误,定义 SIG_BLOCK 的头文件<signal.h>
都没 include 吧
这个错误我猜应该是用到 SIG_BLOCK 的源文件中没 include,而是隐藏在其它3方库里include了,而我这里恰恰没有这个3方库。但是不得不说,这是一个不好的编码习惯。比如
#include <iostream>
之后用到了std::string,不包含 <string>
也是能用 std::string 的,但这不是个好习惯,不能依赖间接包含,何况你这是间接依赖了一个3方库中的 include。
出错的地方有点多,麻烦看bash录制的视频。
希望你们能弄的健壮一点,像pg那样一次编译通过,无需折腾
Industry: Traditional
Functional Expectations: After the logon failure exceeds the limits, lock the user
What issues to solve: Improve security, prevent brute force attacks, and be compatible with Oracle
Alternative: auth_delay
Development Tip: Login Hook?
Divide WAL into Database and implement Database Pluggable function in the future
38/5000
Index Skip Scan Optimizer, which currently requires CTE recursive simulation
大概看了一下相对熟悉的几个folder的code,除了distributed_txn,polar_datamax,polar_dma这三个目录,其它的目录是不是基本就是pg的code,没有改动?
Industry: General
Function Expectations:
Database, table, schema-level snapshot, time point clone, time point flash back function.
Scene:
In the enterprise PaaS service scenario, a different table or schema or database for each database corresponds to an enterprise in the software service, and when the enterprise's data needs to be rolled back, the entire database instance does not need to be rolled back.
Industry: General
Expected function :pg_stat_statements provides metrics for P99, P95
What problem to solve: Monitor the execution stability of PG statements as a reference for business metrics.
The "main" branch generated by Github is useless since you have "master" branch.
I followed the README, but pgxc_ctl binary was not generated.
So I took advice from others, run "make install to generate pgxc_ctl binary file in PGXC_CTL_HOME".
It does generate pgxc_ctl binary.
But if you use it, you will get an error:
/usr/bin/bash
/home/postgres/polardb/polardbhome/bin/pgxc_ctl is not directory. Check your configurfation
环境:centos 7.6.1810
问题:编译安装执行 ./onekey.sh all时,报错如下:
/home/polardb/PolarDB-for-PostgreSQL-master/src/backend/polar_dma/libconsensus/polar_wrapper/polar_consensus_log.cc:21:19: fatal error: paxos.h: No such file or directory
#include "paxos.h"
Industry: Internet industry
Function Expectations:
The ability to limit flow based on SQL characteristic rules.
Implement at the hook level.
Timeout is only supported at user, db, and global statement levels. Large granularity.
Hope to be able to configure SQL, QPS to achieve.
SQL1: QPS < 10000,
SQL2: QPS < 1000
Scene:
Prevent avalanches.
Prevent some businesses or individuals from exhausting resources by submitting some SQL.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.