PG-Strom - Master development repository
Home Page: http://heterodb.github.io/pg-strom/
License: Other
PG-Strom is an extension for PostgreSQL database. It is designed to accelerate mostly batch and analytics workloads with utilization of GPU and NVME-SSD, and Apache Arrow columnar.
For more details, see the documentation below. http://heterodb.github.io/pg-strom/
Software is fully open source, distributed under PostgreSQL License.
text型カラムのソートを試したところ、ソート結果が正しくありませんでした。
さらに、ソート結果がランダムに変わっているような感じがします。
SELECT * FROM text_test_null WHERE str < '!9' ORDER BY str; Table "public.text_test_null"
Column | Type | Modifiers
--------+----------+-----------
id | integer |
len | smallint |
str | text |
id | len | str
---------+-----+----------------------
4714751 | 1 | !
3239043 | 3 | !%C
2752075 | 2 | !#
4006855 | 2 | !4
1451770 | 7 | !2bza8)
4359686 | 1 | !
1025680 | 6 | !#lJQr
1892999 | 2 | !0
656532 | 1 | !
4707357 | 1 | !
3596320 | 1 | !
3725305 | 1 | !
904560 | 1 | !
4115748 | 1 | !
671368 | 1 | !
391973 | 1 | !
4189961 | 1 | !
2744618 | 1 | !
(snip)
id | len | str
---------+-----+----------------------
4935705 | 3 | !%X
508460 | 11 | !7kqID_-7C3
2913026 | 1 | !
209857 | 1 | !
2496844 | 1 | !
770890 | 1 | !
3305833 | 1 | !
3333653 | 1 | !
2571933 | 1 | !
1956452 | 1 | !
721247 | 1 | !
3104838 | 1 | !
4149266 | 1 | !
1458619 | 1 | !
3850674 | 1 | !
1824199 | 1 | !
3520604 | 1 | !
1132854 | 1 | !
(snip)
id | len | str
---------+-----+----------------------
2498205 | 2 | !"
4928480 | 2 | !&
428458 | 1 | !
2609068 | 1 | !
1171356 | 1 | !
2688475 | 1 | !
1978959 | 1 | !
3742408 | 1 | !
3245354 | 1 | !
3503182 | 1 | !
897478 | 1 | !
1119357 | 1 | !
1166894 | 1 | !
2592404 | 1 | !
3972008 | 1 | !
3937692 | 1 | !
2481119 | 1 | !
4894788 | 1 | !
psql上で、"\d"にてテーブル定義表示を行おうとすると、AssertionFailedとなります。
少しのバージョンでは、正常に動作していました。
※ 現在は、grafer.cの#if 1を#if 0にすることで回避しています。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000948a59 in ExceptionalCondition (conditionName=0x7f9513995298 "!(gsortstate->scan_done)",
errorType=0x7f9513994844 "FailedAssertion", fileName=0x7f951399489f "gpusort.c", lineNumber=1808) at assert.c:54
#3 0x00007f9513981b7d in gpusort_exec (node=0x29afdf0) at gpusort.c:1808
#4 0x0000000000680dcb in ExecCustomPlan (cpstate=0x29afdf0) at nodeCustom.c:37
#5 0x0000000000668ae5 in ExecProcNode (node=0x29afdf0) at execProcnode.c:453
#6 0x000000000066652c in ExecutePlan (estate=0x29afcd8, planstate=0x29afdf0, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x2997528) at execMain.c:1475
#7 0x00000000006643dc in standard_ExecutorRun (queryDesc=0x29bf148, direction=ForwardScanDirection, count=0) at execMain.c:308
#8 0x000000000066421f in ExecutorRun (queryDesc=0x29bf148, direction=ForwardScanDirection, count=0) at execMain.c:256
#9 0x00000000007f9e7e in PortalRunSelect (portal=0x29c1cf8, forward=1 '\001', count=0, dest=0x2997528) at pquery.c:946
#10 0x00000000007f9aa5 in PortalRun (portal=0x29c1cf8, count=9223372036854775807, isTopLevel=1 '\001', dest=0x2997528,
altdest=0x2997528, completionTag=0x7fff7461bc20 "") at pquery.c:790
#11 0x00000000007f3212 in exec_simple_query (
query_string=0x294f3d8 "SELECT c.oid::pg_catalog.regclass FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i WHERE c.oid=i.inhrelid AND i.inhparent = '24636' ORDER BY c.oid::pg_catalog.regclass::pg_catalog.text;") at postgres.c:1045
#12 0x00000000007f7992 in PostgresMain (argc=1, argv=0x28dfa60, dbname=0x28df8c0 "pgstrom_test", username=0x28df8a0 "tkondo")
at postgres.c:4004
#13 0x000000000076e8e9 in BackendRun (port=0x290a720) at postmaster.c:4117
#14 0x000000000076df9f in BackendStartup (port=0x290a720) at postmaster.c:3791
#15 0x000000000076a366 in ServerLoop () at postmaster.c:1570
#16 0x00000000007699b9 in PostmasterMain (argc=1, argv=0x28de430) at postmaster.c:1223
#17 0x00000000006b9166 in main (argc=1, argv=0x28de430) at main.c:227
最新版のコードで、WHERE句にnumeric型のカラムに対する比較演算を記述すると、コンパイルエラーとなることがあります。
SELECT * FROM stock_6701t WHERE turnover = 10000000::numeric
SELECT * FROM stock_6701t WHERE turnover < 10000000::numeric
SELECT * FROM stock_6701t WHERE turnover > 10000000::numeric
SELECT * FROM stock_6701t WHERE turnover <= 10000000::numeric
SELECT * FROM stock_6701t WHERE turnover >= 10000000::numeric Table "public.stock_6701t"
Column | Type | Modifiers
----------+---------+-----------
date | date | not null
startp | money | not null
highp | money | not null
lowp | money | not null
endp | money | not null
turnover | numeric |
totalp | money |
Indexes:
"stock_6701t_pkey" PRIMARY KEY, btree (date)
ERROR: PG-Strom: OpenCL execution error (build program failure)
#ifndef PG_BYTEA_TYPE_DEFINED
STROMCL_VARLENA_TYPE_TEMPLATE(bytea)
#endif
#ifndef PG_BOOL_TYPE_DEFINED
STROMCL_SIMPLE_TYPE_TEMPLATE(bool,cl_bool)
#endif
#ifndef PG_NUMERIC_TYPE_DEFINED
STROMCL_VARLENA_TYPE_TEMPLATE(numeric)
#endif
#define KPARAM_0 pg_bytea_param(kparams,errcode,0)
#define KPARAM_1 pg_numeric_param(kparams,errcode,1)
#define KVAR_0 pg_numeric_vref(kcs,toast,errcode,0,kcs_index)
static pg_bool_t
gpuscan_qual_eval(__private cl_int *errcode,
__global kern_gpuscan *kgscan,
__global kern_column_store *kcs,
__global kern_toastbuf *toast,
size_t kcs_index)
{
__global kern_parambuf *kparams
= KERN_GPUSCAN_PARAMBUF(kgscan);
return pgfn_numeric_eq(errcode, KVAR_0, KPARAM_1);
}
DETAIL: Compilation started
5:1890:10: error: implicit declaration of function 'pgfn_numeric_eq' is invalid in OpenCL
5:1890:10: note: did you mean 'pg_numeric_vref'?
5:1873:1: note: 'pg_numeric_vref' declared here
5:918:2: note: expanded from macro 'STROMCL_VARLENA_TYPE_TEMPLATE'
5:884:2: note: expanded from macro 'STROMCL_VARLENA_VARREF_TEMPLATE'
<scratch space>:79:1: note: expanded from macro 'pg_'
5:1890:10: error: returning 'int' from a function with incompatible result type 'pg_bool_t'
Compilation failed
以下のSQL文では、正しい結果が表示されます。
SELECT * FROM stock_6701t WHERE turnover - 10000000::numeric > 0::numericJOIN操作を含むSQL文を実行すると、まれにAssertionFailedとなることがあります。
psqlから実行したもので、多重実行は行っていません。
SELECT gender,avg(score) FROM member_score JOIN member_gender USING(id) GROUP BY gender; Table "public.member_score"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
date | date |
score | integer |
Table "public.member_gender"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
gender | integer |
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (
conditionName=0xac4e28 "!(( ((void) ((bool) ((! assert_enabled) || ! (!((buffer) <= NBuffers && (buffer) >= -NLocBuffer)) || (ExceptionalCondition(\"!((buffer) <= NBuffers && (buffer) >= -NLocBuffer)\", (\"FailedAssertion\"), \"e"...,
errorType=0xac4d45 "FailedAssertion", fileName=0xac4d38 "execTuples.c", lineNumber=341) at assert.c:54
#3 0x0000000000674c55 in ExecStoreTuple (tuple=0x2cc7f58, slot=0x2cc0048, buffer=182, shouldFree=1 '\001') at execTuples.c:341
#4 0x00007f078bfdab68 in gpuscan_next_tuple (gss=0x2cbfab0, slot=0x2cc0048) at gpuscan.c:1128
#5 0x00007f078bfdb2bc in gpuscan_fetch_tuple (node=0x2cbfab0) at gpuscan.c:1157
#6 0x00007f078bfdb354 in gpuscan_exec (node=0x2cbfab0) at gpuscan.c:1307
#7 0x00000000006807e7 in ExecCustomPlan (cpstate=0x2cbfab0) at nodeCustom.c:37
#8 0x00000000006684b5 in ExecProcNode (node=0x2cbfab0) at execProcnode.c:453
#9 0x0000000000684a9a in ExecHashJoin (node=0x2cbee00) at nodeHashjoin.c:154
#10 0x00000000006684f4 in ExecProcNode (node=0x2cbee00) at execProcnode.c:468
#11 0x000000000067de7f in agg_fill_hash_table (aggstate=0x2cbe500) at nodeAgg.c:1351
#12 0x000000000067d9b1 in ExecAgg (node=0x2cbe500) at nodeAgg.c:1113
#13 0x0000000000668548 in ExecProcNode (node=0x2cbe500) at execProcnode.c:487
#14 0x0000000000665efe in ExecutePlan (estate=0x2cbe3e8, planstate=0x2cbe500, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x2caf180) at execMain.c:1475
#15 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x2ca2748, direction=ForwardScanDirection, count=0) at execMain.c:308
#16 0x0000000000663beb in ExecutorRun (queryDesc=0x2ca2748, direction=ForwardScanDirection, count=0) at execMain.c:256
#17 0x00000000007f964e in PortalRunSelect (portal=0x2c99a58, forward=1 '\001', count=0, dest=0x2caf180) at pquery.c:946
#18 0x00000000007f9275 in PortalRun (portal=0x2c99a58, count=9223372036854775807, isTopLevel=1 '\001', dest=0x2caf180,
altdest=0x2caf180, completionTag=0x7fff342c4f10 "") at pquery.c:790
#19 0x00000000007f29e2 in exec_simple_query (
query_string=0x2c556b8 "select gender,avg(score) from member_score join member_gender using(id) group by gender;")
at postgres.c:1045
#20 0x00000000007f7162 in PostgresMain (argc=1, argv=0x2be95b8, dbname=0x2be9418 "pgstrom_test", username=0x2be93f8 "tkondo")
at postgres.c:4004
#21 0x000000000076e18a in BackendRun (port=0x2c11230) at postmaster.c:4104
#22 0x000000000076d840 in BackendStartup (port=0x2c11230) at postmaster.c:3778
#23 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#24 0x0000000000769251 in PostmasterMain (argc=1, argv=0x2be83b0) at postmaster.c:1222
#25 0x00000000006b8aee in main (argc=1, argv=0x2be83b0) at main.c:223
Bulkデータを受け渡す際のデータ構造が十分に練られておらず、
Hash-Joinの結果を Sort に渡す場合など、うまく連携ができない。
そこで、Bulkデータを受け渡しには以下の vrelation 構造体を使う事にする。
vrelation の目的は、複数の row-/column-store から成る結果セットを一律の
方法で取り扱う手段を提供し、さらに結果セットの生成の際には、データそれ
自体のコピーを行わずに済むようにする事。
vrelation は、row-/column-store への参照と、結果セットの各列が何番目の
レコードから構成されているかという情報を持つ。
最終的にPG-Stromの世界からPostgreSQLの世界に戻るときには、行指向
データへと変換するが、PG-Stromの世界にいる間は vrelation のまま上位の
ノードへとデータを受け渡しできる。
typedef struct {
cl_int nrels;
cl_int ncols;
cl_int nitems;
cl_int nrooms;
cl_char has_recheck; /* true, if any rows to be rechecked /
cl_int errcode; / space to write back*/
struct {
cl_char attnotnull;
cl_char attalign;
cl_short attlen;
cl_short vrelidx;
cl_short vattnum;
} colmeta[FLEXIBLE_ARRAY_MEMBER];
} kern_vrelation;
typedef struct
{
StromObject sobj; /* =StromTag_VirtRelation */
slock_t lock;
cl_int refcnt;
cl_int rcsnums;
StromObject *rcstore[VRELATION_MAX_RELS];
kern_vrelation kern;
} pgstrom_vrelation;
#37 の調査中に発見しました。
tcache.c内のQuickSortロジックであるtcache_sort_tcnode_internal()では、
pivotの値をソート範囲の**の値のポインタとしています。
この時、左か右の片方が**まで到達して停止し、もう片方が**まで到達せずに停止した場合、
値交換のコードが実行され、pivotが指しているデータが変更されることになります。
結果としてソートの途中でpivotの値が変化することとなり、正しくソートされない場合が発生します。
PostgreSQL標準のソートと一部結果が異なる場合がある。
pgstrom_test=# explain select id from bytea_test where bin between '\xff00'::byt ea and '\xffff'::bytea order by id;
LOG: (gpusort.c:633) colmeta[0] attname=id {attnotnull=0, attalign=8, attlen=8, cs_ofs=48}
QUERY PLAN
Custom (GpuSort) (cost=129657.73..129701.35 rows=17449 width=8)
Sort keys: id
Sort keys width: 8
Rows per chunk: 21408
-> Custom (GpuScan) on bytea_test (cost=10000.00..129645.44 rows=17449 width=8)
Filter: ((bin >= '\xff00'::bytea) AND (bin <= '\xffff'::bytea))
Host References: id, bin
Device References:
Filter: ((bin >= '\xff00'::bytea) AND (bin <= '\xffff'::bytea)) Planning time: 1.464 ms
(10 rows)
pg_strom.enabled = on | off を切り替えて実行したところ、
--- /home/kaigai/hoge1.out 2014-06-06 15:46:38.120934370 +0900
+++ /home/kaigai/hoge2.out 2014-06-06 15:46:48.200885168 +0900
：
(略)
：
@@ -18735,5 +18724,16 @@ SET
4999537 | \xff5b9eb9c3c6193e6dbc4a4c636ff5a9f17917f89d957e584cd9c2ac8f744f8ecfed48
4999632 | \xffe5754d7a75737cb2773a0a896cebddbf0670629c610c132d9408287dd94d7cbec2c93838
4999963 | \xffe2ec08910173855918
最後の方に少しソートされていない要素が発生している。
実行計画からわかるように、GpuScanの処理結果をGpuSortに回すような構造に
なっており、データの件数は 18734 件。これを GTX のブロックサイズ 32 で
割った余りは 14 なので、kernelを起動する際のworkgroup-sizeが少し臭い。
bytea型を含むテーブルにおいて、ソートを含むSQL文を発行すると、エラーとなり実行できないことがあります。
pg_strom.soを読み込まない状態では実行できるので、PG-Stromでの問題ではないかと思います。
SELECT * FROM bytea_test WHERE bin BETWEEN '\xff00'::bytea AND '\xffff'::bytea ORDER BY id;
SELECT * FROM bytea_test2 WHERE bin BETWEEN '\xff00'::bytea AND '\xffff'::bytea ORDER BY id;ERROR: unrecognized node type: 221
Table "public.bytea_test"
Column | Type | Modifiers
--------+--------+-----------
id | bigint | not null
bin | bytea |
Indexes:
"bytea_test_pkey" PRIMARY KEY, btree (id)
Table "public.bytea_test2"
Column | Type | Modifiers
--------+---------+-----------
id | integer | not null
bin | bytea |
Indexes:
"bytea_test2_pkey" PRIMARY KEY, btree (id)
なお、以下の条件のいずれかにあてはまると再現しません。
#37 の調査中に発見しました。
tcache_split_node()は、tc_nodeの空きがなくなった場合に呼ばれ、
blkno_maxの分のデータを新しいtc_nodeに異動させることで空きを作ります。
tcacheの構築中にheap_getnext()が返すtupleのBlock Noが0に戻った場合、
以下のような動きをすることになり、不都合が生じます。
この結果、ほどなくして再びtc_nodeが満杯となり、新しいtc_nodeが作成され、
結果として、1 Block分のデータしか格納されないtc_nodeが大量に作成されることになります。
tc_node作成の際の要求メモリサイズは固定であるため、この現象が発生すると、
本来共有メモリに格納できる量のデータであっても、tc_node分割の際にメモリ不足となり、
エラー終了するという結果が生じます。
エラー発生時に、例外コンテキストから呼ばれる ResourceRelease() は、
コールバックの呼出し前にLWLockReleaseAll()をコールする。
tcacheがビルド中（未完成）の場合、Ex-Lock下で構築半ばのキャッシュを
解放したいが、既にEx-Lockが解放されてしまっている。
Lwlockの代わりに、トランザクションロックを使用し、TCacheHeadの
エラー回復は RESOURCE_RELEASE_BEFORE_LOCKS で行う
必要がある。
GPU Sortを使用して、integer型のカラムをorder byに指定したクエリーを発行すると、
途中までソートされますが、その行以降ソートされていないように見えます。
先ほど口頭でも報告しました問題です。
id | score
--------+-------
19590 | 0
990241 | 0
341683 | 0
556892 | 0
382012 | 0
672646 | 0
546548 | 0
434079 | 0
556878 | 0
30494 | 0
247808 | 0
(snip)
505549 | 2
259299 | 2
53781 | 2
338375 | 2
986495 | 2
83320 | 2
733178 | 1
33509 | 0
102637 | 2
23007 | 1
874270 | 1
480436 | 1
616056 | 1
371905 | 1
460373 | 1
104010 | 0
399264 | 0
(snip)
SELECT id,score FROM member_score WHERE score < 3 ORDER BY score; Table "public.member_score"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
date | date |
score | integer |
#37 の調査中に発見しました。
tcache_create_column_store()で共有メモリからColumn Store用にメモリを確保していますが、
確保する際のサイズ(tcache.c:433)と、その直後に0で初期化する際のサイズ(tcache.c:436)が異なっていおり、
tcsのcdata以降の領域が初期化されことがあります。
直後のコードで具体的な値が代入されていくため、問題がないように見えましたが、
tcache.c:456からのfor文中のbms_is_member()がfalseを返却した場合、
cdataにゴミデータが残ったままとなってしまいます。
varchar型カラムのソートを試したところ、OpenCLコードのコンパイルエラーとなりました。
ERROR: PG-Strom: OpenCL execution error (build program failure)
#ifndef PG_BYTEA_TYPE_DEFINED
STROMCL_VARLENA_TYPE_TEMPLATE(bytea)
#endif
#ifndef PG_VARCHAR_TYPE_DEFINED
STROMCL_VARLENA_TYPE_TEMPLATE(varchar)
#endif
#ifndef PG_INT4_TYPE_DEFINED
STROMCL_SIMPLE_TYPE_TEMPLATE(int4,cl_int)
#endif
#ifndef PG_TEXT_TYPE_DEFINED
STROMCL_VARLENA_TYPE_TEMPLATE(text)
#endif
#define KPARAM_0 pg_bytea_param(kparams,errcode,0)
#define KPARAM_1 pg_bytea_param(kparams,errcode,1)
static cl_int
gpusort_comp(__private int *errcode,
__global kern_column_store *kcs_x,
__global kern_toastbuf *ktoast_x,
__private cl_int x_index,
__global kern_column_store *kcs_y,
__global kern_toastbuf *ktoast_y,
__private cl_int y_index)
{
pg_varchar_t keyval_x1;
pg_varchar_t keyval_y1;
pg_int4_t comp;
keyval_x1 =pg_varchar_vref(kcs_x,ktoast_x,errcode,0,x_index);
keyval_y1 =pg_varchar_vref(kcs_y,ktoast_y,errcode,0,y_index);
if (!keyval_x1.isnull && !keyval_y1.isnull)
{
comp = pgfn_text_cmp(errcode, keyval_x1, keyval_y1);
if (comp.value != 0)
return comp.value;
}
else if (keyval_x1.isnull && !keyval_y1.isnull)
return 1;
else if (!keyval_x1.isnull && keyval_y1.isnull)
return -1;
return 0;
}
DETAIL: Compilation started
1:2352:35: error: passing 'pg_varchar_t' to parameter of incompatible type 'pg_text_t'
1:2307:52: note: passing argument to parameter 'arg1' here
Compilation failed
SELECT * FROM varchar_test WHERE str < '!9' ORDER BY str; Table "public.varchar_test"
Column | Type | Modifiers
--------+-----------------------+-----------
id | integer |
len | smallint |
str | character varying(64) |
「explain(analyze」で実行時間を表示したが、kernelのじっこ時間が出力されない
QUERY PLAN
Custom (GpuScan) on t1 (cost=10000.00..12298.20 rows=20 width=53) (actual time=21.317..21.344 rows=6 loops=1)
Host References: id, x, y, z
Device References: x
Device Filter: (x < 0.003::double precision)
number of requests: 1
total time to load: 8us
max time to build kernel: 2us
average time in send-mq: 29us
DMA send: total length 1587.00KB, total time 647us, avg time 161us
DMA recv: total length 781.26KB, total time 122us, avg time 122us
DMA band: send 2395.38MB/sec unitsz 396.75KB, recv 6253.70MB/sec unitsz 781.26KB
average time in recv-mq: 78us
time for post device exec: 27us
Planning time: 0.232 ms
Execution time: 21.429 ms
column storeの場合のみ、num_kern_exec を加算し忘れているのではないか？
[a-hasega@saba pg_strom]$ git diff
diff --git a/gpuscan.c b/gpuscan.c
index e99329e..98d3088 100644
--- a/gpuscan.c
+++ b/gpuscan.c
@@ -2912,6 +2912,7 @@ clserv_process_gpuscan_column(pgstrom_message *msg)
gpuscan->msg.pfm.bytes_dma_recv = bytes_dma_recv;
gpuscan->msg.pfm.num_dma_send = clgsc->ev_index - 2;
gpuscan->msg.pfm.num_dma_recv = 1;
gpuscan->msg.pfm.num_kern_exec = 1;
}
GpuHashJoinなどの下位ノード(inner/outer)がGpuScanで、デバイス実行可能な条件句が
付いており、かつ、ホスト実行する条件句が付いていない場合、この条件句の評価を上位の
GpuHashJoinなどにpull-upする事ができる。
上位ノードに pull-up する事で、GpuHashJoinの評価と同時にデバイス実行可能な条件句の
評価を行う事が可能となり、DMA Send => Kernel Exec => DMA Recvのサイクルを
GpuScan側で省略できる。
最大10MBの共有メモリをallocしてfreeする処理を各クライアント並列で繰り返し実行すると、SIGSEGVでクラッシュする。
backtrace:
#0 0x00007f4e4903127c in dlist_delete (node=0x7f4dc8c702e0)
at /usr/local/nanzai/include/postgresql/server/lib/ilist.h:387
#1 0x00007f4e490312b8 in dlist_pop_head_node (head=0x7f4dc8a6a200)
at /usr/local/nanzai/include/postgresql/server/lib/ilist.h:401
#2 0x00007f4e4903157f in pgstrom_shmem_zone_block_alloc (zone=0x7f4dc8a6a000, size=2877645)
at shmem.c:201
#3 0x00007f4e490318d2 in pgstrom_shmem_alloc (size=2877645) at shmem.c:315
#4 0x00007f4e4903694c in pgstrom_shmem_alloc_func (fcinfo=0x7fffa3ea63c0) at debug.c:113
#5 0x000000000060cf10 in ExecMakeTableFunctionResult (funcexpr=0x2b4a480, econtext=0x2b4a110,
expectedDesc=0x2b4ad10, randomAccess=0 '\000') at execQual.c:2164
#6 0x000000000062bb64 in FunctionNext (node=0x2b4a000) at nodeFunctionscan.c:95
#7 0x0000000000613e06 in ExecScanFetch (node=0x2b4a000, accessMtd=0x62babb ,
recheckMtd=0x62be91 <FunctionRecheck>) at execScan.c:78
#8 0x0000000000613e7a in ExecScan (node=0x2b4a000, accessMtd=0x62babb ,
recheckMtd=0x62be91 <FunctionRecheck>) at execScan.c:128
#9 0x000000000062bec9 in ExecFunctionScan (node=0x2b4a000) at nodeFunctionscan.c:266
#10 0x000000000060953d in ExecProcNode (node=0x2b4a000) at execProcnode.c:433
#11 0x00000000006074dc in ExecutePlan (estate=0x2b49ef0, planstate=0x2b4a000,
operation=CMD_SELECT, sendTuples=1 '\001', numberTuples=1, direction=ForwardScanDirection,
dest=0xc9c480) at execMain.c:1475
#12 0x00000000006058af in standard_ExecutorRun (queryDesc=0x2b2d170,
direction=ForwardScanDirection, count=1) at execMain.c:308
#13 0x00000000006057a0 in ExecutorRun (queryDesc=0x2b2d170, direction=ForwardScanDirection,
count=1) at execMain.c:256
#14 0x000000000063c0a7 in _SPI_pquery (queryDesc=0x2b2d170, fire_triggers=1 '\001', tcount=1)
at spi.c:2375
#15 0x000000000063bbde in _SPI_execute_plan (plan=0x2b46240, paramLI=0x2b17540, snapshot=0x0,
crosscheck_snapshot=0x0, read_only=0 '\000', fire_triggers=1 '\001', tcount=1) at spi.c:2163
#16 0x0000000000638c87 in SPI_execute_plan_with_paramlist (plan=0x2b46240, params=0x2b17540,
read_only=0 '\000', tcount=1) at spi.c:449
#17 0x00007f4dc792ecd7 in exec_stmt_execsql (estate=0x7fffa3ea7030, stmt=0x2b30e48)
at pl_exec.c:3333
#18 0x00007f4dc792b2c9 in exec_stmt (estate=0x7fffa3ea7030, stmt=0x2b30e48) at pl_exec.c:1469
#19 0x00007f4dc792afff in exec_stmts (estate=0x7fffa3ea7030, stmts=0x2b30c18) at pl_exec.c:1368
#20 0x00007f4dc792bcf4 in exec_stmt_while (estate=0x7fffa3ea7030, stmt=0x2b313e8) at pl_exec.c:1839
#21 0x00007f4dc792b1bb in exec_stmt (estate=0x7fffa3ea7030, stmt=0x2b313e8) at pl_exec.c:1429
#22 0x00007f4dc792afff in exec_stmts (estate=0x7fffa3ea7030, stmts=0x2b31438) at pl_exec.c:1368
#23 0x00007f4dc792aeaa in exec_stmt_block (estate=0x7fffa3ea7030, block=0x2b314b8)
at pl_exec.c:1306
#24 0x00007f4dc79290d2 in plpgsql_exec_function (func=0x2b08f40, fcinfo=0x2a98690,
simple_eval_estate=0x0) at pl_exec.c:344
#25 0x00007f4dc7923a53 in plpgsql_call_handler (fcinfo=0x2a98690) at pl_handler.c:243
#26 0x000000000060c6fd in ExecMakeFunctionResult (fcache=0x2a98620, econtext=0x2a98430,
isNull=0x2a99028 "", isDone=0x2a99140) at execQual.c:1795
#27 0x000000000060d484 in ExecEvalFunc (fcache=0x2a98620, econtext=0x2a98430, isNull=0x2a99028 "",
isDone=0x2a99140) at execQual.c:2380
#28 0x0000000000613687 in ExecTargetList (targetlist=0x2a99110, econtext=0x2a98430,
values=0x2a99010, isnull=0x2a99028 "", itemIsDone=0x2a99140, isDone=0x7fffa3ea7514)
at execQual.c:5267
#29 0x0000000000613c63 in ExecProject (projInfo=0x2a99040, isDone=0x7fffa3ea7514)
at execQual.c:5482
#30 0x000000000062d006 in ExecResult (node=0x2a98320) at nodeResult.c:155
#31 0x0000000000609456 in ExecProcNode (node=0x2a98320) at execProcnode.c:380
#32 0x00000000006074dc in ExecutePlan (estate=0x2a98210, planstate=0x2a98320,
operation=CMD_SELECT, sendTuples=1 '\001', numberTuples=0, direction=ForwardScanDirection,
dest=0x2adea18) at execMain.c:1475
#33 0x00000000006058af in standard_ExecutorRun (queryDesc=0x2aa4570,
direction=ForwardScanDirection, count=0) at execMain.c:308
#34 0x00000000006057a0 in ExecutorRun (queryDesc=0x2aa4570, direction=ForwardScanDirection,
count=0) at execMain.c:256
#35 0x0000000000756390 in PortalRunSelect (portal=0x2aa2560, forward=1 '\001', count=0,
dest=0x2adea18) at pquery.c:946
#36 0x0000000000756068 in PortalRun (portal=0x2aa2560, count=9223372036854775807,
isTopLevel=1 '\001', dest=0x2adea18, altdest=0x2adea18, completionTag=0x7fffa3ea7860 "")
at pquery.c:790
#37 0x000000000075047d in exec_simple_query (query_string=0x2add0d0 "select shmem_open_loop(1);")
at postgres.c:1045
#38 0x0000000000754482 in PostgresMain (argc=1, argv=0x2a7ee68, dbname=0x2a7ed98 "nanzai",
username=0x2a7ed80 "nanzai") at postgres.c:4004
#39 0x00000000006e7f85 in BackendRun (port=0x2aa6540) at postmaster.c:4104
#40 0x00000000006e76e0 in BackendStartup (port=0x2aa6540) at postmaster.c:3778
#41 0x00000000006e4106 in ServerLoop () at postmaster.c:1569
#42 0x00000000006e3885 in PostmasterMain (argc=1, argv=0x2a7dde0) at postmaster.c:1222
#43 0x000000000064d0d7 in main (argc=1, argv=0x2a7dde0) at main.c:223
tcacheに載ったデータに対してSQLを多重実行すると、エラーが発生することがあります。
現在のところ、bytea型のデータに対する操作で発生します。
ERROR: StromObject 0x7fa5f3c22018 (tag=1006) is not tracked
SELECT * FROM bytea_test WHERE bin BETWEEN '\xff00'::bytea AND '\xffff'::bytea ORDER BY id;これを、pgbenchで多重実行。3多重程度から発生する。
Table "public.bytea_test"
Column | Type | Modifiers
--------+--------+-----------
id | bigint | not null
bin | bytea |
Indexes:
"bytea_test_pkey" PRIMARY KEY, btree (id)
pgbenchにて、全件scanとなるクエリーを6多重にて行ったところ、pgstrom_shmem_free()でAssertionFailedとなった。
正常に処理が終了することもあるが、感覚的には、かなり高い確率で再現する。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (
conditionName=0x7fadd9f0b978 "!(*((cl_uint *)((uintptr_t)address + block->blocksz)) == 0xdeadbeaf)",
errorType=0x7fadd9f0b6b4 "FailedAssertion", fileName=0x7fadd9f0b6dd "shmem.c", lineNumber=247) at assert.c:54
#3 0x00007fadd9ef4d55 in pgstrom_shmem_zone_block_free (zone=0x7fad59834000, address=0x7fadd1c54000) at shmem.c:246
#4 0x00007fadd9ef5290 in pgstrom_shmem_free (address=0x7fadd1c54000) at shmem.c:362
#5 0x00007fadd9f0a2f5 in pgstrom_get_devprog_key (
source=0x1323518 "STROMCL_SIMPLE_TYPE_TEMPLATE(float8,cl_double)\n\nstatic pg_bool_t pgfn_float8lt(pg_float8_t arg1, pg_float8_t arg2)\n{\n pg_bool_t result;\n result.value = (cl_bool)(arg1.value < arg2.value);\n re"..., extra_flags=512)
at opencl_devprog.c:522
#6 0x00007fadd9f00822 in gpuscan_begin (node=0x12e3660, estate=0x1333e78, eflags=16) at gpuscan.c:607
#7 0x000000000068072e in ExecInitCustomPlan (custom_plan=0x12e3660, estate=0x1333e78, eflags=16) at nodeCustom.c:26
#8 0x0000000000668023 in ExecInitNode (node=0x12e3660, estate=0x1333e78, eflags=16) at execProcnode.c:250
#9 0x000000000067e463 in ExecInitAgg (node=0x1321fe0, estate=0x1333e78, eflags=16) at nodeAgg.c:1578
#10 0x00000000006680fc in ExecInitNode (node=0x1321fe0, estate=0x1333e78, eflags=16) at execProcnode.c:291
#11 0x0000000000664cf4 in InitPlan (queryDesc=0x13190d8, eflags=16) at execMain.c:887
#12 0x0000000000663b5b in standard_ExecutorStart (queryDesc=0x13190d8, eflags=16) at execMain.c:208
#13 0x0000000000663926 in ExecutorStart (queryDesc=0x13190d8, eflags=0) at execMain.c:122
#14 0x00000000007f8b89 in PortalStart (portal=0x1331e68, params=0x0, eflags=0, snapshot=0x0) at pquery.c:533
#15 0x00000000007f28d6 in exec_simple_query (query_string=0x12e1e48 "SELECT count(*) FROM oku_table WHERE data < 0.01;")
at postgres.c:1006
#16 0x00000000007f7162 in PostgresMain (argc=1, argv=0x1273fe0, dbname=0x1273ed8 "pgstrom_test", username=0x1273eb8 "tkondo")
at postgres.c:4004
#17 0x000000000076e18a in BackendRun (port=0x129bb10) at postmaster.c:4104
#18 0x000000000076d840 in BackendStartup (port=0x129bb10) at postmaster.c:3778
#19 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#20 0x0000000000769251 in PostmasterMain (argc=1, argv=0x1272e70) at postmaster.c:1222
#21 0x00000000006b8aee in main (argc=1, argv=0x1272e70) at main.c:223
SELECT count(*) FROM oku_table WHERE data < 0.01;varchar(10) など、文字列長に上限が設定されており、かつそれが十分に小さい時は、
（現在の）可変長バッファ + ポインタの組でデータを表現するのではなく、固定長の
データとして処理する方がDMA回数を減らすことができ、パフォーマンスを上げられる。
（DWHではよくあるパターンのデータ構造）
textタイプのカラムにNULL値を含むテーブルを検索したところ、Segmentation Faultが発生しました。
#0 0x00007f593f8f972d in do_insert_tuple (tc_head=0x7f5937682018, tc_node=0x7f5937682278, tuple=0x254ec98) at tcache.c:2003
#1 0x00007f593f8fa454 in tcache_insert_tuple (tc_head=0x7f5937682018, tc_node=0x7f5937682278, tuple=0x254ec98) at tcache.c:2090
#2 0x00007f593f8fa76a in tcache_build_main (tc_scan=0x2562188) at tcache.c:2170
#3 0x00007f593f8fac34 in tcache_scan_next (tc_scan=0x2562188) at tcache.c:2302
#4 0x00007f593f90262b in pgstrom_load_gpuscan (gss=0x2560e98) at gpuscan.c:874
#5 0x00007f593f90363e in gpuscan_fetch_tuple (node=0x2560e98) at gpuscan.c:1175
#6 0x00007f593f903c80 in gpuscan_exec (node=0x2560e98) at gpuscan.c:1331
#7 0x00000000006807e7 in ExecCustomPlan (cpstate=0x2560e98) at nodeCustom.c:37
#8 0x00000000006684b5 in ExecProcNode (node=0x2560e98) at execProcnode.c:453
#9 0x000000000067dab4 in agg_retrieve_direct (aggstate=0x2560700) at nodeAgg.c:1164
#10 0x000000000067d9cb in ExecAgg (node=0x2560700) at nodeAgg.c:1117
#11 0x0000000000668548 in ExecProcNode (node=0x2560700) at execProcnode.c:487
#12 0x0000000000665efe in ExecutePlan (estate=0x25605e8, planstate=0x2560700, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x24c8590) at execMain.c:1475
#13 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x2553d18, direction=ForwardScanDirection, count=0) at execMain.c:308
#14 0x0000000000663beb in ExecutorRun (queryDesc=0x2553d18, direction=ForwardScanDirection, count=0) at execMain.c:256
#15 0x00000000007f964e in PortalRunSelect (portal=0x25781b8, forward=1 '\001', count=0, dest=0x24c8590) at pquery.c:946
#16 0x00000000007f9275 in PortalRun (portal=0x25781b8, count=9223372036854775807, isTopLevel=1 '\001', dest=0x24c8590,
altdest=0x24c8590, completionTag=0x7fffb2d12eb0 "") at pquery.c:790
#17 0x00000000007f29e2 in exec_simple_query (query_string=0x2510778 "select count(*) from text_test_null where str like 'A%';")
at postgres.c:1045
#18 0x00000000007f7162 in PostgresMain (argc=1, argv=0x24a45c8, dbname=0x24a4428 "pgstrom_test", username=0x24a4408 "tkondo")
at postgres.c:4004
#19 0x000000000076e18a in BackendRun (port=0x24cc2f0) at postmaster.c:4104
#20 0x000000000076d840 in BackendStartup (port=0x24cc2f0) at postmaster.c:3778
#21 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#22 0x0000000000769251 in PostmasterMain (argc=1, argv=0x24a33c0) at postmaster.c:1222
#23 0x00000000006b8aee in main (argc=1, argv=0x24a33c0) at main.c:223
pgstrom_test=# \d text_test_null
Table "public.text_test_null"
Column | Type | Modifiers
--------+----------+-----------
id | integer |
len | smallint |
str | text |
select count(*) from text_test_null where str like 'A%';データ型がnameのカラムを含むテーブルを検索・出力すると、name型のカラムに意味不明な文字列が出力されます。
(ただ、name型は非推奨の型なので、優先して対応する必要はないとは思います。)
SELECT * FROM member_pref_name;pgstrom_test=# \d member_pref_name
Table "public.member_pref_name"
Column | Type | Modifiers
-----------+---------+-----------
pref | integer |
pref_name | name |
pref | pref_name
------+---------------
0 | |\x05F\x7F
1 | \x1C\x05F\x7F
2 | \x04F\x7F
3 | \\x04F\x7F
4 | \x03F\x7F
5 | \x03F\x7F
6 | <\x03F\x7F
7 | \x02F\x7F
8 | |\x02F\x7F
9 | \x1C\x02F\x7F
10 | \x01F\x7F
11 | \\x01F\x7F
12 |
13 |
14 | <
15 | F\x7F
16 | |F\x7F
17 | \x1CF\x7F
16 | F\x7F
19 | \F\x7F
20 | F\x7F
21 | F\x7F
22 | <F\x7F
23 | F\x7F
24 | |F\x7F
25 | \x1CF\x7F
26 | F\x7F
27 | \F\x7F
28 | F\x7F
29 | F\x7F
30 | <F\x7F
31 | F\x7F
32 | |F\x7F
33 | \x1CF\x7F
34 | F\x7F
35 | \F\x7F
37 | F\x7F
38 | F\x7F
39 | <F\x7F
40 | F\x7F
41 | |F\x7F
42 | \x1CF\x7F
43 | F\x7F
44 | \F\x7F
45 | F\x7F
46 | F\x7F
47 | <F\x7F
48 | F\x7F
#22 からの分離です。
最新のコードで以下のSQL文をnVidia環境にて実行したところ、Assertion Failedとなりました。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc3408d in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (conditionName=0x7fab3123c2c1 "!(rc == 0)",
errorType=0x7fab3123b884 "FailedAssertion", fileName=0x7fab3123b8ad "gpuscan.c", lineNumber=2269) at assert.c:54
#3 0x00007fab31224138 in clserv_respond_gpuscan_column (event=0x7faa84bc0e80, ev_status=0, private=0x7faa84845930) at gpuscan.c:2269
#4 0x00007faaa6eeb283 in ?? () from /usr/lib64/libnvidia-opencl.so.1
#5 0x00007faaa6eefa73 in ?? () from /usr/lib64/libnvidia-opencl.so.1
#6 0x00007faaa6ef01e4 in ?? () from /usr/lib64/libnvidia-opencl.so.1
#7 0x00007faaa767bf89 in ?? () from /usr/lib64/libnvidia-opencl.so.1
#8 0x0000003cdc0079d1 in start_thread () from /lib64/libpthread.so.0
#9 0x0000003cdbce8b6d in clone () from /lib64/libc.so.6
(gdb) p rc
$1 = -9999
SELECT * FROM temperature_tokyo WHERE high - low >= 15 AND high_time < low_time; Table "public.temperature_tokyo"
Column | Type | Modifiers
-----------+-----------------------------+-----------
date | date |
high | real |
high_time | timestamp without time zone |
low | real |
low_time | timestamp without time zone |
#6 の再現テスト中に、#6 とは異なる箇所でAssertionFailedとなることがありました。
こちらは、#6 と比較すると、発生確率がかなり低いと感じます。
※この箇所は、ソースコード上のコメントにTODOと書いてある箇所になりますが、一応報告いたします。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (conditionName=0x7f33469ebc48 "!(rs_index > 0 && rs_index <= rstore->kern.nrows)",
errorType=0x7f33469eb534 "FailedAssertion", fileName=0x7f33469eb55d "gpuscan.c", lineNumber=793) at assert.c:54
#3 0x00007f33469de111 in gpuscan_next_tuple_row (gss=0x17d6718, slot=0x17d7588) at gpuscan.c:793
#4 0x00007f33469de6e5 in gpuscan_exec (node=0x17d6718) at gpuscan.c:810
#5 0x00000000006807e7 in ExecCustomPlan (cpstate=0x17d6718) at nodeCustom.c:37
#6 0x00000000006684b5 in ExecProcNode (node=0x17d6718) at execProcnode.c:453
#7 0x000000000067dbda in agg_retrieve_direct (aggstate=0x17d5f80) at nodeAgg.c:1230
#8 0x000000000067d9cb in ExecAgg (node=0x17d5f80) at nodeAgg.c:1117
#9 0x0000000000668548 in ExecProcNode (node=0x17d5f80) at execProcnode.c:487
#10 0x0000000000665efe in ExecutePlan (estate=0x17d5e68, planstate=0x17d5f80, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x17cc008) at execMain.c:1475
#11 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x17bb0c8, direction=ForwardScanDirection, count=0) at execMain.c:308
#12 0x0000000000663beb in ExecutorRun (queryDesc=0x17bb0c8, direction=ForwardScanDirection, count=0) at execMain.c:256
#13 0x00000000007f964e in PortalRunSelect (portal=0x17d3e58, forward=1 '\001', count=0, dest=0x17cc008) at pquery.c:946
#14 0x00000000007f9275 in PortalRun (portal=0x17d3e58, count=9223372036854775807, isTopLevel=1 '\001', dest=0x17cc008,
altdest=0x17cc008, completionTag=0x7fff39e5d440 "") at pquery.c:790
#15 0x00000000007f29e2 in exec_simple_query (query_string=0x1783e38 "SELECT count(*) FROM oku_table WHERE data < 0.01;")
at postgres.c:1045
#16 0x00000000007f7162 in PostgresMain (argc=1, argv=0x1715fd0, dbname=0x1715ec8 "pgstrom_test", username=0x1715ea8 "tkondo")
at postgres.c:4004
#17 0x000000000076e18a in BackendRun (port=0x173db00) at postmaster.c:4104
#18 0x000000000076d840 in BackendStartup (port=0x173db00) at postmaster.c:3778
#19 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#20 0x0000000000769251 in PostmasterMain (argc=1, argv=0x1714e60) at postmaster.c:1222
#21 0x00000000006b8aee in main (argc=1, argv=0x1714e60) at main.c:223
(gdb) up
#3 0x00007f33469de111 in gpuscan_next_tuple_row (gss=0x17d6718, slot=0x17d7588) at gpuscan.c:793
793 Assert(rs_index > 0 && rs_index <= rstore->kern.nrows);
(gdb) p rs_index
$1 = 0
boolean型のカラムをtext型にキャストし、そのカラムをorder byに指定したSQLを実行したところ、
エラーが表示されました。
INFO: now building tcache...
INFO: tcache build done...
ERROR: Bug? response of OpenCL server too late
SELECT id, pass::text FROM member_score_pass ORDER BY pass;テーブル "public.member_score_pass"
列 | 型 | 修飾語
-----------+---------+----------
id | integer | not null
max_score | integer |
pass | boolean |
インデックス:
"member_score_pass_pkey" PRIMARY KEY, btree (id)
QUERY PLAN
---------------------------------------------------------------------------------------------
Custom (GpuSort) (cost=29002.58..31502.58 rows=1000000 width=5)
Sort keys: ((pass)::text)
Sort keys width: 36
Rows per chunk: 1000000
-> Custom (GpuScan) on member_score_pass (cost=10000.00..28006.00 rows=1000000 width=5)
Host References: id, pass
Device References:
Planning time: 0.205 ms
#7 の再現確認を行ったところ、元々の問題とは別の場所でSegmentation Faultとなりました。
pgbenchによる、6多重での実行で、まれに発生します。
#0 0x00007fa99bb69739 in do_insert_tuple (tc_head=0x7fa9938e4018, tc_node=0x7fa9938e4200, tuple=0x22a3f38) at tcache.c:1936
#1 0x00007fa99bb6a958 in tcache_insert_tuple (tc_head=0x7fa9938e4018, tc_node=0x7fa9938e4200, tuple=0x22a3f38) at tcache.c:2110
#2 0x00007fa99bb6aa77 in tcache_build_main (tc_scan=0x22a19a0) at tcache.c:2149
#3 0x00007fa99bb6af41 in tcache_scan_next (tc_scan=0x22a19a0) at tcache.c:2281
#4 0x00007fa99bb72bd0 in pgstrom_load_gpuscan (gss=0x22a0788) at gpuscan.c:953
#5 0x00007fa99bb73ce4 in gpuscan_fetch_tuple (node=0x22a0788) at gpuscan.c:1278
#6 0x00007fa99bb74163 in gpuscan_exec (node=0x22a0788) at gpuscan.c:1373
#7 0x00000000006807e7 in ExecCustomPlan (cpstate=0x22a0788) at nodeCustom.c:37
#8 0x00000000006684b5 in ExecProcNode (node=0x22a0788) at execProcnode.c:453
#9 0x000000000067dab4 in agg_retrieve_direct (aggstate=0x229fff0) at nodeAgg.c:1164
#10 0x000000000067d9cb in ExecAgg (node=0x229fff0) at nodeAgg.c:1117
#11 0x0000000000668548 in ExecProcNode (node=0x229fff0) at execProcnode.c:487
#12 0x0000000000665efe in ExecutePlan (estate=0x229fed8, planstate=0x229fff0, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x2297558) at execMain.c:1475
#13 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x2286928, direction=ForwardScanDirection, count=0) at execMain.c:308
#14 0x0000000000663beb in ExecutorRun (queryDesc=0x2286928, direction=ForwardScanDirection, count=0) at execMain.c:256
#15 0x00000000007f964e in PortalRunSelect (portal=0x229dec8, forward=1 '\001', count=0, dest=0x2297558) at pquery.c:946
#16 0x00000000007f9275 in PortalRun (portal=0x229dec8, count=9223372036854775807, isTopLevel=1 '\001', dest=0x2297558,
altdest=0x2297558, completionTag=0x7fff5eeef820 "") at pquery.c:790
#17 0x00000000007f29e2 in exec_simple_query (query_string=0x224f868 "SELECT count(*) FROM oku_table WHERE data < 0.01;")
at postgres.c:1045
#18 0x00000000007f7162 in PostgresMain (argc=1, argv=0x21e1520, dbname=0x21e1418 "pgstrom_test", username=0x21e13f8 "tkondo")
at postgres.c:4004
#19 0x000000000076e18a in BackendRun (port=0x2209420) at postmaster.c:4104
#20 0x000000000076d840 in BackendStartup (port=0x2209420) at postmaster.c:3778
#21 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#22 0x0000000000769251 in PostmasterMain (argc=1, argv=0x21e03b0) at postmaster.c:1222
#23 0x00000000006b8aee in main (argc=1, argv=0x21e03b0) at main.c:223
SELECT count(*) FROM oku_table WHERE data < 0.01;(gdb) p/x tcs
$1 = 0x0
(gdb) p *tc_head
$2 = {sobj = {stag = StromTag_TCacheHead}, chain = {prev = 0x0, next = 0x0}, lru_chain = {prev = 0x0, next = 0x0}, pending_chain = {
prev = 0x0, next = 0x0}, refcnt = 0, locktag = {locktag_field1 = 16384, locktag_field2 = 475770, locktag_field3 = 1024,
locktag_field4 = 0, locktag_type = 1 '\001', locktag_lockmethodid = 1 '\001'}, tcs_root = 0x7fa9938e4200, lock = 0 '\000',
is_ready = 0 '\000', free_list = {head = {prev = 0x7fa9938e5f68, next = 0x7fa9938e4208}}, block_list = {head = {
prev = 0x7fa9938e4088, next = 0x7fa9938e4088}}, pending_list = {head = {prev = 0x7fa9938e4098, next = 0x7fa9938e4098}},
trs_list = {head = {prev = 0x7fa9938e40a8, next = 0x7fa9938e40a8}}, trs_curr = 0x0, datoid = 16384, reloid = 475770, ncols = 1,
i_cached = 0x7fa9938e40e0, tupdesc = 0x7fa9938e40e8, data = 0x7fa9938e4018 "\354\003"}
(gdb) p *tc_node
$3 = {sobj = {stag = 0}, chain = {prev = 0x7fa9938e4078, next = 0x7fa9938e4258}, tv = {tv_sec = 0, tv_usec = 0}, right = 0x0,
left = 0x0, r_depth = 0, l_depth = 0, lock = 0 '\000', tcs = 0x0}
(gdb) p *tuple
$4 = {t_len = 40, t_self = {ip_blkid = {bi_hi = 7, bi_lo = 43886}, ip_posid = 34}, t_tableOid = 475770, t_data = 0x7fa914084070}
この時のCoreは、以下にbzip2圧縮して置いてありますので、解析にご利用ください。
/home/tkondo/coding/postgres/pg_strom_test/core/core.502.2014-05-15_14-54.bz2
#29 からの分割です。
NOT NULL制約があるカラムをORDER BYに指定すると、GpuScanとGpuSortで実行される場合にエラーが表示されます。
ERROR: Bug? processed gpusort chunk(s) are not sorted
SELECT * FROM bytea_test WHERE bin BETWEEN '\xff00'::bytea AND '\xffff'::bytea ORDER BY id; Table "public.bytea_test"
Column | Type | Modifiers
--------+--------+-----------
id | bigint | not null
bin | bytea |
Indexes:
"bytea_test_pkey" PRIMARY KEY, btree (id)
pgstrom_test=# explain SELECT * FROM bytea_test WHERE bin BETWEEN '\xff00'::bytea AND '\xffff'::bytea ORDER BY id;
INFO: now ready to run bulk-load
QUERY PLAN
--------------------------------------------------------------------------------------
Custom (GpuSort) (cost=129659.52..129707.06 rows=19016 width=40)
Sort keys: id
Sort keys width: 8
Rows per chunk: 21408
-> Custom (GpuScan) on bytea_test (cost=10000.00..129646.00 rows=19016 width=40)
Filter: ((bin >= '\xff00'::bytea) AND (bin <= '\xffff'::bytea))
Host References: id, bin
Device References:
Filter: ((bin >= '\xff00'::bytea) AND (bin <= '\xffff'::bytea))
Planning time: 1.728 ms
boolean型のカラムをtext型にキャストし、そのカラムをgroup byとorder byに同時に指定したSQLを実行したところ、
Assertion Failedとなりました。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000948a59 in ExceptionalCondition (conditionName=0x7f0bf4e51b30 "!(gsortstate->scan_done)",
errorType=0x7f0bf4e510e4 "FailedAssertion", fileName=0x7f0bf4e5113f "gpusort.c", lineNumber=1856) at assert.c:54
#3 0x00007f0bf4e39b03 in gpusort_exec (node=0x16c3590) at gpusort.c:1856
#4 0x0000000000680dcb in ExecCustomPlan (cpstate=0x16c3590) at nodeCustom.c:37
#5 0x0000000000668ae5 in ExecProcNode (node=0x16c3590) at execProcnode.c:453
#6 0x000000000066652c in ExecutePlan (estate=0x16c3478, planstate=0x16c3590, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x16a8b50) at execMain.c:1475
#7 0x00000000006643dc in standard_ExecutorRun (queryDesc=0x161f758, direction=ForwardScanDirection, count=0) at execMain.c:308
#8 0x000000000066421f in ExecutorRun (queryDesc=0x161f758, direction=ForwardScanDirection, count=0) at execMain.c:256
#9 0x00000000007f9e7e in PortalRunSelect (portal=0x16d6408, forward=1 '\001', count=0, dest=0x16a8b50) at pquery.c:946
#10 0x00000000007f9aa5 in PortalRun (portal=0x16d6408, count=9223372036854775807, isTopLevel=1 '\001', dest=0x16a8b50,
altdest=0x16a8b50, completionTag=0x7fffd1307930 "") at pquery.c:790
#11 0x00000000007f3212 in exec_simple_query (
query_string=0x1665918 "select pass::text, count(*) from member_score_pass group by pass order by pass;") at postgres.c:1045
#12 0x00000000007f7992 in PostgresMain (argc=1, argv=0x15f6638, dbname=0x15f6498 "pgstrom_test", username=0x15f6478 "tkondo")
at postgres.c:4004
#13 0x000000000076e8e9 in BackendRun (port=0x1621010) at postmaster.c:4117
#14 0x000000000076df9f in BackendStartup (port=0x1621010) at postmaster.c:3791
#15 0x000000000076a366 in ServerLoop () at postmaster.c:1570
#16 0x00000000007699b9 in PostmasterMain (argc=1, argv=0x15f5430) at postmaster.c:1223
#17 0x00000000006b9166 in main (argc=1, argv=0x15f5430) at main.c:227
SELECT pass::text, count(*) FROM member_score_pass GROUP BY pass ORDER BY pass;テーブル "public.member_score_pass"
列 | 型 | 修飾語
-----------+---------+----------
id | integer | not null
max_score | integer |
pass | boolean |
インデックス:
"member_score_pass_pkey" PRIMARY KEY, btree (id)
QUERY PLAN
---------------------------------------------------------------------------------------------------
Custom (GpuSort) (cost=30506.03..30506.03 rows=2 width=1)
Sort keys: ((pass)::text)
Sort keys width: 36
Rows per chunk: 0
-> HashAggregate (cost=30506.00..30506.03 rows=2 width=1)
Group Key: pass
-> Custom (GpuScan) on member_score_pass (cost=10000.00..25506.00 rows=1000000 width=1)
Host References: pass
Device References:
Planning time: 0.431 ms
最新のコードで、char型のカラムをORDER BYに指定したSQLを実行したところ、
メモリ不足で実行が中断されました。
SELECT * FROM clock_md5 WHERE id BETWEEN 100 AND 200 ORDER BY time_md5; テーブル "public.clock_md5"
列 | 型 | 修飾語
----------+---------------+----------
id | integer | not null
time_md5 | character(32) | not null
INFO: now ready to run bulk-load
QUERY PLAN
--------------------------------------------------------------------------------
Custom (GpuSort) (cost=94833.99..94833.99 rows=1 width=37)
Sort keys: time_md5
Sort keys width: 140
Rows per chunk: 32
-> Custom (GpuScan) on clock_md5 (cost=10000.00..94833.99 rows=1 width=37)
Host References: id, time_md5
Device References: id
Device Filter: ((id >= 100) AND (id <= 200))
Planning time: 0.976 ms
(9 行)
QUERY PLAN
---------------------------------------------------------------------
Sort (cost=233332.93..233332.93 rows=1 width=37)
Sort Key: time_md5
-> Seq Scan on clock_md5 (cost=0.00..233332.92 rows=1 width=37)
Filter: ((id >= 100) AND (id <= 200))
Planning time: 0.145 ms
(5 行)
#15 と似た試験になりますが、pgbenchによる多重実行をシェルスクリプトでループさせて走らせていたところ、発生しました。
pgbenchで、2多重での実行で、高確率で発生します。(まれに、発生しないこともあります)
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (conditionName=0x7f34909a754e "!(nremain > 0 && nmoved > 0)",
errorType=0x7f34909a6f44 "FailedAssertion", fileName=0x7f34909a6fa9 "tcache.c", lineNumber=1579) at assert.c:54
#3 0x00007f349098afbd in tcache_split_tcnode (tc_head=0x7f3488702018, tc_node_old=0x7f3488702200) at tcache.c:1579
#4 0x00007f349098d96d in tcache_insert_tuple (tc_head=0x7f3488702018, tc_node=0x7f3488702200, tuple=0x16c1e08) at tcache.c:2118
#5 0x00007f349098d7f5 in tcache_insert_tuple (tc_head=0x7f3488702018, tc_node=0x7f34887065c8, tuple=0x16c1e08) at tcache.c:2086
#6 0x00007f349098d7f5 in tcache_insert_tuple (tc_head=0x7f3488702018, tc_node=0x7f3488706528, tuple=0x16c1e08) at tcache.c:2086
#7 0x00007f349098d7f5 in tcache_insert_tuple (tc_head=0x7f3488702018, tc_node=0x7f34887062a8, tuple=0x16c1e08) at tcache.c:2086
#8 0x00007f349098d7f5 in tcache_insert_tuple (tc_head=0x7f3488702018, tc_node=0x7f3488706028, tuple=0x16c1e08) at tcache.c:2086
#9 0x00007f349098d7f5 in tcache_insert_tuple (tc_head=0x7f3488702018, tc_node=0x7f34887035b0, tuple=0x16c1e08) at tcache.c:2086
#10 0x00007f349098da77 in tcache_build_main (tc_scan=0x16bf870) at tcache.c:2149
#11 0x00007f349098df41 in tcache_scan_next (tc_scan=0x16bf870) at tcache.c:2281
#12 0x00007f3490995bd0 in pgstrom_load_gpuscan (gss=0x16be658) at gpuscan.c:953
#13 0x00007f3490996ce4 in gpuscan_fetch_tuple (node=0x16be658) at gpuscan.c:1278
#14 0x00007f3490997163 in gpuscan_exec (node=0x16be658) at gpuscan.c:1373
#15 0x00000000006807e7 in ExecCustomPlan (cpstate=0x16be658) at nodeCustom.c:37
#16 0x00000000006684b5 in ExecProcNode (node=0x16be658) at execProcnode.c:453
#17 0x000000000067dab4 in agg_retrieve_direct (aggstate=0x16bdec0) at nodeAgg.c:1164
#18 0x000000000067d9cb in ExecAgg (node=0x16bdec0) at nodeAgg.c:1117
#19 0x0000000000668548 in ExecProcNode (node=0x16bdec0) at execProcnode.c:487
#20 0x0000000000665efe in ExecutePlan (estate=0x16bdda8, planstate=0x16bdec0, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x16b5448) at execMain.c:1475
#21 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x16a47f8, direction=ForwardScanDirection, count=0) at execMain.c:308
#22 0x0000000000663beb in ExecutorRun (queryDesc=0x16a47f8, direction=ForwardScanDirection, count=0) at execMain.c:256
#23 0x00000000007f964e in PortalRunSelect (portal=0x16bbd98, forward=1 '\001', count=0, dest=0x16b5448) at pquery.c:946
#24 0x00000000007f9275 in PortalRun (portal=0x16bbd98, count=9223372036854775807, isTopLevel=1 '\001', dest=0x16b5448,
altdest=0x16b5448, completionTag=0x7fff14bd5a40 "") at pquery.c:790
#25 0x00000000007f29e2 in exec_simple_query (query_string=0x166d568 "SELECT count(*) FROM million_table_i8f8 WHERE data < 0.01;")
at postgres.c:1045
#26 0x00000000007f7162 in PostgresMain (argc=1, argv=0x15ff520, dbname=0x15ff418 "pgstrom_test", username=0x15ff3f8 "tkondo")
at postgres.c:4004
#27 0x000000000076e18a in BackendRun (port=0x1627230) at postmaster.c:4104
#28 0x000000000076d840 in BackendStartup (port=0x1627230) at postmaster.c:3778
#29 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#30 0x0000000000769251 in PostmasterMain (argc=1, argv=0x15fe3b0) at postmaster.c:1222
#31 0x00000000006b8aee in main (argc=1, argv=0x15fe3b0) at main.c:223
SELECT count(*) FROM million_table_i8f8 WHERE data < 0.01; Table "public.million_table_i8f8"
Column | Type | Modifiers
--------+------------------+-----------
id | bigint |
data | double precision |
※ million_table ( #10 で使用したもの)をCOPYでファイルに出力し、million_table_i8f8に入力したものです。
#!/bin/bash
RES=0;
COUNT=1;
while [ $RES -eq 0 ];
do
echo -n Try count $COUNT Start at ;
date;
pgbench $@;
RES=$?;
COUNT=`expr $COUNT + 1`;
done;この時のCoreは、以下にbzip2圧縮して置いてありますので、解析にご利用ください。
/home/tkondo/coding/postgres/pg_strom_test/core/core.502.2014-05-15_18-12.bz2
#27 の再現確認の際に発生しました。
GpuScanとGpuSortが行われるSQLで、SIGSEGVが発生します。
SeqScanなど、GpuScan以外のPlanになると発生しません。
#0 0x0000000000669c62 in ExecEvalScalarVar (exprstate=0x116c7a0, econtext=0x116bd28, isNull=0x116c338 "", isDone=0x0)
at execQual.c:627
#1 0x000000000066c6a1 in ExecMakeFunctionResultNoSets (fcache=0x116bf88, econtext=0x116bd28, isNull=0x7ffffc57cb06 "", isDone=0x0)
at execQual.c:1961
#2 0x000000000066d24e in ExecEvalOper (fcache=0x116bf88, econtext=0x116bd28, isNull=0x7ffffc57cb06 "", isDone=0x0) at execQual.c:2406
#3 0x0000000000673b75 in ExecQual (qual=0x116c8c0, econtext=0x116bd28, resultForNull=0 '\000') at execQual.c:5155
#4 0x00007fae48d2342c in gpuscan_exec_multi (node=0x116bb10) at gpuscan.c:1611
#5 0x0000000000680e46 in MultiExecCustomPlan (cpstate=0x116bb10) at nodeCustom.c:44
#6 0x0000000000668d31 in MultiExecProcNode (node=0x116bb10) at execProcnode.c:573
#7 0x00007fae48d29c19 in gpusort_preload_subplan_bulk (gsortstate=0x116b740, overflow=0x7ffffc57ce20) at gpusort.c:1349
#8 0x00007fae48d2b487 in gpusort_exec (node=0x116b740) at gpusort.c:1800
#9 0x0000000000680dcb in ExecCustomPlan (cpstate=0x116b740) at nodeCustom.c:37
#10 0x0000000000668ae5 in ExecProcNode (node=0x116b740) at execProcnode.c:453
#11 0x000000000066652c in ExecutePlan (estate=0x116b628, planstate=0x116b740, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x1173220) at execMain.c:1475
#12 0x00000000006643dc in standard_ExecutorRun (queryDesc=0x10cd948, direction=ForwardScanDirection, count=0) at execMain.c:308
#13 0x000000000066421f in ExecutorRun (queryDesc=0x10cd948, direction=ForwardScanDirection, count=0) at execMain.c:256
#14 0x00000000007f9e7e in PortalRunSelect (portal=0x10c9f08, forward=1 '\001', count=0, dest=0x1173220) at pquery.c:946
#15 0x00000000007f9aa5 in PortalRun (portal=0x10c9f08, count=9223372036854775807, isTopLevel=1 '\001', dest=0x1173220,
altdest=0x1173220, completionTag=0x7ffffc57d290 "") at pquery.c:790
#16 0x00000000007f3212 in exec_simple_query (
query_string=0x11143d8 "select * from bytea_test where bin between '\\xff00'::bytea and '\\xffff'::bytea order by id;")
at postgres.c:1045
#17 0x00000000007f7992 in PostgresMain (argc=1, argv=0x10a4a60, dbname=0x10a48c0 "pgstrom_test", username=0x10a48a0 "tkondo")
at postgres.c:4004
#18 0x000000000076e8e9 in BackendRun (port=0x10cf750) at postmaster.c:4117
#19 0x000000000076df9f in BackendStartup (port=0x10cf750) at postmaster.c:3791
#20 0x000000000076a366 in ServerLoop () at postmaster.c:1570
#21 0x00000000007699b9 in PostmasterMain (argc=1, argv=0x10a3430) at postmaster.c:1223
#22 0x00000000006b9166 in main (argc=1, argv=0x10a3430) at main.c:227
SELECT * FROM bytea_test WHERE bin BETWEEN '\xff00'::bytea AND '\xffff'::bytea ORDER BY id; Table "public.bytea_test"
Column | Type | Modifiers
--------+--------+-----------
id | bigint | not null
bin | bytea |
Indexes:
"bytea_test_pkey" PRIMARY KEY, btree (id)
pgstrom_test=# explain select * from bytea_test where bin between '\xff00'::bytea and '\xffff'::bytea order by id;
INFO: now ready to run bulk-load
QUERY PLAN
--------------------------------------------------------------------------------------
Custom (GpuSort) (cost=129673.73..129721.27 rows=19019 width=40)
Sort keys: id
Sort keys width: 8
Rows per chunk: 21404
-> Custom (GpuScan) on bytea_test (cost=10000.00..129660.21 rows=19019 width=40)
Filter: ((bin >= '\xff00'::bytea) AND (bin <= '\xffff'::bytea))
Host References: id, bin
Device References:
Filter: ((bin >= '\xff00'::bytea) AND (bin <= '\xffff'::bytea))
Planning time: 1.548 ms
現時点の最新のソースで、psql上から大量のデータを持つテーブルの検索を行い、共有メモリ不足に陥ると、それまでに確保した領域が解放されない状況となりました。
explainでGpuScanとなることを確認した直後に、発生しているように見えます。
使用したテーブルは、 #4 で使用したものと同一のものです。
pgstrom_test=# \d oku_table
Table "public.oku_table"
Column | Type | Modifiers
--------+------------------+-----------
id | integer |
data | double precision |
pgstrom_test=# explain select id from oku_table where data < 0.1;
INFO: tc_scan->heapscan = 0x1fb87e0
QUERY PLAN
---------------------------------------------------------------------------------
Custom (GpuScan) on oku_table (cost=10000.00..664461.62 rows=10142061 width=4)
Host References: id
Device References: data
Device Filter: (data < 0.1::double precision)
Planning time: 0.225 ms
(5 rows)
pgstrom_test=# select id from oku_table where data < 0.1;
INFO: tc_scan->heapscan = 0x1fb5f90
INFO: now building tcache...
ERROR: out of shared memory
pgstrom_test=# select * from pgstrom_shmem_info();
zone | size | active | free
------+------+--------+------
0 | 8K | 4 | 1
0 | 16K | 0 | 1
0 | 32K | 0 | 1
0 | 64K | 0 | 1
0 | 128K | 0 | 1
0 | 256K | 0 | 0
0 | 512K | 0 | 1
0 | 1M | 0 | 0
0 | 2M | 0 | 1
0 | 4M | 0 | 1
0 | 8M | 0 | 1
0 | 16M | 119 | 0
0 | 32M | 0 | 0
0 | 64M | 0 | 0
0 | 128M | 0 | 0
0 | 256M | 0 | 0
0 | 512M | 0 | 0
0 | 1G | 0 | 0
0 | 2G | 0 | 0
0 | 4G | 0 | 0
0 | 8G | 0 | 0
0 | 16G | 0 | 0
(22 rows)
typedef struct {
cl_uint length; /* length of this kernel row_store /
cl_uint ncols; / number of columns in the source relation /
cl_uint nrows; / number of rows in this store /
kern_colmeta colmeta[FLEXIBLE_ARRAY_MEMBER]; / metadata of columns */
} kern_row_store;
row-store 上に保持しているタプルのデータ形式を表現する kern_colmeta フィールドの形式に問題。
column-storeにキャッシュされている列か否かを示すフラグをkern_colmeta->flags に持っていたが、column-storeで使用する列は問い合わせの度に変わるので、本来は row-store に保持すべき情報ではない。
GpuScanでは bytea 型の KPARAM_0 を持つようにし、この中にどの列が参照されたのかを示すフラグを持つ事で、デバイス上の row => column 変換でコピーすべき列が明らかになる。
TCacheのスキャン時、取り出したタプルが現在のトランザクションから visible であるかどうか、
一行ごとに評価しなければならない。
OLAPワークロードでは一旦データをDWHにロードし、一括してこれを解析する事が多いと考え
られるので、実際にはキャッシュに載っているタプルが全て visible である頻度が高いはずで
ある。したがって、予めキャッシュの作成時（や参照時にも）に全てのトランザクションからvisibleで
あった場合には 『all-visible』 のフラグを ON にしてチェックを省略する。
textデータ型に対する操作を含む where 句を含み、nvidiaプラットフォームを使用した
場合に、LLVMコンパイラが異常終了する。
Intel環境の場合は再現せず、さらに OpenCL コンパイラの最適化抑止オプションを
付加した場合にも再現しない。
platform-name: NVIDIA CUDA
platform-version: OpenCL 1.1 CUDA 6.0.1
postgres=# explain analyze select count() from t1 where z > 'abc';
INFO: tc_scan->heapscan = 0x184a950
INFO: now building tcache...
INFO: tcache build done...
PHI node operands are not the same type as the result!
%105 = phi i8 [ %101, %99 ], [ %103, %102 ]
PHI node operands are not the same type as the result!
%117 = phi i8* [ %113, %111 ], [ %115, %114 ]
Broken module found, compilation aborted!
Stack dump:
0. Running pass 'Function Pass Manager' on module ''.
Running pass 'Module Verifier' on function '@gpuscan_qual_cs'
LOG: worker process: PG-Strom OpenCL Server (PID 25693) was terminated by signal 6: Aborted
LOG: terminating any other active server processes
WHERE句を含むSQL文を発行すると、WHERE句の条件に合致しない行が結果に表れることがあります。
CREATE TABLE million_table AS SELECT generate_series(1,1000000) AS id, random() AS data ORDER BY random();pgstrom_test=# \d million_table;
Table "public.million_table"
Column | Type | Modifiers
--------+------------------+-----------
id | integer |
data | double precision |
SELECT id FROM million_table WHERE id < 100 ORDER BY id; id
--------
454
1645
2900
3447
3619
4091
4369
4896
5997
(snip)
999234
999748
(1611 rows)
SELECT data FROM million_table WHERE data < 0.2 ORDER BY data; data
----------------------
2.33296304941177e-07
8.09784978628159e-07
(snip)
0.199998688418418
0.199999979231507
0.200000166893005
0.200001429300755
0.262048755306751
0.262048778589815
0.523575029801577
0.523575314786285
(200306 rows)
SELECT id FROM million_table WHERE 900000 < id ORDER BY id; id
--------
1
43
61
69
77
80
86
92
98
99
(snip)
999977
999981
999997
(99838 rows)
SELECT id FROM million_table WHERE id = 100; id
--------
354873
(1 row)
SELECT id FROM million_table WHERE id + 2 = 100; id
--------
168393
(1 row)
#40 と関連して。
TCacheの生成時に、現在はPostgreSQL標準のheap-accessメソッドを使用して
一行ずつ読み出しを行っている。これを Block 単位での読み出しに変更する。
バルクでのロードを行う事で、
○ 一行単位で処理するよりも、処理速度の向上が期待できる。
○ TCacheノード分割時に、ブロックを読む前に次のノードが必要かどうかの
判断ができる。
× 将来の version up で buffer 上のレイアウトが変わる可能性がある。
⇒ 大きな変更が高頻度で入るものではないので、これは大丈夫か？
カラムのデータタイプにvarcharを含むテーブルに対してSQLを実行したところ、
tcache_insert_tuple()内でAssertionFailedとなりました。
今のところ、再現率100%です。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (conditionName=0x7f68694b2cc0 "!(tc_head->tcs_root == tc_node)",
errorType=0x7f68694b1d24 "FailedAssertion", fileName=0x7f68694b1d89 "tcache.c", lineNumber=2090) at assert.c:54
#3 0x00007f6869493428 in tcache_insert_tuple (tc_head=0x7f686121c018, tc_node=0x7f686121c5e8, tuple=0x285add8) at tcache.c:2090
#4 0x00007f68694935bb in tcache_insert_tuple (tc_head=0x7f686121c018, tc_node=0x7f686121c598, tuple=0x285add8) at tcache.c:2124
#5 0x00007f68694935bb in tcache_insert_tuple (tc_head=0x7f686121c018, tc_node=0x7f686121c548, tuple=0x285add8) at tcache.c:2124
#6 0x00007f68694935bb in tcache_insert_tuple (tc_head=0x7f686121c018, tc_node=0x7f686121c4a8, tuple=0x285add8) at tcache.c:2124
#7 0x00007f68694935bb in tcache_insert_tuple (tc_head=0x7f686121c018, tc_node=0x7f686121c368, tuple=0x285add8) at tcache.c:2124
#8 0x00007f6869493755 in tcache_build_main (tc_scan=0x285aa40) at tcache.c:2171
#9 0x00007f6869493c1f in tcache_scan_next (tc_scan=0x285aa40) at tcache.c:2303
#10 0x00007f686949b601 in pgstrom_load_gpuscan (gss=0x2859700) at gpuscan.c:898
#11 0x00007f686949c715 in gpuscan_fetch_tuple (node=0x2859700) at gpuscan.c:1223
#12 0x00007f686949cb94 in gpuscan_exec (node=0x2859700) at gpuscan.c:1318
#13 0x00000000006807e7 in ExecCustomPlan (cpstate=0x2859700) at nodeCustom.c:37
#14 0x00000000006684b5 in ExecProcNode (node=0x2859700) at execProcnode.c:453
#15 0x0000000000665efe in ExecutePlan (estate=0x28595e8, planstate=0x2859700, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x27bffa0) at execMain.c:1475
#16 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x285f6b8, direction=ForwardScanDirection, count=0) at execMain.c:308
#17 0x0000000000663beb in ExecutorRun (queryDesc=0x285f6b8, direction=ForwardScanDirection, count=0) at execMain.c:256
#18 0x00000000007f964e in PortalRunSelect (portal=0x286f1b8, forward=1 '\001', count=0, dest=0x27bffa0) at pquery.c:946
#19 0x00000000007f9275 in PortalRun (portal=0x286f1b8, count=9223372036854775807, isTopLevel=1 '\001', dest=0x27bffa0,
altdest=0x27bffa0, completionTag=0x7fff82577fe0 "") at pquery.c:790
#20 0x00000000007f29e2 in exec_simple_query (query_string=0x2807778 "select * from varchar_test where len = 10;") at postgres.c:1045
#21 0x00000000007f7162 in PostgresMain (argc=1, argv=0x279b5c8, dbname=0x279b428 "pgstrom_test", username=0x279b408 "tkondo")
at postgres.c:4004
#22 0x000000000076e18a in BackendRun (port=0x27c32f0) at postmaster.c:4104
#23 0x000000000076d840 in BackendStartup (port=0x27c32f0) at postmaster.c:3778
#24 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#25 0x0000000000769251 in PostmasterMain (argc=1, argv=0x279a3c0) at postmaster.c:1222
#26 0x00000000006b8aee in main (argc=1, argv=0x279a3c0) at main.c:223
SELECT * FROM varchar_test WHERE len = 10;pgstrom_test=# \d varchar_test
Table "public.varchar_test"
Column | Type | Modifiers
--------+-----------------------+-----------
id | integer |
len | smallint |
str | character varying(64) |
カラムstrは、1文字~64文字の半角文字が含まれています。
(記号を含みます。)
この時のCoreは、以下にbzip2圧縮して置いてありますので、解析にご利用ください。
/home/tkondo/coding/postgres/pg_strom_test/core/core.502.2014-05-22_16-25.bz2
8KB固定サイズでshmem_allocを約8万回行った後、shmem_freeで1つずつ開放する処理を3並列で行ったところ、すでに割り当てられ、かつ未開放の領域が割り当たることがある。(ダブルブッキングする)
この後、shmem_freeが行われた時点でAssertionFailedによるSIGABRTが発生する。
テスト方法は、以下のshmem_test.cを使用し、pgbenchでshmem_test.cの関数を呼び出すSQL文(shmem_test.sql)を実行3並列(クライアント数3、スレッド数3、トランザクション数2)した。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (conditionName=0x7f8f7b03f2b8 "!(block->active.nullmark == ((void *)0))",
errorType=0x7f8f7b03f174 "FailedAssertion", fileName=0x7f8f7b03f19d "shmem.c", lineNumber=228) at assert.c:54
#3 0x00007f8f7b02bd16 in pgstrom_shmem_zone_block_free (zone=0x7f8efa96c000, address=0x7f8f2f11c000) at shmem.c:228
#4 0x00007f8f7b02c1d3 in pgstrom_shmem_free (address=0x7f8f2f11c000) at shmem.c:343
#5 0x00007f8ef170bafc in pgstrom_shmem_test_8Kfree (fcinfo=0x2c23710) at shmem_test.c:78
#6 0x000000000066c190 in ExecMakeFunctionResultNoSets (fcache=0x2c236a0, econtext=0x2c23478,
isNull=0x2c24050 "\177~\177\177\177\177\177\177xJ\301\002", isDone=0x2c24190) at execQual.c:2001
#7 0x000000000066cbbe in ExecEvalFunc (fcache=0x2c236a0, econtext=0x2c23478,
isNull=0x2c24050 "\177~\177\177\177\177\177\177xJ\301\002", isDone=0x2c24190) at execQual.c:2385
#8 0x00000000006737dd in ExecTargetList (targetlist=0x2c24158, econtext=0x2c23478, values=0x2c24030,
isnull=0x2c24050 "\177~\177\177\177\177\177\177xJ\301\002", itemIsDone=0x2c24190, isDone=0x7fff35a28600) at execQual.c:5267
#9 0x0000000000673dfa in ExecProject (projInfo=0x2c24070, isDone=0x7fff35a28600) at execQual.c:5482
#10 0x0000000000691ec9 in ExecResult (node=0x2c23360) at nodeResult.c:155
#11 0x0000000000668365 in ExecProcNode (node=0x2c23360) at execProcnode.c:380
#12 0x0000000000665efe in ExecutePlan (estate=0x2c23248, planstate=0x2c23360, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x2be8448) at execMain.c:1475
#13 0x0000000000663da8 in standard_ExecutorRun (queryDesc=0x2c1fda8, direction=ForwardScanDirection, count=0) at execMain.c:308
#14 0x0000000000663beb in ExecutorRun (queryDesc=0x2c1fda8, direction=ForwardScanDirection, count=0) at execMain.c:256
#15 0x00000000007f964e in PortalRunSelect (portal=0x2b9bbd8, forward=1 '\001', count=0, dest=0x2be8448) at pquery.c:946
#16 0x00000000007f9275 in PortalRun (portal=0x2b9bbd8, count=9223372036854775807, isTopLevel=1 '\001', dest=0x2be8448,
altdest=0x2be8448, completionTag=0x7fff35a289e0 "") at pquery.c:790
#17 0x00000000007f29e2 in exec_simple_query (query_string=0x2be6b08 "select pgstrom_shmem_test_8Kfree();") at postgres.c:1045
#18 0x00000000007f7162 in PostgresMain (argc=1, argv=0x2b78f30, dbname=0x2b78e28 "pgstrom_test", username=0x2b78e08 "tkondo")
at postgres.c:4004
#19 0x000000000076e18a in BackendRun (port=0x2ba07d0) at postmaster.c:4104
#20 0x000000000076d840 in BackendStartup (port=0x2ba07d0) at postmaster.c:3778
#21 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#22 0x0000000000769251 in PostmasterMain (argc=1, argv=0x2b77dc0) at postmaster.c:1222
#23 0x00000000006b8aee in main (argc=1, argv=0x2b77dc0) at main.c:223
#include "postgres.h"
#include "fmgr.h"
#include <string.h>
#include "pg_strom.h"
PG_MODULE_MAGIC;
#define ALLOC_MAX_COUNT 262144 /* 256 * 1024 */
void *pgstrom_test8K_addrs[ALLOC_MAX_COUNT];
bool pgstrom_test8K_alloc_done = false;
PG_FUNCTION_INFO_V1(pgstrom_shmem_test_8K);
Datum
pgstrom_shmem_test_8K(PG_FUNCTION_ARGS)
{
int64 i = 0;
int64 count = PG_GETARG_INT64(0);
/* Already allocated */
if (pgstrom_test8K_alloc_done)
{
PG_RETURN_BOOL(false);
}
/* Out of range */
if (count > ALLOC_MAX_COUNT)
{
PG_RETURN_BOOL(false);
}
/* Clear all addresses */
for (i = 0; i < ALLOC_MAX_COUNT; i++)
{
pgstrom_test8K_addrs[i] = NULL;
}
for (i = 0; i < count; i++)
{
pgstrom_test8K_addrs[i] = pgstrom_shmem_alloc((Size)(8192 - 4));
elog(INFO, "Alloc No.%d (addr=%p)", i, pgstrom_test8K_addrs[i]);
if (pgstrom_test8K_addrs[i] == NULL)
break;
}
elog(LOG, "Allocated %d partition(s).", i);
pgstrom_test8K_alloc_done = true;
PG_RETURN_BOOL(true);
}
PG_FUNCTION_INFO_V1(pgstrom_shmem_test_8Kfree);
Datum
pgstrom_shmem_test_8Kfree(PG_FUNCTION_ARGS)
{
int64 i = 0LL;
/* Not allocated yet */
if (!pgstrom_test8K_alloc_done)
{
PG_RETURN_BOOL(false);
}
/* Free now! */
do {
elog(INFO, "Free No.%d (addr=%p)", i, pgstrom_test8K_addrs[i]);
pgstrom_shmem_free(pgstrom_test8K_addrs[i]);
pgstrom_test8K_addrs[i++] = NULL;
} while ((i < ALLOC_MAX_COUNT) && (pgstrom_test8K_addrs[i] != NULL));
elog(LOG, "Freed %d partition(s).", i);
pgstrom_test8K_alloc_done = false;
PG_RETURN_BOOL(true);
}set log_min_duration_statement=0;
set log_min_messages=info;
set client_min_messages=error;
select pg_backend_pid();
select pgstrom_shmem_test_8K(81947);
select pgstrom_shmem_test_8Kfree();
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:22 JST, 32656, 534e3b28.7f90 INFO: Alloc No.73129 (addr=0x7f8f2f11c000)
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:24 JST, 32656, 534e3b28.7f90 INFO: Free No.73129 (addr=0x7f8f2f11c000)
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:24 JST, 32656, 534e3b28.7f90 INFO: Alloc No.193 (addr=0x7f8f2f11c000)
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:25 JST, 32655, 534e3b28.7f8f INFO: Alloc No.22065 (addr=0x7f8f2f11c000)
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:25 JST, 32655, 534e3b28.7f8f INFO: Alloc No.54966 (addr=0x7f8f2f11c000)
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:26 JST, 32656, 534e3b28.7f90 INFO: Free No.193 (addr=0x7f8f2f11c000)
pgbench, 2014-04-16 17:11:20 JST, 2014-04-16 17:11:27 JST, 32655, 534e3b28.7f8f INFO: Free No.22065 (addr=0x7f8f2f11c000)
GPU側でソート処理を行った後、本体側へ戻ってきた pgstrom_gpusort オブジェクトから、
pgstrom_gpusort_chunk オブジェクトのみをデタッチして、GpuSortState->sorted_chunks
リストに結合する。
ここから gpusort_end が呼ばれるまでの間、resource trackerに捕捉されないため、
潜在的なメモリリークの危険性がある。
6末以降に対応予定。
現時点の最新のソースでpgbenchによる多重実行を行ったところ、AssertionFailedとなりました。
今のところ、再現確率100%です。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000949085 in ExceptionalCondition (
conditionName=0x7f16eea97348 "!(sobject->tracker.prev != ((void *)0) && sobject->tracker.next != ((void *)0))",
errorType=0x7f16eea970ec "FailedAssertion", fileName=0x7f16eea97115 "restrack.c", lineNumber=208) at assert.c:54
#3 0x00007f16eea77c87 in pgstrom_untrack_object (sobject=0x7f16e67f6018) at restrack.c:208
#4 0x00007f16eea894ae in gpuscan_end (node=0x1073e48) at gpuscan.c:1392
#5 0x00000000006808dd in ExecEndCustomPlan (cpstate=0x1073e48) at nodeCustom.c:51
#6 0x0000000000668911 in ExecEndNode (node=0x1073e48) at execProcnode.c:693
#7 0x000000000067f6b7 in ExecEndAgg (node=0x10736b0) at nodeAgg.c:2041
#8 0x0000000000668988 in ExecEndNode (node=0x10736b0) at execProcnode.c:727
#9 0x0000000000665d17 in ExecEndPlan (planstate=0x10736b0, estate=0x1073598) at execMain.c:1374
#10 0x0000000000664165 in standard_ExecutorEnd (queryDesc=0x10587f8) at execMain.c:437
#11 0x0000000000664060 in ExecutorEnd (queryDesc=0x10587f8) at execMain.c:408
#12 0x00000000006186f6 in PortalCleanup (portal=0x1071588) at portalcmds.c:280
#13 0x000000000097e4d3 in PortalDrop (portal=0x1071588, isTopCommit=0 '\000') at portalmem.c:494
#14 0x00000000007f2a0d in exec_simple_query (query_string=0x1021568 "SELECT count(*) FROM million_table WHERE data < 0.01;")
at postgres.c:1054
#15 0x00000000007f7162 in PostgresMain (argc=1, argv=0xfb3520, dbname=0xfb3418 "pgstrom_test", username=0xfb33f8 "tkondo")
at postgres.c:4004
#16 0x000000000076e18a in BackendRun (port=0xfdb230) at postmaster.c:4104
#17 0x000000000076d840 in BackendStartup (port=0xfdb230) at postmaster.c:3778
#18 0x0000000000769bfe in ServerLoop () at postmaster.c:1569
#19 0x0000000000769251 in PostmasterMain (argc=1, argv=0xfb23b0) at postmaster.c:1222
#20 0x00000000006b8aee in main (argc=1, argv=0xfb23b0) at main.c:223
psql上から、1億件のデータがあるテーブルに対して全件Scanを実行し、途中でCtrl+Cで中止すると、8MBサイズの共有メモリ領域がいくつかActiveのまま残る。
SELECT count(*) FROM oku_table WHERE data >= 0.9;CREATE TABLE oku_table AS SELECT generate_series(1,100000000) AS id, random() AS data ORDER BY random(); zone | size | active | free
------+------+--------+------
0 | 8K | 1 | 0
0 | 16K | 0 | 1
0 | 32K | 0 | 0
0 | 64K | 0 | 0
0 | 128K | 0 | 0
0 | 256K | 0 | 1
0 | 512K | 0 | 1
0 | 1M | 0 | 0
0 | 2M | 0 | 1
0 | 4M | 0 | 1
0 | 8M | 0 | 1
0 | 16M | 0 | 1
0 | 32M | 0 | 1
0 | 64M | 0 | 1
0 | 128M | 0 | 0
0 | 256M | 0 | 1
0 | 512M | 0 | 1
0 | 1G | 0 | 1
0 | 2G | 0 | 0
0 | 4G | 0 | 0
0 | 8G | 0 | 0
0 | 16G | 0 | 0
(22 rows)
zone | size | active | free
------+------+--------+------
0 | 8K | 3 | 0
0 | 16K | 0 | 0
0 | 32K | 0 | 0
0 | 64K | 0 | 0
0 | 128K | 0 | 0
0 | 256K | 0 | 1
0 | 512K | 0 | 1
0 | 1M | 0 | 0
0 | 2M | 0 | 1
0 | 4M | 0 | 1
0 | 8M | 3 | 4
0 | 16M | 0 | 4
0 | 32M | 0 | 4
0 | 64M | 0 | 2
0 | 128M | 0 | 2
0 | 256M | 0 | 1
0 | 512M | 0 | 2
0 | 1G | 0 | 0
0 | 2G | 0 | 0
0 | 4G | 0 | 0
0 | 8G | 0 | 0
0 | 16G | 0 | 0
(22 rows)
現状、複数のテーブルをinner-joinする場合、GpuHashJoinを繰り返し実行する。
しかし、GPUのRAM上に一度に複数のHash-Tableをロードすれば、
一回のGpuHashJoinで多段Joinを済ませる事ができる。
現状、GpuHashJoinノード間でデータを受け渡す際の materialization が大きな
ボトルネックになっているため、そもそもデータ受け渡しの必要をなくせばよい。
postgres=# explain select * from t7,t8,t9 where t7.aid = t9.aid and t7.aid = t8.bid;
INFO: now ready to run bulk-load
INFO: now ready to run bulk-load
Custom (GpuHashJoin) using Inner Join (cost=30565.00..30565.00 rows=9973079 width=147)
Device references: t7.aid, t9.aid
hash clauses: (t7.aid = t9.aid)
-> Custom (GpuScan) on t9 (cost=10000.00..204458.06 rows=10000006 width=41)
Host References: aid, bid, z
Device References:
-> Custom (GpuHashJoin) using Inner Join (cost=20390.00..20390.00 rows=10000 width=106)
Device references: t7.aid, t8.bid
hash clauses: (t7.aid = t8.bid)
-> Custom (GpuScan) on t7 (cost=10000.00..10215.00 rows=10000 width=53)
Host References: aid, x, y, z
Device References:
-> Custom (GpuScan) on t8 (cost=10000.00..10215.00 rows=10000 width=53)
Host References: bid, x, y, z
Device References:
Planning time: 0.983 ms
(16 rows)
codegen.cが生成するOpenCLコードと、PostgreSQLの数値演算・比較系の部分の例外処理の差異を確認したところ、以下の差異を検出。
特に、float系の比較演算におけるNaNの扱いは、PostgreSQL側と合わせないと予期しない動作をすると考える。
これは Issue Tracker のテストです。
Issue を登録したらどのように見えるのか確認する用途に使ってください。
カピバラは、体長105-135cm、体重35-65kgにまで成長する。性格は非常に穏やかで、人に懐く事からペットとしても人気がある。5cm以上にもなるタワシのような硬い体毛に覆われている。泳ぎが得意で、前足後足には蜘蛛の巣状の水かきがついている。群れを成して泳ぎ、捕食動物から身を隠すために水中に5分以上潜ることができる。鼻先だけを水上に出して眠ることもある。
オスは鼻の上に分泌腺（モリージョ）があり、交尾期になると、メスを惹き付けるために、これを周囲の木の葉にこする。妊娠期間は150日で、齧歯類の中で例外的に巣を作らず、水辺の草むらに直接仔を1-7匹生む。一頭のオスと複数のメスとその仔からなる10頭ほどの群れで生活し、集団で子育てを行う。乾季になると、水場を求めて大移動を行ない、結果的に100頭以上の群れを形成することもある。寿命は5-10年。草食性で、朝夕に活動し、川辺で水中の草や木の葉などを食べて過ごす。
最新のコードにて、文字列の"<"や">"の比較を試していますが、
SeqScanの結果と一致しない場合が出現しました。
SELECT * FROM member_pref_str WHERE pref_name < 'Ehime';pgstrom_test=# \d member_pref_str
Table "public.member_pref_str"
Column | Type | Modifiers
-----------+---------+-----------
pref | integer |
pref_name | text |
pref | pref_name
------+-----------
2 | Aomori
3 | Akita
12 | Chiba
20 | Aichi
(4 rows)
*結果(GpuScan)
pref | pref_name
------+-----------
2 | Aomori
3 | Akita
12 | Chiba
20 | Aichi
37 | Ehime
(5 rows)
JOIN操作を含むSQL文を実行しようとすると、エラーが表示されて実行できないことがあります。
以前(GpuSortが入る前)のバージョンでは、実行できていたSQL文です。
SELECT gender,avg(score) FROM member_score JOIN member_gender USING(id) GROUP BY gender;ERROR: unrecognized node type: 0
Table "public.member_score"
Column | Type | Modifiers
--------+---------+-----------
id | integer | not null
date | date | not null
score | integer | not null
Table "public.member_gender"
Column | Type | Modifiers
--------+----------+-----------
id | integer | not null
gender | smallint | not null
Indexes:
"member_gender_pkey" PRIMARY KEY, btree (id)
最新のコードで以下のSQL文を実行したところと、結果が0 rowsとなりました。
SeqScanで行うと、5 rowsが表示されます。
SELECT * FROM temperature_tokyo WHERE high - low >= 15 AND high_time < low_time;pgstrom_test=# explain select * from temperature_tokyo where high - low >= 15 and high_time < low_time;
QUERY PLAN
--------------------------------------------------------------------------------------
Custom (GpuScan) on temperature_tokyo (cost=10000.00..10048.05 rows=609 width=28)
Host References: date, high, high_time, low, low_time
Device References: high, high_time, low, low_time
Device Filter: ((high_time < low_time) AND ((high - low) >= 15::double precision))
Planning time: 0.335 ms
(5 rows)
pgstrom_test=# select * from temperature_tokyo where high - low >= 15 and high_time < low_time;
INFO: now building tcache...
INFO: tcache build done...
date | high | high_time | low | low_time
------+------+-----------+-----+----------
(0 rows)
pgstrom_test=# explain select * from temperature_tokyo where high - low >= 15 and high_time < low_time;
QUERY PLAN
-------------------------------------------------------------------------------
Seq Scan on temperature_tokyo (cost=0.00..136.88 rows=609 width=28)
Filter: ((high_time < low_time) AND ((high - low) >= 15::double precision))
Planning time: 0.242 ms
(3 rows)
pgstrom_test=# select * from temperature_tokyo where high - low >= 15 and high_time < low_time;
date | high | high_time | low | low_time
------------+------+---------------------+------+---------------------
2001-04-03 | 22.5 | 2001-04-03 13:38:00 | 7.3 | 2001-04-03 23:47:00
2003-05-08 | 26.9 | 2003-05-08 11:11:00 | 11.3 | 2003-05-09 00:00:00
2008-02-23 | 17 | 2008-02-23 13:06:00 | 0.7 | 2008-02-24 00:00:00
2008-12-22 | 19.9 | 2008-12-22 10:17:00 | 4.2 | 2008-12-22 20:20:00
2013-03-10 | 25.3 | 2013-03-10 13:10:00 | 5.7 | 2013-03-10 23:58:00
(5 rows)
Table "public.temperature_tokyo"
Column | Type | Modifiers
-----------+-----------------------------+-----------
date | date |
high | real |
high_time | timestamp without time zone |
low | real |
low_time | timestamp without time zone |
WHERE句の "high - low >= 15" の条件がない場合は、正しい結果が出力されるように見えます。
psql上からSQLを実行し、tcacheのビルド中にCtrl+Cで実行を中止したあと、おなじSQLを実行するとtcacheビルド中にAssertionFailedとなりました。
psql上からの実行であり、多重実行は行っていません。
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x0000000000948a59 in ExceptionalCondition (conditionName=0x7fe2623b6f16 "!(nremain > 0 && nmoved > 0)",
errorType=0x7fe2623b6904 "FailedAssertion", fileName=0x7fe2623b6969 "tcache.c", lineNumber=1612) at assert.c:54
#3 0x00007fe262388fc3 in tcache_split_tcnode (tc_head=0x7fdeda084018, tc_node_old=0x7fdeda0844a8) at tcache.c:1612
#4 0x00007fe26238b7d3 in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fdeda0844a8, tuple=0x1cf61f8) at tcache.c:2169
#5 0x00007fe26238b65b in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fe1336831a8, tuple=0x1cf61f8) at tcache.c:2137
#6 0x00007fe26238b65b in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fe133683068, tuple=0x1cf61f8) at tcache.c:2137
#7 0x00007fe26238b65b in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fe133682de8, tuple=0x1cf61f8) at tcache.c:2137
#8 0x00007fe26238b65b in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fe1336828e8, tuple=0x1cf61f8) at tcache.c:2137
#9 0x00007fe26238b65b in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fdfca951e78, tuple=0x1cf61f8) at tcache.c:2137
#10 0x00007fe26238b65b in tcache_insert_tuple (tc_head=0x7fdeda084018, tc_node=0x7fdfca950a78, tuple=0x1cf61f8) at tcache.c:2137
#11 0x00007fe26238b8dd in tcache_build_main (tc_scan=0x1cf6160) at tcache.c:2200
#12 0x00007fe26238bda7 in tcache_scan_next (tc_scan=0x1cf6160) at tcache.c:2332
#13 0x00007fe26239439a in pgstrom_load_gpuscan (gss=0x1cf4fc0) at gpuscan.c:895
#14 0x00007fe2623955f9 in gpuscan_fetch_tuple (node=0x1cf4fc0) at gpuscan.c:1260
#15 0x00007fe262395b17 in gpuscan_exec (node=0x1cf4fc0) at gpuscan.c:1390
#16 0x0000000000680dcb in ExecCustomPlan (cpstate=0x1cf4fc0) at nodeCustom.c:37
#17 0x0000000000668ae5 in ExecProcNode (node=0x1cf4fc0) at execProcnode.c:453
#18 0x00007fe26239c2fe in gpusort_preload_subplan (gsortstate=0x1cf4bf0, overflow=0x7fffbc3e0558) at gpusort.c:1019
#19 0x00007fe26239e228 in gpusort_exec (node=0x1cf4bf0) at gpusort.c:1621
#20 0x0000000000680dcb in ExecCustomPlan (cpstate=0x1cf4bf0) at nodeCustom.c:37
#21 0x0000000000668ae5 in ExecProcNode (node=0x1cf4bf0) at execProcnode.c:453
#22 0x000000000066652c in ExecutePlan (estate=0x1cf4ad8, planstate=0x1cf4bf0, operation=CMD_SELECT, sendTuples=1 '\001',
numberTuples=0, direction=ForwardScanDirection, dest=0x1cf2720) at execMain.c:1475
#23 0x00000000006643dc in standard_ExecutorRun (queryDesc=0x1cbb618, direction=ForwardScanDirection, count=0) at execMain.c:308
#24 0x000000000066421f in ExecutorRun (queryDesc=0x1cbb618, direction=ForwardScanDirection, count=0) at execMain.c:256
#25 0x00000000007f9e7e in PortalRunSelect (portal=0x1c54f08, forward=1 '\001', count=0, dest=0x1cf2720) at pquery.c:946
#26 0x00000000007f9aa5 in PortalRun (portal=0x1c54f08, count=9223372036854775807, isTopLevel=1 '\001', dest=0x1cf2720,
altdest=0x1cf2720, completionTag=0x7fffbc3e09c0 "") at pquery.c:790
#27 0x00000000007f3212 in exec_simple_query (query_string=0x1ca0588 "SELECT str FROM text_test_null WHERE str < '!9' ORDER BY str;")
at postgres.c:1045
#28 0x00000000007f7992 in PostgresMain (argc=1, argv=0x1c2f638, dbname=0x1c2f498 "pgstrom_test", username=0x1c2f478 "tkondo")
at postgres.c:4004
#29 0x000000000076e8e9 in BackendRun (port=0x1c5a040) at postmaster.c:4117
#30 0x000000000076df9f in BackendStartup (port=0x1c5a040) at postmaster.c:3791
#31 0x000000000076a366 in ServerLoop () at postmaster.c:1570
#32 0x00000000007699b9 in PostmasterMain (argc=1, argv=0x1c2e430) at postmaster.c:1223
#33 0x00000000006b9166 in main (argc=1, argv=0x1c2e430) at main.c:227
SELECT str FROM text_test_null WHERE str < '!9' ORDER BY str;テーブル "public.text_test_null"
列 | 型 | 修飾語
-----+----------+----------
id | integer | not null
len | smallint | not null
str | text |
QUERY PLAN
-----------------------------------------------------------------------------------------
Custom (GpuSort) (cost=40623.99..40664.75 rows=16305 width=11)
Sort keys: str
Sort keys width: 36
Rows per chunk: 16320
-> Custom (GpuScan) on text_test_null (cost=10000.00..40612.58 rows=16305 width=11)
Host References: str
Device References: str
Device Filter: (str < '!9'::text)
Planning time: 0.423 ms
件数の大きなデータを集約する際には、GROUP BY指定されたカラムで
データをソートしてから GroupAggregate が呼ばれるが、これは割と重い
処理。
代わりに、GPUでいったんサブセットのデータに対する集約処理を実行し、
件数を小さくまとめてから HashAggregate に渡すようクエリ実行計画を
書き換えれば、集約関数のパフォーマンスを大幅に向上できるはず。
利点） データ件数を減らす方向の処理なので、PG-Stromの苦手とする
CPUでのメモリコピーが発生しない。
利点） 前処理で必要なのはチャンク内ソートなので、チャンク間のマージ
は必要ではなくなる。
postgres=# explain select count(*), sum(aid), floor(bid) from t10 group by floor(bid);
HashAggregate (cost=16390.00..16539.25 rows=9950 width=8)
Group Key: floor((bid)::double precision)
-> Custom (GpuScan) on t10 (cost=10000.00..14890.00 rows=200000 width=8)
Host References: aid, bid
Device References:
Planning time: 0.423 ms
(6 rows)
postgres=# explain select count(*), sum(x), floor(y) from t1 group by floor(y);
GroupAggregate (cost=681215.55..771216.45 rows=4000040 width=16)
Group Key: (floor(y))
-> Custom (GpuSort) (cost=110241.86..120241.96 rows=4000040 width=16)
Sort keys: (floor(y))
Sort keys width: 8
Rows per chunk: 5592000
-> Custom (GpuScan) on t1 (cost=10000.00..105855.50 rows=4000040 width=16)
Host References: x, y
Device References:
Planning time: 0.579 ms
(10 rows)
こんにちわ。
pgbenchが作るテーブルに対するGpuScanでAssertionFailedが発生しました。
nanzai=# select * from pgbench_tellers ;
tid | bid | tbalance | filler
-----+-----+----------+--------
1 | 1 | 0 |
2 | 1 | 0 |
3 | 1 | 0 |
4 | 1 | 0 |
5 | 1 | 0 |
6 | 1 | 0 |
7 | 1 | 0 |
8 | 1 | 0 |
9 | 1 | 0 |
10 | 1 | 0 |
(10 rows)
set enable_seqscan to off;
select * from pgbench_tellers ;nanzai=# select * from pgbench_tellers ;
INFO: now building tcache...
INFO: tcache build done...
The connection to the server was lost. Attempting reset: Failed.
!>
nanzai=# explain select * from pgbench_tellers ;
QUERY PLAN
----------------------------------------------------------------------------------
Custom (GpuScan) on pgbench_tellers (cost=10000.00..10001.10 rows=10 width=352)
Host References: tid, bid, tbalance, filler
Device References:
Planning time: 1.274 ms
(4 rows)
(gdb) bt
#0 0x0000003cdbc32925 in raise () from /lib64/libc.so.6
#1 0x0000003cdbc34105 in abort () from /lib64/libc.so.6
#2 0x00000000008d7a26 in ExceptionalCondition (conditionName=0x7f2abc8b5b50 "!((uintptr_t)body % (1UL << 13) == 0)",
errorType=0x7f2abc8b5588 "FailedAssertion", fileName=0x7f2abc8b55b1 "shmem.c", lineNumber=813) at assert.c:54
#3 0x00007f2abc87c935 in pgstrom_shmem_sanitycheck (address=0x7f2ab4707bf0) at shmem.c:813
#4 0x00007f2abc893b61 in pgstrom_load_gpuscan (gss=0x2ce9f00) at gpuscan.c:965
#5 0x00007f2abc894768 in gpuscan_fetch_tuple (node=0x2ce9f00) at gpuscan.c:1210
#6 0x00007f2abc894cde in gpuscan_exec (node=0x2ce9f00) at gpuscan.c:1331
#7 0x0000000000658585 in ExecCustomPlan (cpstate=0x2ce9f00) at nodeCustom.c:37
#8 0x0000000000642400 in ExecProcNode (node=0x2ce9f00) at execProcnode.c:453
#9 0x00000000006400f1 in ExecutePlan (estate=0x2ce9de8, planstate=0x2ce9f00, operation=CMD_SELECT, sendTuples=1 '\001', numberTuples=0,
direction=ForwardScanDirection, dest=0xd5f9e0) at execMain.c:1475
#10 0x000000000063e1ab in standard_ExecutorRun (queryDesc=0x2cf1448, direction=ForwardScanDirection, count=0) at execMain.c:308
#11 0x000000000063e011 in ExecutorRun (queryDesc=0x2cf1448, direction=ForwardScanDirection, count=0) at execMain.c:256
#12 0x00000000005dc0cf in ExplainOnePlan (plannedstmt=0x2cf13b0, into=0x0, es=0x7fff1c9c4490,
queryString=0x2ca98c8 "explain analyze select * from pgbench_tellers ;", params=0x0, planduration=0x7fff1c9c4400) at explain.c:481
#13 0x00000000005dbd97 in ExplainOneQuery (query=0x2c67ea0, into=0x0, es=0x7fff1c9c4490,
queryString=0x2ca98c8 "explain analyze select * from pgbench_tellers ;", params=0x0) at explain.c:336
#14 0x00000000005db9ff in ExplainQuery (stmt=0x2caa648, queryString=0x2ca98c8 "explain analyze select * from pgbench_tellers ;", params=0x0, dest=0x2c67dd0)
at explain.c:227
#15 0x00000000007ae16d in standard_ProcessUtility (parsetree=0x2caa648, queryString=0x2ca98c8 "explain analyze select * from pgbench_tellers ;",
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, dest=0x2c67dd0, completionTag=0x7fff1c9c46f0 "") at utility.c:650
#16 0x00000000007ad88d in ProcessUtility (parsetree=0x2caa648, queryString=0x2ca98c8 "explain analyze select * from pgbench_tellers ;",
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, dest=0x2c67dd0, completionTag=0x7fff1c9c46f0 "") at utility.c:310
#17 0x00000000007ac935 in PortalRunUtility (portal=0x2c64c68, utilityStmt=0x2caa648, isTopLevel=1 '\001', dest=0x2c67dd0, completionTag=0x7fff1c9c46f0 "")
at pquery.c:1187
#18 0x00000000007ac684 in FillPortalStore (portal=0x2c64c68, isTopLevel=1 '\001') at pquery.c:1061
#19 0x00000000007abf8a in PortalRun (portal=0x2c64c68, count=9223372036854775807, isTopLevel=1 '\001', dest=0x2cab748, altdest=0x2cab748,
completionTag=0x7fff1c9c48d0 "") at pquery.c:785
#20 0x00000000007a6015 in exec_simple_query (query_string=0x2ca98c8 "explain analyze select * from pgbench_tellers ;") at postgres.c:1045
#21 0x00000000007aa214 in PostgresMain (argc=1, argv=0x2c40fa0, dbname=0x2c40e08 "nanzai", username=0x2c40de8 "nanzai") at postgres.c:4004
#22 0x000000000072ee75 in BackendRun (port=0x2c68c40) at postmaster.c:4117
#23 0x000000000072e59f in BackendStartup (port=0x2c68c40) at postmaster.c:3791
#24 0x000000000072ad79 in ServerLoop () at postmaster.c:1570
#25 0x000000000072a4a5 in PostmasterMain (argc=1, argv=0x2c3fdb0) at postmaster.c:1223
#26 0x000000000068acf3 in main (argc=1, argv=0x2c3fdb0) at main.c:227
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.