hkuds / gpt-st

[NeurIPS'2023] "GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks"

Hi there 👋

✨Welcome to the Data Intelligence Lab @ HKU!✨

🚀 Our Lab is Passionately Dedicated to Exploring the Forefront of the Data Science & AI 👨‍💻

       

gpt-st's Issues

Error for code

Traceback (most recent call last):
File "", line 1, in
AttributeError: 'Namespace' object has no attribute 'scaler_zeros'

Overflow problem

The output of Hypergraph_encoder block produce inf in squash function,

for the PEMS08 dataset:

flow_encode_eb:

tensor([[[[-5.2653e+09, 9.3891e+11, -5.5095e+09, ..., 7.2950e+11,
-3.1244e+09, -1.7719e+08],
[-1.2422e+10, 5.4581e+11, 6.6494e+10, ..., 6.2498e+11,
-7.0435e+09, 9.0786e+11],
[ 1.3344e+11, 7.8447e+10, 9.6840e+11, ..., -1.4184e+09,
-1.7943e+08, 6.5809e+11],
...,
[ 5.5459e+10, -1.0283e+08, -4.2533e+08, ..., 1.7518e+10,
1.5606e+10, -4.0996e+08],
[-1.9864e+09, 2.3029e+11, -1.0385e+10, ..., 2.3933e+11,
2.1303e+10, -4.0887e+09],
[-2.0802e+10, 8.5787e+11, -7.5113e+09, ..., 1.1368e+12,
-7.5130e+09, 5.3748e+11]],
[[-1.7175e+10, 9.4165e+11, 1.0968e+12, ..., 1.2070e+11,
-1.0944e+10, 1.9299e+12],
[-2.5238e+10, 2.0915e+11, 1.2227e+12, ..., 1.3333e+12,
-1.2979e+10, 2.2329e+12],
[-2.9424e+09, -7.6849e+09, -2.2214e+08, ..., -2.9996e+08,
5.4478e+11, -5.0003e+08],
...,
[-7.6789e+08, 1.4989e+10, 5.0317e+10, ..., -1.0244e+08,
-1.8085e+08, 5.5189e+10],
[ 5.8449e+10, -2.2103e+09, 5.0724e+11, ..., -9.2613e+08,
-2.7623e+07, 1.7494e+11],
[-2.3659e+10, 7.8041e+11, -6.2711e+09, ..., 1.8937e+12,
-1.2461e+10, 9.2876e+11]],
[[-9.5941e+09, 8.3174e+11, -4.1092e+09, ..., 1.8217e+11,
-1.5162e+09, 3.5043e+11],
[-8.1781e+09, -2.9769e+09, 8.4290e+11, ..., 5.1741e+11,
-3.4328e+09, 8.6847e+11],
[-3.4403e+10, 2.3496e+12, 6.4393e+10, ..., 4.4681e+12,
-1.2721e+10, 3.1503e+12],
...,
[-1.0615e+09, 1.4454e+10, -3.3334e+08, ..., -6.4495e+07,
-9.1145e+07, 2.8557e+10],
[-2.8808e+09, 1.4480e+11, -5.3969e+08, ..., 4.0438e+11,
-1.7176e+09, 2.6920e+11],
[-1.4652e+09, 1.1365e+11, -6.2235e+09, ..., -1.1800e+07,
2.5693e+10, -2.7570e+09]],
...,
[[-8.1441e+09, 6.1352e+11, 2.4874e+11, ..., 7.2296e+11,
-7.8808e+09, 7.3540e+11],
[ 5.5829e+11, 2.2956e+11, -5.3680e+09, ..., -3.2939e+09,
2.7393e+11, -6.4277e+09],
[ 3.3010e+11, 7.0728e+11, 1.2370e+12, ..., 1.9730e+11,
-1.2123e+09, 5.4335e+11],
...,
[-8.7209e+07, 1.3545e+09, 3.7040e+09, ..., 2.9861e+08,
-6.2985e+07, 4.8487e+09],
[ 3.0255e+11, -1.4527e+09, 7.9548e+11, ..., -3.4367e+09,
3.5435e+10, 2.5829e+11],
[ 4.4405e+11, -3.1164e+09, 2.7430e+11, ..., -1.4864e+09,
1.4797e+11, -5.0589e+08]],
[[-9.0427e+09, 1.3244e+11, 6.7902e+11, ..., 2.1199e+11,
-6.2866e+09, 1.0076e+12],
[ 1.4364e+11, -4.2985e+09, 2.7069e+11, ..., -4.3986e+07,
1.5554e+11, 4.6971e+10],
[ 1.6946e+11, -1.0585e+09, 1.6464e+12, ..., 1.6484e+10,
-1.1237e+07, 1.3758e+12],
...,
[-8.1306e+07, 3.4464e+09, -3.2758e+07, ..., -1.2622e+06,
-7.6931e+07, 1.1145e+09],
[-3.1198e+09, -2.6004e+08, 1.7828e+11, ..., 2.4246e+11,
-8.5265e+08, 2.4522e+11],
[-2.3768e+09, 5.7859e+10, 4.6476e+11, ..., 1.5984e+11,
-1.2724e+09, 3.6888e+11]],
[[ 8.3994e+11, 1.6872e+11, -5.2307e+09, ..., 2.1716e+11,
7.3251e+11, -7.9507e+09],
[ 3.9185e+11, -2.2590e+09, 8.6533e+10, ..., -2.4395e+09,
2.3023e+11, -2.6549e+09],
[ 1.6281e+12, 3.9403e+11, 4.2729e+11, ..., -1.9017e+10,
6.0303e+11, -1.9731e+10],
...,
[-5.1985e+08, 6.8942e+09, 3.1177e+10, ..., -3.7462e+07,
-1.0247e+08, 4.0438e+10],
[ 9.1139e+10, -9.6727e+08, 7.1794e+11, ..., 9.2616e+10,
-1.3988e+09, 4.5414e+11],
[ 6.3977e+10, 6.3177e+10, -2.8426e+09, ..., -1.0993e+09,
3.2036e+10, -2.1908e+09]]],
[[[-9.4192e+09, 8.5863e+11, -9.6346e+09, ..., 1.0767e+12,
-3.5570e+09, -1.1293e+09],
[-1.3120e+10, 5.1021e+11, 6.2822e+10, ..., 7.0319e+11,
-7.0327e+09, 8.7349e+11],
[ 1.3374e+11, -9.7839e+08, 1.2032e+12, ..., 3.3066e+11,
-6.2516e+08, 9.0085e+11],
...,
[ 5.3048e+10, 2.1270e+09, -4.2695e+08, ..., 9.0383e+09,
8.5565e+09, -3.9655e+08],
[-1.3814e+08, 1.6149e+11, -9.0171e+09, ..., 1.1326e+11,
1.0808e+11, -4.5542e+09],
[-2.0838e+10, 8.5669e+11, -7.5193e+09, ..., 1.1188e+12,
-7.5517e+09, 5.5570e+11]],
[[-2.4087e+10, 8.8711e+11, 9.5400e+11, ..., 6.7293e+11,
-1.4059e+10, 2.4996e+12],
[-2.6555e+10, 1.2034e+11, 1.2952e+12, ..., 1.5598e+12,
-1.2970e+10, 2.2264e+12],
[-1.3454e+09, -4.6950e+09, 3.0269e+11, ..., -1.1635e+09,
4.2978e+11, -2.8828e+09],
...,
[-8.9329e+08, 5.3833e+09, 4.1122e+10, ..., 5.7571e+09,
-1.3560e+08, 5.3837e+10],
[ 4.5369e+10, -1.7564e+09, 4.4253e+11, ..., -6.6925e+08,
-3.3431e+08, 1.8225e+11],
[-2.3934e+10, 7.8469e+11, -6.5418e+09, ..., 1.8951e+12,
-1.2545e+10, 9.4312e+11]],
[[-1.2271e+10, 5.6691e+11, -7.9976e+09, ..., 1.4428e+11,
-1.9438e+09, 3.9093e+11],
[-8.6583e+09, -3.2973e+09, 8.6646e+11, ..., 6.2791e+11,
-3.6920e+09, 9.1090e+11],
[-5.2748e+10, 2.2959e+12, -9.8083e+09, ..., 6.0520e+12,
-2.4692e+10, 4.2248e+12],
...,
[-1.5766e+09, 2.1928e+10, -5.3539e+08, ..., 3.6774e+10,
-2.1427e+08, 5.5673e+10],
[-2.9765e+09, 1.3335e+11, -6.6653e+08, ..., 2.8810e+11,
-1.3247e+09, 1.7413e+11],
[-1.4350e+09, 1.1349e+11, -6.1666e+09, ..., -6.3567e+07,
2.5410e+10, -2.7089e+09]],
...,
[[-1.0133e+10, 6.3976e+11, 3.3061e+11, ..., 1.0406e+12,
-8.4710e+09, 1.0253e+12],
[ 5.7545e+11, 2.5708e+11, -6.0712e+09, ..., -3.8490e+09,
2.5262e+11, -6.5651e+09],
[-5.0625e+08, 2.3240e+11, 1.1795e+12, ..., 1.0406e+12,
-6.3023e+09, 1.2622e+12],
...,
[-1.7659e+08, 7.3531e+06, 1.0318e+10, ..., 1.1937e+10,
-2.6830e+07, 1.2130e+10],
[ 3.2851e+11, -1.2840e+09, 7.2542e+11, ..., -1.7533e+09,
4.2801e+10, 2.4790e+11],
[ 4.4212e+11, -3.0921e+09, 2.6986e+11, ..., -1.4147e+09,
1.4871e+11, -5.6146e+08]],

     [[-1.0733e+10,  3.5432e+10,  8.4972e+11,  ...,  4.1215e+11,
       -7.1463e+09,  1.4012e+12],
      [ 1.5490e+11, -4.3152e+09,  3.2413e+11,  ...,  7.5080e+07,
        1.7190e+11,  7.4626e+10],
      [-3.6430e+09, -2.0520e+09,  2.0288e+12,  ...,  8.2582e+11,
       -3.0221e+09,  1.8655e+12],
      ...,
      [-1.6824e+08,  9.6719e+09, -8.1633e+06,  ...,  1.2217e+10,
       -8.1665e+07,  5.9053e+09],
      [-4.1940e+09,  9.7910e+09,  8.8177e+10,  ...,  2.6284e+11,
       -1.8999e+09,  2.6723e+11],
      [-2.3743e+09,  5.6972e+10,  4.6316e+11,  ...,  1.5755e+11,
       -1.2622e+09,  3.6932e+11]],

     [[ 1.4046e+12,  2.6915e+11, -3.8877e+09,  ..., -2.5874e+09,
        9.5870e+11, -9.8621e+09],
      [ 4.1669e+11, -2.1158e+09,  9.5867e+10,  ..., -2.7183e+09,
        2.3366e+11, -2.5589e+09],
      [ 3.2784e+12, -3.9107e+09,  1.3846e+11,  ..., -2.3535e+10,
        1.2301e+12, -2.3632e+10],
      ...,
      [-8.8327e+08,  1.0330e+10,  4.3150e+10,  ...,  3.1753e+10,
       -1.8406e+08,  7.2255e+10],
      [-8.9286e+08, -5.8699e+08,  6.0844e+11,  ...,  8.7490e+10,
       -1.7401e+09,  4.2118e+11],
      [ 6.2839e+10,  6.4362e+10, -2.8154e+09,  ..., -1.0807e+09,
        3.3338e+10, -2.1842e+09]]],


    [[[-2.1893e+09,  9.4219e+11, -2.8986e+09,  ...,  4.5507e+11,
       -1.8595e+09,  9.7841e+10],
      [-1.2287e+10,  5.3027e+11,  3.4157e+10,  ...,  6.5532e+11,
       -7.1322e+09,  8.5476e+11],
      [-2.3602e+09, -3.2034e+09,  1.3787e+12,  ...,  4.5475e+11,
       -2.0664e+09,  1.0279e+12],
      ...,
      [ 3.1172e+10, -4.6981e+05, -1.4839e+08,  ...,  3.7402e+10,
        8.4692e+09, -1.8876e+08],
      [ 2.7060e+10,  1.5803e+11, -1.2051e+10,  ..., -7.2443e+08,
        5.0649e+10, -6.1316e+09],
      [-2.0930e+10,  8.4954e+11, -7.4759e+09,  ...,  1.0983e+12,
       -7.6290e+09,  5.7981e+11]],

     [[-1.7854e+10,  9.5240e+11,  9.8380e+11,  ..., -9.1964e+08,
       -9.6824e+09,  1.8568e+12],
      [-2.4641e+10,  2.1880e+11,  1.1181e+12,  ...,  1.4332e+12,
       -1.3077e+10,  2.0802e+12],
      [-2.4202e+09, -3.3951e+09,  2.3664e+11,  ...,  2.9724e+10,
        4.0993e+11, -2.1152e+09],
      ...,
      [-6.5134e+08,  5.2344e+09,  8.6965e+09,  ..., -2.5191e+08,
       -1.2048e+08,  2.9865e+10],
      [ 7.5161e+10, -2.1900e+09,  6.4004e+11,  ..., -2.7723e+08,
        9.6922e+09,  2.6885e+11],
      [-2.3454e+10,  7.7933e+11, -6.5473e+09,  ...,  1.8540e+12,
       -1.2415e+10,  9.4083e+11]],

     [[-1.0652e+10,  7.9177e+11, -3.7253e+09,  ...,  1.8442e+11,
       -2.4431e+09,  4.6641e+11],
      [-8.6843e+09, -2.6224e+09,  7.4573e+11,  ...,  6.2365e+11,
       -3.9247e+09,  8.5665e+11],
      [-7.3245e+10,  2.5991e+12, -1.2282e+10,  ...,  7.8762e+12,
       -3.4835e+10,  4.0184e+12],
      ...,
      [-5.7342e+08,  2.5121e+10, -1.8301e+08,  ..., -3.0084e+08,
        1.3923e+09,  1.7435e+10],
      [-3.9774e+09,  1.6052e+11, -7.0406e+08,  ...,  3.8858e+11,
       -1.8563e+09,  2.4366e+11],
      [-1.4348e+09,  1.1393e+11, -6.0928e+09,  ..., -7.3272e+07,
        2.2333e+10, -2.6413e+09]],

     ...,

     [[-5.2311e+09,  6.5767e+11,  3.6572e+11,  ...,  3.9492e+11,
       -5.8626e+09,  6.0650e+11],
      [ 4.7303e+11,  2.3027e+11, -5.4838e+09,  ..., -3.2000e+09,
        2.3040e+11, -5.6923e+09],
      [-4.6000e+09, -9.4782e+08,  1.4083e+12,  ...,  9.9698e+11,
       -7.8564e+09,  1.3911e+12],
      ...,
      [-1.1914e+08, -1.6607e+07,  5.8858e+09,  ...,  6.3982e+08,
       -1.5460e+07,  6.4720e+09],
      [ 4.0109e+11, -2.7560e+09,  9.9577e+11,  ..., -2.3879e+09,
        1.1110e+11,  3.8327e+11],
      [ 4.5156e+11, -3.0508e+09,  2.6589e+11,  ..., -1.3925e+09,
        1.5315e+11, -6.6490e+08]],

     [[-8.8090e+09,  1.8876e+11,  5.8744e+11,  ...,  3.7510e+10,
       -5.5773e+09,  8.5064e+11],
      [ 2.1742e+11, -4.1756e+09,  2.8904e+11,  ..., -3.0450e+08,
        2.0684e+11,  9.3541e+09],
      [-1.0075e+10, -4.9728e+09,  2.3481e+12,  ...,  1.4027e+12,
       -6.7797e+09,  2.1131e+12],
      ...,
      [-1.1179e+08,  3.0366e+09, -3.4328e+07,  ...,  6.3145e+09,
       -5.9917e+06,  1.0533e+09],
      [-5.9569e+09,  1.2845e+11,  1.6025e+11,  ...,  4.5629e+11,
       -1.8234e+09,  3.3452e+11],
      [-2.4574e+09,  5.4820e+10,  4.5740e+11,  ...,  1.5671e+11,
       -1.2743e+09,  3.7093e+11]],

     [[ 9.3786e+11,  2.5534e+11, -5.5306e+09,  ...,  1.5376e+11,
        6.9083e+11, -8.1840e+09],
      [ 3.9118e+11, -2.2280e+09,  1.0882e+11,  ..., -2.6560e+09,
        2.3081e+11, -2.5494e+09],
      [ 4.7218e+12, -7.4998e+09,  8.4514e+11,  ..., -3.7836e+10,
        2.0993e+12, -2.1672e+10],
      ...,
      [-5.2946e+08,  7.7882e+09,  2.8906e+10,  ..., -2.8254e+07,
        8.8099e+08,  3.8150e+10],
      [-1.5570e+09, -5.2429e+08,  8.6122e+11,  ...,  2.6475e+11,
       -1.6419e+09,  5.9703e+11],
      [ 6.3615e+10,  6.7003e+10, -2.8046e+09,  ..., -1.0850e+09,
        3.2339e+10, -2.2131e+09]]],


    ...,


    [[[-2.5347e+10,  1.3258e+12, -1.9853e+10,  ...,  1.0199e+12,
       -1.3633e+10,  1.3451e+12],
      [-1.8695e+09,  2.9655e+11, -2.6234e+09,  ...,  3.8114e+11,
       -5.2636e+09,  4.6898e+11],
      [-3.8590e+09, -3.8353e+09,  8.1085e+11,  ...,  3.7843e+11,
       -2.2914e+09,  8.4360e+11],
      ...,
      [ 3.8966e+10,  3.7509e+10, -9.9483e+08,  ..., -1.2330e+08,
        1.0660e+08, -5.4407e+08],
      [-2.0997e+09,  7.0337e+10, -5.6233e+09,  ...,  6.5645e+10,
        1.3591e+11,  1.9371e+10],
      [-1.5330e+10,  3.3383e+11, -7.0775e+09,  ...,  2.4228e+11,
       -9.6654e+09,  1.0639e+12]],

     [[-2.9845e+10,  6.4809e+11,  1.2602e+12,  ...,  9.7542e+11,
       -2.4331e+10,  3.5917e+12],
      [ 1.0644e+11,  4.1329e+11, -4.8751e+08,  ...,  9.3726e+11,
       -1.0052e+10,  1.0357e+12],
      [-6.8574e+09,  8.2503e+10, -2.6066e+09,  ...,  4.3057e+11,
       -1.8156e+09,  4.1985e+11],
      ...,
      [-1.3920e+09,  1.4148e+09,  5.6420e+10,  ...,  7.0269e+10,
       -2.6763e+08,  1.0225e+11],
      [ 5.0144e+10, -6.3334e+07,  9.2801e+10,  ...,  1.4365e+10,
        3.1446e+10, -7.2564e+08],
      [-1.6401e+10,  4.3506e+11, -9.1643e+09,  ...,  6.1941e+11,
       -1.1965e+10,  1.4173e+12]],

     [[-2.3105e+10,  6.4085e+11, -9.6746e+09,  ...,  5.6172e+11,
       -1.4733e+10,  1.6728e+12],
      [ 3.1417e+11,  8.0506e+10,  2.1039e+11,  ...,  4.0535e+11,
       -2.6212e+09,  3.4268e+11],
      [-8.8271e+10,  2.6373e+12, -4.1295e+10,  ...,  5.8921e+12,
       -6.2104e+10,  7.7965e+12],
      ...,
      [-2.7841e+09,  1.2705e+11, -1.0945e+09,  ...,  9.9183e+10,
       -1.7108e+09,  2.9280e+11],
      [-1.2817e+09,  1.0343e+11,  3.0835e+09,  ...,  9.3434e+10,
       -1.8138e+09,  2.3648e+11],
      [-7.5431e+08,  1.1020e+11, -3.3122e+09,  ..., -2.0976e+08,
       -5.4143e+08, -9.3660e+08]],

     ...,

     [[-1.5603e+10,  7.8911e+11,  3.4713e+11,  ...,  5.8903e+11,
       -1.3287e+10,  1.8881e+12],
      [-8.5787e+08, -2.6150e+08, -9.3901e+08,  ..., -2.5867e+09,
        2.4819e+11, -3.0183e+09],
      [-6.4645e+09, -2.7111e+09,  6.5189e+11,  ...,  6.2382e+11,
       -6.7245e+09,  9.7201e+11],
      ...,
      [-3.1870e+08, -2.5328e+07,  2.4463e+10,  ...,  3.6012e+10,
        4.7651e+09,  3.5142e+10],
      [-7.2548e+07,  1.5751e+10,  1.4780e+11,  ..., -1.6134e+09,
       -2.0148e+09,  1.5501e+11],
      [ 3.7842e+11, -1.1211e+09,  1.7189e+11,  ...,  1.8579e+10,
        2.4236e+11, -1.8588e+09]],

     [[-1.1218e+10,  7.5936e+10,  1.2923e+12,  ...,  4.1362e+11,
       -1.0737e+10,  1.6893e+12],
      [ 1.0328e+11, -1.3370e+09,  1.7519e+11,  ...,  8.4299e+10,
        6.3660e+10,  6.4677e+10],
      [-1.4593e+10, -3.8212e+09,  1.1999e+12,  ...,  1.3949e+12,
       -7.3151e+09,  2.3509e+12],
      ...,
      [-4.7442e+08,  1.0520e+10,  1.6690e+10,  ...,  3.2117e+10,
       -1.2230e+08,  5.6691e+10],
      [ 1.0535e+11,  2.0038e+10,  5.2376e+11,  ...,  3.1615e+11,
        2.3542e+11,  6.3380e+10],
      [-2.0331e+09, -4.7692e+08,  2.1969e+11,  ...,  3.5644e+10,
       -1.0289e+09,  2.5455e+11]],

     [[ 1.0437e+12,  1.6666e+11, -5.3497e+09,  ..., -4.6010e+09,
        1.1462e+12, -1.0386e+10],
      [ 8.0269e+10, -1.1632e+09,  9.0640e+10,  ..., -1.6995e+09,
        1.7252e+11, -1.7673e+09],
      [ 6.0444e+12, -1.7520e+10,  2.7133e+12,  ..., -3.3086e+10,
        3.4502e+12, -5.5503e+10],
      ...,
      [-1.6534e+09,  3.4699e+10,  6.8339e+10,  ...,  1.3161e+11,
       -5.5683e+08,  1.9581e+11],
      [ 1.4019e+11, -1.8408e+08,  4.0969e+11,  ..., -3.0963e+08,
       -1.7223e+09,  7.4093e+10],
      [ 3.7620e+09,  9.0540e+10, -1.6924e+09,  ..., -3.7405e+08,
        5.5561e+09, -1.0311e+09]]],


    [[[-6.6930e+09,  5.9551e+11, -9.5671e+09,  ...,  6.8873e+11,
       -6.0083e+09,  4.7846e+11],
      [-9.1036e+09,  2.3810e+11, -3.3651e+09,  ...,  3.8476e+11,
       -6.2549e+09,  1.0637e+12],
      [ 1.3978e+11, -1.1303e+09,  5.5608e+11,  ...,  4.5399e+11,
       -2.1888e+09,  7.0781e+11],
      ...,
      [ 2.2803e+10, -2.4589e+07,  5.0886e+09,  ...,  7.5280e+09,
        1.5649e+10, -2.2538e+08],
      [-3.8471e+09,  1.7935e+11, -4.2868e+09,  ...,  5.6299e+11,
       -9.0098e+08, -1.8734e+09],
      [-1.5846e+10,  3.7288e+11, -7.4849e+09,  ...,  2.5362e+11,
       -1.0075e+10,  1.1175e+12]],

     [[-1.4440e+10,  3.9596e+11, -1.6765e+09,  ...,  5.8761e+11,
       -1.1362e+10,  1.6464e+12],
      [-1.8845e+10,  3.9133e+11, -3.3021e+09,  ...,  1.3930e+12,
       -1.0886e+10,  2.2821e+12],
      [ 8.2954e+10, -2.2175e+09,  4.3715e+11,  ...,  7.0758e+10,
        4.1630e+11, -5.6454e+09],
      ...,
      [-3.6508e+08,  9.4639e+09, -1.5771e+08,  ...,  6.2021e+09,
       -2.0836e+08,  3.6794e+10],
      [ 2.8941e+10, -6.4736e+08,  1.5825e+11,  ..., -9.1214e+08,
        7.5351e+10, -6.8175e+07],
      [-1.6216e+10,  4.6190e+11, -9.1941e+09,  ...,  6.0833e+11,
       -1.2114e+10,  1.4025e+12]],

     [[-9.4249e+09,  3.0783e+11, -4.5341e+09,  ...,  2.4279e+11,
       -6.2084e+09,  6.3262e+11],
      [-9.8383e+09,  2.1676e+11, -1.3692e+09,  ...,  8.4466e+11,
       -5.0601e+09,  1.1153e+12],
      [-4.5289e+10,  2.2974e+12, -3.3578e+10,  ...,  5.3516e+12,
       -4.7414e+10,  6.2797e+12],
      ...,
      [-3.5543e+08,  9.2505e+08, -1.1757e+08,  ...,  1.3958e+10,
       -6.9412e+07,  1.6058e+10],
      [-2.0608e+09,  1.0101e+11, -1.1672e+09,  ...,  2.7194e+11,
       -1.6343e+09,  3.5730e+11],
      [-7.9350e+08,  1.1139e+11, -3.3791e+09,  ..., -2.0723e+08,
       -5.8001e+08, -8.9241e+08]],

     ...,

     [[-3.4413e+09,  4.3065e+11, -2.5035e+09,  ...,  6.7786e+11,
       -5.3152e+09,  6.9742e+11],
      [ 2.0765e+11,  6.4168e+09, -1.2208e+09,  ..., -4.3521e+09,
        6.3764e+10, -3.0160e+09],
      [ 1.6411e+11,  1.7133e+11,  2.0205e+11,  ...,  9.3318e+11,
       -7.2439e+09,  1.2878e+12],
      ...,
      [-4.0944e+07,  4.1266e+09,  8.8050e+07,  ..., -3.2010e+06,
       -7.6343e+07,  2.6045e+09],
      [-6.7068e+08, -1.1596e+09,  2.5235e+11,  ..., -2.5172e+09,
        1.1890e+10,  1.0141e+10],
      [ 3.9749e+11, -1.2612e+09,  1.9021e+11,  ...,  1.7096e+10,
        2.5588e+11, -2.0554e+09]],

     [[-4.5518e+09,  3.8167e+10,  1.2992e+11,  ...,  3.3115e+11,
       -4.2669e+09,  7.1159e+11],
      [ 1.4434e+11, -2.7185e+08,  1.6379e+11,  ...,  2.9769e+11,
        1.9205e+11, -2.8744e+09],
      [-6.8184e+08, -1.1213e+09,  1.0410e+12,  ...,  1.2097e+12,
       -6.9337e+09,  1.5642e+12],
      ...,
      [-5.2693e+07,  3.9806e+09, -4.5814e+07,  ...,  1.9728e+08,
       -5.9103e+07,  1.9269e+09],
      [ 6.3414e+10, -1.7663e+08,  1.8600e+11,  ...,  3.4464e+09,
       -1.5193e+09,  1.0266e+11],
      [-2.1712e+09, -3.9121e+08,  2.1156e+11,  ...,  3.8612e+10,
       -1.0927e+09,  2.6725e+11]],

     [[ 1.0361e+12,  1.9326e+11,  1.6478e+11,  ...,  6.1522e+10,
        4.1706e+11, -5.2314e+09],
      [ 5.0765e+11, -2.0765e+09,  2.2610e+11,  ..., -3.0338e+09,
        3.3017e+11, -4.6017e+09],
      [ 2.6571e+12, -9.8119e+09,  9.6588e+11,  ..., -2.4737e+10,
        2.1214e+12, -3.0270e+10],
      ...,
      [-2.1989e+08,  1.3429e+10,  7.3580e+08,  ...,  1.6007e+10,
       -1.5727e+08,  2.2728e+10],
      [ 7.9655e+10, -4.4549e+08,  2.2490e+11,  ..., -1.6964e+09,
       -4.4920e+08,  3.2137e+11],
      [ 7.7026e+09,  8.8357e+10, -1.6514e+09,  ..., -3.3472e+08,
        9.4216e+09, -1.0344e+09]]],


    [[[-6.0862e+09,  3.8691e+11, -5.2678e+09,  ...,  5.2127e+11,
       -2.9961e+09,  2.8873e+11],
      [-6.6508e+09,  2.9480e+11, -2.0601e+09,  ...,  1.9939e+11,
       -5.8823e+09,  7.1467e+11],
      [ 1.9496e+11,  4.5957e+10,  5.7836e+11,  ...,  3.3471e+10,
       -5.6642e+08,  2.5977e+11],
      ...,
      [ 4.3017e+10,  3.8735e+10, -5.5557e+08,  ..., -9.0509e+08,
       -4.4149e+07, -7.9349e+08],
      [-1.7535e+09,  8.6028e+10, -4.2015e+09,  ..., -5.3992e+08,
        3.2805e+11, -2.1209e+09],
      [-1.5237e+10,  3.6046e+11, -7.2527e+09,  ...,  2.3167e+11,
       -9.8746e+09,  1.0742e+12]],

     [[-6.0950e+09,  3.6948e+11,  3.2411e+11,  ...,  3.5775e+11,
       -4.8411e+09,  6.5723e+11],
      [-1.2751e+10,  5.6651e+11,  7.3168e+10,  ...,  5.8541e+11,
       -1.1053e+10,  1.7232e+12],
      [-4.0308e+09, -5.4738e+09, -1.9349e+09,  ...,  5.2030e+11,
       -2.3041e+09,  3.5089e+11],
      ...,
      [-9.2366e+08, -2.8772e+08,  4.9176e+10,  ...,  1.0876e+11,
       -1.8715e+08,  9.6827e+10],
      [ 1.3452e+11, -1.5527e+08,  1.3643e+11,  ...,  7.7090e+09,
       -6.3393e+08, -1.2348e+08],
      [-1.7448e+10,  4.9618e+11, -9.7148e+09,  ...,  6.7532e+11,
       -1.2646e+10,  1.5301e+12]],

     [[-7.5562e+09,  3.5645e+11, -3.5173e+09,  ...,  3.3042e+11,
       -3.6222e+09,  4.4826e+11],
      [-7.0562e+09,  2.3808e+11,  9.9487e+10,  ...,  3.9518e+11,
       -5.3996e+09,  9.9114e+11],
      [-1.8159e+10,  2.7511e+12, -1.4330e+10,  ...,  3.1170e+12,
       -2.1891e+10,  2.8215e+12],
      ...,
      [-2.6588e+09,  7.2310e+10, -1.1256e+09,  ...,  1.9049e+11,
       -1.1892e+09,  2.9009e+11],
      [-2.9263e+08,  6.4129e+10, -2.0283e+08,  ...,  1.3579e+11,
       -1.0087e+09,  1.8545e+11],
      [-6.6394e+08,  1.0839e+11, -3.3324e+09,  ..., -2.8693e+08,
       -5.4105e+08, -9.6124e+08]],

     ...,

     [[-2.3624e+09,  2.6150e+11,  6.5184e+10,  ...,  4.2481e+11,
       -2.2346e+09,  2.9386e+11],
      [ 7.2819e+10, -9.8933e+08, -2.1938e+09,  ..., -1.0727e+09,
        7.0446e+10, -2.6725e+09],
      [ 3.8182e+11,  6.2986e+11,  7.9217e+11,  ..., -4.8056e+08,
        1.7333e+11, -9.6867e+08],
      ...,
      [-3.1505e+08, -1.0228e+07,  1.4509e+10,  ...,  3.9852e+10,
        1.3815e+10,  3.4597e+10],
      [ 4.8238e+10,  3.2595e+10,  2.4364e+11,  ..., -7.2559e+08,
       -1.9649e+09,  1.1855e+11],
      [ 3.6643e+11, -1.1649e+09,  1.7557e+11,  ...,  3.1998e+10,
        2.4436e+11, -1.7686e+09]],

     [[-1.3028e+09,  9.7525e+10,  3.4656e+11,  ...,  1.4875e+11,
       -1.6537e+09,  2.7661e+11],
      [ 1.8280e+11,  1.7359e+10,  2.1112e+11,  ...,  4.7727e+10,
        2.4419e+11, -1.3106e+09],
      [ 5.4858e+11, -1.4787e+09,  9.4358e+11,  ...,  3.6515e+11,
       -3.5482e+09,  6.4555e+11],
      ...,
      [-5.8247e+08,  1.7022e+10, -8.9132e+07,  ...,  5.0501e+10,
       -7.8562e+07,  5.6358e+10],
      [-4.7032e+08,  5.9142e+10,  1.2843e+11,  ...,  3.2640e+11,
       -1.0235e+09,  1.9908e+11],
      [-1.9675e+09, -4.5512e+08,  2.2173e+11,  ...,  2.8949e+10,
       -1.0155e+09,  2.4508e+11]],

     [[ 3.3498e+11,  1.3946e+11, -2.8348e+09,  ...,  2.1367e+11,
       -5.8016e+08,  1.3514e+10],
      [ 3.3158e+11, -2.1205e+09,  1.4036e+11,  ..., -1.3339e+09,
        2.4158e+11, -2.9523e+09],
      [ 7.3816e+11,  2.2258e+11,  1.1799e+12,  ..., -2.0772e+10,
        1.5610e+12, -2.0085e+10],
      ...,
      [-1.4994e+09,  1.4466e+10,  3.6015e+10,  ...,  1.4090e+11,
       -6.1795e+08,  1.6893e+11],
      [ 2.3953e+11, -4.8849e+08,  3.2231e+11,  ...,  9.3758e+09,
       -2.3517e+09,  1.3778e+11],
      [ 1.0885e+10,  8.4943e+10, -1.5894e+09,  ..., -2.8764e+08,
        1.2211e+10, -9.6989e+08]]]], device='cuda:0')

is it normal?

How to pretrain in a new dataset?

Dear author,

Thank you for open-sourcing this very meaningful work. I have a couple of questions, and I would appreciate it if you could provide some clarification:

Is GPT-ST only applicable to the four transportation datasets mentioned in your paper? Can I modify certain parameters and run it on other datasets after converting them into time and spatial feature files?

In your project introduction document, I noticed that you employ different running modes and baseline models. Is this a fixed combination? For instance, if I want to use the STGCN baseline model to pre-train my own spatiotemporal dataset, is it feasible?

Thank you very much for your response!