Incorrect stats computed/stored for partitioned Hive tables about presto

[aaneja]

The reason for this is that we do partition sampling (default 100) and extrapolate our stats based on the fetched partition stats. By increasing the sample count to greater than actual partitions, we see that the table stats match the unpartitioned counterpart -

presto:tpcds_sf1_parquet> set session hive.partition_statistics_sample_size=1825;
SET SESSION
presto:tpcds_sf1_parquet> show stats for hive.tpcds_sf1000_parquet_varchar_part.store_sales;
      column_name      | data_size | distinct_values_count |    nulls_fraction    |   row_count   | low_value | high_value | histogram 
-----------------------+-----------+-----------------------+----------------------+---------------+-----------+------------+-----------
 ss_sold_time_sk       | NULL      |               47961.0 |  0.04499776111740666 | NULL          | 28800     | 75599      | NULL      
 ss_item_sk            | NULL      |              297612.0 |                  0.0 | NULL          | 1         | 300000     | NULL      
 ss_customer_sk        | NULL      |           1.2124495E7 |  0.04499698125304584 | NULL          | 1         | 12000000   | NULL      
 ss_cdemo_sk           | NULL      |             1890006.0 |  0.04500093578341331 | NULL          | 1         | 1920800    | NULL      
 ss_hdemo_sk           | NULL      |                7082.0 | 0.044998298272422764 | NULL          | 1         | 7200       | NULL      
 ss_addr_sk            | NULL      |             5947530.0 | 0.044996645487757815 | NULL          | 1         | 6000000    | NULL      
 ss_store_sk           | NULL      |                 513.0 |  0.04499048261485481 | NULL          | 1         | 1000       | NULL      
 ss_promo_sk           | NULL      |                1483.0 |  0.04499917917887129 | NULL          | 1         | 1500       | NULL      
 ss_ticket_number      | NULL      |           1.0071624E8 |                  0.0 | NULL          | 1         | 240000000  | NULL      
 ss_quantity           | NULL      |                 100.0 |  0.04499472152140729 | NULL          | 1         | 100        | NULL      
 ss_wholesale_cost     | NULL      |               10091.0 |  0.04499681007177697 | NULL          | 1.0       | 100.0      | NULL      
 ss_list_price         | NULL      |               19495.0 |  0.04499915278987244 | NULL          | 1.0       | 200.0      | NULL      
 ss_sales_price        | NULL      |               19348.0 | 0.044999514249711985 | NULL          | 0.0       | 200.0      | NULL      
 ss_ext_discount_amt   | NULL      |              529048.0 |  0.04500334759901894 | NULL          | 0.0       | 19778.0    | NULL      
 ss_ext_sales_price    | NULL      |              600147.0 |  0.04499642951463563 | NULL          | 0.0       | 19972.0    | NULL      
 ss_ext_wholesale_cost | NULL      |              388752.0 |  0.04499845764808689 | NULL          | 1.0       | 10000.0    | NULL      
 ss_ext_list_price     | NULL      |              731384.0 |  0.04499803472965792 | NULL          | 1.0       | 20000.0    | NULL      
 ss_ext_tax            | NULL      |              115731.0 |  0.04499627500010287 | NULL          | 0.0       | 1797.48    | NULL      
 ss_coupon_amt         | NULL      |              529048.0 |  0.04500334759901894 | NULL          | 0.0       | 19778.0    | NULL      
 ss_net_paid           | NULL      |              786675.0 |   0.0449999816127706 | NULL          | 0.0       | 19972.0    | NULL      
 ss_net_paid_inc_tax   | NULL      |             1098165.0 |  0.04500332989061181 | NULL          | 0.0       | 21769.48   | NULL      
 ss_net_profit         | NULL      |             1023699.0 | 0.044990789213354636 | NULL          | -10000.0  | 9986.0     | NULL      
 ss_sold_date_sk       | NULL      |                1823.0 |  0.04500048022595944 | NULL          | 2450816   | 2452642    | NULL      
 NULL                  | NULL      | NULL                  | NULL                 | 2.879987999E9 | NULL      | NULL       | NULL      
(24 rows)