prestodb / presto-go-client Goto Github PK

It gives me presto: Error loading krb config: error processing libdefaults section: invalid krb5 config libdefaults section line (}) error.

Has anyone tested it?
My keb config file is fine and I tested it on python.

When my user use the presto-go-client to query the presto,the Source is null. And When I use the presto-python-client, the Source value is presto-python-client. I think the Source should have a default value and do not should be null. Because if the default Source value is not null ,the presto maintainer will be more easier to manage the queries.

The following picture shows the question:

How to set the parameter "krb5-disable-remote-service-hostname-canonicalization" for Kerberos authentication ?

SQL：
select id from table where year = '2019' and month = '08' and day = '05' and id = 360287970235245893 limit 1;

but the result is:
360287970235245888

The id in table is bigint type.

Is there support for JWT as an authentication mechanism? i.e adding a Authorization: Bearer <jwt> header to the http request for authentication or an access_token=<jwt> query parameter?

If not, is there any notable downsides to this / curious why it's not yet implemented. If there are no apparent downsides, I would be happy to add support for JWT based auth to this library.

Example Code:

	prestoConfig := presto.Config{
		PrestoURI:          "https://username:[email protected]:443",
		Source:             "",
		Catalog:            "hive",
		Schema:             "public",
		SessionProperties:  nil,
		CustomClientName:   "",
		KerberosEnabled:    "",
		KerberosKeytabPath: "",
		KerberosPrincipal:  "",
		KerberosRealm:      "",
		KerberosConfigPath: "",
		SSLCertPath:        "/tmp/certs/datagateway_cert.pem",
	}
	dsn, err := prestoConfig.FormatDSN()

	helper.Check(err)
	fmt.Println("Going to call with DSN: ", dsn)
	db, err := sql.Open("presto", dsn)
	helper.Check(err)

	ctx := context.Background()
	conn, err := db.Conn(ctx)
	helper.Check(err)

	groupID := 129565
	rows, err := conn.QueryContext(ctx, "SELECT driver_id, updated_at FROM public.driver_groups_drivers WHERE driver_group_id = " + strconv.Itoa(groupID) + " AND deleted_at IS NULL")
	helper.Check(err)

	result := make([]int64, 0, 1000)
	for rows.NextResultSet() {
		fmt.Println("Got next result set")
		for rows.Next() {
			fmt.Println("Loading record...")
			var daxID int64
			var updatedAt time.Time
			rows.Scan(&daxID, &updatedAt)
			result = append(result, daxID)
		}
	}

	err = rows.Close()
	helper.Check(err)

	fmt.Println("Results: ", result)

What I Get

Empty Result

What I Expect

the query result with data

A column with a struct type, e.g.

struct_col struct<i:int,s:string>

gets treated as a

row(i integer, s varchar)

which is currently unsupported by the ConvertValue function (

I see from #56 that you can query individual fields from there, but not the col as a whole.

Might use db.begin() to start a transaction.

First off, great work this lib is super easy to use & works like a charm.

One question though,

I've got a column of type ROW & am trying to query it. However when I have a query that includes it, rows.Next() always returns false, even when there are results.

Example:
some_table has 10 rows

rows, _ := db.Query(`SELECT row_type_column FROM some_table`)
for rows.Next() {
    // never gets called; rows.Next() returns false (though there is no error)
}

rows, _ := db.Query(`SELECT primitive_column FROM some_table`)
for rows.Next() {
        // loops through 10x as expected
}

I know the docs don't mention support for a presto ROW but I'm wondering if there would be a way to do so that I'm overlooking, or any pointers towards adding support for that myself.

Add support for fixed-precision type: https://prestodb.io/docs/current/language/types.html#fixed-precision

@skogtwin found this bug where queries using variadic arguments (e.g. SELECT x FROM y WHERE z = ?) fail when the columns selected are not star or more than a single column. Beautiful.

There's a comment in #29 on the line that needs fix.

You can get an unknown column type by running the query select null;. Instead of being able to scan these into a string the driver returns an error, even though it's a valid query. I fixed it by including in the switch statement with the other character types, although I'm not sure that's the right fix.

Support Basic Authentication when the Presto end-point is configured for LDAP Authentication.

select user_id
from my_table
where pdate='2022-10-09'
and newuser_in_day = 0 limit 1000

github.com/prestodb/[email protected]/presto/presto.go:767 +0x338

how to free memory shared, waiting for help

go tool pprof -alloc_space http://xxxxx:8899/debug/pprof/heap

package main

import (
"database/sql"
"fmt"
_ "github.com/prestodb/presto-go-client/presto"
"net/http"
_ "net/http/pprof"
"time"
)

var SerNum int

func ServerNum(app string, dt time.Time) int {

dsn := "http://root@xxxxx:8080?catalog=hive&schema=tds_tmp"
db, err := sql.Open("presto", dsn)
if err != nil {
	fmt.Println(err)
}
defer db.Close()

s := fmt.Sprintf(
	`select  count(ip) from xxx where dt='%s' and  appname='%s'`, dt.Format("20060102"), app)

err = db.QueryRow(s).Scan(&SerNum)

if err != nil {
	fmt.Println(err)
} else {
	return SerNum
}

return 0

}

func main() {

go func() {
	_ = http.ListenAndServe("0.0.0.0:8899", nil)
}()

for i := 0; i < 1000; i++ {
	num := ServerNum("xxx", time.Now())
	fmt.Println(num)
}

}

Is there any plan for support of canceling with context?
This feature is implemented on other database/sql drivers like on this one for PG.

A common scenario for the usage of this feature is when a user closes the request but the query remains on running state and it will cause to occupying system resources on heavy queries.

Hi,
There is a potential bug found in presto repo describe as following:
prestodb/presto#14807
And try to fix as following:
prestodb/presto#14817

I am not sure anything need to be updated in the go client repo. Thanks.

Presto now may returns a list of warnings in the warnings field of the JSON struct returned by then coordinator when the client polls the status of a query.

The warning are useful to surface issues in queries at runtime.

Currently, using query parameters in the popular sqlx library will result in an ErrOperationNotSupported due to QueryContext returning the error if it encounters a non-empty args array.

The following arbitrary SQL statements will return ErrOperationNotSupported error:
db.Select(&interface, `SELECT * FROM table WHERE amount > $1`, 100)
or such:
db.Select(&interface, `SELECT * FROM table WHERE amount > ?`, 100)

Is there any plans to implement these operations? This is the relevant section on the sqlx on this functionality.

https://github.com/prestodb/presto/wiki/Presto-v2-client-protocol

It currently links to the source code, not the rendered documentation.

https://prestodb.io/docs/current/overview/concepts.html

Hi,i was trying to connect to presto using this repo ,but i was getting authorization error for user which was registered.The reason for this i think is the quotes added against the username.for eg -> if my user name is abc ,it was becoming 'abc' while sending call hence causing unauthorzied

I solved the problem by cloning repo to my local and making two changes to it.

1. changing file serial.go line no 89 as->
   case string:
   return strings.Replace(x, "'", "''", -1), nil

2. removing line 614->query = "EXECUTE " + preparedStatementName + " USING " + strings.Join(ss, ", ")
   from presto.go file

is there a better way of doing this?

thank you

The current typeConverter.ConvertValue function always return the default value for types, as opposed to nil, which causes types such as sql.Null* to always have Valid=true even when the value returned by the query is NULL.

e.g.

// ConvertValue implements the driver.ValueConverter interface.
func (c *typeConverter) ConvertValue(v interface{}) (driver.Value, error) {
	switch c.parsedType[0] {
	case "boolean":
		vv, err := scanNullBool(v)
		return vv.Bool, err

The last line causes the returned sql.NullBool to always be valid. In order to fix, we have to:

if !vv.Valid { return nil, err }

Our integration tests for type conversion should test this. I found this by accident.

We use Query() for the write operations, of which the result could be empty.
It is wired that we use other libs like thrift to do write operations of presto.
I think many users need this feature. So that we can use this lib just as we do by database/sql

Is there any plan to support Exec function? "presto: operation not supported " is returned when calling exec function, thanks!

CREATE TABLE test_dl (
userip STRING,
dmname STRING,
time STRING,
serverip STRING
)
STORED AS PARQUET
TBLPROPERTIES ("serialization.null.format"="")

insert SQL:

insert into test_dl
select
userip userip,
dmname dmname,
time time,
coalesce(serverip, '0.0.0.0') serverip
from tmp_1516270919359966300_rxu4khl0

code:

var i interface{}
err := db.QueryRow(sql).Scan(&i)
if err != nil {
return err
}
fmt.Println("info:", i)

run this program is success, but uesd presto-cli run sql "select count(*) from test_dl" rows count is 0?

There is one struct but not used named infoResponse.

type infoResponse struct { QueryID string json:"queryId"State stringjson:"state" }

presto supports the ROW type.
It seems it is not supported by this go driver, yet no error is reported to the caller.
I test all error code and nothing happens. It seems to be ignoring the data and be happy about it.
I've been able to use it without issues on other tables with simpler schema. :/