lib / pq Goto Github PK

type Domain struct {
    UpdatedAt        time.Time
}

$ psql maestro
maestro=# SELECT * FROM domains;
 id |        name        | domain_group_id | base_domain |         created_at         |         updated_at         |         deleted_at         |  txid   
----+--------------------+-----------------+-------------+----------------------------+----------------------------+----------------------------+---------
  1 | herro.com          |               1 | herro.com   | 2011-12-07 10:42:49.398783 |                            |                            | 1558846

2012/03/28 11:25:51 level=error event=scan_error error="sql: Scan error on column index 5: unsupported driver -> Scan pair: <nil> -> *time.Time"

I've tried parsing it to bool, sql.NullString, time.Time, none of them work. It's the same basic "Scan error" for any type.

If I perform

err := txn.QueryRow(query, args...).Scan(&out) // no error, returns rows

on the transaction txn the rows are correctly returned. If I do the same with a prepared statement they're not:

stmt, err := txn.Prepare(query)
if err != nil {
  panic(err)
}
err := stmt.QueryRow(args...).Scan(&out) // yields ErrNoRows error

Is it possible to support array types?
http://www.postgresql.org/docs/9.2/static/arrays.html

From a cursory look at encode.go, both encode() and decode() functions would need to be extended to take or return a slice of any of the current types they handle. Perhaps other parts need changing as well?

Do you think it is feasible to extend this library to support arrays, or would I be better to use the cgo wrapper around libpq for such native/non-standard SQL support? (though ARRAY is actually part of SQL99 standard, so maybe other DBs that don't support it should be called non-std :-)

Ideally, I'd like to do something like:

db.Exec("INSERT INTO t (k, v) VALUES ($1, $2)", "key", []string{"value1", "value2"})

Thanks,

Hi,

There is no mention of license on which pq is released.

I'm afraid this means the code is copyrighted and thus it's adoption is severely limited

With connection options:

application_name=users_tests dbname=test user=testuser password=testpass host=localhost port=5432 connect_timeout=3 client_encoding=UTF8

Emit:
_, err = db.Exec("select users.id where sessions.user_id = users.id and users.email = ($1)", "[email protected]");

where there is a syntax error that is called out by the postgres process:

ERROR:  missing FROM-clause entry for table "users" at character 8
STATEMENT:  select users.id where sessions.user_id = users.id and users.email = ($1)

However, no error is returned by db.Exec. Instead, the next query (that's correct) on the connection will block the code and the process will never get past it:

_, err = db.Exec("insert into users (id, email) values (($1), ($2), "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "");

We used the name with capital letters, such as a tables tb_Users , tb_WorkLists or functions f_GetUserName, f_SetPosition and etc. Trying to use them get errors, and in the text shows that the names are used with lower-case letters, for example:
Error request: pq: P:"8" H:"No function matches the given name and argument types. You might need to add explicit type casts." F:"parse_func.c" S:"ERROR" C:"42883" M:"function main.f_getusername(unknown) does not exist" R:"ParseFuncOrColumn" L:"304"

If the database to change the name to lowercase all works, but then stop running other programs. Maybe somehow disable translation in small letters?

I'm getting a panic in Go tip with code that works with 1.0.3.

Relevant code:

rows, err := tdb.Db.Query("SELECT "+siteColumns+" FROM site, member WHERE member.\"user\" = $1 AND member.\"level\" >= 100 AND site.id = member.\"site\"", userId)
if err != nil {
    return nil, err
}

sites := make([]*common.Site, 0)
for rows.Next() {
    ...
}

Panic happens at rows.Next()

Traceback:

panic: runtime error: slice bounds out of range [recovered]
    panic: runtime error: slice bounds out of range

goroutine 25 [running]:
github.com/bmizerany/pq.errRecover(0x108d8950, 0x108d8100)
    /Users/samuel/go/src/github.com/bmizerany/pq/error.go:89 +0xda
----- stack segment boundary -----
github.com/bmizerany/pq.(*rows).Next(0xc2002f0410, 0xc2002ee6c0, 0x11, 0x11, 0x0, ...)
    /Users/samuel/go/src/github.com/bmizerany/pq/conn.go:574 +0x182
database/sql.(*Rows).Next(0xc2002ea7e0, 0x0, 0x0, 0x80f728)
    /Users/samuel/go/src/pkg/database/sql/sql.go:893 +0xc1

I'm getting unexpected results from ORDER BY queries. I'm running PostgreSQL and Go v1.0.3 on OSX.

I have a table with two columns: name varchar, a int

Inserted rows are: ("Alice", 2) and ("Bob", 1)

If I run my query like this: rows, err := db.Query("SELECT name FROM foo ORDER BY a") I get what I want ("Bob", "Alice")

But if I run it like this rows, err = db.Query("SELECT name FROM foo ORDER BY $1", "a") I get them by insertion order ("Alice", "Bob")

Am I doing something wrong?

Complete code: http://paste2.org/p/2537881

in conn.go:220

which makes
make([]byte,-4)
The problem appears intermittent

panic: runtime error: makeslice: len out of range [recovered]
    panic: runtime error: makeslice: len out of range [recovered]
    panic: runtime error: makeslice: len out of range

goroutine 4 [running]:
testing._func_003(0x7fd7439b6fa8, 0x7fd7439b6100, 0x7fd7439b6650, 0x7fd7439b6570)
    /usr/lib/go/src/pkg/testing/testing.go:268 +0xf3
----- stack segment boundary -----
github.com/bmizerany/pq.errRecover(0x7fd7439b68c0, 0x7fd7439b6100)
    /home/ttallman/workspace/newRuleEngine/src/github.com/bmizerany/pq/error.go:61 +0xec
----- stack segment boundary -----
github.com/bmizerany/pq.(_conn).recv1(0xf840056980, 0xf840055a44, 0xf840058b40, 0x481e71)
    /home/ttallman/workspace/newRuleEngine/src/github.com/bmizerany/pq/conn.go:221 +0x247
github.com/bmizerany/pq.(_conn).recv(0xf840056980, 0x0, 0x0, 0x8, 0x7fd7439b6910, ...)
    /home/ttallman/workspace/newRuleEngine/src/github.com/bmizerany/pq/conn.go:198 +0x2a
github.com/bmizerany/pq.(_rows).Next(0xf840059930, 0xf8400961e0, 0x500000005, 0x0, 0x0, ...)
    /home/ttallman/workspace/newRuleEngine/src/github.com/bmizerany/pq/conn.go:459 +0x8a
database/sql.(_Rows).Next(0xf8400763c0, 0x7fd7439b6c10, 0x500000005, 0x0)
    /usr/lib/go/src/pkg/database/sql/sql.go:912 +0x100
rules.TestDb(0xf840000230, 0x2359ce0)
    /home/ttallman/workspace/newRuleEngine/src/rules/temp_test.go:91 +0x2fd
testing.tRunner(0xf840000230, 0x6c9af0, 0x0, 0x0)
    /usr/lib/go/src/pkg/testing/testing.go:273 +0x6f
created by testing.RunTests
    /usr/lib/go/src/pkg/testing/testing.go:349 +0x77c

goroutine 1 [chan receive]:
testing.RunTests(0x400c00, 0x6c9af0, 0x100000001, 0x1, 0x6ba838, ...)
    /usr/lib/go/src/pkg/testing/testing.go:350 +0x79f
testing.Main(0x400c00, 0x6c9af0, 0x100000001, 0x6cf100, 0x0, ...)
    /usr/lib/go/src/pkg/testing/testing.go:285 +0x7a
main.main()
    /tmp/go-build872041093/rules/_test/_testmain.go:43 +0x91

goroutine 2 [syscall]:
created by runtime.main
    /build/buildd/golang-weekly-2012-03-27/src/pkg/runtime/proc.c:221

goroutine 3 [select]:
code.google.com/p/tcgl/monitoring.backend()
    /usr/lib/go/src/pkg/code.google.com/p/tcgl/monitoring/monitoring.go:297 +0x2bd
created by code.google.com/p/tcgl/monitoring.init·1
    /usr/lib/go/src/pkg/code.google.com/p/tcgl/monitoring/monitoring.go:97 +0x144

goroutine 5 [syscall]:
syscall.Syscall6()
    /build/buildd/golang-weekly-2012-03-27/src/pkg/syscall/asm_linux_amd64.s:40 +0x5
syscall.EpollWait(0xf800000008, 0xf840077170, 0xa0000000a, 0xffffffff, 0xc, ...)
    /usr/lib/go/src/pkg/syscall/zerrors_linux_amd64.go:1781 +0xa1
net.(_pollster).WaitFD(0xf840077160, 0xf840055b40, 0x0, 0x0, 0x0, ...)
    /usr/lib/go/src/pkg/net/fd_linux.go:146 +0x110
net.(_pollServer).Run(0xf840055b40, 0x0)
    /usr/lib/go/src/pkg/net/fd.go:236 +0xe4
created by net.newPollServer
    /usr/lib/go/src/pkg/net/newpollserver.go:35 +0x382
exit status 2

```
```

Please modify driver's registration logic.
If I want to work with database/sql package only (which in my opinion is the "better" way) currently I have to add
import _ "github.com/bmizerany/pq"
to trigger pq's init() to call the driver registration on line 33.

I believe that _ imports are in general "bad" as a design decision, and using standard go's lib's sql package is the preferred way to work with sql databases.

At the very least, please put a note in the readme to avoid confusion caused by the lack of (exported) Driver.

Proposed solutions:

• export the driver, remove sql.Register from init()
• add explicit func Register(), remove sql.Register from init()
• leave as is, but document somewhere

When I send a query parameter that is the wrong type, I get an (expected) error message like "invalid input syntax for integer". But then any queries I run after that return errors too: usually something like "pq: unexpected parse response: '3'"

So apparently when the driver receives an "invalid input syntax" error, it needs to read some more messages before returning—or just close the connection and open a new one.

I'm using PostgreSQL 9.1alpha2

Sample program that exhibits this error:

package main

import (
    "database/sql"
    "fmt"
    _ "github.com/bmizerany/pq"
)

func main() {
    db, err := sql.Open("postgres", "user=postgres dbname=postgres sslmode=disable")
    if err != nil {
        fmt.Println(err)
        return
    }

    _, err = db.Exec("create table test (i integer)")
    if err != nil {
        fmt.Println(err)
        return
    }

    _, err = db.Query("select * from test where i=$1", "hhh")
    fmt.Println(err)

    _, err = db.Query("select * from test where i=$1", 1)
    fmt.Println(err)
}

I'm finding that I can execute both regex and equality queries like this successfully:

rows, err := db.Query("select id from people where firstname ~* '^pete' and lastname ~* '^pu'")
rows, err := db.Query("select id from people where firstname ~* '^pe'")
rows, err := db.Query("select id from people where firstname = 'Peter' and lastname = 'Puma'")

(etc.) and then I can iterate through rows.Next() to see the results.

But when I try these same queries as prepared statements, I get these kinds of errors from the go executable:

pq: C:"42601" R:"scanner_yyerror" S:"ERROR" M:"syntax error at or near "and"" P:"43" L:"1001" F:"scan.l"
pq: L:"993" P:"42" S:"ERROR" M:"syntax error at end of input" C:"42601" F:"scan.l" R:"scanner_yyerror"

and this is what's in the postgres logfile:

ERROR: syntax error at or near "and" at character 44
STATEMENT: select id from people where firstname ~* ? and lastname ~* ?
ERROR: syntax error at or near "and" at character 43
STATEMENT: select id from people where firstname = ? and lastname = ?
ERROR: syntax error at end of input at character 42
STATEMENT: select id from people where firstname = ?

Here's how I'm using Prepare and Query in my code (db is the result of a successful sql.Open() statement):

    stmt, err := db.Prepare("select id from people where firstname = ?")
    if err != nil {
        fmt.Println(err)
        return
    }
    defer stmt.Close()

    rows, err := stmt.Query("Peter")
    if err != nil {
        fmt.Println(err)
        return
    }

    for rows.Next() {
        var id string
        rows.Scan(&id)
        fmt.Println(id)
    }

I've varied the lines with db.Prepare() and stmt.Query() like this, but they all result in the same type of error:

    stmt, err := db.Prepare("select id from people where firstname = ? and lastname = ?")
    rows, err := stmt.Query("Peter", "Puma")

    stmt, err := db.Prepare("select id from people where firstname ~* ? and lastname ~* ?")
    rows, err := stmt.Query("^pete", "^pu")

    stmt, err := db.Prepare("select id from people where firstname = ?")
    rows, err := stmt.Query("Peter")

Accessing a server without MD5-encrypted password gives the following error: pq: unknown authentication response: 3

Patch: http://paste.debian.net/203995

I've run pq on both Linux (Ubuntu 12.10) and Windows7. I've downloaded the latest version as of 15-Jan-2012.

On Linux, no problems encountered, but I haven't done a lot of testing.

On Windows, it is very slow. eg. simple SELECT on small table taking (say) 2 seconds. A minor update involving an update on one table and an insert on another with COMMIT taking 4.5 seconds.

psql runs find on Windows, no slowness detected.

Running "GO test" takes 0.321 seconds (Linux), and it takes 39.287 seconds (Windows). Subsequent runs on Linux get it just under 0.100 seconds, and on Windows just under 39 seconds.

I had to alter conn_test.go (on Windows) because running the test gave repeated errors of 'pq: Role "ta10\Brian" does not exist'. I attempted to create role 'ta10\Brian', but psql gave error. I created role 'Brian', but still got error. Therefore, I altered "conn_test.go" as follows:

    ////conn, err := sql.Open("postgres", "")
    conn, err := sql.Open("postgres",
    "user=postgres dbname=postgres password=super")

Working on a new project and wanted to try go. Simple benchmark alternating between insert and select are not what I expected for the go version. Am I doing something wrong?

The go version results in 13.46s user 12.16s system 56% cpu 45.257 total for this code

db, err := sql.Open("postgres", "user=pgdev dbname=db_dev password=password sslmode=disable")
defer db.Close()

_, err = db.Exec("BEGIN")

for i := 0; i < 100000; i++ {

    if i % 2 == 0 {
        _, err := db.Exec("INSERT INTO users (user_name, first_name, last_name) VALUES ('go', 'is', 'it')")
    } else {
        rows, err := db.Query("SELECT user_name, first_name, last_name from users LIMIT 50")

        for rows.Next() {
            if i == 3 {
                var user_name string
                var first_name string
                var last_name string
                err = rows.Scan(&user_name, &first_name, &last_name)
                fmt.Println(user_name, first_name, last_name)
            }
        }

    }
}
db.Exec("COMMIT")

The Python (libpg) version result 2.67s user 1.01s system 28% cpu 12.691 total

conn = psycopg2.connect(database="db_dev", user="pgdev", password="password")
cur = conn.cursor()
for i in range(100000):
    if i % 2 == 0:
        cur.execute("INSERT INTO users (user_name, first_name, last_name) VALUES (%s, %s, %s);", ('psycopg2', 'bar', 'baz'))
        if i == 2:
            pass
            # print i
    else:
        cur.execute("SELECT user_name, first_name, last_name FROM users LIMIT 50;")
        for row in cur:
            # pass
            if i == 3:
                print row
            else:
                pass

conn.commit()
cur.close()
conn.close()

A query like select to_tsquery('english', 't') results in "NOTICE: text-search query contains only stop words or doesn't contain lexemes, ignored" depending on the log level you have set.

The PQ driver when using Query() the other hand results in odd error message pq: unexpected bind response: 'N'

I assume the 'N' is the first letter of the NOTICE but I wouldn't have expected an error to be produced at all. The following code demonstrates:

package main

import (
  "database/sql"
  _ "github.com/bmizerany/pq"
  "log"
)

func main() {
  db, err := sql.Open("postgres", "dbname=nd sslmode=disable")
  if err != nil {
    log.Fatal(err)
  }

  // Unexpected error for NOTICE.
  if _, err := db.Query("select to_tsquery('english', $1)", "t"); err != nil {
    log.Fatal(err)
  }
}

Have you guys tried to add the COPY command at all?

I want to use pq in a middleware program that will serve more than 100 web clients simultaneously and it's possible to need a pool of 10 or 20 active connections to a PostgreSQL database and pool the connections between goroutines serving different web rest-services.
Is there an example how to do that using your pq driver?
In the golang docs there is something written about "database/sql" package:
"If the underlying database driver has the concept of a connection and per-connection session state, the sql package manages creating and freeing connections automatically, including maintaining a free pool of idle connections. "

Best regards,
Constantin Teodorescu

Wouldn't be nice to have the chance of getting the rows.Next() as a whole map[fieldname]interface{}.

Teo

I'm not actively using pq as much as I used to because my current Go projects do not use SQL databases. Therefore, I'm having trouble context switching back to SQL land to review and merge all the great patches everyone is submitting. I greatly appreciate all of them and want to see them get in ASAP. I still love my postgres, so I want this project to keep moving forward and be the best driver for PG in Go.

I'm looking for volunteers ( @fdr and other submitters) to help. Please let me know if you're interested.

In https://github.com/kr/hk/blob/4ecd30859355794be94eb7eb46545c0f1a7efeff/hkdist/web.go#L146-L163 I'm selecting a bytea value, and I get back this error:

web.go:163: pq: encoding/hex: odd length hex string

Data:

\\230\\357\\031Br\\333Q\\344s\\236\\003G\\264\\273\\323\\267-\\275_\\261\\240\\217^\\007;%\\2304\\022@\\311*

Will this library provides SQL-sanitization that helps avoid SQL-injection?

Executing a parameterized query

https://gist.github.com/4140487#

fails with the following output

https://gist.github.com/4140483#

The commented line works fine. Am I doing this wrong?

The time.Time parsing seems to be having issues with postgres' variable number of significant figures in timestamps. Some debug output I've gathered:

parsing time: 2012-04-03 22:52:44.390114  using  2006-01-02 15:04:05
parsing time: 2012-04-03 22:52:44.465705  using  2006-01-02 15:04:05
parsing time: 2012-04-03 22:52:44.494043  using  2006-01-02 15:04:05
parsing time: 2012-04-03 22:52:44.499289  using  2006-01-02 15:04:05
parsing time: 2012-04-03 22:52:44.5  using  2006-01-02 15:04:05
2012/04/06 10:27:24 level=error event=pq_rows_iteration error="pq: decode: parsing time \"2012-04-03 22:52:44.5\": extra text: .5"

So it can parse the string if there's 6 digits after the decimal, but for some reason it barfs if there's just one. Is this a time.Parse() issue?

In the current tip of Go, int is 64 bits on amd64. This causes problems for pq, because an uint32 converted to int is checked if it is -1. Since 0xffffffff in int32 is -1 but 4294967295 in a 64 bit signed int, checks for -1 don't work.

I see several ways to fix this:

1. Check for -1 or 4294967295
2. Change buf.go to do an extra conversion:
   n = int(int32(binary.BigEndian.Uint32(*b)))
3. Change buf.go to use int32, int16 etc. instead of int.

id | name | server_id | created_at | updated_at
------+---------------------+-----------+---------------------------+---------------------------
1501 | test | 0 | 2013-01-04 20:14:58.80033 | 2013-01-04 20:14:58.80033
(1 row)

r, e := db.Query("SELECT name FROM hosts WHERE name='test'")
r.Next() retrue true

id | name | server_id | created_at | updated_at
------+---------------------+-----------+---------------------------+---------------------------
1501 | test | 0 | 2013-01-04 20:14:58 | 2013-01-04 20:14:58.80033
(1 row)

r, e := db.Query("SELECT name FROM hosts WHERE name='test'")
r.Next() retrue false

For streaming a large amount of data to insert into Postgresql, the COPY command is the most efficient. http://www.postgresql.org/docs/9.2/static/sql-copy.html
Ideally, pq would support the BINARY encoding which is faster than having to convert from native types to text in pq, and then back from text to native type in postgresql (especially for large volumes of floating point data). With pq's goroutine support, it should be possible to have multiple concurrent binary COPY streams insert into the same table for super fast bulk upload.

It seems like doing something like:

value := 2
rows, err := db.Query("select * from table where value > ?", value)

Returns a syntax error from pq, but doing this works:

value := 2
rows, err := db.Query("select * from table where value > $1", value)

Which is the postgres-specific of prepared statements, but doesn't match go's documentation (they have a sample under NullString).

If I send erroneous SQL to the PostgreSQL server (maybe an extra parenthesis, leave out a quote, etc.), the connection hangs on the Go side I think around the time it tries to close.

PostgreSQL is returning an error message, which I can print out. For example, if I call err := tx.QueryRow(...), I can print out the message:

Failed to save: pq: F:"scan.l" R:"scanner_yyerror" C:"42601" P:"113" M:"syntax error at or near \"default\"" S:"ERROR" L:"1001"

But when my test app is ready to exit, it just hangs, I think waiting for the connection to close.

Obviously the fix is to correct my SQL, but this seems like a dangerous state to get into for the PostgreSQL client.

For example:

func TestMultiple(t *testing.T) {
    db := openTestConn(t)
    defer db.Close()

    _, err := db.Exec("CREATE TEMP TABLE temp2 (a int)")
    if err != nil {
        t.Fatal(err)
    }   

    _, err = db.Exec("INSERT INTO temp2 VALUES (1), (2), (3)")
    if err != nil {
        t.Fatal(err)
    }   

    rows, err := db.Query("SELECT * FROM temp2")
    if err != nil {
        t.Fatal(err)
    }   
    defer rows.Close()

    rows2, err := db.Query("SELECT * FROM temp2")
    if err != nil {
        t.Fatal(err)
    }   
    rows2.Close()
}

The err result from the second query is:

pq: S:"ERROR" R:"parserOpenTable" F:"parse_relation.c" M:"relation \"temp2\" does not exist" L:"885" C:"42P01" P:"15

In another case where I used a real table and a Tx instead of a DB, the second Query just blocks indefinitely.

Even with a completely invalid connection string:

db, err := sql.Open("postgres", "foo bar zub")
if err != nil {
    fmt.Printf("Cannot connnect to PostgreSQL: %s\n", err)
}

err is never set to nil.

When a QueryRow and Scan is done inside a transaction, and the Scan fails with ErrNoRows, a Commit won't yield an error.

From pq:

driver: bad connection

On the postgres side:

FATAL:  database "my_database" does not exist

It would be nice to rationalize this a bit.

Just thought I'd file this as a nice to have.

Given the following query, where I do not have the schema defined:

        res, err := db.Query("INSERT INTO sample (group, class, letter, array) VALUES ('test','AB',A','{0.500,0.500,0.500,0.500}');")

I get:

pg_test.go:29:  pq: S:"ERROR" L:"526" M:"prepared statement \"1\" does not exist" R:"FetchPreparedStatement" C:"26000" F:"prepare.c"
        ok      _/home/fdr/codes/goenv/src/exp  0.016s

I was perplexed for a while until I instrumented prepareTo's switch for the error response byte, 'E'. This is the error I get in there:

BenchmarkPGSQL  preparing, ""
wat
preparing, "1"
err is "pq: R:\"scanner_yyerror\" F:\"scan.l\" S:\"ERROR\" M:\"syntax error at or near \\\"group\\\"\" C:\"42601\" L:\"1001\" P:\"21\""--- FAIL: BenchmarkPGSQL

So this makes a lot more sense, but I had to instrument pq to get it.

in the database/sql
the result has define

type Result

type Result interface {
    LastInsertId() (int64, error)
    RowsAffected() (int64, error)
}

but the pg drive don't support LastInsertId()

I want to make an insert into a table with constraints. I also want to get the inserted id value if successful. The problem is if QueryRow() with an INSERT...RETURNING fails, then the error is a rather unhelpful no rows in result set.

The error I want would be something like:
duplicate key violates unique constraint "user_email_unique"
which is the error that's returned by Exec() but then I don't get the last inserted id if everything goes well.

When trying to use noeq as an ID generator and pq as a data persistence layer, it's possible to generate an ID that can't be stored. PostgreSQL only supports bigint (int64), not uint64. From what I understand, database/sql converts uint64s to int64s transparently, but throws an error when the uint64 is above the limit of int64, effectively wasting half the space in the uint64:

sql: converting argument #1's type: uint64 values with high bit set are not supported

Seeing as @bmizerany worked on both the ID generator and the database driver, I figured he may have a solution, workaround, or suggestion for using uint64 IDs with pq. Any ideas?

TLS encryption was broken by a6c22b1 in #34. With that commit and a connection string including sslmode=require, I receive the following error after a fairly long timeout (30s?) on a SELECT 1:

unexpected EOF

With that commit reverted, the query completes without issue.

/cc @jgallagher

Hi,
I was previously able to connect to my PostgreSQL Database and prepare a statement with the code in commit edea8d2. However, with the latest commit, the statement prepare call hangs (for any statement). Here's my stack trace:

Thanks for your help!

10:46:42 web.1 | started with pid 15270
10:47:33 web.1 | SIGABRT: abort
10:47:33 web.1 | PC=0x423c69
10:47:33 web.1 |
10:47:33 web.1 |
10:47:33 web.1 | goroutine 1 [chan receive]:
10:47:33 web.1 | net.(_pollServer).WaitRead(0xf8400c3e40, 0xf840087360, 0xf84006ea80, 0xb, 0x1, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:268 +0x73
10:47:33 web.1 | net.(_netFD).accept(0xf840087360, 0x4fa4c2, 0x0, 0xf84006b240, 0xf840081040, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:622 +0x20d
10:47:33 web.1 | net.(_TCPListener).AcceptTCP(0xf8400813b8, 0xf84006fb60, 0x0, 0x0, 0x8, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/tcpsock_posix.go:322 +0x71
10:47:33 web.1 | net.(_TCPListener).Accept(0xf8400813b8, 0x0, 0x0, 0x0, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/tcpsock_posix.go:332 +0x49
10:47:33 web.1 | net/http.(_Server).Serve(0xf8400c3dc0, 0xf8400c3f00, 0xf8400813b8, 0x0, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:1012 +0x88
10:47:33 web.1 | net/http.(_Server).ListenAndServe(0xf8400c3dc0, 0xf8400c3dc0, 0x1, 0x406dc5)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:1002 +0xb6
10:47:33 web.1 | net/http.ListenAndServe(0xf840081350, 0xf800000005, 0x0, 0x0, 0xf800000004, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:1074 +0x69
10:47:33 web.1 | main.main()
10:47:33 web.1 | $GOPATH/src/github/proj/main.go:60 +0xcc
10:47:33 web.1 |
10:47:33 web.1 | goroutine 2 [syscall]:
10:47:33 web.1 | created by runtime.main
10:47:33 web.1 | /usr/lib/go/src/pkg/runtime/proc.c:221
10:47:33 web.1 |
10:47:33 web.1 | goroutine 3 [syscall]:
10:47:33 web.1 | syscall.Syscall6()
10:47:33 web.1 | /usr/lib/go/src/pkg/syscall/asm_linux_amd64.s:40 +0x5
10:47:33 web.1 | syscall.EpollWait(0xf800000009, 0xf8400c20c0, 0xa0000000a, 0xffffffff, 0xc, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/syscall/zerrors_linux_amd64.go:1781 +0xa1
10:47:33 web.1 | net.(_pollster).WaitFD(0xf8400c20b0, 0xf8400c3e40, 0x0, 0x0, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd_linux.go:146 +0x110
10:47:33 web.1 | net.(_pollServer).Run(0xf8400c3e40, 0x0)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:236 +0xe4
10:47:33 web.1 | created by net.newPollServer
10:47:33 web.1 | /usr/lib/go/src/pkg/net/newpollserver.go:35 +0x382
10:47:33 web.1 |
10:47:33 web.1 | goroutine 4 [chan receive]:
10:47:33 web.1 | net.(_pollServer).WaitRead(0xf8400c3e40, 0xf8400e7630, 0xf84006ea80, 0xb, 0x1, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:268 +0x73
10:47:33 web.1 | net.(_netFD).Read(0xf8400e7630, 0xf84012f021, 0x300fffc0300fffc, 0xf8ffffffff, 0xf84006b240, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:428 +0x1ec
10:47:33 web.1 | net.(_TCPConn).Read(0xf8400817c0, 0xf84012f021, 0x300fffc0300fffc, 0xffb00000ffb, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/tcpsock_posix.go:87 +0xce
10:47:33 web.1 | bufio.(_Reader).Read(0xf840069c80, 0xf84012f021, 0x300fffc0300fffc, 0x300fffc, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/bufio/bufio.go:135 +0x102
10:47:33 web.1 | io.ReadAtLeast(0xf8400cf720, 0xf840069c80, 0xf84012f000, 0x301001d0301001d, 0x7f900301001d, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/io/io.go:254 +0xc3
10:47:33 web.1 | io.ReadFull(0xf8400cf720, 0xf840069c80, 0xf84012f000, 0x301001d0301001d, 0xf840069c80, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/io/io.go:273 +0x69
10:47:33 web.1 | github.com/bmizerany/pq.(_conn).recv1(0xf8400f4c00, 0xf8400011e1, 0x4f00000035, 0xf8400b9720)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:293 +0x1b6
10:47:33 web.1 | github.com/bmizerany/pq.(_conn).recv(0xf8400f4c00, 0xf8400fdfa0, 0x5000000035, 0xf840081800, 0xf800000001, ...)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:270 +0x2a
10:47:33 web.1 | github.com/bmizerany/pq.(_conn).startup(0xf8400f4c00, 0xf8400f1540, 0xf840069c80, 0xf84006e120)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:342 +0x31d
10:47:33 web.1 | github.com/bmizerany/pq.Open(0xf8400a1e60, 0x94, 0x0, 0x0, 0x0, ...)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:78 +0x3dc
10:47:33 web.1 | github.com/bmizerany/pq.(_drv).Open(0xf8400811f8, 0xf8400a1e60, 0x94, 0x0, 0xf800000001, ...)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:29 +0x34
10:47:33 web.1 | database/sql.(_DB).conn(0xf8400fa0c0, 0x7f90cd9f0888, 0x7f90cd9f0878, 0x425ca9)
10:47:33 web.1 | /usr/lib/go/src/pkg/database/sql/sql.go:248 +0x208
10:47:33 web.1 | database/sql.(_DB).prepare(0xf8400fa0c0, 0x6a6564, 0x84, 0x0, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/database/sql/sql.go:310 +0x48
10:47:33 web.1 | database/sql.(_DB).Prepare(0xf8400fa0c0, 0x6a6564, 0x84, 0x7, 0xf8400a1e60, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/database/sql/sql.go:295 +0x57
10:47:33 web.1 | main.MustPrepare(0xf8400fa0c0, 0x6a6564, 0x84, 0x7f90cd9f09a0, 0x40ead0, ...)
10:47:33 web.1 | $GOPATH/src/github/proj/db.go:47 +0x39
10:47:33 web.1 | main.fetchCompanies(0x6a6564, 0x454c455300000084, 0x0, 0x0, 0xf800000000, ...)
10:47:33 web.1 | $GOPATH/src/github/proj/company.go:163 +0x5b
10:47:33 web.1 | main.companyDatastoreRetrieve(0xf8400d0900, 0xf8400d0280, 0xf8400acf00, 0x54454700000003, 0x0, ...)
10:47:33 web.1 | $GOPATH/src/github/proj/company.go:295 +0x969
10:47:33 web.1 | main.companyDatastore(0xf8400d0900, 0xf8400d0280, 0xf8400acf00, 0x64fbfc, 0x746e6f430000000c, ...)
10:47:33 web.1 | $GOPATH/src/github/proj/company.go:216 +0x87
10:47:33 web.1 | main._func_002(0xf840081240, 0x40b318, 0xf8400d0900, 0xf8400d0280, 0xf8400acf00, ...)
10:47:33 web.1 | $GOPATH/src/github/proj/main.go:74 +0x112
10:47:33 web.1 | main._func_016(0xf840081248, 0x44c64b, 0xf8400d0900, 0xf8400d0280, 0xf8400acf00, ...)
10:47:33 web.1 | $GOPATH/src/github/proj/route.go:99 +0x40
10:47:33 web.1 | net/http.HandlerFunc.ServeHTTP(0xf8400c3500, 0xf8400d0900, 0xf8400d0280, 0xf8400acf00, 0xf800000000, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:690 +0x3d
10:47:33 web.1 | net/http.(_ServeMux).ServeHTTP(0xf84006c900, 0xf8400d0900, 0xf8400d0280, 0xf8400acf00, 0xf8400d0280, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:924 +0xe7
10:47:33 web.1 | net/http.(_conn).serve(0xf8400a4300, 0x0)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:656 +0x621
10:47:33 web.1 | created by net/http.(_Server).Serve
10:47:33 web.1 | /usr/lib/go/src/pkg/net/http/server.go:1040 +0x430
10:47:33 web.1 |
10:47:33 web.1 | goroutine 5 [chan receive]:
10:47:33 web.1 | net.(_pollServer).WaitRead(0xf8400c3e40, 0xf8400e7090, 0xf84006ea80, 0xb, 0x1, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:268 +0x73
10:47:33 web.1 | net.(_netFD).Read(0xf8400e7090, 0xf843162021, 0x300fffc0300fffc, 0xf8ffffffff, 0xf84006b240, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/fd.go:428 +0x1ec
10:47:33 web.1 | net.(_TCPConn).Read(0xf8400816c0, 0xf843162021, 0x300fffc0300fffc, 0xffb00000ffb, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/net/tcpsock_posix.go:87 +0xce
10:47:33 web.1 | bufio.(_Reader).Read(0xf840069e80, 0xf843162021, 0x300fffc0300fffc, 0x300fffc, 0x0, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/bufio/bufio.go:135 +0x102
10:47:33 web.1 | io.ReadAtLeast(0xf8400cf720, 0xf840069e80, 0xf843162000, 0x301001d0301001d, 0x7f900301001d, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/io/io.go:254 +0xc3
10:47:33 web.1 | io.ReadFull(0xf8400cf720, 0xf840069e80, 0xf843162000, 0x301001d0301001d, 0xf840069e80, ...)
10:47:33 web.1 | /usr/lib/go/src/pkg/io/io.go:273 +0x69
10:47:33 web.1 | github.com/bmizerany/pq.(_conn).recv1(0xf84012abe0, 0xf840001051, 0x4f00000035, 0xf8400b9720)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:293 +0x1b6
10:47:33 web.1 | github.com/bmizerany/pq.(_conn).recv(0xf84012abe0, 0xf84006fbd0, 0x5000000035, 0xf840081700, 0xf800000001, ...)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:270 +0x2a
10:47:33 web.1 | github.com/bmizerany/pq.(_conn).startup(0xf84012abe0, 0xf84012a180, 0xf840069e80, 0xf84006e120)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:342 +0x31d
10:47:33 web.1 | github.com/bmizerany/pq.Open(0xf8400a1140, 0x94, 0x0, 0x0, 0x0, ...)
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:78 +0x3dc
10:47:33 web.1 | process terminated
10:47:33 web.1 | github.com/bmizerany/pq.(_drv).Open(0xf8400811f8, 0xf8400a1140, 0x94, 0x7f90cba4cb88, 0x0, ...)
10:47:33 system | sending SIGTERM to all processes
10:47:33 web.1 | $GOPATH/src/github.com/bmizerany/pq/conn.go:29 +0x34
10:47:33 web.1 | database/sql.(*DB).conn(0xf8400c36c0, 0x563cb0, 0x0, 0x412104)

ijt@web:~$ PGHOST=/var/run/postgresql go test github.com/bmizerany/pq
--- FAIL: TestExec (0.06 seconds)
conn_test.go:39: driver: bad connection
--- FAIL: TestStatment (0.06 seconds)
conn_test.go:67: driver: bad connection
--- FAIL: TestRowsCloseBeforeDone (0.06 seconds)
conn_test.go:126: driver: bad connection
--- FAIL: TestEncodeDecode (0.06 seconds)
conn_test.go:168: driver: bad connection
--- FAIL: TestNoData (0.07 seconds)
conn_test.go:225: driver: bad connection
--- FAIL: TestErrorOnExec (0.06 seconds)
conn_test.go:294: expected PGError, was: &errors.errorString{s:"driver: bad connection"}
--- FAIL: TestErrorOnQuery (0.06 seconds)
conn_test.go:310: driver: bad connection
--- FAIL: TestBindError (0.06 seconds)
conn_test.go:339: driver: bad connection
--- FAIL: TestNullAfterNonNull (0.06 seconds)
conn_test.go:380: driver: bad connection
FAIL
FAIL github.com/bmizerany/pq 0.616s

Am I running this incorrectly?

See:

http://www.postgresql.org/docs/9.1/static/libpq-pgpass.html

I see there's a comment in conn.go:

    // skip PGPASSFILE, PGSERVICE, PGSERVICEFILE,
    // PGREALM

so I take that it was kind of planned to support it?

Thanks.

If the consensus is that this isn't an issue with pq, I'll submit it to the go devs.

I've come across some weirdness on linux/x86 when runtime.GOMAXPROCS(N) where N > 2 on a four core machine. I'm getting crashes of 2 types. It appears to be related to garbage collection. The good news is that when the call to runtime.GOMAXPROCS is omitted, the crashing stops.

Here's the gist: https://gist.github.com/3831248 . For people with postgres and pq already installed, the only step before go build is to setup a test database.

 $ psql

 postgres=# CREATE USER warnerb WITH PASSWORD 'passport';
 postgres=# CREATE DATABASE testdb123;

After that, the call to GOMAXPROCS on first line of main can be changed to produce a crash, maybe.

package main

import (
    _ "github.com/bmizerany/pq"
    //"runtime"
    "database/sql"
    "log"
    "math/rand"
)

/*


 8< ------------------------------------------------------------------
    TYPE A 

 throw: gosched of g0
 [signal 0xb code=0x1 addr=0x0 pc=0x804ab44]

 goroutine 1 [running]:

 goroutine 2 [syscall]:
 created by runtime.main
      /usr/custom/goroot/src/pkg/runtime/proc.c:220

 goroutine 3 [syscall]:
 syscall.Syscall6()
      /usr/custom/goroot/src/pkg/syscall/asm_linux_386.s:0 +0x27
 syscall.EpollWait(0x6, 0x188430a8, 0xa, 0xa, 0xffffffff, ...)
      /usr/custom/goroot/src/pkg/syscall/env_unix.go:0 +0x7d
 net.(*pollster).WaitFD(0x188430a0, 0x1883f740, 0x0, 0x0, 0x0, ...)
      /usr/custom/goroot/src/pkg/net/fd_linux.go:146 +0x103
 net.(*pollServer).Run(0x1883f740, 0x0)
      /usr/custom/goroot/src/pkg/net/fd_unix.go:216 +0xdb
 created by net.newPollServer
      /usr/custom/goroot/src/pkg/net/newpollserver_unix.go:33 +0x2c4


 8< ------------------------------------------------------------------
    TYPE B

 panic: invalid memory address or nil pointer dereference
 throw: panic during gc
 [signal 0xb code=0x1 addr=0x0 pc=0x804ace4]

 goroutine 1 [running]:
 github.com/bmizerany/pq.(*conn).recv1(0x18850950, 0x188edff0, 0x5)
      /home/derek/dev/gopath/src/github.com/bmizerany/pq/conn.go:292 +0x104
 github.com/bmizerany/pq.(*stmt).exec(0x188eacc0, 0x188edef0, 0x2)
      /home/derek/dev/gopath/src/github.com/bmizerany/pq/conn.go:479 +0x29f
 github.com/bmizerany/pq.(*stmt).Exec(0x188eacc0, 0x188edef0, 0x2, 0x2, 0x0, ...)
      /home/derek/dev/gopath/src/github.com/bmizerany/pq/conn.go:428 +0xbe
 github.com/bmizerany/pq.(*conn).Exec(0x18850950, 0x188eac80, 0x30, 0x188edef0, 0x2, ...)
      /home/derek/dev/gopath/src/github.com/bmizerany/pq/conn.go:245 +0x137
 database/sql.(*DB).exec(0x188013c0, 0x188eac80, 0x30, 0x62dee0, 0x2, ...)
      /usr/custom/goroot/src/pkg/database/sql/sql.go:357 +0x1af
 database/sql.(*DB).Exec(0x188013c0, 0x188eac80, 0x30, 0x62dee0, 0x2, ...)
      /usr/custom/goroot/src/pkg/database/sql/sql.go:335 +0x74
 main.Insert(0x188013c0, 0x188edd40, 0xa, 0x188e77e0, 0x19, ...)
      /home/derek/dev/gopath/src/pq-bugtest/main.go:122 +0x1eb
 main.main()
      /home/derek/dev/gopath/src/pq-bugtest/main.go:99 +0x327

 goroutine 2 [syscall]:
 created by runtime.main
      /usr/custom/goroot/src/pkg/runtime/proc.c:220

 goroutine 3 [syscall]:
 syscall.Syscall6()
      /usr/custom/goroot/src/pkg/syscall/asm_linux_386.s:0 +0x27
 syscall.EpollWait(0x6, 0x188430a8, 0xa, 0xa, 0xffffffff, ...)
      /usr/custom/goroot/src/pkg/syscall/env_unix.go:0 +0x7d
 net.(*pollster).WaitFD(0x188430a0, 0x1883f740, 0x0, 0x0, 0x0, ...)
      /usr/custom/goroot/src/pkg/net/fd_linux.go:146 +0x103
 net.(*pollServer).Run(0x1883f740, 0x0)
      /usr/custom/goroot/src/pkg/net/fd_unix.go:216 +0xdb
 created by net.newPollServer
      /usr/custom/goroot/src/pkg/net/newpollserver_unix.go:33 +0x2c4

 8< ------------------------------------------------------------------

 The following listing shows the argument to GOMAXPROCS and how many 
 insertions were performed prior to crashing. PASS means no crash occured

 with N = 4
 Trials: 
 1: 436
 2: 1555
 3: 1095
 4: 4752
 5: 871
 6: 3604
 7: 1779
 8: 3151
 9: 642

 with N = 3
 Trials:
 1: 1271
 2: 5972
 3: 3801
 4: 1061
 5: 853
 6: 855
 7: 2541
 8: 2757
 9: 1064

 with N = 2
 Trials:
 1: PASS
 2: PASS
 3: PASS
 4: PASS
 5: PASS
 6: PASS
 7: PASS
 8: PASS
 9: PASS

 with N = 1
 Trials:
 1: PASS
 2: PASS
 3: PASS
 4: PASS
 5: PASS
 6: PASS
 7: PASS
 8: PASS
 9: PASS

 !! Omitting call to GOMAXPROCS altogether !!
 Trials:
 1: PASS
 2: PASS
 3: PASS
 4: PASS
 5: PASS
 6: PASS
 7: PASS
 8: PASS
 9: PASS

*/

const createperson = `
create table person (
        did SERIAL,
        unique(did),

        name varchar(255),
        email varchar(255),
        unique(email)

        );
`

func main() {
    //runtime.GOMAXPROCS(3)

    cstr := `user=warnerb dbname=testdb123 dbpasswd=passport sslmode=disable`
    db, err := sql.Open("postgres", cstr)
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Drop the table if it exists
    _, err = db.Exec(`drop table if exists person cascade;`)
    if err != nil {
        log.Fatal(err, "\ncan't drop table person")
    }

    // Create the table
    _, err = db.Exec(createperson)
    if err != nil {
        log.Fatal(err)
    }

    // Insert a bunch of records. 
    for i := 0; i < 10000; i++ {
        log.Print(i, " ")
        // This is where it will fail, if it does.
        err := Insert(db, RandomName(), RandomEmail())
        if err != nil {
            log.Fatal(err, "\nProblem inserting person")
        }
    }
}

func RandomName() string {
    alphas := `abcdefghijklmnopqrstuvwxyz`
    name := ``
    for i := 0; i < 10; i++ {
        name += string(alphas[rand.Int()%26])
    }
    return name
}

func RandomEmail() string {
    return RandomName() + "@" + RandomName() + ".com"
}

func Insert(db *sql.DB, name, email string) error {
    log.Println("Inserting: ", name, email)
    cols := `(name, email) values ($1,$2) `
    _, err := db.Exec(`INSERT INTO person `+cols, name, email)
    if err != nil {
        return err
    }
    return nil
}

Support for ~/.pgpass files would be really useful (description: https://wiki.postgresql.org/wiki/Pgpass)

I've run pq on both Linux (Ubuntu 12.10) and Windows7. I've downloaded the latest version as of 15-Jan-2012.

On Linux, no problems encountered, but I haven't done a lot of testing.

On Windows, it is very slow. eg. simple SELECT on small table taking (say) 2 seconds. A minor update involving an update on one table and an insert on another with COMMIT taking 4.5 seconds.

psql runs fine on Windows, no slowness detected.

Running "GO test" takes 0.321 seconds (Linux), and it takes 39.287 seconds (Windows). Subsequent runs on Linux get it just under 0.100 seconds, and on Windows just under 39 seconds.

I had to alter conn_test.go (on Windows) because running the test gave repeated errors of 'pq: Role "ta10\Brian" does not exist'. I attempted to create role 'ta10\Brian', but psql gave error. I created role 'Brian', but still got error. Therefore, I altered "conn_test.go" as follows:

    ////conn, err := sql.Open("postgres", "")
    conn, err := sql.Open("postgres",
    "user=postgres dbname=postgres password=super")

If you insert a []byte in a Text field in a database, querying the same field later returns the encoded version. I guess this is because decode() doesn't deal with t_text at all. I noticed that when encoding values of type string this is not a problem, but I would prefer not to have to convert all my []byte to string in all cases.

I'm not sure what the best way to solve this problem would be or if this is just working as designed.

Hi,

Execution sql.Rollback(),appear dead lock.

First of all not sure if this is a bug, did not know everybody does have has had the same question!

I use the scene is like this:

Two tables do insert operation, have a mistake to return to execute rolled back.

First things go in perfectly correct logical operation is possible!

But when I delete a list of test has been stuck in a Rollback () method invocation place

Is the following code comments place, have been waiting for!

Test a day to find problems in here, also don't error!

You think this is bug?

Basic thought the HTML code is as follows.
http://play.golang.org/p/DrOD-cplBO