oracle / odpi

ODPI-C: Oracle Database Programming Interface for Drivers and Applications

ODPI-C

Oracle Database Programming Interface for C (ODPI-C) is an open source library of C code that simplifies access to Oracle Database for applications written in C or C++. It is a wrapper over Oracle Call Interface (OCI) that makes applications and language interfaces easier to develop.

ODPI-C supports basic and advanced features of Oracle Database and Oracle Client. See the homepage for a list.

Installation

See ODPI-C Installation.

Documentation

See the ODPI-C Documentation and Release Notes.

Samples

See /samples.

Help

Please report bugs and ask questions using GitHub issues.

Tests

See /test.

Contributing

See CONTRIBUTING.

Drivers Using ODPI-C

Oracle Drivers:

• python-oracledb Python interface (previously known as cx_Oracle).
• node-oracledb Node.js module.

Third-party Drivers:

• godror Go Driver.
• odpic-raw Haskell Raw Bindings.
• ruby-ODPI Ruby Interface.
• rust-oracle Driver for Rust.
• Oracle.jl Driver for Julia.
• oranif Driver for Erlang.
• nimodpi Driver for Nim.

License

Copyright (c) 2016, 2024, Oracle and/or its affiliates.

This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl and Apache License 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. You may choose either license.

If you elect to accept the software under the Apache License, Version 2.0, the following applies:

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

odpi's Issues

Add support for ORACLE_HOME

It would be nice if odpi would support ORACLE_HOME instead of fiddling with LD_LIBRARY_PATH for those who have the database or full client installed.

a example

hello, I'm new
Could you please write a minimal example connecting to database and run a query select?
Thank you.

Custom error message when incorrect-architecture OCI.DLL is in PATH on Windows

When incorrect-architecture OCI.DLL is in the PATH and correct-architecture OCI.DLL isn't there, GetLastError() returns ERROR_BAD_EXE_FORMAT(%1 is not a valid Win32 application). How about custom error message saying "c:\full\path\of\OCI.DLL is 32-bit, which isn't available in x64 programs" or so in this case?
This will reduce issues such as oracle/node-oracledb#752.

1. Search OCI.DLL as the step 1 in #48 (comment).
2. If all OCI.DLLs are incorrect-architecture, create a custom error message. Otherwise use message from FormatMessage().

Architecture could be checked by nt_hdr->FileHeader.Machine in #48 (comment) or by the following function.

#include <windows.h>

static void print_arch(const char *name)
{
    FILE *fp = fopen(name, "rb");
    IMAGE_DOS_HEADER dos_hdr;
    IMAGE_NT_HEADERS nt_hdr;

    if (fp == NULL) {
        printf("failed to open: %s\n", name);
        return;
    }
    fread(&dos_hdr, sizeof(dos_hdr), 1, fp);
    if (dos_hdr.e_magic != IMAGE_DOS_SIGNATURE) {
        printf("invalid DOS signature: 0x%x\n", dos_hdr.e_magic);
        fclose(fp);
        return;
    }
    fseek(fp, dos_hdr.e_lfanew, SEEK_SET);
    fread(&nt_hdr, sizeof(nt_hdr), 1, fp);
    fclose(fp);
    if (nt_hdr.Signature != IMAGE_NT_SIGNATURE) {
        printf("invalid NT signature: 0x%x\n", nt_hdr.Signature);
        return;
    }
    switch (nt_hdr.FileHeader.Machine) {
    case IMAGE_FILE_MACHINE_I386:
        printf("architecture is x86.\n");
        break;
    case IMAGE_FILE_MACHINE_AMD64:
        printf("architecture is x64.\n");
        break;
    case IMAGE_FILE_MACHINE_IA64:
        printf("architecture is IA64.\n");
        break;
    default:
        printf("invalid architecture: 0x%x\n", nt_hdr.FileHeader.Machine);
        break;
    }
}

Custom error message when Visual Studio distributable package is not installed on Windows

How about custom error message when Visual Studio distributable package is not installed?
IMO, oracle/node-oracledb#404 (comment) should be checked in ODPI-C.

When GetLastError() returns ERROR_MOD_NOT_FOUND(The specified module could not be found), do the following.

1. Search OCI.DLL in the following directories in this order:
   1. Executable directory (Get the full path of the executable by GetModuleFileName(NULL, buf, sizeof(buf)) and truncate the file name.)
   2. Current directory
   3. each directory in the environment variable PATH.
2. If OCI.DLL is not found, use the message from FormatMessage().
3. If OCI.DLL is found, get the bit of the DLL (see the function below) and skip if it isn't same with the executable.
4. If step 3 passes, get import DLL files (see the function below) and get the DLL whose name is 'MSVCR*.DLL'.
5. Call LoadLibrary with the 'MSVCR*.DLL' at step 4.
6. If step 5 fails, map the 'MSVCR*.DLL' name to VS distributable package name and create a custom error message.

The following is a sample function which prints bit and dependent DLL files.
It is easy to customize this to fit this issue's need.

// link dbghelp.lib also
#include <dbghelp.h>

static void print_import_dlls(const char *name)
{
    HANDLE hFile = CreateFile(name, GENERIC_READ, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE,
                              NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    HANDLE hFileMapping = NULL;
    void *base = NULL;
    if (hFile == INVALID_HANDLE_VALUE) {
        printf("failed to open: %s\n", name);
        return;
    }
    hFileMapping = CreateFileMapping(hFile, NULL, PAGE_READONLY, 0, 0, NULL);
    if (hFileMapping == NULL) {
        printf("failed to create file mapping: %s\n", name);
        goto cleanup;
    }
    base = MapViewOfFile(hFileMapping, FILE_MAP_READ, 0, 0, 0);
    if (base == NULL) {
        printf("failed to map file: %s\n", name);
        goto cleanup;
    }
    IMAGE_NT_HEADERS *nt_hdr = ImageNtHeader(base);
    if (nt_hdr == NULL) {
        printf("not executable nor DLL: %s\n", name);
        goto cleanup;
    }

    printf("%s (%s bit):\n", name, nt_hdr->FileHeader.Machine == IMAGE_FILE_MACHINE_I386 ? "32" : "64");
    ULONG uSize = 0;
    IMAGE_IMPORT_DESCRIPTOR *desc = (IMAGE_IMPORT_DESCRIPTOR *)ImageDirectoryEntryToDataEx(base, FALSE, IMAGE_DIRECTORY_ENTRY_IMPORT, &uSize, NULL);

    while (desc->Characteristics != 0) {
        char *name = (char*)ImageRvaToVa(nt_hdr, base, desc->Name, NULL);
        printf("  %s\n", name);
        desc++;
    }
cleanup:
    if (base != NULL) {
        UnmapViewOfFile(base);
    }
    if (hFileMapping != NULL) {
        CloseHandle(hFileMapping);
    }
    CloseHandle(hFile);
}

Oracle object subtype support

ODPI-C doesn't seem to support Oracle object subtype properly.

Consider the table created by the following statements.

create type test_inheritance_base as object (
   name varchar2(30)
) not final
/
create type test_inheritance_sub under test_inheritance_base (
   surname varchar2(30)
)
/
create table test_inheritance (id integer, name test_inheritance_base);
insert into test_inheritance values (1, test_inheritance_base('John'));
insert into test_inheritance values (2, test_inheritance_sub('John', 'Smith'));
commit;

When the name column where id = 1 is fetched, its type is same with the type information retrieved by dpiStmt_getQueryInfo.
But when the name column where id = 2 is fetched, there is no way to know its actual type.

IMO, the problem will be fixed if dpiObject_getObjectType(), which internally uses OCIObjectGetTypeRef and OCIDescribeAny with OCI_OTYPE_REF, is added.

It may be better to add isFinalType member to dpiObjectTypeInfo to check whether the type is a final type. When a column in a query is a final type, there is no need to call dpiObject_getObjectType() for each fetched value. When it isn't a final type, dpiObject_getObjectType() must be called for each value.

will you provide new functions to repeat using dpiStmt

I find function:
// prepare a statement and return it for subsequent execution/fetching
int dpiConn_prepareStmt(dpiConn *conn, int scrollable, const char *sql,
uint32_t sqlLength, const char *tag, uint32_t tagLength,
dpiStmt **stmt);
that means when executing a sql, we should call dpiConn_prepareStmt one time.
will you some functions likes：
1.allocate dpiStmt
2.execute sql
3.get affectedrows
4.get resultset
5.close resultset
6.repeat 1-5
7.close dpiStmt
thinks.

Closing connection hangs in dpiConn_release

I use the go-goracle/goracle driver for a golang project. This version uses the version 2.0.3 of ODPI-C. The program is running on Centos 7 and I'm use the Instant Client 12.2.0.1 from Oracle.

The program is running as expected, however when I try to stop the program with pressing Ctrl+C the program is calling my Close() method as expected but than the program hangs forever.

After a while (approximately 10 minutes) I open a second terminal and send kill -ABRT <pid> and than the stacktrace below will be printed.

goroutine 62 [syscall, locked to thread]:
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle%2ev2._Cfunc_dpiConn_release(0x46da9c0, 0x0)
	github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle.v2/_obj/_cgo_gotypes.go:1448 +0x49
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle%2ev2.(*conn).Close.func1(0x46da9c0, 0x2f5d950)
	/home/mguggi/work/beats/src/github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle.v2/conn.go:112 +0x60
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle%2ev2.(*conn).Close(0xc420230300, 0x0, 0x0)
	/home/mguggi/work/beats/src/github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle.v2/conn.go:112 +0xa9
database/sql.(*driverConn).finalClose.func2()
	/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:467 +0x49
database/sql.withLock(0x405f9e0, 0xc420011ce0, 0xc42019bd38)
	/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:2867 +0x65
database/sql.(*driverConn).finalClose(0xc420011ce0, 0xc42019be58, 0xc42019be48)
	/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:465 +0x128
database/sql.(finalCloser).(database/sql.finalClose)-fm(0xc42008d108, 0xc420346ea0)
	/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:564 +0x2f
database/sql.(*DB).Close(0xc42008d0e0, 0x0, 0x0)
	/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:678 +0x31b
github.com/mguggi/oraclebeat/module/oracle/sgastats.(*MetricSet).Close(0xc42008d7c0, 0x406ec20, 0xc42008d7c0)
	/home/mguggi/go/src/github.com/mguggi/oraclebeat/module/oracle/sgastats/sgastats.go:110 +0xd7
github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module.(*metricSetWrapper).close(0xc420118d40, 0x0, 0x0)
	/home/mguggi/go/src/github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:257 +0x64
github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module.(*Wrapper).Start.func1(0xc4200193e0, 0xc420020ba0, 0xc4202504e0, 0xc420118d40)
	/home/mguggi/go/src/github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:111 +0xbe
created by github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module.(*Wrapper).Start
	/home/mguggi/go/src/github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:106 +0x146

I've already opened the issue on the driver repository #33 but in the driver code conn.go the last call ist the the C function dpiConn_release.

When I execute the procedure above and create an tcpdump on my interface and port 1521 I receive the following line:

153 31.614966 192.168.12.12 172.16.1.108 TCP 67 57578 → 1521 [PSH, ACK, URG] Seq=2790 Ack=87582 Win=182272 Urg=1 Len=1 TSval=32936936 TSecr=717866813[Malformed Packet: TNS]

The URG flag is set when I pressing Ctrl+C ...

But the curious thing is that this code works when the database is running on the same host. It looks like a vpn tunnel or firewall issue ... maybe???

Additionally the code works when I remove the db.Close() stuff from the ´Close()` method, but this is probably not the best idea.

Is there a way or a procedure how to activate a trace so that I can further narrow the root cause?

change object transaction id

The Database Change Notification docs seem to indicate that transaction ID should be available on a change message. I would like to request this be plumbed through to the odpi layer (next I'll ask for it in cx_Oracle.

I see DPI_OCI_ATTR_CHDES_XID exists in the headers, so I've been stumbling around in the code seeing if it is populated.

/tmp/cx_Oracle/odpi ((v2.0.0))$ git diff
diff --git a/include/dpi.h b/include/dpi.h
index d8f8a60..6739766 100644
--- a/include/dpi.h
+++ b/include/dpi.h
@@ -587,6 +587,8 @@ struct dpiSubscrMessage {
     dpiEventType eventType;
     const char *dbName;
     uint32_t dbNameLength;
+    const char *TxID;
+    uint32_t TxIDLength;
     dpiSubscrMessageTable *tables;
     uint32_t numTables;
     dpiSubscrMessageQuery *queries;
diff --git a/src/dpiSubscr.c b/src/dpiSubscr.c
index e35c29d..8eb9cdc 100644
--- a/src/dpiSubscr.c
+++ b/src/dpiSubscr.c
@@ -289,6 +289,13 @@ static int dpiSubscr__populateMessage(dpiSubscr *subscr,
             DPI_OCI_ATTR_CHDES_DBNAME, "get DB name", error) < 0)
         return DPI_FAILURE;
 
+    // determine transaction id
+    if (dpiOci__attrGet(descriptor, DPI_OCI_DTYPE_CHDES,
+            (void*) &message->TxID, &message->TxIDLength,
+            DPI_OCI_ATTR_CHDES_XID, "get transaction id", error) < 0)
+        return DPI_FAILURE;
+
+
     // populate event specific attributes
     switch (message->eventType) {
         case DPI_EVENT_OBJCHANGE:

This seems to give the transaction ID, but not quite correclty. For a transaxction ID of 4.22.470 I get back 0x80 0x00 0x00 0x00 THEN the transaction 0x04 0x00 0x16 0x00 0x68 0x04 0x00 0x00.

In other words if I look in message->TxID+4 I see what seem like a transaction ID.

So, any chance of getting this added, and any idea why my test returned a weird shifted result?

more data type information about object types

Could you add precision and scale members to dpiObjectAttrInfo and dpiObjectTypeInfo?
dpiOracleTypeNum is in dpiObjectAttrInfo, dpiObjectTypeInfo and dpiQueryInfo structures. However precision and scale are only in dpiQueryInfo.
Language bindings may refer these members to determine data types in its language.
For example if a member of an object type is NUMBER(20), a language binding may get the data as integer. (I'm making ruby-odpi, which is not a replacement of ruby-oci8. I'm doing it just to know ODPI.)

I prefer to make a structure as follows and put it in dpiObjectAttrInfo, dpiObjectTypeInfo and dpiQueryInfo. But I could not imagine use cases using size and nullOk members except just information for users as in the #22 (comment) example. So I don't adhere to my idea.

struct dpiDataTypeInfo
    dpiOracleTypeNum oracleTypeNum;
    dpiNativeTypeNum defaultNativeTypeNum;
    uint32_t dbSizeInBytes;
    uint32_t clientSizeInBytes;
    uint32_t sizeInChars;
    int16_t precision;
    int8_t scale;
    int nullOk;
    dpiObjectType *objectType;
};

The last argument of dpiConn_newSubscription must be `uint64_t *`.

I found a bug when I made a test code about #27.
dpiConn_newSubscription overwrites additional four bytes after the address passed as the last argument.

Look at https://gist.github.com/kubo/394af98c4c541b2e069467a97f0aaec9#file-subscr-test-c-L51-L55.
If the last argument of dpiConn_newSubscription is uint32_t *, subscrId.next4byte must be 0xdeadbeaf at line 53. However it became 0x00000000.
I guess that the type is uint64_t * so the next four bytes were overwritten with 0x00000000.

Set rpath on Linux and Solaris

Could you set -Wl,-rpath,$(OCI_LIB_DIR) to LDFLAGS on Linux and -R,$(OCI_LIB_DIR) on Solaris in addition to macOS?

	UNAME_S := $(shell uname -s)
	ifeq ($(UNAME_S), Linux)
		LDFLAGS += -Wl,-rpath,$(OCI_LIB_DIR)
	endif
	ifeq ($(UNAME_S), SunOS)
		LDFLAGS += -R,$(OCI_LIB_DIR)
	endif

If rpath is set, client programs depending on odpi have no need to use LD_LIBRARY_PATH to know the directory containing Oracle client libraries.

OCINumber as a native type

Would you have a plan to expose OCINumber as a native type?
It is required for drivers which implement JDBC-like API.

Assume the following code:

ResultSet rs = stmt.executeQuery("select NUMBER_COL from TABLE_NAME");
while (rs.next()) {
     ... use rs.getInt(1), rs.getLong(1), rs.getDouble(1) or rs.getBigDecimal(1) ...
}

The native type of NUMBER_COL is not decided at stmt.executeQuery(). It depends on methods such as getInt after the column data is fetched. However such API cannot be implemented by ODPI-C, which must declare native types explicitly or implicitly at SQL execution time.

If ODPI-C exposes OCINumber as a native type and provides conversion functions between OCINumber and integer, double and text, drivers can convert the OCINumber to int, long, double and BigDecimal via text after it is fetched.

`warning: unused parameter` when -Wextra is set to CFLAGS

When -Wextra is added to CFLAGS, gcc complains many unused parameter warnings on Linux.

$ make
mkdir build
mkdir lib
gcc -c -Iinclude -O2 -g -Wall -fPIC -Wextra src/dpiConn.c -obuild/dpiConn.o
src/dpiConn.c: In function ‘dpiConn__decrementOpenChildCount’:
src/dpiConn.c:222:63: warning: unused parameter ‘error’ [-Wunused-parameter]
 int dpiConn__decrementOpenChildCount(dpiConn *conn, dpiError *error)
                                                               ^
gcc -c -Iinclude -O2 -g -Wall -fPIC -Wextra src/dpiContext.c -obuild/dpiContext.o
src/dpiContext.c: In function ‘dpiContext__initCommonCreateParams’:
src/dpiContext.c:33:58: warning: unused parameter ‘context’ [-Wunused-parameter]
 int dpiContext__initCommonCreateParams(const dpiContext *context,
                                                          ^
src/dpiContext.c:34:50: warning: unused parameter ‘error’ [-Wunused-parameter]
         dpiCommonCreateParams *params, dpiError *error)
                                                  ^
...

Could you suppress these warnings?

Add the following macro to src/dpiImpl.h.

#ifdef __GNUC__
#define UNUSED __attribute((unused))
#else
#define UNUSED
#endif

And then use the macro after unused parameters as follows:

int dpiConn__decrementOpenChildCount(dpiConn *conn, dpiError *error UNUSED)

Connection pool, performance tests

I would like to make a question regarding performance

Below a small "demo" file that is used to measure execution time. I want to update a sequence with nextvval 100K times.

CREATE SEQUENCE  "TEST_SEQ"  
    MINVALUE 1 
    MAXVALUE 10000000000
    INCREMENT BY 1 
    START WITH 1 
    CACHE 200 
    NOORDER  
    CYCLE;

The sequence is created with the above script, and using the workbench like this

declare local_1 INT;
BEGIN
 
 
  FOR i IN 1..100000 LOOP
    select TEST_SEQ.nextval into local_1 from dual;
  END LOOP;
 
END;
/

it takes less than 2 seconds for the counter to reach 100K.

However the following code wants more than 7 seconds to reach 10K (an order of magnitude less).
I am wondering if there is something better I can do, or if I miss something. Is it possible using the library to reach better number, ie 2K-3K per second?

In order to compile please use
g++ t_main.cpp -std=c++11 -lboost_system -lboost_thread -lodpic -ldl

I used a simple thread pool from boost.

#include <boost/array.hpp>
#include <iostream>
#include <boost/asio/io_service.hpp>
#include <boost/bind.hpp>
#include <boost/thread/thread.hpp>
#include <boost/chrono/thread_clock.hpp>
#include <memory>
#include <chrono>

dpiContext *context;
dpiPool *pool;

int ShowError(dpiContext *context)
{
    return 1;
}


int icalc(int x) {

    dpiConn *conn;
    dpiStmt *stmt;
    uint32_t numQueryColumns;

    const char *sql = "select TEST_SEQ.nextval from dual";


    if (dpiPool_acquireConnection(pool, NULL, 0, NULL, 0, NULL, &conn) < 0 )
        return ShowError(context);

    if (dpiConn_prepareStmt(conn, 0, sql, strlen(sql), NULL, 0, &stmt) < 0)
        return ShowError(context);

    if ( dpiStmt_execute(stmt, DPI_MODE_EXEC_DEFAULT, &numQueryColumns) < 0)
        return ShowError(context);

    if ( dpiStmt_release(stmt) < 0 )
        return ShowError(context);

    if( conn ) {
            dpiConn_release(conn) ;
            return 0; //"Hello world";
    } else {
        return 1; //"err";
    }

}

int main(int argc, char** argv)
{

    dpiErrorInfo errorInfo;
    //dpiContext *context;

    if (dpiContext_create(DPI_MAJOR_VERSION, DPI_MINOR_VERSION, &context, &errorInfo) < 0) {
        return -1;
    }

    dpiCommonCreateParams commonParams;
    dpiPoolCreateParams createParams;

    //dpiPool *pool;
    
    if (dpiContext_initCommonCreateParams(context, &commonParams) < 0) {
        std::cout << "cannot init common params" << std::endl;
        return 1;
    }
    commonParams.createMode = DPI_MODE_CREATE_THREADED;

    if (dpiContext_initPoolCreateParams(context, &createParams) < 0) 
    {
        std::cout << "cannot init the params" << std::endl;
        return 1; //dpiTestCase_setFailedFromError(testCase);
    }
    createParams.minSessions = 15; //MINSESSIONS;
    createParams.maxSessions = 20; //MAXSESSIONS;
    createParams.sessionIncrement = 1; //SESSINCREMENT;
    
    if (dpiPool_create(context, "DB", strlen("DB"),
            "DB", strlen("DB"), "1.7.7.1:1521/DEV",
            strlen("10.70.7.146:1521/L10DEV"), &commonParams, &createParams,  &pool) < 0)
        return 1; 

    if (dpiPool_setGetMode(pool, DPI_MODE_POOL_GET_WAIT) < 0)
        return 1;     

    int maxthreads = 50;
    boost::asio::io_service ioService;
    std::unique_ptr<boost::asio::io_service::work> work( new boost::asio::io_service::work(ioService));
    boost::thread_group threadpool; //pool
    std::cout << "thread starts with max threads: : " << maxthreads << std::endl;
    for (std::size_t i = 0; i < maxthreads; ++i)
        threadpool.create_thread(boost::bind(&boost::asio::io_service::run, &ioService));

    using namespace boost::chrono;

    //thread_clock::time_point start = thread_clock::now();

    std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();
    for(int i = 0 ; i < 10000; i++) 
        ioService.post(boost::bind(icalc, 42));


    work.reset();
    threadpool.join_all();
    ioService.stop();
    std::chrono::steady_clock::time_point end= std::chrono::steady_clock::now();
    
    std::cout << "Time difference = " << std::chrono::duration_cast<std::chrono::seconds>(end - begin).count() <<std::endl;
    
    return 0;
}

Do we misuse the library somehow?

It is a little bit critical for us, since we want to decide if we will use oracle for a high performance server.

Thank you very much in advance.

Question: varray support

I am evaluating ODPI-C for use in our project using Master 2.1.0-dev
We have an Oracle custom data type that is a VARRAY of BINARY_DOUBLE[60].
I am following the TestFetchObjects.c example.

Not able to retrieve the array attributes from the object. After executing the query i get the query info and object type info

    if (dpiStmt_getQueryInfo(stmt, 29, &queryInfo) < 0)
        return dpiSamples_showError();
    printf("('%*s', %d, %d, %d, %d, %d, %d)\n",
    		queryInfo.nameLength,
            queryInfo.name,
			queryInfo.typeInfo.oracleTypeNum,
            queryInfo.typeInfo.sizeInChars,
            queryInfo.typeInfo.clientSizeInBytes,
            queryInfo.typeInfo.precision,
			queryInfo.typeInfo.scale,
            queryInfo.nullOk);

    if (dpiObjectType_getInfo(queryInfo.typeInfo.objectType, &typeInfo) < 0)
        return dpiSamples_showError();
    printf("Fetching objects of type %.*s.%.*s\n   is collection %d\n   number of attributes %d\n",
    		typeInfo.schemaLength,
            typeInfo.schema,
			typeInfo.nameLength,
			typeInfo.name,
			typeInfo.isCollection,
			typeInfo.numAttributes);

And this is the output

('VECTOR_COVAR', 2023, 0, 0, 0, 0, 1)
Fetching objects of type DATA_OWNER.SV_VARRAY
   is collection 1
   number of attributes 0

Notice that the number of attributes is zero. That is not correct. And I am unable to get values.
dpiObjectType_getAttributes()
dpiObjectAttr_getInfo()
dpiObject_getAttributeValue()

I am using Instant Client 12.2 installed locally
My development environment is Linux RHEL6

Does ODPI-C support VARRAY? If so, can you point me to an example?

1000+ Valgrind Error

==27524== Use of uninitialised value of size 8
==27524== at 0x9D3DE91: r0_des_encp_k8_64 (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9D13958: r0_cipher_cbc8lup_enc (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9D13C3B: r0_cipher_cbc8lup (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9D13C7A: R1_CIPH_CTX_cipher (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9CFEB89: r_ck_cipher_update (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF6DFB: ztca_RSAAdpEncrypt (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9E36FA4: ztcebn (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9E360C8: ztcecrypto_2 (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BDEE0B: ztceenc (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF8914: ztcrbm (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF838F: ztcrbh (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF8136: ztcrbp (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== Uninitialised value was created by a stack allocation
==27524== at 0x9BF8586: ztcrsg (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524==
==27524==
==27524== More than 1000 different errors detected. I'm not reporting any more.
==27524== Final error counts will be inaccurate. Go fix your program!
==27524== Rerun with --error-limit=no to disable this cutoff. Note
==27524== that errors may occur in your program without prior warning from
==27524== Valgrind, because errors are no longer being displayed.
==27524==
Done.
==27524==
==27524== HEAP SUMMARY:
==27524== in use at exit: 1,228,491 bytes in 368 blocks
==27524== total heap usage: 2,725 allocs, 2,357 frees, 2,084,971 bytes allocated
==27524==
==27524== 43 bytes in 1 blocks are definitely lost in loss record 100 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB46A115: slzsetevar (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB46E4DA: lfvSetOHome (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33EB3C: slpmloclfv (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33E6A1: slpmloc (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C047: lpmloadpkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E09A: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D384: lfvinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0x63BD768: kpummpin (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524== by 0x5E486A3: kpuenvcr (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524==
==27524== 192 bytes in 1 blocks are possibly lost in loss record 203 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB31755E: sltsmxi (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB3380F6: lmmhpinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB336FD6: lmmcis (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C589: lpmpali (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33B7F7: lpminitm (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E041: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D384: lfvinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0x63BD768: kpummpin (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524== by 0x5E486A3: kpuenvcr (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524==
==27524== 192 bytes in 1 blocks are possibly lost in loss record 204 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB31755E: sltsmxi (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB3380F6: lmmhpinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB336FD6: lmmcis (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C589: lpmpali (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C13C: lpmloadpkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E09A: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D384: lfvinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0x63BD768: kpummpin (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524== by 0x5E486A3: kpuenvcr (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524==
==27524== 139,264 bytes in 1 blocks are possibly lost in loss record 297 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB33AB14: lmmstvrt (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB339585: lmmstchnk (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33A909: lmmstsml (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB55516E: lmmstmalloc (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB554885: lmmmalloc (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB336EC5: lmmcis (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C589: lpmpali (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33B7F7: lpminitm (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E041: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524==
==27524== LEAK SUMMARY:
==27524== definitely lost: 43 bytes in 1 blocks
==27524== indirectly lost: 0 bytes in 0 blocks
==27524== possibly lost: 139,648 bytes in 3 blocks
==27524== still reachable: 1,088,800 bytes in 364 blocks
==27524== suppressed: 0 bytes in 0 blocks
==27524== Reachable blocks (those to which a pointer was found) are not shown.
==27524== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==27524==
==27524== For counts of detected and suppressed errors, rerun with: -v
==27524== ERROR SUMMARY: 1129 errors from 1004 contexts (suppressed: 0 from 0)
-bash-4.2$

DPI_NATIVE_TYPE_UINT64 is handled as signed int.

When a dpiVar is created with DPI_NATIVE_TYPE_UINT64 and DPI_ORACLE_TYPE_NATIVE_INT, it is defined or bound as SQLT_INT. It should be SQLT_UIN.
When a dpiVar is created with DPI_NATIVE_TYPE_UINT64 and DPI_ORACLE_TYPE_NUMBER, OCINumberFromInt() and OCINumberToInt are called with OCI_NUMBER_SIGNED. It should be OCI_NUMBER_UNSIGNED.

Compiling samples and tests.

Hello.

I'm trying to compile the samples and tests and error occurs:

Make
Gcc -c -I ../ include -O2 -g -Wall -obuild / TestContext.TestContext.c
Gcc -c -I ../ include -O2 -g -Wall -obuild / TestLib.o TestLib.c
Gcc -L ../ lib -lodpic -ldl build / TestContext.o -obuild / TestContext build / TestLib.o
Build / TestContext.o: in function dpiTest_100_validMajorMinor ': /odpi/test/TestContext.c:35: undefined reference to dpiContext_create '
/odpi/test/TestContext.c:38: undefined reference to `dpiContext_destroy '
...

OS: Ubuntu 16.4,
InstantClient: /usr/lib/oracle/12.2/client64/lib

Any tips?
Thank you.

Texts retrieved by FormatMessage on Windows

Some issues about FormatMessage used here on Windows
I might have to create separate issues. But some issues are related so I put all in one.

A potential bug

The 6th argument FormatMessageW is sizeof(wLoadError) but it should be sizeof(wLoadError)/sizeof(wchar_t) because the argument is the number of characters, not bytes, allocated for the 5th argument.

Message is incorrectly stripped on Japanese.

Trailing period and carriage return are stripped here. However, when the last character before carriage return is a multi-byte character, the character is broken. It should be stripped only when it is really period.

Message language

The error message retrieved by FormatMessageW depends on the OS language. Could you change the messages to English or add a compile-time option to change it?
The message is changed to English when the 4th argument is MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT).

Programs in the samples directory print garbled text in Japanese Windows command prompt when OCI.DLL is not found. If the message language is English, it isn't garbled.

Add error number to the message

When the message isn't changed to English, could you add the error number to the message?
For example when the error code is 126:

DPI-1047: 64-bit Oracle Client library cannot be loaded: Win32 error 126: "The specified module could not be found". See https://oracle.github.io/odpi/doc/installation.html#windows for help

When an issue is posted with a localized error message and you don't know the language, you cannot read the message. Moreover if it is garbled as messages in Japanese Windows command prompt, nobody read it. If the error number is included in the text, you can know the error as follows.

DPI-1047: 64-bit Oracle Client library cannot be loaded: Win32 error 126: "指定されたモジュールが見つかりません。". See https://oracle.github.io/odpi/doc/installation.html#windows for help

node-oracledb is now implemented with ODPI-C

If anyone wants another 'bigger' example that uses ODPI-C, the first development branch of the Node.js node-oracledb interface is available, see the release announcement.

The code is currently on the dev-2.0 branch.

Cannot bind negative time zone minute offset such as `-03:30`.

When a timestamp with negative time zone minute offset such as -03:30 is fetched, dpiTimestamp.tzMinuteOffset is negative. However when the fetched timestamp is set to dpiData_setTimestamp, dpiStmt_execute fails with ORA-01875: time zone minute must be between -59 and 59.

I made a test program.
https://gist.github.com/kubo/3d8cfb06a71454d137e3014475e62674

It printed:

Timestamp value:
  year: 2017
  month: 1
  day: 1
  hour: 0
  minute: 0
  second: 0
  fsecond 0
  tzHourOffset: -3
  tzMinuteOffset: -30

ERROR at line 84
  dpiStmt_execute(stmt, 0, &numQueryColumns)
  ORA-01875: time zone minute must be between -59 and 59

This issue is fixed if ts.tzMinuteOffset in this line is replaced with abs(ts.tzMinuteOffset). However this is strange workaround.

App crashes when acquiring connection from pool

Hello,

I what to use the driver in a multithreaded project. So I created a pool and I am acquiring a connection from the pool in order to create a statement and then to execute this statement. However the code crashes when sending multiple request (via an http interface). It never crashes so far when sending the requests one by one.
The method where the crash begins is:

int test_method( ... , *context ) {
  if (dpiPool_acquireConnection(pool, NULL, 0, NULL, 0, NULL, &conn) < 0 )
        return ShowError(context);
 
    if( conn ) {
            dpiConn_release(conn) ;
     }
}

and the pool is created like this

dpiErrorInfo errorInfo;
  dpiContext *context;

  if (dpiContext_create(DPI_MAJOR_VERSION, DPI_MINOR_VERSION, &context, &errorInfo) < 0) {
      return -1;
  }
  dpiCommonCreateParams commonParams;
  dpiPoolCreateParams createParams;

  dpiPool *pool;

  if (dpiContext_initCommonCreateParams(context, &commonParams) < 0) {
      std::cout << "cannot init common params" << std::endl;
      return 1;
  }
  commonParams.createMode = DPI_MODE_CREATE_THREADED;

  if (dpiContext_initPoolCreateParams(context, &createParams) < 0)
  {
      std::cout << "cannot init the params" << std::endl;
      return 1; //dpiTestCase_setFailedFromError(testCase);
  }
  createParams.minSessions = 50; //MINSESSIONS;
  createParams.maxSessions = 100; //MAXSESSIONS;
  createParams.sessionIncrement = 20; //SESSINCREMENT;

  if (dpiPool_create(context, "DB", strlen("DB"),
          "DB", strlen("DB"), "1.7.7.16:1521/DEV",
          strlen("1.7.7.16:1521/DEV"), NULL, &createParams,  &pool) < 0)
      return 1; //dpiTestCase_setFailedFromError(testCase);

Is there a problem with the way I am creating the pool or the problem is one the way I am acquiring the connection?

I am working with oracle 12c in a Linux machine, and as I said the app works w/o problem when the requests are made sequentially.

@kubo's ruby-odpi interface

@kubo has been busy (surprise!) and his ruby-odpi driver is taking shape at https://github.com/kubo/ruby-odpi

Separate LDFLAGS in Makefile

Could you separate LDFLAGS in Makefile to LDFLAGS and LIBS and use $LIBS at the last position of the link command as follows?
Conventionally LDFLAGS doesn't include dependent libraries and they are set to LIBS.

from

	LDFLAGS=-L$(OCI_LIB_DIR) -lclntsh -shared
   ... skip ...
$(LIB_DIR)/$(LIB_NAME): $(OBJS)
	$(LD) $(LDFLAGS) $(LIB_OUT_OPTS) $(OBJS)

to

	LDFLAGS=-shared
	LIBS=-L$(OCI_LIB_DIR) -lclntsh
   ... skip ...
$(LIB_DIR)/$(LIB_NAME): $(OBJS)
	$(LD) $(LDFLAGS) $(LIB_OUT_OPTS) $(OBJS) $(LIBS)

When odpi is compiled on Linux without the above changes, the -L$(OCI_LIB_DIR) -lcltnsh options is set just after gcc.
But it takes no effect as follows.

$ make
mkdir build
mkdir lib
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC  src/dpiConn.c -obuild/dpiConn.o
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC  src/dpiContext.c -obuild/dpiContext.o
   ... skip ...
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC  src/dpiMsgProps.c -obuild/dpiMsgProps.o
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC  src/dpiRowid.c -obuild/dpiRowid.o
gcc -L/opt/instantclient -lclntsh -shared -o lib/libodpic.so build/dpiConn.o build/dpiContext.o build/dpiData.o build/dpiEnv.o build/dpiError.o build/dpiGen.o build/dpiGlobal.o build/dpiLob.o build/dpiObject.o build/dpiObjectAttr.o build/dpiObjectType.o build/dpiPool.o build/dpiStmt.o build/dpiUtils.o build/dpiVar.o build/dpiOracleType.o build/dpiSubscr.o build/dpiDeqOptions.o build/dpiEnqOptions.o build/dpiMsgProps.o build/dpiRowid.o
$ ldd lib/libodpic.so 
	linux-vdso.so.1 =>  (0x00007ffd581f0000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f788b44d000)
	/lib64/ld-linux-x86-64.so.2 (0x00005572bb26f000)

libodpic.so doesn't depend on libclntsh.so.

If -L/opt/instantclient -lclntsh is set at last. libodpic.so depends on libclntsh.so.

$ gcc -shared -o lib/libodpic.so build/dpiConn.o build/dpiContext.o build/dpiData.o build/dpiEnv.o build/dpiError.o build/dpiGen.o build/dpiGlobal.o build/dpiLob.o build/dpiObject.o build/dpiObjectAttr.o build/dpiObjectType.o build/dpiPool.o build/dpiStmt.o build/dpiUtils.o build/dpiVar.o build/dpiOracleType.o build/dpiSubscr.o build/dpiDeqOptions.o build/dpiEnqOptions.o build/dpiMsgProps.o build/dpiRowid.o -L/opt/instantclient -lclntsh
$ ldd lib/libodpic.so 
	linux-vdso.so.1 =>  (0x00007ffdeb6bd000)
	libclntsh.so.12.1 => /opt/instantclient/libclntsh.so.12.1 (0x00007f0b285c5000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0b281d8000)
	libmql1.so => /opt/instantclient/libmql1.so (0x00007f0b27f61000)
	libipc1.so => /opt/instantclient/libipc1.so (0x00007f0b27be3000)
	libnnz12.so => /opt/instantclient/libnnz12.so (0x00007f0b274d9000)
	libons.so => /opt/instantclient/libons.so (0x00007f0b27293000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0b2708f000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0b26d86000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0b26b68000)
	libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f0b2694f000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0b26747000)
	/lib64/ld-linux-x86-64.so.2 (0x0000562a4e5cf000)
	libaio.so.1 => /lib/x86_64-linux-gnu/libaio.so.1 (0x00007f0b26544000)
	libclntshcore.so.12.1 => /opt/instantclient/libclntshcore.so.12.1 (0x00007f0b25fd2000)

Off-topic: When I write Makefile, I won't use EXTRA_CFLAGS. I'll use CFLAGS += -DDPI_TRACE_REFS instead.

DPI_ORACLE_TYPE_LONG_NVARCHAR?

LONG corresponds to DPI_ORACLE_TYPE_LONG_VARCHAR.
LONG RAW corresponds to DPI_ORACLE_TYPE_LONG_RAW.
What Oracle data type corresponds to DPI_ORACLE_TYPE_LONG_NVARCHAR?

amount of characters read by dpiLob_readBytes()

Is it guaranteed that dpiLob_readBytes() reads characters up to the specified amount if it doesn't reach the end of the LOB?
In other words, does the following code work? (Error checks are omitted for simplicity.)

    dpiLob *lob = ...;
    FILE *fp = ...;
    const uint64_t char_size = 8192;
    uint64_t byte_size;
    char *buf;
    uint64_t offset = 1;

    dpiLob_getBufferSize(lob, char_size, &byte_size);
    buf = malloc(byte_size);

    while (1) {
        uint64_t read_byte_len = byte_size;
        dpiLob_readBytes(lob, offset, char_size, buf, &read_byte_len);
        if (read_byte_len == 0) {
            break;
        }
        fwrite(buf, 1, read_byte_len, fp);
        offset += char_size;
    }

This assume that dpiLob_readBytes() always reads 8192 characters if it doesn't reach the end of the LOB. If this assumption is incorrect, could you change dpiLobReadBytes() or add a new function which returns the number of read characters as OCILobRead2() does? Otherwise, it must be counted in the caller to increment offset as follows.

        offset += number_of_characters_in_buf(buf, read_byte_len);

Help using ODPI-C

I'm trying to write a Go database/sql/driver from scratch (I've created a cx_Oracle-like driver back as goracle, and later joined rana/ora, rewritten as ora.v4, and now trying to rewrite it from scratch using ODPI-C. cx_Oracle's quality is what I'm chasing).

I've stuck right at the beginning: get the Data from a Query.

I use dpiConn_newVar to get a *dpiVar and an array of *dpiData, store them for later use

    C.dpiStmt_setFetchArraySize(st.dpiStmt, fetchRowCount)
    var info C.dpiQueryInfo
    for i := 0; i < colCount; i++ {
        if C.dpiStmt_getQueryInfo(st.dpiStmt, C.uint32_t(i+1), &info) == C.DPI_FAILURE {
            return nil, st.getError()
        }
        typ, numTyp := info.oracleTypeNum, info.defaultNativeTypeNum
        bufSize := maxArraySize * info.clientSizeInBytes
        switch typ {
        case C.DPI_ORACLE_TYPE_VARCHAR, C.DPI_ORACLE_TYPE_NVARCHAR, C.DPI_ORACLE_TYPE_CHAR, C.D
PI_ORACLE_TYPE_NCHAR:
            bufSize *= 4
        }
        var dataArr *C.dpiData
        if C.dpiConn_newVar(
            st.conn.dpiConn, typ, numTyp, maxArraySize,
            bufSize, 1, 0, info.objectType,
            &r.vars[i], &dataArr,
        ) == C.DPI_FAILURE {
            return nil, st.getError()
        }
        if C.dpiStmt_define(st.dpiStmt, C.uint32_t(i+1), r.vars[i]) == C.DPI_FAILURE {
            return nil, st.getError()
        }
        r.data[i] = (*((*[maxArraySize]*C.dpiData)(unsafe.Pointer(dataArr))))[:]
    }
    if C.dpiStmt_addRef(st.dpiStmt) == C.DPI_FAILURE {
        return &r, st.getError()
    }

then I use dpi_fetch (tried dpi_fetchRows, with no avail)

    var found C.int
    if C.dpiStmt_fetch(r.dpiStmt, &found, &r.bufferRowIndex) == C.DPI_FAILURE {
        return r.getError()
    }
    if found == 0 {
        r.finished = true
        return io.EOF
    }
     for i, col := range r.columns {
        var typ C.dpiNativeTypeNum
        var d *C.dpiData
        if C.dpiStmt_getQueryValue(r.dpiStmt, C.uint32_t(i+1), &typ, &d) == C.DPI_FAILURE {
            return r.getError()
        }
        if d.isNull == 1 {
            dest[i] = nil
            continue
        }
        switch typ {
        case C.DPI_ORACLE_TYPE_VARCHAR, C.DPI_ORACLE_TYPE_NVARCHAR, C.DPI_ORACLE_TYPE_CHAR, C.D
PI_ORACLE_TYPE_NCHAR, C.DPI_ORACLE_TYPE_LONG_VARCHAR:
            b := C.dpiData_getBytes(d)
            fmt.Printf("b=%p[%d]\n", b.ptr, b.length)
            if b.ptr == nil {
                dest[i] = ""
                continue
            }
            dest[i] = C.GoStringN(b.ptr, C.int(b.length))
 ...

Fails with SIGSEGV and when not, then data's bytes are zero.

1. any obvious failures?

2. is dpiStmt_define's position starts from 1?

3. what is bufferRowIndex? What shall I index it with? The array of dpiData? But then why is that needed in dpiStmt_fetch ?

4. I read cx_Oracle/src/Cursor.c, and it increments bufferRowIndex right after the fetch - why does this not skip the first value? As I printed out the dpiData array after the fetch, the first element is nil, so this indicates such thing, but I don't know.

Thanks in advance,
Tamás Gulácsi

Answer the following questions:

1. What version of ODPI-C are you using (see dpi.h)?
   2.0.0-beta.4

2. What error(s) you are seeing?
   It's all my application code's fault.

Could not get object type attribute number as int64 or bytes

Object type attribute number could be get as double but not as int64 or bytes.

I made a test program.
https://gist.github.com/kubo/aea497d746d3d1603882c86b450accc0

It printed:

Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_DOUBLE
  => 1
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_INT64
  => ERROR: DPI-1014: conversion between Oracle type 2010 and native type 3000 is not implemented
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_BYTES
  => ERROR: DPI-1014: conversion between Oracle type 2010 and native type 3004 is not implemented

What I want is:

Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_DOUBLE
  => 1
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_INT64
  => 1
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_BYTES
  => 1

Cannot build project in Cygwin

1. Downloaded and unzipped the Instant Client to "C:\instant_client"
2. Added "C:\instant_client" to PATH system variable
3. Copied the include files to Cygwin include folder
4. Copied src folder to the project folder
5. Created a simple program main.c:

<<<
#include <stdio.h>
#include <dpi.h>

int main()
{
dpiErrorInfo errorInfo;
dpiContext *gContext = NULL;
if (dpiContext_create(DPI_MAJOR_VERSION, DPI_MINOR_VERSION, &gContext, &errorInfo) < 0) {
fprintf(stderr, "ERROR: %.*s (%s : %s)\n", errorInfo.messageLength, errorInfo.message, errorInfo.fnName, errorInfo.action);
return -1;
}
return 0;
}

in the CMakeList.txt:
<<<
cmake_minimum_required(VERSION 3.7)
project(odpi)
set(CMAKE_C_STANDARD 11)

set(SOURCE_FILES
main.c
src/dpiConn.c
src/dpiContext.c
src/dpiData.c
src/dpiDeqOptions.c
src/dpiEnqOptions.c
src/dpiEnv.c
src/dpiError.c
src/dpiGen.c
src/dpiGlobal.c
src/dpiLob.c
src/dpiMsgProps.c
src/dpiObject.c
src/dpiObjectAttr.c
src/dpiObjectType.c
src/dpiOci.c
src/dpiOracleType.c
src/dpiPool.c
src/dpiRowid.c
src/dpiStmt.c
src/dpiSubscr.c
src/dpiUtils.c
src/dpiVar.c
)
add_executable(odpi ${SOURCE_FILES})

When I try to build project I get the following error:
ERROR: DPI-1047: Oracle Client library cannot be loaded: No such file or directory. See https://oracle.github.io/odpi/doc/installation.html for help (dpiContext_create : load library)

How I can load client library?

Suggestions about debug message format

There are suggestions about debug message format: (1) thread ID and (2) date format.

When DPI_DEBUG_LEVEL was set, the following messages were printed on Linux.

ODPI [140274862778112] 2017/09/02 14:13:44.244: ref 0x167c1d0 (dpiPool) -> 1 [NEW]
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x17026b0 (dpiConn) -> 1 [NEW]
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x167c1d0 (dpiPool) -> 2
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x167c1d0 (dpiPool) -> 1
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x16ed170 (dpiVar) -> 1 [NEW]
...

Thread ID

The thread ID 140274862778112 is available to identify a thread but it is a too big number.
That's because it is an address of struct pthread in the process, not a thread ID in the OS.
If it is a thread ID in the OS, the messages are printed as follows:

ODPI [11318] 2017/09/02 14:36:30.607: ref 0x151a1d0 (dpiPool) -> 1 [NEW]
ODPI [11318] 2017/09/02 14:36:30.696: ref 0x15a06b0 (dpiConn) -> 1 [NEW]
ODPI [11318] 2017/09/02 14:36:30.696: ref 0x151a1d0 (dpiPool) -> 2
ODPI [11318] 2017/09/02 14:36:30.697: ref 0x151a1d0 (dpiPool) -> 1
ODPI [11318] 2017/09/02 14:36:30.697: ref 0x158b170 (dpiVar) -> 1 [NEW]
...

Thread IDs in the OS is retrieved by syscall(SYS_gettid) on Linux.

Thread IDs in the macOS seems to be retrieved by pthread_threadid_np.
It is available since OS X 10.6 Snow Leopard. The source of the man page about pthread_threadid_np is here but formatted page could not be found in the apple site.

Date format

I prefer YYYY-MM-DD to YYYY/MM/DD because the former is same with standards.
The latter is widely used in Japan. But I don't know whether it is acceptable in other countries. (I have not lived outside of Japan.)

init 0 for last byte for padding terminator

Please init 0 for last byte while initialize xxx->value.asBytes.ptr. It always display the weird thing at the last character. For example
strIngColval->value.asBytes.ptr[strIngColval->value.asBytes.length] = 0;

Bind PL/SQL array types

HI,

I need some help: I want to bind PL/SQL array types - use of DBMS_OUTPUT.get_lines works, so OUT is OK.
For IN OUT types (TYPE vc_tab_typ IS TABLE OF VARCHAR2(100) INDEX BY PLS_INTEGER; PROCEDURE x(p_vc IN OUT NOCOPY vc_tab_typ)),

1. dpiConn_newVar" typ=2001 natTyp=3004 arraySize=2 bufSize=32767 isArray=1,
2. dpiVar_setFromBytes pos=1 then pos=2, and then call
3. dpiVar_setNumElementsInArray n=2
4. dpiStmt_bindByPos
5. dpiStmt_execute result is ORA-06513: PL/SQL: index for PL/SQL table out of range for host language array

        stmt.go:259: enter="bindVars" args=[]driver.NamedValue{driver.NamedValue{Name:"", Ordinal:1, Value:sql.Out{_Named_Fields_Required:struct {}{}, Dest:(*[]string)(0xc42000c380), In:true}}}
        stmt.go:341: msg="bindVars" i=0 in=true out=true value="[]string []string{\"string\", \"bring\"}"
        stmt.go:482: msg="newVar" i=0 plSQLArrays=true typ=2001 natTyp=3004 sliceLen=2 bufSize=32767
        conn.go:179: C="dpiConn_newVar" conn=&goracle._Ctype_struct_dpiConn{} typ=2001 natTyp=3004 arraySize=2 bufSize=32767 isArray=1 v=(*goracle._Ctype_struct_dpiVar)(nil)
        stmt.go:511: msg="set" i=0 j=1 n=2 v="string=\"string\""
        stmt.go:670: C="dpiVar_setFromBytes" dv=&goracle._Ctype_struct_dpiVar{} pos=1 p=(*goracle._Ctype_char)(0xc4200126a8) len=6
        stmt.go:511: msg="set" i=0 j=2 n=2 v="string=\"bring\""
        stmt.go:670: C="dpiVar_setFromBytes" dv=&goracle._Ctype_struct_dpiVar{} pos=2 p=(*goracle._Ctype_char)(0xc420012710) len=5
        stmt.go:519: C="dpiVar_setNumElementsInArray" n=2
        stmt.go:530: C="dpiStmt_bindByPos" dpiStmt=&goracle._Ctype_struct_dpiStmt{} i=0 v=&goracle._Ctype_struct_dpiVar{}
        stmt.go:174: C="dpiStmt_execute" mode=0x20 colCount=0x0
        z_test.go:236: ORA-06513: PL/SQL: index for PL/SQL table out of range for host language array
                ORA-06512: at line 1

                dpiStmt_execute(mode=32 arrLen=1024)
                gopkg.in/goracle%2ev2.(*statement).ExecContext
                        /home/gthomas/src/gopkg.in/goracle.v2/stmt.go:179

I've tried ofsetting the pos at dpiVar_setFromBytes, changing n of setNumElementsInArray, resulting in different errors.

What am I doing wrong?

Thanks in advance,
Tamás Gulácsi

GetObject and other object-related API questions

      objType := (*C.dpiObjectType)(C.malloc(C.sizeof_void))
      if C.dpiConn_getObjectType(c.dpiConn, cName, C.uint32_t(len(name)), (**C.dpiObjectType)(unsafe.Pointer(&objType))) == C.DPI_FAILURE {
          C.free(unsafe.Pointer(objType))
          return nil, errors.Wrapf(c.getError(), "getObjectType(%q) conn=%p", name, c.dpiConn)
      }

result:

$ go install && go test -run=Object
--- FAIL: TestObject (3.58s)
        conn.go:147: msg="PrepareContext" shortcut="--GET_OBJECT_TYPE--"
        stmt.go:235: msg="QueryContext" args=[]driver.NamedValue{driver.NamedValue{Name:"", Ordinal:1, Value:"test_pkg_obj.int_tab_typ"}}
        z_test.go:710: ORA-04043: DPI-1002: invalid dpiStmt handle
                getObjectType("test_pkg_obj.int_tab_typ") conn=0x1778470
                gopkg.in/goracle%2ev2.(*conn).GetObjectType
                        /home/gthomas/src/gopkg.in/goracle.v2/obj.go:165
                gopkg.in/goracle%2ev2.(*statement).QueryContext

here c.dpiConn is a proper *dpiConn. Why is an "invalid dpiStmt handle" error returned?

precision information of timestamp and intarval

Could you add lfprec and fsprec to dpiQueryInfo?
If they are added, users can know precisions of timestamp and interval data types.
There is no merit for driver developers. Drivers can handle these Oracle data types without precisions. This is just information for users.

ruby-oci8 can print such information as follows:

conn = OCI8.new('username/password')
conn.exec(<<EOS)
  create table test_types (
    col1 timestamp not null,
    col2 timestamp(9),
    col3 interval day to second,
    col4 interval day(2) to second(3)
  )
EOS
cursor = conn.exec('select * from test_types where 1 = 2')
# string representation of column types which could be put in create table statements
cursor.column_metadata[0].data_type_string # => TIMESTAMP NOT NULL
cursor.column_metadata[1].data_type_string # => TIMESTAMP(9)
cursor.column_metadata[2].data_type_string # => INTERVAL DAY TO SECOND
cursor.column_metadata[3].data_type_string # => INTERVAL DAY(2) TO SECOND(3)

Long string converted from number corrupts memory.

ODPI-C defines DPI_NUMBER_AS_TEXT_CHARS as 135 in dpiImpl.h. However it is too short. It must be at least 170.

I made a test code and put it here.
The following is the result.

define as DPI_ORACLE_TYPE_VARCHAR and DPI_NATIVE_TYPE_BYTES
  isNull=0x0
  len=170
  str=-.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000009999999999999999999999999999999999999999

define as DPI_ORACLE_TYPE_NUMBER and DPI_NATIVE_TYPE_BYTES
  isNull=0x39393939
  len=960051553
  str=0x3939393939393939

When a number column UTL_RAW.CAST_TO_NUMBER('7F0202020202020202020202020202020202020202') is defined as DPI_ORACLE_TYPE_VARCHAR and DPI_NATIVE_TYPE_BYTES, it is fetched as a string whose length is 170.
When the same number is defined as DPI_ORACLE_TYPE_NUMBER and DPI_NATIVE_TYPE_BYTES, isNull is 0x39393939 and str is 0x3939393939393939. isNull must be either 0 or 1. It is apparently overwritten with "9999" (code point of '9' is 0x39). str is also overwritten with "99999999".

dpiConn_startupDatabase() pfile parameter

Do you have any plan to make dpiConn_startupDatabase() accept pfile as Data Provider for .NET does?

The priority of this feature is extremely low. I sometimes use startup pfile=xxx on sqlplus to check effect of initialization parameters without changing persistent parameters. However I could not imagine who use it in applications.

ODPI-C implemented in cx_Oracle

For those interested in viewing an example of a driver implemented using ODPI-C you can take a look at the Python Oracle Database driver cx_Oracle. You can compare the code between the 5.x branch (direct OCI) and master branch (ODPI-C) to see the differences in the two approaches. I believe you'll see that the code is considerably simpler with ODPI-C.

Is OCIThread supported?

Just wanted to know whether OCIThread is supported in ODPI for now, I see no APIs offering multi-threading using OCIThread, don't know whether I am missing something here!

Rust bindings over ODPI

Looks like you have new implementations announcements here, so I'll toss mine in the mix.

https://github.com/rustyhorde/mimir

Still a work in progress, but the initial implementation is complete. Needs documentation.

query re api stability

While preparing the Solaris Userland (https://github.com/oracle/solaris-userland) integration
of ODPI-C, one of my code reviewers asked about the stability of the API.

• Is it anticipated that there will be incompatible changes to the API in future?
• Is it anticipated that customers might want to have multiple versions of this library installed?

Answer the following questions:

1. What version of ODPI-C are you using (see dpi.h)?

2.0.2

2. What error(s) you are seeing?

None

3. What exact command caused the problem (e.g. what command did you try to install with)? Who were you logged in as?

n/a

4. What environment variables did you set? How exactly did you set them?

n/a

5. What is your version of the Oracle client (e.g. Instant Client)? How was it installed? Where it is installed?

12.1 and 12.2, via IPS packages

6. What is your OS and version?

Solaris 11.4 and 11.3

7. What compiler version did you use?

gcc 5.4

Incorrect NVARCHAR2 object attribute information

dpiObjectAttr_getInfo() returns incorrect dpiOracleTypeNum about NVARCHAR2 attributes.

I made a test program which gets attribute information about udt_ObjectDataTypes.
https://gist.github.com/kubo/17189baff057bb8fdd5797ae80750518
It printed:

STRINGCOL - 2001
UNICODECOL - 2001
FIXEDCHARCOL - 2003
FIXEDUNICODECOL - 2004
INTCOL - 2010
NUMBERCOL - 2010
DATECOL - 2011
TIMESTAMPCOL - 2012
TIMESTAMPTZCOL - 2013
TIMESTAMPLTZCOL - 2014
BINARYFLTCOL - 2007
BINARYDOUBLECOL - 2008

The datatype of UnicodeCol is nvarchar2(60). Its oracleTypeNum should be 2002(DPI_ORACLE_TYPE_NVARCHAR). However it is 2001(DPI_ORACLE_TYPE_VARCHAR).

Some questions about odpi

Hi,

ODPI seems to be a really good initiave :)

I'm the author of OCILIB and I had a quick look at odpi code in order to check if moving OCILIB code to ODPI could be an option :)

1 - Why restrict Oracle client version to 11gR2 and above ?
Lot's of application using Oracle often reliy on heterogenous Oracle versions and often not the latest ones. Imposing this limit is a very limiting from my point of view

2 - Why not permitting runtime loading of OCI shared libraries ?
Can be limiting, especially on Windows platforms

2 - Has it been heavily tested with Environment set to UTF16 ?
OCI had shown a numerous amount of bugs over the years in Unicode mode.
And Some are still present many years after been reported.
2 examples : Retrieving column name form a describe call is leaking, calling OCIObjectSetAttr() and OCIObjectGetAttr() still fail.
In OCILIB, I had to find workarounds (sometimes ugly) for these issues (re-implementing OCIObjectSetAttr/OCIObjectGetAttr for example).
Shall odpi try to "compensate" OCI internal issues or shall it remain a generic wrapper on top of it ?
In latter case, trying to concentrate any driver implementation on odpi seems to me preventing from using OCI in UTF16 mode in real life.

3 - Lack of support for some types such as collections
Nested tables and varrays seems to not be supported.
Do you plan to support them in a short term time frame ?

Do you have any documented plans/roadmap ?
In its current state, some OCI features seems to not be supported yet : Direct Path, full SQL/PLSQL type support, user callback tracing, ....

The only purpose of these questions is to find out if I could make OCILIB based on ODPI or not at some point :)

Best regards,

Vincent

information about merge statement

dpiStmt_getInfo() returns incorrect information about merge.

I made a test program.
https://gist.github.com/kubo/d8b623072b755edd508896940be86e81
It printed:

SQL: merge info ...
  isQuery: 0
  isPLSQL: 0
  isDDL: 0
  isDML: 0
  statementType: 16
  isReturning: 0

Merge is DML but isDML is zero and statementType is a value not listed here.
In this example the statement is not a valid SQL but it returns same results even for valid statements.

Well, I cannot say that this is a bug due to odpi. That's because OCI_STMT_MERGE is not in OCI manual.
I hope that the Oracle team says 16 is the statement type of merge.

soname in libodpic.so

Do you have any plan to add soname such as libodpic.so.2?

• Link libodpic.so with -hlibodpic.so.2 linker flag.
• Install libodpic.so as libodpic.so.2.0.x.
• Make two symbolic links libodpic.so.2 and libodpic.so which point to libodpic.so.2.0.x.
• When the major version is changed to 3, the soname is also changed to libodpic.so.3.

Soname gets it anticipated that customers might want to have incompatible versions such as ODPI-C 2.x and ODPI-C 3.x installed. (the second question in #42)

Go Language driver uses ODPI-C

Followers of ODPI-C will have seen questions from @tgulacsi who was working on a Go driver using ODPI-C. He has it up at https://gopkg.in/goracle.v2 with the code at https://github.com/go-goracle/goracle/tree/v2.0.0

dpiSubscr_prepareStmt doesn't increment openChildCount.

dpiSubscr_prepareStmt doesn't increment openChildCount in dpiConn.
It should be incremented as dpiConn_prepareStmt does.
Or it should not be decremented when a dpiStmt created by dpiSubscr_prepareStmt is closed.

TestSuiteRunner.c uses system(), should use posix_spawn() instead

While examining the odpi source in preparation for creating an IPS packaged version, I came across the test suite's TestSuiteRunner.c file, which contains these lines:

 98     // run executable and return success only if all tests pass
 99     result = system(executableName);

This violates https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=2130132, aka " ENV33-C. Do not call system()".

The posix_spawn() function (see http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html) is available on Linux, Mac osx, Solaris and other UNIX-like systems. For MS Windows, one of the functions noted at https://en.wikipedia.org/wiki/Spawn_(computing) would be preferable.

Commit 292d09bdf breaks dpiDeqOptions_getWait and dpiMsgProps_getNumAttempts

The changes to the dpiDeqOptions__getAttrValue function where the valueLength is checked for null cause the dpiDeqOptions_getWait call to always fail because the valueLength is hardcoded to NULL.

int dpiDeqOptions_getWait(dpiDeqOptions *options, uint32_t *value)
{
    return dpiDeqOptions__getAttrValue(options, DPI_OCI_ATTR_WAIT, __func__,
   value, NULL);
}

Similar case in dpiMsgProps

int dpiMsgProps_getNumAttempts(dpiMsgProps *props, int32_t *value)
{
    return dpiMsgProps__getAttrValue(props, DPI_OCI_ATTR_ATTEMPTS, __func__,
            value, NULL);
}

Those two fell out of my tests, but a quick glance at some other functions where a numeric is the out value have the same issue.

Temporary LOB caching

Dear Anthony,

There is a problem in the folowing function (dpiLob.c:1427):

int dpiOci__lobCreateTemporary(dpiLob *lob, dpiError *error)
{
    uint8_t lobType;
    int status;

    DPI_OCI_LOAD_SYMBOL("OCILobCreateTemporary",
            dpiOciSymbols.fnLobCreateTemporary)
    if (lob->type->oracleTypeNum == DPI_ORACLE_TYPE_BLOB)
        lobType = DPI_OCI_TEMP_BLOB;
    else lobType = DPI_OCI_TEMP_CLOB;
    status = (*dpiOciSymbols.fnLobCreateTemporary)(lob->conn->handle,
            error->handle, lob->locator, DPI_OCI_DEFAULT,
            lob->type->charsetForm, lobType, 0, DPI_OCI_DURATION_SESSION);
    return dpiError__check(error, status, lob->conn, "create temporary LOB");
}

It creates temporary lob with cache=false, which leads to unnecessary physical io every time such a variable is created.

For example, if we try to pass empty string from python code to plsql procedure with CLOB in parameter using cx_Oracle and ODPI, it would generate physical IO (could be easily seen in 10046 trace).

I believe it should be changed to chache=true, or, which is probably better, make this parameter configurable.

Regards,
Andrey

Make libodpic.so depend on libdl and libpthread on Unix.

Could you add LIBS=-ldl -lpthread to Makefile and remove -ldl -lpthread in test/Makefile and samples/Makefile?

When libodpic.so depends on libdl and libpthread, there is no need to link with -ldl -lpthread to build programs depending on libodpic.so.