Comments (6)
You are correct that our .thrift file is not present in this repo nor any of our other OSS Thrift-based connectors. Databricks have not chosen to open source it at this time.
Are you blocked by not having this file? How so?
from databricks-sql-python.
That is disappointing that they have decided not to open source the contract
I was actually trying to write a similar library to this but for dotnet. This repo was being used as inspiration to how it can be done. I was using the hive projects thrift file for generation but it has issues connecting to sql warehouse instances (although connects to other clusters fine).
Sounds like what I was trying to do is not really possible until databricks shares the thrift file or they develop their own dotnet native connector
from databricks-sql-python.
Thanks for that context.
You can use the open source Hive thrift definition to interact with Databricks because the Databricks runtime is backwards compatible with it. But you're right that trying to implement the same features as our OSS connectors that were developed post-fork will be impossible without access to our Thrift contract. I've drawn attention to this with my team, which maintains all our OSS connectors (currently for Python, Node, and Go).
To connect with Databricks from C#, I think your best bet is to connect using our ODBC driver and the C# ODBC tooling.
I'm going to keep your issue closed on this repository since it's not relevant for Python development. But this is an ongoing discussion for us internally and you are welcome to respond here or ping my @
if you need. Thanks for starting this discussion.
from databricks-sql-python.
@alex-davies we are actually going to release a new REST API with well documented interfaces very soon, and then we will transition all the connectors to that new API. If you can just wait a little bit, we'd love to work with you to get a dotnet version based on that new API.
from databricks-sql-python.
@susodapop @rxin Are you guys still going to move the connectors over to the rest api? I would also like to make a databricks connector in C#. The ODBC driver is not easy to configure if you are running your application in a container. Would it be possible for you guys to generate the thift files in dotnet and check them into a repo so that I can make a dotnet connector for databricks? It would be ideal if I could implement using the IDbConnection so I could use like any other data source in dotnet.
from databricks-sql-python.
https://github.com/apache/arrow-adbc/tree/main/csharp/src/Drivers/Apache/Spark this is an .net ADBC driver against thrift under development.
from databricks-sql-python.
Related Issues (20)
- cursor.execute returing error 'NoneType' object is not iterable HOT 15
- [Document] Support databricks-cli authentication HOT 1
- Dash/Minus in column name causes UNBOUND_SQL_PARAMETER in bind values HOT 2
- Extremely slow import times on Python 3.12 HOT 9
- [sqlalchemy] execute("select 1") gives TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' HOT 8
- `databricks.sql.connect` hangs in a long retrying loop when an invalid access token is used HOT 3
- Idea: arrow_record_batches cursor method
- SQLAlchemy engine from workspace level service principle? HOT 2
- Unable to write list/array type data HOT 2
- Issue with version 3.1.1
- Failure on cursor.fetchall() HOT 2
- Fixing a couple type problems. (adding py.typed, typing connect, returning Any from fetchall (which I failed to fix!)) HOT 2
- Connector reads 0 rows although Cluster returned results HOT 18
- support new Cursor attribute that provides information on completed commands HOT 1
- loosen, update, or widen pyarrow dependencies HOT 2
- ImportError: cannot import name 'sql' from partially initialized module 'databricks' HOT 2
- Unpin Thrift
- Original thrift file HOT 3
- Invalid SessionHandle Error
- PyCharm IDE + pandas exit code 139
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from databricks-sql-python.