Coder Social home page Coder Social logo

Comments (3)

aherlihy avatar aherlihy commented on May 29, 2024

The main goal of this ticket was to reduce the amount of code and rely more on the Numpy C API instead of manually manipulating pointers and using memcpy. The bottom line is because we are using named dtypes, a lot of the macros will not work.

PyArray_SetItem: requires a PyObject which would introduce overhead for converting BSON objects into PyObjects only so we could use this instead of memcpy. Additionally, we need to separate out the validation depending on type, so even if we use this instead of memcpy we could not get rid of separate loading functions. Lastly, flexible dtypes that may be shorter than the assigned size (i.e. string, bytes, etc) need to be zero-padded. Could swap out memcpy in simple scalar cases, but would not end up making the code any less complex.

PyArray_GetPtr: See #30

NpyIter_*: We iterate through the BSON object instead of through the ndarray. Can revisit this while implementing #5.

Ultimately -- the issues that we have with the major Numpy C API Macros are not ones that are going away anytime soon, so I believe we can close this issue.

from bson-numpy.

prashantmital avatar prashantmital commented on May 29, 2024

It is likely that the Numpy macros now support dtypes since it has had a while to mature. We should reinvestigate doing this.

from bson-numpy.

prashantmital avatar prashantmital commented on May 29, 2024

Closing this as we have discontinued development of BSON-NumPy. PyMongoArrow is now the recommended way to materialize MongoDB query results as NumPy ndarrays as well as tabular formats like Pandas' DataFrames and PyArrow Tables.

from bson-numpy.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.