Comments (2)
@sam-s I discovered why this bug has been elusive in the past. When a dataframe has multiple columns and one of those columns is of dtype float32, float64, string, or object, calling dataframe.values
causes pandas to find a common dtype for a numpy array to hold values from all the columns. And since float32, float64, string, and object dtypes all result in values that are JSON serializable, there's no bug. Only for a dataframe whose columns are all non-JSON-seralizable dtypes (like int64) does the bug surface.
The solution in PR #36 decides on 'object' as the common dtype in all cases, and uses the recommended dataframe.to_numpy
method instead of the deprecated dataframe.values
.
from gspread-dataframe.
@sam-s: Reading through all the pandas documentation, I'm torn between two possible approaches to solve this problem:
- Use a custom JSON serializer that supports all the pandas/numpy data types.
- Make pandas do the work of converting the dtypes to Python basic data types.
I'm leaning toward option 2, and I'm wondering if you think it's a good idea to use dataframe.to_numpy('object')
and dataframe.index.to_numpy('object')
to let pandas/numpy do the necessary type conversions. I'm getting correct results with this technique, you can see it on pull request #36 for branch int64-json-35
.
from gspread-dataframe.
Related Issues (20)
- buggy behavior when dealing with decimals? HOT 5
- Bug: Object of type int64 is not JSON serializable - introduced by Version 3.1.1 HOT 8
- Bug: escaping when allow_formulas = False is not returning any value HOT 1
- Sheets API erroneously rejects new rowCount if (new rowCount * old columnCount) > 5000000 (was: Bug in the number of cells of the dataframe HOT 4
- Cannot use gspread's add_row() method before using set_with_dataframe() HOT 8
- Can't import, "No module named gspread.models" HOT 1
- Consider allowing option to drop duplicates in set_with_dataframe HOT 4
- Data gets interpreted even when using dtype=str HOT 1
- ModuleNotFoundError: No module named 'gspread.models' HOT 7
- Write header rows for DataFrames with MultiIndex columns so that Pandas reads them properly HOT 1
- Include formatting options in set_with_dataframe
- Consider offering a reader/writer object based on a given dataframe
- return API response from set_with_dataframe HOT 5
- No module named 'gspread' HOT 2
- Feature Request: Append feature to an existing sheet. HOT 1
- Max Cell Count
- Requests to sheets API cause 400 error is worksheet name is valid cell reference (was: Error using get_as_dataframe) HOT 1
- Newest version gspread not compatible HOT 7
- get_as_dataframe() is reading the whole sheet data HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gspread-dataframe.