queryverse / statfiles.jl Goto Github PK
View Code? Open in Web Editor NEWFileIO.jl integration for Stata, SPSS, and SAS files
License: Other
FileIO.jl integration for Stata, SPSS, and SAS files
License: Other
I am trying to read stata panel datasets of Russian households and invividuals.
They can be downloaded freely after simple registration.
The houeseholds dataset is ~1gb and individuals is ~4gb. Machine has 16 gb.
Even 1gb file take too long to load (have waited > 15 minutes without any results).
using StatFiles
a = load("USER_RLMS-HSE_HH_1994_2017nt_eng.dta")
Package is usefull , Thanks !
When
data= load("data.dta")
how to make clear data (Julia Array e.g {Any} ) from data? I have big file and Data Frames and other are slowly :/
I hope is posible to fast and easy to do!
Paul
Presumably we only need to register the file type in FileIO, and for that we need to find out whether there is some magic byte sequence for that file type.
Rough first steps:
Is there a way to automatically change the type of all float variables to Float64
?
Due to a bug, one cannot fit linear models with Float32
in GLM.jl
?
using ReadStat
file = download("http://www.stata-press.com/data/r15/fullauto.dta",
"data/ologit.dta")
data = read_dta(file)
using StatFiles, DataFrames
output = load(file) |> DataFrame
If you take a look at data
you will see that categorical variables have a mapping to labels given by val_labels_keys
and val_label_dict
. Without taking into account that nuance, the default behavior specified here yields the values instead of the labels (e.g., rep77 gives [1, 2, 3, 4, 5]
instead of ["Poor", "Fair", "Average", "Good", "Excellent"])
. It might be the case for other file formats, but this is confirmed for Stata's dta.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.