Comments (6)
This PR is one step towards this issue: #126
as we are easing the input format assumption for the simple linear model driver .
from photon-ml.
@XianXing - is this still an issue we need to look at? Please let me know - we might work on it soon.
from photon-ml.
I think recent refactoring (client/core/lib) on Photon should be related to the idea proposed here, basically we'd like to make sure Photon is not restricted to Avro serialized/formatted data.
But you guys have more context than me now, so please feel free to close this issue once the client/core/lib refactoring is done (is it done already?).
from photon-ml.
from photon-ml.
The "master plan" is to provide a library of data readers for various format that would all feed GameEstimator.fit
with DataFrames
and ancillary data structures. This is already exemplified by [AvroDataReader](https://github.com/linkedin/photon-ml/blob/8702fea9a436d4979a3e672fe20ae30e32078f43/photon-client/src/main/scala/com/linkedin/photon/ml/data/AvroDataReader.scala)
.
I'm going improve on the recent client refactoring by isolating AvroDataReader
in an avro
package, and colocate in that package the model writer too. We can then have parallel modules for parquet
... and others.
from photon-ml.
While we don't have support for Parquet or LIBSVM data in the client code, sufficient decoupling has been done. For a while now, the input to the GameEstimator
has been a DataFrame
with features stored as Vector
objects.
For that reason, I am going to close this issue and separate ones can be opened for adding specific kinds of readers derived from DataReader
.
from photon-ml.
Related Issues (20)
- Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
- error in running job:"Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class" HOT 3
- Variances in FixedEffectsModel are Scaled Oddly HOT 5
- TRON Optimizer Does Not Support Coefficient Constraints
- Item-item Similarity HOT 1
- GaussianProcessSearch does not substract mean from the prior evaluations HOT 1
- Receiving ClassCastException in getGameDatumFromRow HOT 4
- Getting exception while fitting the Random Effect Model HOT 3
- Example of a Mixed effects model? HOT 4
- Can't run a GAME HOT 5
- 'model-metadata.json' not saved in GCS HOT 1
- GLMix in tensorflow HOT 1
- Value modelsRDD in class RandomEffectModel cannot be accessed HOT 2
- GameTrainingDriver will not run HOT 2
- Examples for hyperparameter tuning HOT 5
- style sheet missing
- featureShardConfigurations when scoring a mixed effects model with random intercept only HOT 1
- The best model's RMSE score doesn't match .. HOT 1
- Using/Refactoring for Standalone Model Runs in Spark-Scala HOT 2
- Cannot build photon-ml with Java 11 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from photon-ml.