jiwoncpark / deep-learning-notebooks Goto Github PK

Hello!

On your blog post you write
"For AdamOptimizer, it shows two methods are effectively identical. This is what we would expect naturally unless there is a dynamic state of an optimizer, which changed during minibatch loop. It turns out this is the case for the three other optimizers we tested here. So we do expect red and blue not to completely overlap. "

I cannot seem to find any information online explaining what you write here. Could you detail? How does the minibatch size does not affect an adam optimizer?

Best,
Franco

Kaggle Knowhow 2-part articles (https://github.com/zzsza/Kaggle-knowhow/blob/master/01.Kaggle-Intro.md and https://github.com/zzsza/Kaggle-knowhow/blob/master/02.Kaggle-Flow.md).

• Usually submit two entries, record hyperparameters, etc. in the description field of submission, and .zip files are good for download and upload speed.
• Look at the evaluation metric (e.g. for accuracy, MAP/MSE/log error).
• Do an exploratory data analysis (EDA). Reference kernels but also your own, e.g. groupby each column and do a feature count, plot distributions, find missing data and think about substitution methods.
• Data preprocessing for missing data, outliers, and nominal variables. Adjust each data size or data augmentation.
• Implement a single model, e.g. random forest, xgboost, LGBM, Catboost
• Feature engineering is important!!! Apply many transformations and see if model improves. Iterate many times.
• Try many architectures and also try stacking predictions from various models, e.g. 0.3model_A_pred + 0.7model_B_pred. See StackNet.

Website

Youtube video on the world model by Schmidhuber and Ha. The takeaway point is that, when trained inside a "dream" environment, i.e. an environment modeled by the MDN-RNN , the agent could learn a policy that had a higher score than when trained on "real" scenarios. The tau "temperature" parameter determines the degree of uncertainty. It seems the uncertainty helped the agent learn well. Also, training inside the simulated latent-space dream world is efficient! The world models were trained incrementally to simulate reality that is useful for transferring policies back to the real world.
Will this be useful for simulating PLAsTiCC data?

Kaggle kernel
Paper

Syllabus, slides, and video links here

Repo
Kaggle article

Repo

Youtube link