bzantium / ea-lstm

EA-LSTM: Evolutionary attention-based LSTM for time series prediction

[2023.08 - Current] LLM Researcher @Kakaobrain

• Developing Language Foundation Model a.k.a. KoGPT2

[2020.03 - 2023.08] Machine Learning Engineer @SK Telecom & EleutherAI

• Developing Large Language Model for SK Telecom
• Developed Multimodal AI Service at SKT A.
  • AI Eraser (Object Removal with Image Inpainting)
    • role: project manager & implementation of pipeline algorithm (segmentation postprocessing & enhancement in inpainting performance).
• Developing polyglot and oslo project at EleutherAI
  • polyglot: Large Language Models of Well-balanced Competence in Multi-languages
    • role: distributed training of LM with Megatron LM & data crawling, preprocessing and model evaluaton. Published 1.3B, 3.8B, 5.8B, 12.8B polyglot-ko models.
  • oslo: Open Source for Large-scale Optimization
    • role: tensor parallel 1D, 2D, 3D implementation.

Interest

• Foundation Model / NLP / Multimodal AI

ea-lstm's Issues

Some of the problems

Is this code reproduced according to the original paper?There are a few points in the code that I did not understand. Can you give me your thoughts?