jaduplessis / article-fine-tuning

fine-tune-article

Data PreProcessing for Fine-Tuning.

Project requirements

Pyenv and Python 3.11.7

• Install pyenv to manage your Python versions and virtual environments:

  curl -sSL https://pyenv.run | bash

  • If you are on MacOS and experiencing errors on python install with pyenv, follow this comment
  • Add these lines to your ~/.bashrc or ~/.zshrc to be able to activate pyenv virtualenv:

    eval "$(pyenv init -)"
    eval "$(pyenv virtualenv-init -)"
    eval "$(pyenv init --path)"

  • Restart your shell

• Install the right version of Python with pyenv:

  pyenv install 3.11.7

Poetry

• Install Poetry to manage your dependencies and tooling configs:

  curl -sSL https://install.python-poetry.org | python - --version 1.5.1

  If you have not previously installed any Python version, you may need to set your global Python version before installing Poetry:

  pyenv global 3.11.7

Installation

Create a virtual environment

Create your virtual environment and link it to your project folder:

pyenv virtualenv 3.11.7 fine-tune-article
pyenv local fine-tune-article

Now, every time you are in your project directory your virtualenv will be activated thanks to pyenv!

Install Python dependencies through poetry

poetry install --no-root