Coder Social home page Coder Social logo

erow / ffcv Goto Github PK

View Code? Open in Web Editor NEW

This project forked from libffcv/ffcv

0.0 0.0 0.0 3.38 MB

FFCV: Fast Forward Computer Vision (and other ML workloads!)

Home Page: https://ffcv.io

License: Apache License 2.0

Shell 0.06% C++ 3.41% Python 96.21% Dockerfile 0.33%

ffcv's Introduction

Fast Forward Computer Vision for Pretraining

[install] [new features] [docs] [paper]

This library is derived from FFCV to optimize the memory usage and accelerate data loading.

Installation

Running Environment

conda create -y -n ffcv "python>=3.9" cupy pkg-config "libjpeg-turbo>=3.0.0" opencv numba -c conda-forge
conda activate ffcv
conda install pytorch-cuda=11.3 torchvision  -c pytorch -c nvidia
pip install .

Prepackaged Computer Vision Benchmarks

From gridding to benchmarking to fast research iteration, there are many reasons to want faster model training. Below we present premade codebases for training on ImageNet and CIFAR, including both (a) extensible codebases and (b) numerous premade training configurations.

Make Dataset

We provide a script to make the dataset examples/write_dataset.py, which provides three mode:

  • jpg: The script will compress all the images to jpg format.
  • png: The script will compress all the images to png format. This format is too slow.
  • raw: The script will not compress the images.
  • smart: The script will compress the images larger than the threshold.
  • proportion: The script will compress a random subset of the data with size specified by the compress_probability argument.
python examples/write_dataset.py --cfg.write_mode=smart --cfg.threshold=206432  --cfg.jpeg_quality=90  \
    --cfg.num_workers=40 --cfg.max_resolution=500 \
    --cfg.data_dir=$IMAGENET_DIR/train \
    --cfg.write_path=$write_path 

ImageNet

We provide a self-contained script for training ImageNet fast. Above we plot the training time versus accuracy frontier, and the dataloading speeds, for 1-GPU ResNet-18 and 8-GPU ResNet-50 alongside a few baselines.

TODO:

Link to Config top_1 top_5 # Epochs Time (mins) Architecture Setup
Link 0.784 0.941 88 77.2 ResNet-50 8 x A100
Link 0.780 0.937 56 49.4 ResNet-50 8 x A100
Link 0.772 0.932 40 35.6 ResNet-50 8 x A100
Link 0.766 0.927 32 28.7 ResNet-50 8 x A100
Link 0.756 0.921 24 21.7 ResNet-50 8 x A100
Link 0.738 0.908 16 14.9 ResNet-50 8 x A100
Link 0.724 0.903 88 187.3 ResNet-18 1 x A100
Link 0.713 0.899 56 119.4 ResNet-18 1 x A100
Link 0.706 0.894 40 85.5 ResNet-18 1 x A100
Link 0.700 0.889 32 68.9 ResNet-18 1 x A100
Link 0.688 0.881 24 51.6 ResNet-18 1 x A100
Link 0.669 0.868 16 35.0 ResNet-18 1 x A100

Train your own ImageNet models! You can use our training script and premade configurations to train any model seen on the above graphs.

CIFAR-10

We also include premade code for efficient training on CIFAR-10 in the examples/ directory, obtaining 93% top1 accuracy in 36 seconds on a single A100 GPU (without optimizations such as MixUp, Ghost BatchNorm, etc. which have the potential to raise the accuracy even further). You can find the training script here.

Features

Compared to the original FFCV, this library has the following new features:

  • crop decode: RandomCrop and CenterCrop are now implemented to decode the crop region, which can save memory and accelerate decoding.

  • cache strategy: There is a potential issue that the OS cache will be swapped out. We use FFCV_DEFAULT_CACHE_PROCESS to control the cache process. The choices for the cache process are:

    • 0: os cache
    • 1: process cache
    • 2: Shared Memory
  • lossless compression: PNG is supported for lossless compression. We use RGBImageField(mode='png') to enable the lossless compression.

  • few memory: We optimize the memory usage and accelerate data loading.

ffcv's People

Contributors

afspies avatar andrewilyas avatar anooptp avatar carmocca avatar dskhudia avatar elicassion avatar erow avatar fcossio avatar giangnguyen2412 avatar guillaumeleclerc avatar hadisalman avatar jasonlee1995 avatar kellerjordan avatar kristian-georgiev avatar kwentar avatar kynk94 avatar lengstrom avatar mvpatel2000 avatar nicolashug avatar numpee avatar pmeier avatar seva100 avatar sung-max avatar vishu26 avatar vtjeng avatar warner-benjamin avatar yvsriram avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.