bigscience-workshop Goto Github PK

amazon-sagemaker-mlflow-fargate

Managing your machine learning lifecycle with MLflow and Amazon SageMaker

architecture-objective

bibliography

A list of BigScience publications

bigscience

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

bigscience-workshop.github.io

Alternative to https://github.com/Dynalon/mdwiki-seed

biomedical

Tools for curating biomedical training data for large-scale language modeling

bloom-dechonk

A repo for running model shrinking experiments

carbon-footprint

A repository for `codecarbon` logs.

catalogue_data

Scripts to prepare catalogue data

codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.

data-preparation

Code used for sourcing and cleaning the BigScience ROOTS corpus

data_sourcing

This directory gathers the tools developed by the Data Sourcing Working Group

data_tooling

Tools for managing datasets for governance and training.

datasets_stats

Generate statistics over datasets used in the context of BS

evaluation

Code and Data for Evaluation WG

evaluation-robustness-consistency

Tools for evaluating model robustness and consistency

historical_texts

BigScience working group on language models for historical texts

interpretability-ideas

lam

Libraries, Archives and Museums (LAM)

lm-evaluation-harness

A framework for few-shot evaluation of autoregressive language models.

massive-probing-framework

Framework for BLOOM probing

megatron-deepspeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

metadata

Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.

model_card

multilingual-modeling

BLOOM+1: Adapting BLOOM model to support a new unseen language

multilingual-modeling-1

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

pii_processing

PII Processing code to detect and remediate PII in BigScience datasets. Reference implementation for the PII Hackathon

promptsource

Toolkit for creating, sharing and using natural language prompts.

scaling-laws-tokenization

scaling-laws-tokenization