stjordanis / broken_neural_scaling_laws

Code Release for "Broken Neural Scaling Laws" paper

Code Release for "Broken Neural Scaling Laws" (BNSL) paper (arxiv.org/abs/2210.14891)

To reproduce the Fitting and Extrapolation of BNSL on 4 Digit Addition from Figure 4 Left, run

python fit_bnsl_and_extrapolate__4_digit_addition__dataset_size_x-axis.py

To reproduce the Fitting and Extrapolation of BNSL on a noiseless simulation of the scaling behavior of 4 Digit Addition from Figure 4 Right, run

python fit_bnsl_and_extrapolate__4_digit_addition__dataset_size_x-axis__noiseless_simulation.py

To reproduce the Decomposition of BNSL into Power Law Segments from Figure 1, run

python make_figure_1__decomposition_of_bnsl_into_power_law_segments.py