Comments (3)
import math
def cosine_annealing_lr(lr, step_count, T_max, eta_min=0.0):
lr = eta_min + (lr - eta_min) * (1 + math.cos(math.pi * step_count / T_max)) / (
1 + math.cos(math.pi * (step_count - 1) / T_max)
)
return lr
lr = 0.1
step_count = 10.0
t_max = 100.0
eta_min = 0.0
assert math.isclose(
0.0995287864543317, cosine_annealing_lr(lr, step_count, t_max, eta_min)
)
use std::f64::consts::PI;
fn cosine_annealing_lr(lr: f64, step_count: f64, t_max: f64, eta_min: f64) -> f64 {
let cosine_arg = PI * step_count / t_max;
let lr = eta_min + (lr - eta_min) * (1.0 + f64::cos(cosine_arg)) / (1.0 + f64::cos(PI * (step_count - 1.0) / t_max));
lr
}
fn main() {
let lr = 0.1;
let step_count = 10.0;
let t_max = 100.0;
let eta_min = 0.0;
let new_lr = cosine_annealing_lr(lr, step_count, t_max, eta_min);
assert_eq!(0.09952878645433175, new_lr);
}
ref https://chat.openai.com/share/1adcedee-7c58-4583-88dc-f504b80fce6b
from fsrs-rs.
The next step is to implement the interface (or trait?):
from fsrs-rs.
I am not so sure about the correctness of implement of this.
use burn::{lr_scheduler::LRScheduler, LearningRate};
#[derive(Clone, Debug)]
pub struct CosineAnnealingLR {
t_max: f64,
eta_min: f64,
init_lr: LearningRate,
step_count: f64,
}
impl LRScheduler for CosineAnnealingLR {
type Record = usize;
fn step(&mut self) -> LearningRate {
self.step_count += 1.0;
use std::f64::consts::PI;
fn cosine_annealing_lr(lr: f64, step_count: f64, t_max: f64, eta_min: f64) -> f64 {
let cosine_arg = PI * step_count / t_max;
let lr = eta_min
+ (lr - eta_min) * (1.0 + f64::cos(cosine_arg))
/ (1.0 + f64::cos(PI * (step_count - 1.0) / t_max));
lr
}
self.init_lr = cosine_annealing_lr(self.init_lr, self.step_count, self.t_max, self.eta_min);
self.init_lr
}
fn to_record(&self) -> Self::Record {
self.step_count as usize
}
fn load_record(mut self, record: Self::Record) -> Self {
self.step_count = record as f64;
self
}
}
from fsrs-rs.
Related Issues (20)
- [Enhancement] Use more splits while training with larger datasets HOT 3
- Request: Ignore reviews before "Forget" HOT 9
- Enhancement: Include incomplete revlogs even when training HOT 4
- Consider time-frame limitation? HOT 3
- TODO: speed up finding optimal retention via Brent's method
- Better outlier filter for trainset HOT 25
- Skip reviews with time = 0 when calculating average answer times HOT 1
- What's the difference between this repo and rs-fsrs? HOT 1
- User guide HOT 3
- Add an option to turn off outlier filter when benchmark HOT 1
- Inference.rs uses the new power curve, but the default parameters are from v4 HOT 17
- Add a example file HOT 4
- Reference usage? HOT 5
- Pre-training Only when the number of reviews is less than 1000 HOT 5
- [BUG] Potential inconsistency in optimal_retention.rs HOT 20
- [Question] How to choose "Days to simulate"? HOT 14
- [Feature Request[ Use two different sets of initial parameters, then average out the results HOT 4
- Use the first revlog in the "known" review history for converting SM-2 ivl & ease to memory states HOT 13
- Achieve parity with the Python optimizer HOT 10
- support WASM HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fsrs-rs.