Error propagation from weights about hep_ml

[acampove]

@acampove do you maybe have sources for

(from studies done by other people)

I would, AFAIU, second the argument of @arogozhnikov that, in general, you can do that

Hi Jonas,

This is what I remember Christoph Langenbruch said once in a meeting the measurement of the Bs-> phi mumu branching ratio. I would check their note or just talk to him, maybe you know him better than I do :)

Cheers.

[arogozhnikov]

Estimating correlation is a dead thing for that many numbers.
Better just model that correlation with training multiple models (e.g. by different cross-val splits), and estimate error of downstream processing by having multiple replicas

val1 = f(x_i, w_1i)
val2 = f(x_i, w_2i)
val2 = f(x_i, w_3i)
etc.

[acampove]

That approach would mean doing probably 200 trainings for the MVA. The real data usually is background subtracted using the sPlot technique. Apparently (from studies done by other people) we cannot bootstrap the sweighted sample, we have to:

1. Bootstrap the unweighted data + simulation
2. Obtain the sweights doing the fit.
3. Train the MVA on the bootstrapped data and simulation

Many times, which seems very challenging computationally. In case of 2D or even 3D reweighting, I think this just means that we should not use hep_ml, given that obtaining uncertainties is highly non trivial and a number without uncertainties is pretty useless. Hep_ml would only be an alternative once you start thinking of reweighting in higher dimensions.

[acampove]

By the way, the bootstrapping argument also applies to k-Folding.

[arogozhnikov]

You surely can subsample sweighted samples. Opposite would mean your fitting is unstable (and hence probably wrong).

[jonas-eschle]

@acampove do you maybe have sources for

(from studies done by other people)

I would, AFAIU, second the argument of @arogozhnikov that, in general, you can do that