pangzecheung / discrete-probability-flow Goto Github PK

Hi Pengze,

I hope you are doing well! Congrats on your recent paper "Formulating Discrete Probability Flow Through Optimal Transport" . I had a good read and found it impressive in theory! However, it is too much math for me to understand the details. Thus I have a few questions and hope to hear your answers.

I compared your code with tauLDR and found that the only change in the sampling code is shown below.

reverse_rates = forward_rates * F.relu(inner_sum - 1) # (N, D, S)

Correct me if I am wrong. I do observe other codes but Is this the only change in terms of the formulation?
Would you explain more about why you implement it this way? Why inner_sum - 1 and why using relu? Do you have an intuitive explanation for that?
I try to read your paper to find the answer, but it is a little hard for me to understand why from the equations.

I have read other flow-matching models that have a preprocessing step called pairing, basically pairing the data samples and prior samples. Thus during the training, they do not sample random noise from prior distribution, instead, they use the pre-sampled noises. Do you think it is possible to use this technique in discrete sequence flow matching?

Best,

Zhangzhi,
University of Missouri,
Columbia, MO, USA