How to constraint the velocity model in optimization. about deepwave

[ar4]

Hi @bbsun

The ability to easily add constraints such as this is one of the advantages of using PyTorch. You can just implement a constraint on the model that you are optimizing, and PyTorch will automatically take this into account when updating the model.

One easy way to do it is to choose a velocity in the middle of the desired range, and then penalize differences from this. For example, I think this should work (but is untested) to penalize differences from 3000m/s, with a weight to balance how important this is compared to matching the data:

model_loss = (model - 3000).norm()
data_loss = criterion(data_predicted, data_true)
total_loss = data_loss + weight * model_loss
total_loss.backward()

If you want to only penalize velocities above and below specific thresholds, but not anything in between, one way to do this is using PyTorch's MarginRankingLoss, or you can construct your own loss function (as long as PyTorch is able to automatically differentiate it).

[bbsun]

Thanks for the in-detail reply. I have another issue, if the source wavelet and the velocity model are the output of another function, i.e., the model and source wavelet is not the leaf node in the graph. In this case, how to use your code for updating the new parameters.

[bbsun]

The deepwave manually compute the gradient of the loss withrespect to the model in order to save time. It do not use the autograd function of pytorch. Do I understand it correctly. Last comment is actually asking the question of how to use your code within the Graph when some part of the Graph still need autograd. For example, the model is created from other function.

[ar4]

Regarding your first question:

It sounds like your actual leaf variables are not your model and source wavelet, but the inputs to the functions that generate them. That should not be a problem, as PyTorch will backpropagate through the functions to update the actual leaf variables. In that case you should not set "requires_grad" yourself on the model or source wavelet (they will inherit it themselves when they are generated from a leaf variable that has "requires_grad" set), only on the variables that you actually want to optimize (presumably some input to the functions that generate your source wavelet and model). One error you might encounter is a complaint that you are trying to backpropagate through the graph twice. This would occur if you generate the source wavelet and model outside your training loop. To avoid that, you just need to make sure you run the entire computational graph from the leaf variables to the loss function (i.e., including the call to the function that generates your source wavelet and model) every step. This is all assuming that what you want to optimize is really the inputs to the functions that generate your source wavelet and model. If that is not the case - you instead just want to run regular FWI - then you need to detach the graph so that PyTorch knows that you don't want to backpropagate further back along the graph than the model and source wavelet. For that, just use .detach() on the inputs to the functions that generate your source wavelet and model that have "requires grad" set.

Regarding your second question:

You are correct that Deepwave manually computes the gradient of the receiver data with respect to the model for improved performance. It is, however, just like any other PyTorch module, and so there is no problem at all with including the Deepwave wave propagation in a chain of other operations that do require PyTorch's automatic differentiation. In the example notebook, for example, I apply a normalization operation to the output of Deepwave's wave propagation. PyTorch automatically differentiates this and backpropagates through the chain of operations correctly. Putting operations before Deepwave in the chain - functions that generate the model and source wavelet - is exactly the same and should not cause any problem.