Question about “Custom cost matrices” about moscot

[MUCDK]

Hi @L-traveling ,

thanks for reporting this. This is due to a change in moscot where we now distinguish between the "FGWProblem", and the "GWProblem".

As you are using the "GWProblem", the set_xy method (setting the linear term) doesn't make sense.
Could you try running this with the FGWProblem?

We will update the example.

[L-traveling]

@MUCDK Thanks for your reply. But when I type the following：
from moscot.problems.generic import FGWProblem
I get the errors:
ImportError: cannot import name 'FGWProblem' from 'moscot.problems.generic'
But the User API tells me I should. Hope to wait for your reply.

[ArinaDanilina]

Hi @L-traveling , are you on the latest version of moscot?
You can try pip install moscot -U

[L-traveling]

Thanks @ArinaDanilina ,I have Successfully installed moscot-0.3.4. But FGW does not seem to solve the problem that the result is the same before and after the change of the cost_matrix.

[MUCDK]

Hi @L-traveling ,

Can you run the following (i.e. adapt the notebook):

import warnings

warnings.simplefilter("ignore", FutureWarning)

from moscot import datasets
from moscot.problems.generic import FGWProblem

import numpy as np
import pandas as pd

import scanpy as sc

adata = datasets.simulate_data(n_distributions=3, key="batch")
sc.pp.pca(adata)

fgw = FGWProblem(adata)
fgw = fgw.prepare(key="batch", x_attr="X_pca", y_attr="X_pca")
fgw["0", "1"].solve()

print(fgw["0", "1"].solution)


rng = np.random.default_rng(seed=42)
obs_names_0 = fgw["0", "1"].adata_src.obs_names
obs_names_1 = fgw["0", "1"].adata_tgt.obs_names

cost_linear_01 = np.abs(rng.normal(size=(len(obs_names_0), len(obs_names_1))))
cost_quad_0 = np.abs(rng.normal(size=(len(obs_names_0), len(obs_names_0))))
np.fill_diagonal(cost_quad_0, 0)
cost_quad_1 = np.abs(rng.normal(size=(len(obs_names_1), len(obs_names_1))))
np.fill_diagonal(cost_quad_1, 0)

cm_linear = pd.DataFrame(data=cost_linear_01, index=obs_names_0, columns=obs_names_1)
cm_quad_0 = pd.DataFrame(data=cost_quad_0, index=obs_names_0, columns=obs_names_0)
cm_quad_1 = pd.DataFrame(data=cost_quad_1, index=obs_names_1, columns=obs_names_1)

fgw["0", "1"].set_xy(cm_linear, tag="cost_matrix")
fgw["0", "1"].set_x(cm_quad_0, tag="cost_matrix")
fgw["0", "1"].set_y(cm_quad_1, tag="cost_matrix")

fgw["0", "1"].solve()

print(fgw["0", "1"].solution)

The first print statement gives me OTTOutput[shape=(20, 20), cost=419.9979, converged=True], while the second gives OTTOutput[shape=(20, 20), cost=1.3961, converged=True]. Hence, the solution is not the same.

I also tried only setting xy above, and I get OTTOutput[shape=(20, 20), cost=372.2607, converged=True] , again a different result, indicating that set_xy does make a difference.

[L-traveling]

Thanks @MUCDK ,I tried the code above, and it worked just like you said.But after running my code I have two questions ：

import warnings
warnings.simplefilter("ignore", FutureWarning)

from moscot import datasets
from moscot.problems.generic import FGWProblem

import numpy as np
import pandas as pd
import scanpy as sc
import moscot.plotting as mtp

adata = datasets.simulate_data(n_distributions=3, key="batch")
sc.pp.pca(adata)
adata.obs['celltype'] = pd.Categorical(adata.obs['celltype'])

fgw = FGWProblem(adata)
fgw = fgw.prepare(key="batch", x_attr="X_pca", y_attr="X_pca")
fgw
a = fgw
fgw["0", "1"].solve()
print(fgw["0", "1"].solution)

new_key = "cell_transition_1"
ct_desc = fgw.cell_transition(
    "0", "1", "celltype", "celltype", forward=False, key_added=new_key
)
mtp.cell_transition(fgw, dpi=80, fontsize=14, key=new_key)

rng = np.random.default_rng(seed=42)
obs_names_0 = fgw["0", "1"].adata_src.obs_names
obs_names_1 = fgw["0", "1"].adata_tgt.obs_names
cost_linear_01 = np.abs(rng.normal(size=(len(obs_names_0), len(obs_names_1))))
cm_linear = pd.DataFrame(data=cost_linear_01, index=obs_names_0, columns=obs_names_1)
fgw["0", "1"].set_xy(cm_linear, tag="cost_matrix")
b = fgw

fgw["0", "1"].solve()
print(fgw["0", "1"].solution)

if a == b:
    print("no change！")
else:
    print("changed！")

new_key = "cell_transition_2"
ct_desc = fgw.cell_transition(
    "0", "1", "celltype", "celltype", forward=False, key_added=new_key
)
mtp.cell_transition(fgw, dpi=80, fontsize=14, key=new_key)

Q1: When I am plotting cell_transition_1 and cell_transition_2, the result is the same at first so that I think the cost_matrix has not changed. However, after I close my program and re-run it , it is correct. And changing the seed produce a different result, which is wonderful. I'll try it again later on the real data set. By the way, it needs to add the following line of code:
adata.obs['celltype'] = pd.Categorical(adata.obs['celltype'])
Q2:Although the result has changed, why is the fgw the same before and after the change. Because the above code will output:
no change！
Thank you very much for your patient guidance which have helped me a lot. Thanks.

[MUCDK]

Regarding Q1: Does this mean it's resolved?
Regarding Q2: This is because you compare the object, which doesn't change. If you compare the solutions, it should change.

[L-traveling]

Thanks @MUCDK ,
Regarding Q1: Yes, it's resolved. When I brought in datasets.hspc (), the results were still correct.
Regarding Q2: Yes, you are right.
I have another question. After I perform the cost_matrix change operation,
fgw["0", "1"].set_xy(cm_linear, tag="cost_matrix")
Is there any way I can output the cost_matrix of the object fgw？Because I don't seem to find this in the dir(fgw)

[MUCDK]

check out https://moscot.readthedocs.io/en/latest/notebooks/examples/problems/100_tagged_arrays.html.

Your cost matrix will be stored in data_src in your TaggedArray which you can access with fgw[0,1].xy

[L-traveling]

Thanks @MUCDK , I see.
Thank you very much for the guidance of your team, I am very interested in your work which is very creative and useful. With best wishes