jarvis73 / moving-least-squares

Numpy & PyTorch implementation of three algorithms of image deformation using moving least squares. http://dl.acm.org/citation.cfm?doid=1179352.1141920

Hi, this is Jarvis73 👋

• 🔭 I'm working on image segmentation in computer vision (medical image segmentation, few-shot segmentation, domain adaptation segmentation).
• 😄 My blog: Jarvis73
• 📫 Contact me: [email protected]
• ⚡ A simple but useful navigation webpage: 路灯

moving-least-squares's Issues

Deformation with Line Segments

thanks for the very helpful implementation , I was wondering if there were an implementation available for the third chapter in the original paper : "Deformation with Line Segments" which the authors applied on an image of the Leaning Tower of Pisa.

有关逆变换

做基于最小二乘变形时，逆变换公式为什么不通过将dst点作为原始输入点，src点作为变形过后的点，求取变形公式？

could you please tell me where can i download the datasets?

Thank you for your sharing.could you please tell me where can i get the file of train-volume.tif and train-labels.tif and test-volume.tif.
thank you !

Overflow Error with mls_rigid_deformation

Hi there,

There is an overflow error, using np.int16 is not enough in this function, which was leading to errors when running on mac or black pixels at the centre of anchor points when using it on Windows.

When I updated it to the following:
q = np.ascontiguousarray(q[:, [1, 0]].astype(np.int32))
p = np.ascontiguousarray(p[:, [1, 0]].astype(np.int32))

I saw that the issue was solved.

Deal with this information however you want!

Best,
Rob

it takes a huge of memory

When I ran the code with a 2000x2000 px image and 64 moving points, it took about 20Gib of ram, which is quite a lot to implement on server.

General coordinate transformation

Would it be possible to factor the coordinate transformation out of the current functions? I'm pretty sure it would just be moving some code out of the current function into an accessory. We use MLS for non-image transformations so coordinates would be helpful!

could

When pTwp is not invertible in affine deformation

Hi, I am a little confused about the code dealing with the situation when pTwp is not invertible in affine deformation (line94 in img_utils.py). If I’m not mistaken, code directly set M to be an identity matrix (according to line 129, 130) when the determinant is close to zero. I supposed this kind of issue should be done by pseudo-inverse. This discussion in stackoverflow should help:
https://stackoverflow.com/questions/49357417/why-is-numpy-linalg-pinv-preferred-over-numpy-linalg-inv-for-creating-invers

Thanks for the implementation.

关于索引图的扭曲变换

@Jarvis73
您好，您的工作很棒。
我有个问题想要请教您，当使用index = pyvips.Image.xyz(width, height)新建一个索引图后，在函数体中设定 height的值为558，width的值为800，再将这张图放到mls_rigid_deformation_inv(image, p, q, alpha=1.0, density=1.0)中，程序会出现如下错误：
Exception in Tkinter callback
Traceback (most recent call last):
File "/home/fl/Python-3.6.4/Lib/tkinter/init.py", line 1699, in call
return self.func(*args)
File "/home/fl/MLS/images/makeindex.py", line 208, in deformPicture
transformed_image = run(mls_rigid_deformation_inv)
File "/home/fl/MLS/images/makeindex.py", line 201, in run
transformed_index = fun_inv(index, p, q, alpha=1, density=1)
File "/home/fl/MLS/images/makeindex.py", line 183, in mls_rigid_deformation_inv
transformed_image = image[tuple(transformers.astype(np.int16))] # [grow, gcol]
File "/home/fl/.local/lib/python3.6/site-packages/pyvips/vimage.py", line 981, in getitem
raise TypeError
TypeError
而这种错误在对图片扭曲的时候是不会发生的，看报错类型应该是格式的问题，而且报错的地方貌似出现在 transformed_image = image[tuple(transformers.astype(np.int16))] # [grow, gcol]这句话上。请问如何优化解决？麻烦您指点一下，不胜感激，谢谢。

>2D case

MLS works in higher dimensionality - how difficult would it be for this implementation to be extended into 3D or even higher?

Different results between Numpy and PyTorch versions

Hi, I have tried to compare the results of the affine deformation of demo() and demo_pytorch() in demo.py using the Monalisa image with the control points given in demo2() and the results are not exactly equal as I expected. For some indices, the pixel values of the two images are slightly different (even for about 40 out of 256) and I can't understand why. Do you have any idea why this happens?

Faster running time

Hi, is there a way to obtain faster running times? Like optimizing some part of the code for example

论文写错的

这个地方的=号应该是+号吧

大佬，有没有局部旋转的代码或论文呀？

如题

Taking too much time to deform

Why it is talking too much time to deform image even I tried pytorch Cuda?

what's the mean of [inverse]?

how to get rid of white dots?

2021-08-10_17-23-05.mp4

Using mesh for the deformation

I was able to run this code on an image. Can I also using it in a 3D/2D mesh?
Meaning, If I only want to deform 1 object (that has mesh) from the image

Thanks!
Efrat

The process of generating a mapping function(matrix) and applying it to image should be separated.

The process of generating a mapping function(matrix) and applying it to image should be separated.

Sometimes we need to deform two or more images with the same mapping function at the same time. For example, in image augmentation which is common in semantic segmentation, one needs to deform both the original image and its pixel-wise labeled image. Under this circumstance, calling the deformation function twice with same p and q is not the best solution.
If the deformation function I choose here is a inverse function, which has a O(n^3) algorithm complexity as it contains a matrix inverse operation, I would have to at least double my time. But once we've separated the generation process and the final mapping process, only one O(n^3) operation would be needed plus two mapping operation which has O(n^2) complexity.

the deformed result is different from the paper?