ikintosh / gabornet

Hi there! I'm Andrey 👋

• 🔭 I’m currently working on Advertisment platform at bol.com
• 🌱 And trying to find time for open source contributions
• 👯 I’m looking to collaborate on developing ETNA package
• 💬 Ask me about time series and open source development

You may find my publication and conference talks:

• PyData 2022
• Medium article about time series demand forecasting with ETNA
• Medium article about fundamentals of Neural Networks part 1, part 2 (to be released)
• PyCon 2021
• TimeSeries MeetUp 2022

📫 How to reach me:

gabornet's Issues

Make weights fixed in eval mode

Is there any way to make GaborLayer not to calculate weights every time in evaluate or no_grads mode?

I guess I should change something in forward method, but I have no idea what to do.

GaborNet/GaborLayers.py

Lines 71 to 93 in 435e031

	def forward(self, input):
	y, x = torch.meshgrid([torch.linspace(-self.x0+1, self.x0, self.kernel_size[0]), torch.linspace(-self.y0+1, self.y0, self.kernel_size[1])])
	x = x.to(device)
	y = y.to(device)
	weight = torch.empty(self.weight.shape, requires_grad=False).to(device)
	for i in range(self.out_channels):
	for j in range(self.in_channels):
	sigma = self.sigma[i, j].expand_as(y)
	freq = self.freq[i, j].expand_as(y)
	theta = self.theta[i, j].expand_as(y)
	psi = self.psi[i, j].expand_as(y)
	rotx = x * torch.cos(theta) + y * torch.sin(theta)
	roty = -x * torch.sin(theta) + y * torch.cos(theta)
	g = torch.zeros(y.shape)
	g = torch.exp(-0.5 * ((rotx ** 2 + roty ** 2) / (sigma + 1e-3) ** 2))
	g = g * torch.cos(freq * rotx + psi)
	g = g / (2*3.14*sigma**2)
	weight[i, j] = g
	self.weight.data[i, j] = g
	return F.conv2d(input, weight, self.bias, self.stride, self.padding, self.dilation, self.groups)

Deal with sanity_check code

sanity_check requires heavy computation (probably with GPU) so it cannot be part of test suite. However it is nice to have sanity_check case to be able to test layer performance.

Add simple test case to test perfomance

When refactor, the performance (in this case, ability to learn proper image representation) should be tested. So it's necessary to find simple dataset and add test case to train simple CNN on it and monitor accuracy metric.

Vectorize kernel creation

Remove loops

GaborNet/GaborNet/GaborLayer.py

Lines 103 to 119 in a104371

	def calculate_weights(self):
	for i in range(self.conv_layer.out_channels):
	for j in range(self.conv_layer.in_channels):
	sigma = self.sigma[i, j].expand_as(self.y)
	freq = self.freq[i, j].expand_as(self.y)
	theta = self.theta[i, j].expand_as(self.y)
	psi = self.psi[i, j].expand_as(self.y)
	rotx = self.x * torch.cos(theta) + self.y * torch.sin(theta)
	roty = -self.x * torch.sin(theta) + self.y * torch.cos(theta)
	g = torch.exp(
	-0.5 * ((rotx ** 2 + roty ** 2) / (sigma + self.delta) ** 2)
	)
	g = g * torch.cos(freq * rotx + psi)
	g = g / (2 * math.pi * sigma ** 2)
	self.conv_layer.weight.data[i, j] = g

Remove private class inheritance

This should be removed from implementation:

from torch.nn.modules.conv import _ConvNd
from torch.nn.modules.utils import _pair

class GaborConv2d(_ConvNd):
...

Difference between layers

Hello,
Could you please tell me what is the difference between "GaborSmallConv2d", "GaborConv2d" and "GaborCurvedConv2d"? from the math point of view.

A question about making weights fixed in the eval mode

Hello, thanks for your great work!
Can you explain to me why the following code can make weights fixed in the eval mode? What is the function of the parameter self.is_calculated？ Thanks very much!

Remove codecov label from readme.md

Minor modifications

Hello,
Thank you @iKintosh for providing your code.
I'm currently trying to use your layer in my code (with torch 1.1.0) and I ran into some minor problems

I think the following lines should be updated in your code but I'm not sure so I decided to just point them out here :

1- Line 19 -> padding_mode should be added

super(GaborSmallConv2d, self).__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, False, _pair(0), groups, bias, padding_mode='zeros')

2- Line 24 -> .type(torch.Tensor) should be added to make learnable parameter floating

self.theta = nn.Parameter(0.628*torch.randint(0, 6, (out_channels, in_channels)).type(torch.Tensor))

3- Line 62 -> same as #1

super(GaborConv2d, self).__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, False, _pair(0), groups, bias, padding_mode='zeros')

4- Line 72 -> second parameter for meshgrid should be self.x0-1 not self.x0 (??)

y, x = torch.meshgrid([torch.linspace(-self.x0+1, self.x0, self.kernel_size[0]), torch.linspace(-self.y0+1, self.y0, self.kernel_size[1])])

5- Line 105 -> same as #1

super(GaborCurvedConv2d, self).__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, False, _pair(0), groups, bias, padding_mode='zeros')

Add test cases to check that in train mode params change

GaborNet/GaborNet/GaborLayer.py

Lines 52 to 60 in a104371

	self.theta = Parameter(
	(math.pi / 8)
	* torch.randint(0, 8, (out_channels, in_channels)).type(torch.Tensor),
	requires_grad=True,
	)
	self.sigma = Parameter(math.pi / self.freq, requires_grad=True)
	self.psi = Parameter(
	math.pi * torch.rand(out_channels, in_channels), requires_grad=True
	)

Any available pretrained weights?

Hi guys,

I was wondering if you have any available pretrained weights for the GaborLayer?

Add poetry as dependency manager

Questions about calculate_weights()

Hi Andrey, I have a couple of questions about calculate_weights().

1. in line 106-109, why do you need expand_as(self.y)?
2. in line 118, why do you need to divide g by (2 * math.pi * sigma ** 2)? I cannot find this in the original expression of Gabor filters.
   Thank you!

Add tests cases to check that in eval mode params stays the same

GaborNet/GaborNet/GaborLayer.py

Lines 52 to 60 in a104371

	self.theta = Parameter(
	(math.pi / 8)
	* torch.randint(0, 8, (out_channels, in_channels)).type(torch.Tensor),
	requires_grad=True,
	)
	self.sigma = Parameter(math.pi / self.freq, requires_grad=True)
	self.psi = Parameter(
	math.pi * torch.rand(out_channels, in_channels), requires_grad=True
	)

Remove numbers

Should have proper names:

• 3.14 / 2
• 1.41
• 8
• 0.5
• 1e-3
• 3.14