mfbalin / concrete-autoencoders Goto Github PK

Not sure if this is the right place to put it, but I think you have a small error when calculating mean_max

Line 107 reads:
if K.get_value(K.mean(K.max(K.softmax(self.concrete_select.logits, axis = -1)))) >= stopper_callback.mean_max_target:
Line 52 reads:
monitor_value = K.get_value(K.mean(K.max(K.softmax(self.model.get_layer('concrete_select').logits), axis = -1)))

The difference is that in Line 107 you have axis=-1 in the K.softmax function, while in Line 52 you have it in the K.max function. From what I can tell, K.max, without axis specified takes the maximum of the entire matrix, instead of the row-wise maximum. For this reason, I have been seeing the program finish and exit when the monitoring value is still only ~0.15

Hi, thanks for your interesting paper.

From my understanding, the concrete autoencoder is an autoencoder, with the latent space constrained by concrete distribution, and now work as a feature selector.

I want to know if it's possible to reconstruct the data, in case I only have the extracted features. For example, I got 2 populations A and B:

• Collect n1 features on population A -> yield dataset A -> train concrete autoencoder on it -> get n2 features of A
• Collect n2 features on population B -> yield dataset B -> reconstruct n1 feature by using only the decoder -> get n2 feature of population B

Would you like to suggest/advise me how to to this? Thank you.

I applied on my dataset and that's shape changed to (K, N), that k is input to your model and n is my X_train data features number. I supposed transform method is like dimensions reduction transform methods.

Hi,

I am looking through your code and I see that for the supersived implementation you use the MSE loss instead of the cross-entropy loss like you mentioned in the paper... There is no condition to use the cross-entropy loss when supervised and it always use the mse loss. Is it a mistake or is it done on purpose?

Hi,

Could you consider adding a permissive license such as MIT to the repository?
Github makes that really easy: https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-license-to-a-repository

Thanks in advance.

Hello! I want to use this auto encoder for unsupervised feature selection, but alaways fails.

If ues fit(X), then reporting error'fit() missing 1 required positional argument: 'Y'';

if use fir(X, Y=None) , reporting 'have no len()'

if comment out the code 'assert len(X) == len(Y)' , reporting error while epochs begin.

So how do we start unsupervised training?

Hello,
thank you for your significant contribution.
I was wondering regarding the epochs of the model: why if I select let's say 1 epoch in the constructor, in the end in the verbose I see 1, 2, 4, 8 and then 16 epochs trained?

Thank you for your great feature selection method. However, I found that your method might lead to such a situation that the same features could be selected for multiple times. For example, after training, the method get_support() might output a result like [3, 4, 6, 2, 2, 5, 1, 0]. In this case, the second feature has been selected twice. Is it possible to avoid such cases using concrete autoencoders? Thank you!

Hello
Can anybody help me to find a way to apply hyperparameters tuning for this algorithm?

Hello,
Really thankful for this amazing work. I want to know that is it possible to implement it on Pytorch?
Thanks again.

Runing the second block, there is a small bug.
cannot import name 'Adam' from 'keras.optimizers' (/usr/local/lib/python3.7/dist-packages/keras/optimizers.py)
Maybe you can fix it.😀

Can you put the download link of geo data?

Hello,
after each epoch the code prints: step - loss: 0.0043 - val_loss: 0.0043. What does this step and value loss mean for each epoch??
Also at the starting of each epoch it shows : mean max of probabilities: 6.54364e-05 - temperature 0.1. what is mean max of probabilities here??
Actually I am selecting 50 features using different no of epochs and trying to find reconstruction error at each epoch. Which one is the reconstruction error ???
Thank you.

Hi, Thank you for providing the paper and the code,

I wanted to ask if you recommend using this approach for a big amount of data, let's say number of features around 2000. I tried the code with such a number of features, setting k=600 as selected features in input, however the code was taking too much time, and it was doubling the epoch time every time since it is not converging apparently. I wanted to ask approximately does this approach usually takes time ? also I wanted to ask about the convergence, what I understood from the code is that the selection will be done once we reach a 'mean_max_target=0.998' otherwise it will keep doubling the epoch time till it reaches that value, right? However I set the tryout limit to only 4 and epoch number to 50, the training stopped therefore after 400 epochs, however the selected features were redundant, i.e I got an array of indexes with for example [ 305, 310, 822, 310, 310, 310, 310, 305, 305, 310, 310, 310, 310, 305, 305, 310, 305, 310, 305, 305, 310, 305, 310, 310, 310, 771, …. ]
The size was indeed 600 features though. is there any idea how to set the tryout limit and the epoch num in case of this amount of features, and what does mean_max_target mean?

Hello, I've read your paper and I would like to do a experiment of L1000 datasets.
I checked this site , but there are seven datasets.

Whick datasets did you use??
And, Could you provide the code of processing datasets??

Thank you.