shenweichen / dsin Goto Github PK

作者你好，以下有两个问题,希望您能解答.

第一点:在你论文中Qk应该指用户的第k个session矩阵, 但是在论文后面

这个Qk好像和原来意思不一样啊？能帮忙解释一下具体含义吗?

第二点：我看过Attention is All your need论文，我发现你论文中的参数矩阵应该是不同session之间是不同的.而你的论文中看起来所有参数矩阵W^Q都一样.

在deepctr库的Transformer类中有：
def call(self, inputs, mask=None, training=None, **kwargs):
if self.supports_masking:
queries, keys = inputs
query_masks, key_masks = mask
print('query_masks:',query_masks.get_shape())
query_masks = tf.cast(query_masks, tf.float32)
key_masks = tf.cast(key_masks, tf.float32)
我在复现DSIN的时候，如果是保持原配置不变的话复现是没有问题的。但是实际上我在将原输入划分成两个序列长为5的序列分别输入时，在执行query_masks, key_masks = mask这一步时就会提示'Nonetype' is not iterable。

python version [3.6]
tensorflow version [1.4.0]
deepctr version [0.4.1]

从2_gen_dien_input.py 和 2_gen_dsin_input.py 这两个文件中，看到dien生成的数据是选择第1个session中最新的数据；而dsin生成的数据是把最新的session中最新数据放至sess_0。
所以，这里我有两个问题：
1.dsin和dien本身输入的数据不同，是否因为这个会导致实验结果有差异
2.dsin是将最新数据放在sess_0，因此在段间兴趣提取层，实际上是学习了一个从现在到过去的演化过程（当然，您论文中用的是bi-lstm，是双向学习）。那如果是换个顺序存储sess数据，那结果是否会有变化。
嗯，上面这些只是一些想法，还没有进行验证。
或许，您能帮忙解释一下，当初您为什么采用这种策略生成输入数据。

DSIN数据处理部分的 from deepctr.utils import SingleFeat 不存在，使用的是0.8的deepctr

hi，DSIN非常棒，还有测试数据，想请教个问题，模型训练后怎么对线上数据进行预测，模型预测是否click，预测数据需要如何组装

你好！我看代码里面Transformer输入是TR([tr_input[i], tr_input[i]])，但是其具体函数定义格式又是：def call(self, inputs, mask=None, training=None, **kwargs)，其中的参数mask，要求是和tr_input[i]同shape或者是(batch_size, 1)，不知道是不是我哪里有遗漏，谢谢！

我将环境配置完毕后，在训练模型时抛出异常：InternalError: GPU sync failed。我觉得可能是一次性训练太多数据内存不够，于是我将DIN模型的BATCH_SIZE改成512，结果为0.629。将DSIN模型的BATCH_SIZE改为256，结果为5.63。百思不得解，也尝试过其它BATCH_SIZE，结果大同小异。求大神不吝赐教。

Hi, I have got an error while run 0_gen_sampled_data.py, the info as follows:

My computer memory is 8GB.Do you know how to fix this, thanks!

你好，请问一下，论文中只是使用了用户的点击序列，那么用户的收藏序列或者购买序列是否也能加进去？如果多个序列的话应该是不能共享参数的？

bug位置在文件2_gen_dsin_input.py第52行：last_sess_idx = i。当用户没有大于2个行为的session时，last_sess_idx = len(user_hist_session[user]) - 1，而不是等于0。导致第56行定位用户前4个session时，取的是最新的4个session，而非当前session前4个session。因此造成部分样本会使用到label时间之后的特征。
“11,1494226737,302383,430548_1007,1,0
11,1494226737,598359,430548_1007,1,0
11,1494226737,684497,430548_1007,1,0
11,1494419569,427488,430548_1007,1,0
11,1494419569,611964,430548_1007,1,0
11,1494419569,739213,430548_1007,1,0”，例如raw_sample中user_id=11，时间=1494226737的3个样本就是这种情况。

hi, very lucky to study your work! And I have some questions about your code:
My TF version is 1.13.0 , cuda 10.0, I got :
AttributeError: 'Tensor' object has no attribute '_uses_learning_phase'
error like this.

So I pip install tf 1.4.0 according to your readme, and got :
ImportError: libcublas.so.8.0: cannot open shared object file: No such file or directory
Seems like my cuda version cannot work with tf version, So do you know how to fix this case?

Thank you for your time !

WARNING:tensorflow:
The following Variables were used a Lambda layer's call (lambda), but
are not present in its tracked objects:
<tf.Variable 'attention_sequence_pooling_layer/local_activation_unit/kernel:0' shape=(16, 1) dtype=float32>
<tf.Variable 'attention_sequence_pooling_layer/local_activation_unit/bias:0' shape=(1,) dtype=float32>
It is possible that this is intended behavior, but it is more likely
an omission. This is a strong indication that this layer should be
formulated as a subclassed Layer rather than a Lambda layer.
WARNING:tensorflow:
The following Variables were used a Lambda layer's call (lambda), but
are not present in its tracked objects:
<tf.Variable 'attention_sequence_pooling_layer/local_activation_unit/kernel:0' shape=(16, 1) dtype=float32>
<tf.Variable 'attention_sequence_pooling_layer/local_activation_unit/bias:0' shape=(1,) dtype=float32>
It is possible that this is intended behavior, but it is more likely
an omission. This is a strong indication that this layer should be
formulated as a subclassed Layer rather than a Lambda layer.
Traceback (most recent call last):
File "/home/lyw/PycharmProjects/DSIN/code/train_din.py", line 49, in
att_hidden_size=(64, 16,))
File "/home/lyw/PycharmProjects/DSIN/code/models/din.py", line 87, in DIN
query_emb, keys_emb])
File "/home/lyw/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 922, in call
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/home/lyw/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 265, in wrapper
raise e.ag_error_metadata.to_exception(e)
AttributeError: in user code:

/home/lyw/anaconda3/envs/py36/lib/python3.6/site-packages/deepctr/layers/sequence.py:198 call  *
    outputs._uses_learning_phase = attention_score._uses_learning_phase

AttributeError: 'Tensor' object has no attribute '_uses_learning_phase'

I ran the train_din.py code, and throw this exception. I have try in two different servers. Tthe problem took place both in two servers.

您好，我想请问一下，在对历史行为做session的划分时，为什么只取时间戳为5.3号以后的数据？

Hi, I have got an error while run train_dsin.py, the info as follows:

Caused by op 'sparse_emb_14-brand/Gather_6', defined at:
File "train_dsin.py", line 52, in
att_embedding_size=1, bias_encoding=False)
File "/home/dedong/pycharmProjects/Emb4RS/models/DSIN/code/_models/dsin.py", line 85, in DSIN
sess_feature_list, sess_max_count, bias_encoding=bias_encoding)
File "/home/dedong/pycharmProjects/Emb4RS/models/DSIN/code/_models/dsin.py", line 154, in sess_interest_division
sparse_fg_list, sess_feture_list, sess_feture_list)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/deepctr/input_embedding.py", line 145, in get_embedding_vec_list
embedding_vec_list.append(embedding_dictfeat_name)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/keras/_impl/keras/engine/topology.py", line 252, in call
output = super(Layer, self).call(inputs, **kwargs)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/keras/_impl/keras/layers/embeddings.py", line 158, in call
out = K.gather(self.embeddings, inputs)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/keras/_impl/keras/backend.py", line 1351, in gather
return array_ops.gather(reference, indices)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 2486, in gather
params, indices, validate_indices=validate_indices, name=name)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1834, in gather
validate_indices=validate_indices, name=name)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/dedong/anaconda3/envs/tf1.4/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): indices[0,0] = 136739 is not in [0, 79963)
[[Node: sparse_emb_14-brand/Gather_6 = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](sparse_emb_14-brand/embeddings/read, sparse_emb_14-brand/Cast_6)]]

do you know how to fix this, thanks!

您好，我想请问一下您在dsin原始论文当中与dien做对比的时候，dien是否有用到auxiliary loss，据我了解在dien的论文当中，这个auxiliary loss对结果影响非常大，用了可以提升好几个百分点。因此，您对比dien模型的时候是否有用到auxiliary loss，还是直接用没有auxiliary loss的dien与dien做对比？

Hello, I am running you baseline train_dien.py
add
` names = [weight.name for layer in model.layers for weight in layer.weights]
weights = model.get_weights()

for name, weight in zip(names, weights):
    print(name, weight.shape)`

to print variable in model and get

why there is no parameter for gru and attention layer? I user model.summary() also show no parameter for these two layer.

Hi,

The nan problem in training, I have no idea. Can you fix it? How many epochs is the model converge?

The nan problem in test, you can change the last line with
print("test LogLoss {0}".format( round(log_loss(test_label, np.array(pred_ans, dtype=np.float64), labels=(0, 1)), 4)))
maybe work.
btw, I wonder if preprocessed pkl will be released.

Thanks!

Hi,
I have set up the enviroment and generated the input data for the DSIN model following the instruction. But I still failed to the train the model executing "train_dsin.py". The error messages are in the attachment.
Thank you for your time!

ImportError: No module named input_embedding when from deepctr.input_embedding import create_singlefeat_inputdict (master/code/models/dsin.py)

pip install deepctr has been done！

python version: Python 2.7.10

想问一下您，在taobao ad数据集上，最终的AUC是能够得到论文中结果的嘛？ 因为我按照您的这个流程走下来，使用了5%的ad数据集（受硬件环境限制），但是最后AUC只有0.5+， 我就很怀疑是我哪里没有弄对吗？