louisscorpio / datamining Goto Github PK
View Code? Open in Web Editor NEWlearn in datamining
learn in datamining
input=tf.reshape(X,[-1,input_size])
input_rnn=tf.matmul(input,w_in)+b_in
input_rnn=tf.reshape(input_rnn,[-1,time_step,rnn_unit])
output_rnn,final_states=tf.nn.dynamic_rnn(cell, input_rnn,initial_state=init_state, dtype=tf.float32)
综合网上教程,我觉的dynamic_cnn中的input_rnn维度应该是[-1,time_step,input_size],tensorflow中是封装好的(参考:https://www.cnblogs.com/zyly/p/9029591.html),但作者您自己写了input 的w 和b,将input_rnn的维度改成了[-1,time_step,rnn_unit],感觉有点奇怪。
我自己写的代码如下:
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
mnist=input_data.read_data_sets("MNIST_data/",one_hot=True)
print("start")
print(tf.version)
print(mnist)
n_inputs=28 #input_size
max_time=28 #也即time_step
lstm_size=100 #num_units
n_classes=10
batch_size=50
n_batch=mnist.train.num_examples //batch_size
x=tf.placeholder(tf.float32,[None,784])
y=tf.placeholder(tf.float32,[None,10])
w=tf.Variable(tf.truncated_normal([lstm_size,n_classes],stddev=0.1))
b=tf.Variable(tf.constant(0.1,shape=[n_classes]))
def RNN(X,w,b):
inputs=tf.reshape(X,[-1,max_time,n_inputs])
lstm_cell=tf.contrib.rnn.BasicLSTMCell(lstm_size)
outputs,final_state=tf.nn.dynamic_rnn(lstm_cell,inputs,dtype=tf.float32)
#final_state[0]=cell state
#final_state[1]=hidden state
results=tf.nn.softmax(tf.matmul(final_state[1],w)+b)
return results
prediction=RNN(x,w,b)
cross_entropy=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction,labels=y))
train_step=tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) #hinton建议设置为1e-3,代表初始学习率
correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))
accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
n=0
with tf.Session() as sess:
init=tf.global_variables_initializer()
sess.run(init)
for epoch in range(6):
print("n:",n)
n+=1
for batch in range(n_batch):
batch_xs,batch_ys=mnist.train.next_batch(batch_size)
sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})
print("epoch:",epoch)
acc=sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})
print(str(epoch)+" times,accuracy:"+str(acc))
print("over")
Attempt to have a second RNNCell use the weights of a variable scope that already has weights: 'rnn/basic_lstm_cell'; and the cell was not constructed as BasicLSTMCell(..., reuse=True). To share the weights of an RNNCell, simply reuse it in your second calculation, or create a new one with the argument reuse=True.
我查了stackoverflow,按照上面问题将rnn.BasicLSTMCell(rnn_unit)修改成cell=rnn.BasicLSTMCell(rnn_unit, reuse=True)或者rnn.BasicLSTMCell(rnn_unit,reuse = tf.get_variable_scope().reuse)还是报错:
ValueError: Variable rnn/basic_lstm_cell/weights already exists, disallowed. Did you mean to set reuse=True in VarScope?
请问是怎么运行该代码啊?
用的tensorflow 1.2.1, 对stock_predict.py进行了一些修改后运行正常。
然而在学习您的stock_predict_2.py时遇到问题了,问题出在第44行。
以下是详细错误信息:
/usr/local/lib/python3.5/dist-packages/numpy/core/fromnumeric.py:2909: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python3.5/dist-packages/numpy/core/_methods.py:73: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/usr/local/lib/python3.5/dist-packages/numpy/core/_methods.py:135: RuntimeWarning: Degrees of freedom <= 0 for slice
keepdims=keepdims)
/usr/local/lib/python3.5/dist-packages/numpy/core/_methods.py:105: RuntimeWarning: invalid value encountered in true_divide
arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
/usr/local/lib/python3.5/dist-packages/numpy/core/_methods.py:125: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/usr/local/lib/python3.5/dist-packages/numpy/lib/function_base.py:1110: RuntimeWarning: Mean of empty slice.
avg = a.mean(axis)
/usr/local/lib/python3.5/dist-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
在网上搜到一些解决方式,尝试修改为
with warnings.catch_warnings():
warnings.simplefilter("ignore", category=RuntimeWarning)
mean=np.mean(data_test,axis=0)
然而并无法解决问题。
想请教这个问题如何处理?
开放一下数据集下载 谢谢啊
ValueError: Variable rnn/basic_lstm_cell/weights already exists, disallowed. Did you mean to set reuse=True in VarScope
训练的时候,报错:
Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.
关于这行代码:loss=tf.reduce_mean(tf.nn.nce_loss(nce_weights, nce_bias, embed, train_labels, num_sampled, num_classes=vocabulary_size))
我想是tensorflow的版本问题还是什么原因?
直接使用了吗?
'gbk' codec can't decode byte 0xb4 in position 35: illegal multibyte sequence
强制换成utf-8 能跑,但是训练到几千次的时候会报类似的错误,也是can't decode之类的
我只想说小姐姐那篇adaboost的文章写的太棒了,顺带看了你的博客和游记,我从来没有见过这么聪明漂亮还会玩的小姑娘,为了评论,博客园刚注册没法评论,你的github博客也没法评论,只能来这了,希望可以改掉文中wm,i˜=exp(−yi(Fm−1(xi)+αm−1Gm−1(xi)))=wm−1,i˜exp(−yiαm−1Gm−1(xi))这个小错误,方便我这样的小白更好的崇拜你
我自己用处理的数据进行训练,没用提供的数据.
我想问下代码里,训练时x传入7列,是不是不对啊,第7列应该是label吧.应该只用传入6列吧,是不是?
另外想问下预测的部分,如果我想用前90%进行训练,,后10%进行预测,应该怎么写啊.
你的数据集链接不能用了啊 小姐姐!求更新
module_file = tf.train.latest_checkpoint()
示例中怎么没有参数,无法运行。。。
这个方法的定义是:
def latest_checkpoint(checkpoint_dir, latest_filename=None):
"""Finds the filename of latest saved checkpoint file.
Args:
checkpoint_dir: Directory where the variables were saved.
latest_filename: Optional name for the protocol buffer file that
contains the list of most recent checkpoint filenames.
See the corresponding argument to Saver.save()
.
Returns:
The full path to the latest checkpoint or None
if no checkpoint was found.
"""
ckpt = get_checkpoint_state(checkpoint_dir, latest_filename)
if ckpt and ckpt.model_checkpoint_path:
# Look for either a V2 path or a V1 path, with priority for V2.
v2_path = _prefix_to_checkpoint_path(ckpt.model_checkpoint_path,
saver_pb2.SaverDef.V2)
v1_path = _prefix_to_checkpoint_path(ckpt.model_checkpoint_path,
saver_pb2.SaverDef.V1)
if file_io.get_matching_files(v2_path) or file_io.get_matching_files(
v1_path):
return ckpt.model_checkpoint_path
else:
logging.error("Couldn't match files for checkpoint %s",
ckpt.model_checkpoint_path)
return None
——————————————————定义神经网络变量——————————————————
X=tf.placeholder(tf.float32, [None,time_step,input_size]) #每批次输入网络的tensor
Y=tf.placeholder(tf.float32, [None,time_step,output_size]) #每批次tensor对应的标签
运行到这里报错:
AttributeError: module 'tensorflow' has no attribute 'placeholder'
如果要进行预测 预测后的数据应该进行归一化还原才是真实数据 但是我们数据是预测的 怎么知道预测数据的std 和 mean呢
stock预测第一部分的代码 进行了后期预测 但是没有还原的操作啊 是不是有问题
你好,请问time_step的取值是依据什么?我看你time_step取的是20
module_file = tf.train.latest_checkpoint()
TypeError: latest_checkpoint() takes at least 1 argument (0 given)
训练集的由于数据量比较多,在归一化时接近真实分布的均值和方差,但是测试集样本量比较少,这样计算的测试集的均值和方差肯定会有误差的。但是我对整体归一化的效果并没有分开归一化的效果好,能请教一下原因吗?
感谢您的代码,能不能说明一下所使用的包的版本?tensorflow是什么版本的呀?
请问po主用的python3.几以及相应的tensorflow版本是多少?我这里会报错
AttributeError: module 'tensorflow.python.ops.nn' has no attribute 'rnn_cell'
stackoverflow上说是因为我的tensorflow版本太新,改动比较大
用了未来数据了。。。你实际上是用今天和后两天的数据在预测今天的数据。
那个stock_predict.py,import tensorflow as tf都没有,后面使用tf怎么能成功?还有那个rnn是哪里来的?
第二个stock_predict_2.py,会发生越界。请问能跑的代码是哪个版本?
你好,我拷贝了整个代码,运行的时候,没反应?也没生成图形,也没提示错误,不知道是怎么回事呢?
data=df.iloc[:,2:10].values #取第3-10列 获取了8列数据,为什么input_size=7呢
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.