Comments (10)
Great question.
First up. I don't think your Crop
layers are correct. The 164 doesn't seem right in CropPlayer
. (The numbers are how many are taken from the left and right, not the resulting width).
Seems like I should add a version of Crop
which works on 1D shapes, you can do this too if you like in your own code.
I believe right now it's probably not possible*. As really, RecNet
should be a recurrent network, with an recurrent Concat
layer (R
instead of F
).
Now the problem is that Concat
isn't an instance of RecurrentLayer
. I can't think of a fundamental reason that it shouldn't be, or at least that a layer just like Concat
(RecConcat
) couldn't exist which takes two layers which are tagged with F
or R
and makes a new recurrent layer.
Would you like to try writing it?
EDIT:
- With the current Concat layer (or without an orphan instance). One can write their own layers downstream
from grenade.
Ah, good to know about how Crop
works.
I'll take a stab at a 1D Crop
and making Concat
an instance of RecurrentLayer
. Hopefully the types should guide me in the right direction (and I'll drop back here for advice if I get stuck).
from grenade.
Ahh, there's actually another problem. I haven't yet written an instance of RecurrentLayer
for RecurrentNetwork
.
I think it's possible, but requires packing all the recurrent (sideways travelling) shapes into a single vector.
from grenade.
You might have to run both LSTM networks forwards individually for now. The GAN mnist example gives a non-recurrent example of something like this.
from grenade.
Ah, ok. I had thought about doing that, but hadn't looked closely enough at runBackwards
/runGradient
to see that they spit out something input-shaped.
Seems like runNetwork
for both LSTMs, then combine their output, and feed that into runNetwork
for the combining network. Then take the target output, and runBackward
through the combining network to get target results for the two LSTMs, and then runGradient
/applyUpdate
for all networks should do the trick. I'll give it a try, see how it works out.
Thanks for your help!
from grenade.
That's right.
Only difference is you'll need runRecurrent
and backPropagateRecurrent
for the LSTM nets.
Edit. Sorry:
runRecurrentForwards
and runRecurrentBackwards
would also be useful.
from grenade.
Cool, I'm making progress on this. One question that came up as I was working is whether there's an easy way to construct an all-zero vector for a particular RecurrentInput
shape. I want to make sure my network is always starting from the same state at the start of every game.
from grenade.
You can just use the literal 0
.
S
is an instance of Num
so has fromInteger
. In fact RecurrentInputs xs
is also an instance of Num
, so that should work for the entire stack.
If you look at the code for backPropagateRecurrent
you can see I do this (for the back propagated sideways gradients at least).
from grenade.
Please see #32
In that branch, this will compile
type R = Recurrent
type F = FeedForward
type ShapeInput = 'D1 10
type LearnPlayer = RecurrentNetwork
'[ R (LSTM 10 20) ]
'[ ShapeInput , D1 20 ]
type LearnOpponent = RecurrentNetwork
'[ R (LSTM 10 20) ]
'[ ShapeInput, D1 20 ]
type RecNet = RecurrentNetwork
'[ R (
ConcatRecurrent
(D1 20)
(R LearnPlayer)
(D1 20)
(R LearnOpponent)
)
]
'[ ShapeInput, 'D1 40 ]
randomNet :: MonadRandom m => m RecNet
randomNet = randomRecurrent
from grenade.
I believe this is fixed, but feel free to follow up with any problems you're having.
from grenade.
Related Issues (20)
- OpenCL support HOT 6
- Connectionist Temporal Classification support?
- mafia build failure HOT 4
- Bug: network for XOR doesn't train correctly HOT 3
- Recurrent — cannot deduce typeclasses HOT 3
- dropout serialization missing HOT 1
- Do you have non-mafia install tips? HOT 2
- Question about backprop HOT 2
- cabal collisions during install HOT 1
- grenade more easily built with stack than mafia HOT 2
- ghc-8.6 compatibility HOT 1
- bug: foreign import type mismatch
- Is it possible to create neural networks in run-time? HOT 2
- mafia build failure HOT 6
- Test suite fails HOT 4
- Shakespeare - no decent results. HOT 1
- Support ONNX? HOT 3
- Shuffling data
- Cannot get mnist example working HOT 6
- Wrong category of family instance; declaration was for a type family HOT 18
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grenade.