knetml / neural-style-transfer

Knet Implementation of Neural Style Transfer

Neural Style Transfer

This notebook implements deep CNN based image style transfer algorithm from "Image Style Transfer Using Convolutional Neural Networks" (Gatys et al., CVPR 2016).

The proposed technique takes two images as input, i.e. a content image (generally a photograph) and a style image (generally an artwork painting). Then, it produces an output image such that the content(objects in the image) resembles the "content image" whereas the style i.e. the texture is similar to the "style image". In order words, it re-draws the "content image" using the artistic style of the "style image".

neural-style-transfer's Issues

Error in last cell

While running the last cell I'm getting this error:

MethodError: Cannot convert an object of type AutoGrad.Rec{Array{Float64,4}} to an object of type Array
This may have arisen from a call to the constructor Array(...),
since type constructors fall back to convert methods.

Stacktrace:
[1] tv_loss(::AutoGrad.Rec{Array{Float64,4}}, ::Float64) at ./In[10]:12
[2] loss(::AutoGrad.Rec{Array{Float64,4}}, ::Float64, ::Int64, ::Array{Float64,4}, ::NTuple{5,Int64}, ::Array{Any,1}, ::Array{Float64,1}, ::Float64) at ./In[11]:6
[3] forward_pass(::Function, ::Tuple{Array{Float64,4},Float64,Int64,Array{Float64,4},NTuple{5,Int64},Array{Any,1},Array{Float64,1},Float64}, ::Array{Any,1}, ::Int64) at /home/subhankar/.julia/v0.6/AutoGrad/src/core.jl:88
[4] (::AutoGrad.##gradfun#4#6{#loss,Int64})(::Array{Any,1}, ::Function, ::Array{Float64,4}, ::Vararg{Any,N} where N) at /home/subhankar/.julia/v0.6/AutoGrad/src/core.jl:57
[5] (::AutoGrad.#gradfun#5)(::Array{Float64,4}, ::Vararg{Any,N} where N) at /home/subhankar/.julia/v0.6/AutoGrad/src/core.jl:57
[6] style_transfer(::String, ::String, ::Int64, ::Int64, ::Int64, ::Float64, ::NTuple{5,Int64}, ::Array{Float64,1}, ::Float64, ::Bool) at ./In[12]:60
[7] style_transfer(::String, ::String, ::Int64, ::Int64, ::Int64, ::Float64, ::NTuple{5,Int64}, ::Array{Float64,1}, ::Float64) at ./In[12]:25
[8] include_string(::String, ::String) at ./loading.jl:522
[9] execute_request(::ZMQ.Socket, ::IJulia.Msg) at /home/subhankar/.julia/v0.6/IJulia/src/execute_request.jl:193
[10] (::Compat.#inner#6{Array{Any,1},IJulia.#execute_request,Tuple{ZMQ.Socket,IJulia.Msg}})() at /home/subhankar/.julia/v0.6/Compat/src/Compat.jl:189
[11] eventloop(::ZMQ.Socket) at /home/subhankar/.julia/v0.6/IJulia/src/eventloop.jl:8
[12] (::IJulia.##13#16)() at ./task.jl:335

I'm on Julia 0.6.4 running on Elementary OS 5

example does not work under Julia >= 1.0

I made some obvious syntax changes to get the example to run under julia 1.3/Knet 132,
but it fails with this error

ERROR: LoadError: MethodError: Cannot `convert` an object of type UnitRange{Int64} to an object of type Colon
Closest candidates are:
  convert(::Type{T}, ::T) where T at essentials.jl:167
Stacktrace:
 [1] convert(::Type{Tuple{Colon,UnitRange{Int64},Colon,Colon}}, ::Tuple{UnitRange{Int64},Colon,Colon,Colon}) at ./essentials.jl:304
 [2] setindex!(::Array{Tuple{Colon,UnitRange{Int64},Colon,Colon},1}, ::Tuple{UnitRange{Int64},Colon,Colon,Colon}, ::Int64) at ./array.jl:766
 [3] copyto!(::Array{Tuple{Colon,UnitRange{Int64},Colon,Colon},1}, ::Int64, ::Array{Tuple{UnitRange{Int64},Colon,Colon,Colon},1}, ::Int64, ::Int64) at ./abstractarray.jl:842
 [4] append!(::Array{Tuple{Colon,UnitRange{Int64},Colon,Colon},1}, ::Array{Tuple{UnitRange{Int64},Colon,Colon,Colon},1}) at ./array.jl:895
 [5] addto!(::AutoGrad.Sparse{Float64,4}, ::AutoGrad.Sparse{Float64,4}) at /root/.julia/packages/AutoGrad/pTNVv/src/addto.jl:44
 [6] #differentiate#3(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(AutoGrad.differentiate), ::Function, ::Param{KnetArray{Float64,4}}, ::Vararg{Any,N} where N) at /root/.julia/packages/AutoGrad/pTNVv/src/core.jl:166
 [7] differentiate(::Function, ::Param{KnetArray{Float64,4}}, ::Vararg{Any,N} where N) at /root/.julia/packages/AutoGrad/pTNVv/src/core.jl:135
 [8] (::getfield(AutoGrad, Symbol("##gradfun#6#8")){typeof(loss),Int64,Bool})(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::getfield(AutoGrad, Symbol("#gradfun#7")){getfield(AutoGrad, Symbol("##gradfun#6#8")){typeof(loss),Int64,Bool}}, ::KnetArray{Float64,4}, ::Vararg{Any,N} where N) at /root/.julia/packages/AutoGrad/pTNVv/src/core.jl:225
 [9] (::getfield(AutoGrad, Symbol("#gradfun#7")){getfield(AutoGrad, Symbol("##gradfun#6#8")){typeof(loss),Int64,Bool}})(::KnetArray{Float64,4}, ::Vararg{Any,N} where N) at /root/.julia/packages/AutoGrad/pTNVv/src/core.jl:221
 [10] style_transfer(::String, ::String, ::Int64, ::Int64, ::Int64, ::Float64, ::NTuple{5,Int64}, ::Array{Float64,1}, ::Float64, ::Bool) at /work/neural_style_transfer.jl:365
 [11] style_transfer(::String, ::String, ::Int64, ::Int64, ::Int64, ::Float64, ::NTuple{5,Int64}, ::Array{Float64,1}, ::Float64) at /work/neural_style_transfer.jl:329
 [12] top-level scope at util.jl:156
 [13] include at ./boot.jl:328 [inlined]
 [14] include_relative(::Module, ::String) at ./loading.jl:1094
 [15] include(::Module, ::String) at ./Base.jl:31
 [16] include(::String) at ./client.jl:431
 [17] top-level scope at REPL[1]:1
in expression starting at /work/neural_style_transfer.jl:397

DimensionMismatch("new array has 225 color channels, must have 3")

hello,

I tried to run the notebook in Julia 0.6.3 in OSX and I get the following error at the last cell (#Demo: Style transfer. [...]):

INFO: Recompiling stale cache file /Users/alage/.julia/lib/v0.6/QuartzImageIO.ji for module QuartzImageIO.
DimensionMismatch("new array has 225 color channels, must have 3")

Stacktrace:
 [1] check_ncolorchan(::Array{ColorTypes.RGB4{FixedPointNumbers.Normed{UInt8,8}},2}, ::Tuple{Int64,Int64,Int64}) at /Users/alage/.julia/v0.6/ImageCore/src/colorchannels.jl:360
 [2] similar(::ImageCore.ChannelView{FixedPointNumbers.Normed{UInt8,8},3,Array{ColorTypes.RGB4{FixedPointNumbers.Normed{UInt8,8}},2}}, ::Type{FixedPointNumbers.Normed{UInt8,8}}, ::Tuple{Int64,Int64,Int64}) at /Users/alage/.julia/v0.6/ImageCore/src/colorchannels.jl:117
 [3] permutedims(::ImageCore.ChannelView{FixedPointNumbers.Normed{UInt8,8},3,Array{ColorTypes.RGB4{FixedPointNumbers.Normed{UInt8,8}},2}}, ::Array{Int64,1}) at ./permuteddimsarray.jl:116
 [4] #preprocess#10(::Int64, ::Function, ::Array{ColorTypes.RGB4{FixedPointNumbers.Normed{UInt8,8}},2}) at ./In[3]:22
 [5] (::#kw##preprocess)(::Array{Any,1}, ::#preprocess, ::Array{ColorTypes.RGB4{FixedPointNumbers.Normed{UInt8,8}},2}) at ./<missing>:0
 [6] style_transfer(::String, ::String, ::Int64, ::Int64, ::Int64, ::Float64, ::NTuple{5,Int64}, ::Array{Float64,1}, ::Float64, ::Bool) at ./In[18]:25 (repeats 2 times)