High number of memory allocations. about resumablefunctions.jl

[BenLauwens]

Hi

Sorry for the delay but it took some time before I could address this issue.
The problem comes from inference of the type of your sieve. Because a @resumable function is a macro it can not include the runtime type of an object. You can remedy this problem by specifying it manually:

@resumable function segmentedsieve(n::Int64)
    segmentstart::Int64 = 0
    segmentend::Int64 = Int(ceil(sqrt(n)))
    initialsieve = basicsieve(segmentend)::BitArray{1}
    segment = copy(initialsieve)
    lastprime::Int64 = 2
    p::Int64 = 3
    while p <= n
        while p <= segmentend
           if segment[p-segmentstart]
                @yield lastprime
                lastprime = p
            end
            p+=1
        end
        segmentstart  += length(segment)
        segmentend += length(segment)
        segmentend = min(segmentend,n)
        segment .= true
        sievesegment(initialsieve,segment,segmentstart)
    end
    lastprime
end

This gives on my computer for n=10000 the following benchmark results:

julia> @btime bench1()
  130.895 μs (7 allocations: 400 bytes)

julia> @btime bench2()
  7.433 ms (3707 allocations: 61.94 KiB)

julia> @btime bench3()
  121.794 μs (7 allocations: 4.41 KiB)

If you do a @macroexpand of the @resumable function you see what the resulting code is and you can find out what the types are.

Can you keep me informed whether this solves your problem? I think I will include a kind of best practice section in the documentation with your segmented sieve as an example (if you allow me).

Kind regards

Ben

[saolof]

Ah right, adding type annotations did indeed eliminate the allocations. I'm impressed by how well it performs.

Interestingly enough, the allocations seemed to be coming exclusively from the function argument n being noninferrable. Once I specified the type of n, all allocations disappeared. Maybe it could be made to work for more general function type parameters with generated function and a parametrically typed struct?

Feel free to add the segmented sieve as an example.

Kind regards,
Olof

[BenLauwens]

Thanks!

Parametric @resumable functions are possible. I hope that one day the Julia compiler is good enough to avoid these type annotations.
I will close the issue.

Kind regards

Ben