multiple threads writing to same disk causes issues with disk cleanup about stenographer

[jtgreen-cse]

I'd be interested in seeing this merged. I've come across this issue and have this use case.

Edit: @rskjetlein I found your fork, here's the commit that fixes it -->rskjetlein@6f6d2cb

[gconnell]

You're right that this is not an expected use-case, but you're also right that it's one we should probably support, since there's nothing saying you CAN'T do it. Given that, if you've got a PR you'd like to send out I'm happy to take a look.

[emillynge]

I can very quickly have a PR on solution 1) :)

Speaking of PRs, would you mind taking a look at #213 ? It has been open for about a month now and fixes a bug I found.

[gconnell]

Done, thanks, and sorry for the late reply!

[rskjetlein]

Was this fixed?

[rskjetlein]

I made a working solution for 1. mentioned. Can assemble a PR for it if any interest

[rskjetlein]

Will have a look. Its been dormant for some time...

[rskjetlein]

I'd be interested in seeing this merged. I've come across this issue and have this use case.

Edit: @rskjetlein I found your fork, here's the commit that fixes it -->rskjetlein@6f6d2cb

For the most part its working fine, during very high traffic rate (+10G) a unbalanced deletion can be observed.

[jtgreen-cse]

I've been testing the random thread picking strategy with 16 threads .. that makes it too random. Threads delete files very unevenly. Like suggested I think moving the logic in env and deciding which thread to pick for delete based on which has the most files may solve this.

[rskjetlein]

I've been testing the random thread picking strategy with 16 threads .. that makes it too random. Threads delete files very unevenly. Like suggested I think moving the logic in env and deciding which thread to pick for delete based on which has the most files may solve this.

I extended the API with a object delete function and had an external process that used the same API to control the disk fill percentage. The external process took into account other things that used the same disk, might be a usable approach for you.

Please reach out if you want to code snippets. [email protected]