Comments (2)
That's really unfortunate for that interface. Anything that takes a pure byte array is just going to be inefficient and almost certainly require copying the bytes to a new array.
That doesn't necessarily mean RMS won't help you. Maybe there is significant internal work that can be covered, and you'll still get some benefit, even with the built-in interface.
To get the full benefit, you'd have to find a way to use a different interface that took a byte range instead. A new version of the library, something that takes Span? completely different serialization interface/method?
I hesitate to provide optimal settings for people because so much depends on more than just buffer size. It's how it's used, how many simultaneous usages, whether you want to avoid LOH allocs, and more.
Certainly, if most of the cases end up being 2 MB, you could "collapse" the two types of buffers into one flat pool and just use a 2 MB (or larger) block size. That would avoid a memory copy whenever you need that full buffer. A little different than the original use case, but I think it could work fine.
Only real answer is to measure and see!
from microsoft.io.recyclablememorystream.
Thanks for your answer. Unfortunately we cannot change the interface, but I could try and open a request on the confluent repo to see if this would be even doable. Even if the interface is changed, if internally the system requires a byte array (and it may, since that's a wrapper around a C++ library), we still wouldn't get very far.
Thanks for the tips - will continue exploring.
from microsoft.io.recyclablememorystream.
Related Issues (20)
- Is there a plan to contribute to the dotnet/runtime? HOT 4
- FeatureRequest: Add opt-in possibility to zeroed-out buffers HOT 10
- Continous benchmarking HOT 1
- Multiple StreamManagers in referenced libraries - how to find total memory usage HOT 1
- Large pool limit should apply to whole pool, not each slot HOT 1
- Humanitarian Organization Team mbrgi
- ResponseTime is almost doubled with RecyclableMemoryStream. HOT 5
- Question: GetStream copy existing buffer use case HOT 1
- How effective is this package for buffers that are usually smaller than 256 bytes? HOT 1
- Consider using ArrayPool<byte> as underlying storage mechanism HOT 4
- Breaking changes at 3.0.0 version break OfficeOpenXml NuGet package HOT 11
- Guideline about cryptography with RMS HOT 5
- Unclear Documentation for MaximumFreeSmallPoolBytes
- Upgrade blocker: Recent change of GetStream() to return RecyclableMemoryStream instead of MemoryStream HOT 1
- why just pool MemoryStream HOT 2
- FileStreamResult: ObjectDisposedException: Cannot access a disposed object. HOT 4
- For many small stream writes, RecyclableMemoryStream is slower than other implementations HOT 4
- ETW event source is not working HOT 2
- RecyclableMemoryStream.TryGetBuffer causes infinite loop when UseExponentialLargeBuffer is true HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from microsoft.io.recyclablememorystream.