ktorio / ktor-io

Collection of IO primitives to work with network and files written in Kotlin using kotlinx.coroutines library

Ktor IO

Collection of IO primitives to work with network and files written in Kotlin using kotlinx.coroutines library

Design Process

Please note, current version of this library is not finished yet. We do not provide any guarantees that it will keep any backward compatibility. To get a motivation why things are implemented as it is, please see the design folder.

ktor-io's Issues

Access to underlying Buffer primitive

As stated in design/BUFFER.md

The main purpose of IO is read and write arrays of bytes. There is a ByteArray - common Kotlin array of bytes abstraction, but it usually is not the efficient way to represent array of bytes on the platform(ByteBuffer on JVM, NSData on iOS, CPointer on Native, ArrayBuffer on Js and so on).

In multiplatform code, it will be ok to use just Buffer, but in platform specific code, near interop with already written Java or Native code, we need to have access to underlying primitive, like ByteBuffer on JVM and CPointer or ByteArray (which can be converted to CPointer) on Native.

My use case is using JDK and OpenSSL for cryptography API, where JDK API need ByteArray or ByteBuffer and OpenSSL native API has CPointer. Of course, we may read ByteArray from Buffer and use it, and also we can create Buffer from ByteArray - but this will be not efficient as copies will be created.

I would think, that such a functionality should be hidden under some UnsafeBufferApi (similar to DelicateCoroutinesApi) opt-in annotation, as such code will be to easy to misuse.

Possible designs:

1. Make specific Buffer implementation public with public property/function to access underlying primitive
2. Make expect/actual to access ONLY platform specific representations
3. Create some other way to safely access underlying primitive, but not tight to specific implementation, so different implementations be it wrapping ByteBuffer or ByteArray or other primitive can be accessed both as ByteArray and ByteBuffer

Revisit location of ByteArray extension methods(get*At, set*At)

Type-safe read-only views

In most of my use-cases, I need a read-only view of buffers (both Buffer and CompositeBuffer) with:

• absolute read methods (i.e. random-access read)
• relative read methods (i.e. sequential read)

It seems from e5l/view-and-refcount and whyoleg/resources that we are taking the route of having reference-counted underlying storages. So the following design proposal is based on this idea.

As a first step, I would propose that we extract a ReadBuffer interface from Buffer containing:

• var readIndex: Int relative read index,
• a new val readLimit: Int exclusive read limit index (always equal to writeIndex in Buffer's default implementation),
• fun read*() relative read methods,
• fun get*At(...) absolute read methods,
• fun copyTo*(...) relative and absolute bulk read methods,
• a new fun duplicate(): ReadBuffer method that returns a duplicate read buffer (with shared underlying storage but independent readIndex),
• new fun slice(...): ReadBuffer methods that returns sliced duplicates.

Additionally, we would add the following to the Buffer interface:

• a new val writeLimit: Int for "symmetry" (always equal to capacity)
• an override override fun duplicate(): Buffer method that returns a duplicate buffer (with shared underlying storage but independent readIndex and writeIndex).