Implement POSIX platform about gneiss

[jklmnn]

A block client example for posix is available on branch issue_24_block_client and can be run via

$ gprbuild -P ada_interface.gpr -XPLATFORM=posix -XTEST=block_client
$ ./build/block_client

[jklmnn]

I looked a bit into aio. It doesn't seem to have any block size mechanism and is able to read and write any buffer size. In this case I would check if the opened file descriptor is a regular file or a device file. In the first case I would set the block size to whatever the underlying file system uses and in the latter case I would use the block size of the device behind the device file.

[jklmnn]

I went through the aio man page and currently I cannot see how we would implement our fail safe enqueue/submit. Some errors are preventable, such as accessing files out of bounds, but I don't see a way to prevent generic errors such as out of resources errors. Preexecuting requests on ready is definitely not a solution.

Also some functions can return ENOSYS if they're not implemented. In this case one could use the assumption to call them with an invalid file descriptor and if they return ENOSYS then they're not implemented since any other implementation would check the arguments and return with another error.

[jklmnn]

The general idea is not to use aio_write|read|fsync but to allocate a local list, fill it via enqueue and then call lio_listio in submit. It seems that this encodes some of the errors of the other procedures in the request status. Furthermore it seems to only trigger a signal once all requests are completed which should improve the performance. The length of this list should be

buffer_size ? min(AIO_LISTIO_MAX, buffer_size / sizeof(struct aiocb)) : 64

[senier]

lio_listio looks interesting. I don't see an equivalent to aio_sync in the manpage, though. How would sync be implemented?

[jklmnn]

sync would not be enqueued regularily with a special marker implemented by us. Before lio_listio is executed the queue will evaluate how many consecutive items can be passed to lio_listio. The implementation is then free to decide to take less but not more items. A sync will break the consecutiveness and needs to be executed via aio_fsync before more items can be taken from the queue. Furthermore syncs will not generate signals as their success is not evaluated.

[jklmnn]

I'm now able to run the block_client test successfully on POSIX. Although the implementation will only run on Linux due to the block device file support.
There are also some parts that I'm not really sure about under which circumstances they work. This is caused by the sometimes quite imprecise documentation, especially in regards to the exact behaviour of errors and undocumented magic values.
But as a proof of concept it already works.

[senier]

Great to see that basic POSIX support works already!

I don't think the Linux-only nature of the code is an issue right now. Should we ever need to run on different POSIXish systems we can adapt the code.

[jklmnn]

I hereby present you the first working run of our correctness test on Posix:

[Correctness] Info: Correctness
[Correctness] Info: Running correctness test over 131072 blocks of 4096 byte size (512 MiB)...
[Correctness] Info: Writing... (2.4%, 12 MiB / 512 MiB)
[Correctness] Info: Elapsed: 2.010211 Remaining: 81.748585
[Correctness] Info: Writing... (4.9%, 25 MiB / 512 MiB)
[Correctness] Info: Elapsed: 4.011784 Remaining: 77.861364
[Correctness] Info: Writing... (7.5%, 38 MiB / 512 MiB)
[Correctness] Info: Elapsed: 6.016067 Remaining: 74.198165
[Correctness] Info: Writing... (10.0%, 51 MiB / 512 MiB)
[Correctness] Info: Elapsed: 8.019245 Remaining: 72.173206
[Correctness] Info: Writing... (12.5%, 64 MiB / 512 MiB)
[Correctness] Info: Elapsed: 10.027866 Remaining: 70.195063
[Correctness] Info: Writing... (15.0%, 77 MiB / 512 MiB)
[Correctness] Info: Elapsed: 12.028451 Remaining: 68.161221
[Correctness] Info: Writing... (17.5%, 89 MiB / 512 MiB)
[Correctness] Info: Elapsed: 14.034591 Remaining: 66.163072
[Correctness] Info: Writing... (20.0%, 102 MiB / 512 MiB)
[Correctness] Info: Elapsed: 16.042945 Remaining: 64.171778
[Correctness] Info: Writing... (22.6%, 115 MiB / 512 MiB)
[Correctness] Info: Elapsed: 18.046256 Remaining: 61.804433
[Correctness] Info: Writing... (25.2%, 129 MiB / 512 MiB)
[Correctness] Info: Elapsed: 20.051460 Remaining: 59.517824
[Correctness] Info: Writing... (27.5%, 140 MiB / 512 MiB)
[Correctness] Info: Elapsed: 22.061890 Remaining: 58.163164
[Correctness] Info: Writing... (29.9%, 153 MiB / 512 MiB)
[Correctness] Info: Elapsed: 24.073110 Remaining: 56.438964
[Correctness] Info: Writing... (32.4%, 166 MiB / 512 MiB)
[Correctness] Info: Elapsed: 26.082692 Remaining: 54.419444
[Correctness] Info: Writing... (35.0%, 179 MiB / 512 MiB)
[Correctness] Info: Elapsed: 28.084252 Remaining: 52.156469
[Correctness] Info: Writing... (37.4%, 191 MiB / 512 MiB)
[Correctness] Info: Elapsed: 30.087285 Remaining: 50.360002
[Correctness] Info: Writing... (39.8%, 203 MiB / 512 MiB)
[Correctness] Info: Elapsed: 32.090791 Remaining: 48.539338
[Correctness] Info: Writing... (42.3%, 216 MiB / 512 MiB)
[Correctness] Info: Elapsed: 34.099086 Remaining: 46.513410
[Correctness] Info: Writing... (44.8%, 229 MiB / 512 MiB)
[Correctness] Info: Elapsed: 36.100058 Remaining: 44.480428
[Correctness] Info: Writing... (47.1%, 241 MiB / 512 MiB)
[Correctness] Info: Elapsed: 38.100815 Remaining: 42.792635
[Correctness] Info: Writing... (49.5%, 253 MiB / 512 MiB)
[Correctness] Info: Elapsed: 40.106829 Remaining: 40.917067
[Correctness] Info: Writing... (52.0%, 266 MiB / 512 MiB)
[Correctness] Info: Elapsed: 42.116247 Remaining: 38.876535
[Correctness] Info: Writing... (54.5%, 279 MiB / 512 MiB)
[Correctness] Info: Elapsed: 44.122774 Remaining: 36.836444
[Correctness] Info: Writing... (57.1%, 292 MiB / 512 MiB)
[Correctness] Info: Elapsed: 46.130227 Remaining: 34.658261
[Correctness] Info: Writing... (59.7%, 305 MiB / 512 MiB)
[Correctness] Info: Elapsed: 48.131434 Remaining: 32.490733
[Correctness] Info: Writing... (62.2%, 318 MiB / 512 MiB)
[Correctness] Info: Elapsed: 50.133454 Remaining: 30.466954
[Correctness] Info: Writing... (64.7%, 331 MiB / 512 MiB)
[Correctness] Info: Elapsed: 52.156991 Remaining: 28.456596
[Correctness] Info: Writing... (66.9%, 342 MiB / 512 MiB)
[Correctness] Info: Elapsed: 54.163079 Remaining: 26.798175
[Correctness] Info: Writing... (69.3%, 354 MiB / 512 MiB)
[Correctness] Info: Elapsed: 56.173683 Remaining: 24.885022
[Correctness] Info: Writing... (71.9%, 367 MiB / 512 MiB)
[Correctness] Info: Elapsed: 58.179987 Remaining: 22.737936
[Correctness] Info: Writing... (74.5%, 381 MiB / 512 MiB)
[Correctness] Info: Elapsed: 60.183759 Remaining: 20.599810
[Correctness] Info: Writing... (77.1%, 394 MiB / 512 MiB)
[Correctness] Info: Elapsed: 62.187824 Remaining: 18.470832
[Correctness] Info: Writing... (79.6%, 407 MiB / 512 MiB)
[Correctness] Info: Elapsed: 64.188178 Remaining: 16.450237
[Correctness] Info: Writing... (81.9%, 419 MiB / 512 MiB)
[Correctness] Info: Elapsed: 66.188591 Remaining: 14.627759
[Correctness] Info: Writing... (84.5%, 432 MiB / 512 MiB)
[Correctness] Info: Elapsed: 68.194642 Remaining: 12.509076
[Correctness] Info: Writing... (87.1%, 446 MiB / 512 MiB)
[Correctness] Info: Elapsed: 70.195784 Remaining: 10.396390
[Correctness] Info: Writing... (89.6%, 459 MiB / 512 MiB)
[Correctness] Info: Elapsed: 72.197122 Remaining: 8.380023
[Correctness] Info: Writing... (92.2%, 471 MiB / 512 MiB)
[Correctness] Info: Elapsed: 74.200879 Remaining: 6.277298
[Correctness] Info: Writing... (94.7%, 484 MiB / 512 MiB)
[Correctness] Info: Elapsed: 76.204122 Remaining: 4.264856
[Correctness] Info: Writing... (97.3%, 498 MiB / 512 MiB)
[Correctness] Info: Elapsed: 78.213298 Remaining: 2.170359
[Correctness] Info: Writing... (99.9%, 511 MiB / 512 MiB)
[Correctness] Info: Elapsed: 80.214587 Remaining: 0.080295
[Correctness] Info: Reading... (12.5%, 64 MiB / 512 MiB)
[Correctness] Info: Elapsed: 82.214620 Remaining: 575.502339
[Correctness] Info: Reading... (23.8%, 121 MiB / 512 MiB)
[Correctness] Info: Elapsed: 84.215451 Remaining: 269.630982
[Correctness] Info: Reading... (36.2%, 185 MiB / 512 MiB)
[Correctness] Info: Elapsed: 86.217127 Remaining: 151.951733
[Correctness] Info: Reading... (48.5%, 248 MiB / 512 MiB)
[Correctness] Info: Elapsed: 88.217345 Remaining: 93.674088
[Correctness] Info: Reading... (61.3%, 314 MiB / 512 MiB)
[Correctness] Info: Elapsed: 90.218708 Remaining: 56.956998
[Correctness] Info: Reading... (74.4%, 381 MiB / 512 MiB)
[Correctness] Info: Elapsed: 92.220164 Remaining: 31.731669
[Correctness] Info: Reading... (87.6%, 448 MiB / 512 MiB)
[Correctness] Info: Elapsed: 94.221601 Remaining: 13.337304
[Correctness] Info: Correctness test succeeded.

The write performance is roughly doubled compared to Genode on the NUC but we have to take into account that this test was running on an NVMe SSD. The read performance is far beyond Genode but this is most probably due to Linux' caching. Also since this was run on a file in the file system and the posix implementation uses in this case the block size of the underlying file system the block size was 4k.

[jklmnn]

The same test on Linux on a loop device with 512 instead of 4096 byte blocks looses ca. 15% performance.

[jklmnn]

After fixing two bugs the latency test also works although I won't post all the XML here ;)

[senier]

Gorgeous to see the tests run already on Linux!

What confuses me: What did we use on the NUC if not the SSD for the previous Genode tests?

[jklmnn]

We used a SATA SSD on the NUC but I tested an NVMe SSD on my laptop, that's what I meant.

[senier]

OK, got it. Eventually, we should do the test on the NUC under Linux to get a fair comparison.

[jklmnn]

Fixed by #55.