smol-rs / async-io Goto Github PK
View Code? Open in Web Editor NEWAsync I/O and timers
License: Apache License 2.0
Async I/O and timers
License: Apache License 2.0
Polling 2.6.0 depends on windows-sys 0.45.0, but async-io 1.12.0 depends on windows-sys 0.42.0. Would it be possible to make a new release of async-io, which has already migrated from windows-sys to rustix? That would let us use async-io on some of the new platforms supported by polling
.
Back before this crate was split from smol, there used to be a with
method which would wait for either a readable or writable event.
This method was deprecated even then, and nowadays there is only read_with
and write_with
.
Is there any alternative way to do the equivalent?
My use case is that I am trying to update the async-ssh2 crate, which tries to provide an async wrapper around the libssh2 C library. My fork can be found here.
This C library provides a non-blocking interface. When using this interface, when you make calls to async functions, they may return LIBSSH2_ERROR_EAGAIN
indicating that you should call back again later when the underlying socket is ready. The problem is you don't know if it is waiting for a read or a write - it could be either depending on the state of the protocol.
The original code, which was written against smol 0.1, used the above mentioned 'with' method, and appeared to work. It was doing something like this:
let res = stream
.read(|_s| match cb() {
Ok(v) => Ok(Ok(v)),
Err(e)
if io::Error::from(ssh2::Error::from_errno(e.code())).kind()
== io::ErrorKind::WouldBlock =>
{
Err(io::Error::new(io::ErrorKind::WouldBlock, e))
}
Err(e) => Ok(Err(e)),
})
.await??;
where cb
is a callback that is calling a libssh2 function that may return LIBSSH2_ERROR_EAGAIN (which is converted to an io::Error of kind WouldBlock).
I am trying to update it to depend on async-io
directly. I've replaced the calls to read
with calls to read_with
or read_write
, but it ends up hanging at various points. I suspect the reason is that the C function wants to write from within read_with
or vice versa.
Sorry for the convoluted explanation ... hopefully it makes sense.
Many programs will still be using async-io
v1 due to difficulties involved with porting over to the new API. So to make sure any future correctness/performance improvements are ported from async-io
v2 throughout the ecosystem, v1 should be implemented in terms of v2.
block_on
and Timer
should be trivial to port since they don't have any semver breaking changes; they can just be imported from v2. v1's Async
has to be implemented as a wrapper around v2.
Potential unaligned read
Details | |
---|---|
Status | unsound |
Package | atty |
Version | 0.2.14 |
URL | softprops/atty#50 |
Date | 2021-07-04 |
On windows, atty
dereferences a potentially unaligned pointer.
In practice however, the pointer won't be unaligned unless a custom global allocator is used.
In particular, the System
allocator on windows uses HeapAlloc
, which guarantees a large enough alignment.
A Pull Request with a fix has been provided over a year ago but the maintainer seems to be unreachable.
Last release of atty
was almost 3 years ago.
The below list has not been vetted in any way and may or may not contain alternatives;
See advisory page for additional details.
In most programs that use async-io
, the user will simply run block_on
at the top level. However, this will still spawn the async-io
thread despite it not being necessary. Would it be possible to avoid that?
There is an (undocumented) NtAssociateWaitCompletionPacket syscall that can be used for the purpose.
Would solve smol-rs/smol#94
Because std::time::Instant
is implemented using libc::clock_gettime(CLOCK_MONOTONIC)
and this time does not advance on Android while the app is in "doze" state, the timer may be effectively paused on Android: https://users.rust-lang.org/t/std-now-with-android/41774
One way to fix this is to use SystemTime
instead and handle time occasionally going backwards properly. Another is to lift this issue to the standard library so it uses Android-specific ioctl
on /dev/alarm
as described here: https://users.rust-lang.org/t/std-now-with-android/41774/2
When looking at the Registration
type for unix , I stumbled across the required invariant:
This describes a valid file descriptor that has not been
close
d. It will not be closed while this object is alive.
and it looks a lot to what is guarantee by the BorrowedFd type:
This has a lifetime parameter to tie it to the lifetime of something that owns the file descriptor. For the duration of that lifetime, it is guaranteed that nobody will close the file descriptor.
So, I thought naively this is maybe a valid design that will avoid the unsafe and ensure the invariant is respected:
pub struct Registration<'fd> {
fd: BorrowedFd<'fd>,
}
But I'm sure you are well aware about BorrowdFd, so my question is, why have you decided to use a RawFd
and unsafe instead of a BorrowedFd
?
Windows has two kinds file descriptors: Handle
and Socket
. It seems smol::Async
currently only support Socket
but not yet Handle
, which is needed to abstract over APIs such as named pipes. What is the plan for Handle
-based APIs in smol?
I have a simple TLS proxy which is using futures_lite::io::copy
to copy source to dest and dest to source. The test does 10 iteration of sending message to between client and server. However, only first iteration works, during second iteration, client send seems to be block
Oct 01 04:48:53.968 DEBUG proxy: client: sleep to give server chance to come up
Oct 01 04:48:53.968 DEBUG proxy: server: binding
Oct 01 04:48:53.968 DEBUG proxy: server: successfully binding. waiting for incoming
Oct 01 04:48:54.168 DEBUG proxy: client: trying to connect
Oct 01 04:48:54.177 DEBUG proxy: client: got connection. waiting
Oct 01 04:48:54.177 DEBUG proxy: client: loop 0 sending test message
Oct 01 04:48:54.178 DEBUG proxy: server: new incoming connection
Oct 01 04:48:54.178 DEBUG proxy: server: loop 0, received from client: 8 bytes
Oct 01 04:48:54.178 DEBUG proxy: sever: send back reply: message0reply
Oct 01 04:48:54.178 DEBUG proxy: client: loop 0, received reply back
Oct 01 04:48:54.178 DEBUG proxy: client: loop 1 sending test message
To reproduce this:
git clone https://github.com/infinyon/flv-tls-proxy
cd flv-tls-proxy
RUST_LOG=proxy=debug cargo test test_proxy
In trace mode, log shows following:
TRACE tokio_util::codec::framed_read: frame decoded from buffer
Oct 01 04:52:52.420 DEBUG proxy: client: loop 0, received reply back
Oct 01 04:52:52.420 DEBUG proxy: client: loop 1 sending test message
Oct 01 04:52:52.420 TRACE tokio_util::codec::framed_write: flushing framed transport
Oct 01 04:52:52.420 TRACE tokio_util::codec::framed_write: writing; remaining=8
Oct 01 04:52:52.420 TRACE async_io::driver: main_loop: sleeping for 1000 us
Oct 01 04:52:52.420 TRACE polling::epoll: interest: epoll_fd=3, fd=8, ev=Event { key: 2, readable: true, writable: false }
Oct 01 04:52:52.420 TRACE polling::epoll: new events: epoll_fd=3, res=1
Oct 01 04:52:52.420 TRACE tokio_util::codec::framed_write: framed transport flushed
Oct 01 04:52:52.420 TRACE polling::epoll: interest: epoll_fd=3, fd=4, ev=Event { key: 18446744073709551615, readable: true, writable: false }
Oct 01 04:52:52.420 TRACE tokio_util::codec::framed_read: attempting to decode a frame
Oct 01 04:52:52.420 TRACE async_io::reactor: react: 1 ready wakers
Oct 01 04:52:52.420 TRACE async_io::driver: block_on: stops hogging the reactor
Oct 01 04:52:52.420 TRACE polling: Poller::notify()
Oct 01 04:52:52.420 TRACE polling::epoll: notify: epoll_fd=3, event_fd=4
Oct 01 04:52:52.420 TRACE async_io::driver: main_loop: notified
Oct 01 04:52:52.420 TRACE async_io::driver: block_on: waiting on I/O
Oct 01 04:52:52.420 TRACE async_io::driver: main_loop: sleeping for 50 us
Oct 01 04:52:52.420 TRACE async_io::reactor: process_timers: 0 ready wakers
Oct 01 04:52:52.420 TRACE polling::epoll: interest: epoll_fd=3, fd=7, ev=Event { key: 1, readable: true, writable: false }
Oct 01 04:52:52.420 TRACE polling: Poller::wait(_, None)
Oct 01 04:52:52.420 TRACE async_io::driver: block_on: sleep until notification
Oct 01 04:52:52.420 TRACE polling::epoll: interest: epoll_fd=3, fd=9, ev=Event { key: 3, readable: true, writable: false }
Oct 01 04:52:52.420 TRACE polling::epoll: wait: epoll_fd=3, timeout=None
Oct 01 04:52:52.421 TRACE async_io::driver: main_loop: sleeping for 75 us
Oct 01 04:52:52.421 TRACE polling::epoll: interest: epoll_fd=3, fd=5, ev=Event { key: 18446744073709551615, readable: true, writable: false }
Oct 01 04:52:52.421 TRACE async_io::driver: block_on: sleep until notification
Oct 01 04:52:52.421 TRACE polling::epoll: new events: epoll_fd=3, res=1
Oct 01 04:52:52.421 TRACE polling::epoll: interest: epoll_fd=3, fd=4, ev=Event { key: 18446744073709551615, readable: true, writable: false }
Oct 01 04:52:52.421 TRACE async_io::reactor: process_timers: 0 ready wakers
Oct 01 04:52:52.421 TRACE async_io::driver: main_loop: sleeping for 100 us
Oct 01 04:52:52.421 TRACE async_io::reactor: react: 0 ready wakers
Oct 01 04:52:52.421 TRACE async_io::driver: block_on: stops hogging the reactor
Oct 01 04:52:52.421 TRACE async_io::driver: main_loop: notified
Oct 01 04:52:52.421 TRACE async_io::driver: main_loop: waiting on I/O
Oct 01 04:52:52.421 TRACE async_io::reactor: process_timers: 0 ready wakers
Oct 01 04:52:52.421 TRACE polling: Poller::wait(_, None)
Oct 01 04:52:52.421 TRACE polling::epoll: wait: epoll_fd=3, timeout=None
Oct 01 04:52:52.421 TRACE polling::epoll: interest: epoll_fd=3, fd=5, ev=Event { key: 18446744073709551615, readable: true, writable: false }
I have a project aims to be pluggable into different async executors through FDs/Handles. It works great, I have a PoC working with tokio
. It also works on linux with smol
. Example of how I'm using Async
.
However, on macOS I'm getting ENOTTY
, which stems from Async::new
unconditionally running fcntl
to set O_NONBLOCK, even on FDs that do not support it (like kqueue).
My usecase of Async
is purely to have async-io poll for read-readiness of the FD, and have my task woken up. That's the whole extent of it. Maybe it would make sense to expose a different type just for polling, but the simpler way would be to add an overload for Async::new
that allows making fcntl
call conditional.
There are rare occasions (like when running with threads in different network namespaces during a network simulation) where it would be beneficial to explicitly create an async-io
reactor to make sure that the reactor runs in the correct network namespace
ticks indicate whether event have happened, after record ticks we register event to reactor and return Pending. if event happen immediately, tick will be:
let tick = self
.reactor
.ticker
.fetch_add(1, Ordering::SeqCst)
.wrapping_add(1);
because reactor block on wait, which will cause ticks.0 equal to tick when we wake this task. and we reregister event to reactor and wake again. this time ticks.0 increase 1, and we return Poll::Ready now.
IMO ticks.0 is useless maybe we can just use ticks.1 to indicate whether event has happened. or we can set state[dir].tick = tick + 1
.
I am in the process of moving swayipc from async-std
to async-io
. While running the unit tests I encountered tests failing with io::ErrorKind::WouldBlock
. After re-running the current tests with the latest version of async-std
I encountered the same behaviour (since async-std
moved to async-io
). The unit tests run without failing on async-std
version 1.5 since this is the last version using mio-uds
. I extracted an example on how to reproduce this bug:
use async_io::Async;
use futures_lite::future;
use std::os::unix::net::UnixStream;
use std::thread;
fn main() {
let mut handles = Vec::new();
for _ in 1..100 {
let handle = thread::spawn(|| {
future::block_on(async {
Async::<UnixStream>::connect("pathtosomeunixsocket").await
})
});
handles.push(handle);
}
for handle in handles {
if let Err(error) = handle.join().unwrap() {
dbg!(error);
}
}
}
See async-rs/async-std#1037. We should probably use checked_add
when calculating the instant instead of regular addition.
Same deal as smol-rs/async-net#17, looks like its usage was removed in this commit.
Mostly for tech porn reasons, it's new and shiny. Apparently it's also faster. Not sure if it fits in async-io, because it works very different from epoll from what I understand by reading withoutboats blog.
tempdir
crate has been deprecated; usetempfile
instead
Details | |
---|---|
Status | unmaintained |
Package | tempdir |
Version | 0.3.7 |
URL | rust-lang-deprecated/tempdir#46 |
Date | 2018-02-13 |
The tempdir
crate has been deprecated
and the functionality is merged into tempfile
.
See advisory page for additional details.
I have an issue with async-native-tls together with async-std, and looking into the backtrace, it points to somewhere in this crate.
Our crate (tiberius) runs tests just fine with tokio and async-std 1.6.3, but updating async-std to 1.6.4 or later randomly throws Io(Kind(WouldBlock))
. We have a few hundred tests, and 1-5 of them randomly fail into this error. The error happens when calling connect
from TlsConnector
(async-native-tls 0.3.3) and passing a TcpStream
to it from async-std 1.6.5 (or 1.6.4).
As I said, when passing a TcpStream
from async-std 1.6.3 or a Compat
version from tokio 0.2.x, the errors never happen. Hopefully this is the right crate to create the issue!
Here's a full backtrace from one of the failing tests:
---- transactions_async_std stdout ----
thread 'transactions_async_std' panicked at 'called `Result::unwrap()` on an `Err` value: Ssl(Error { code: ErrorCode(5), cause: Some(Io(Kind(WouldBlock))) }, X509VerifyResult { code: 18, error: "self signed certificate" })', /home/pimeys/code/tiberius/src/client/connection.rs:387:22
stack backtrace:
0: 0x55f6b0a016d5 - backtrace::backtrace::libunwind::trace::h14d338b30b3ea0a7
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
1: 0x55f6b0a016d5 - backtrace::backtrace::trace_unsynchronized::h73ea91d74a3fd67f
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
2: 0x55f6b0a016d5 - std::sys_common::backtrace::_print_fmt::hd42948c952866e12
at src/libstd/sys_common/backtrace.rs:78
3: 0x55f6b0a016d5 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha8f928866ff7571e
at src/libstd/sys_common/backtrace.rs:59
4: 0x55f6b0a2b75c - core::fmt::write::he0c1e5f7426d2718
at src/libcore/fmt/mod.rs:1076
5: 0x55f6b073d675 - std::io::Write::write_fmt::h1152922039931516
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/io/mod.rs:1537
6: 0x55f6b09f9891 - std::io::impls::<impl std::io::Write for alloc::boxed::Box<W>>::write_fmt::ha927d7e5f294bce5
at src/libstd/io/impls.rs:176
7: 0x55f6b0a03e60 - std::sys_common::backtrace::_print::hfc0110703f3696fd
at src/libstd/sys_common/backtrace.rs:62
8: 0x55f6b0a03e60 - std::sys_common::backtrace::print::h3f77c6990ddfaa22
at src/libstd/sys_common/backtrace.rs:49
9: 0x55f6b0a03e60 - std::panicking::default_hook::{{closure}}::heae49580a8d62d75
at src/libstd/panicking.rs:198
10: 0x55f6b0a03b5a - std::panicking::default_hook::hecc34e3f729e213c
at src/libstd/panicking.rs:214
11: 0x55f6b0a044a3 - std::panicking::rust_panic_with_hook::he82f5d0644692441
at src/libstd/panicking.rs:526
12: 0x55f6b0a0409b - rust_begin_unwind
at src/libstd/panicking.rs:437
13: 0x55f6b0a296c1 - core::panicking::panic_fmt::h09c929f06bb87c98
at src/libcore/panicking.rs:85
14: 0x55f6b0a294e3 - core::option::expect_none_failed::h188f17af6c9f404b
at src/libcore/option.rs:1269
15: 0x55f6b00dbbcc - core::result::Result<T,E>::unwrap::h0d451077fbe85e9f
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/result.rs:1005
16: 0x55f6b053bb63 - tiberius::client::connection::Connection<S>::tls_handshake::{{closure}}::h683d4d491de77824
at /home/pimeys/code/tiberius/src/client/connection.rs:384
17: 0x55f6b02a6072 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::hf56e4ae22867706d
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/future/mod.rs:78
18: 0x55f6b0562bc9 - tiberius::client::connection::Connection<S>::connect::{{closure}}::hc5b1351ccc0c3fa2
at /home/pimeys/code/tiberius/src/client/connection.rs:84
19: 0x55f6b02847d2 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1935640bb1256dc9
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/future/mod.rs:78
20: 0x55f6b033b757 - tiberius::client::Client<S>::connect::{{closure}}::h4f4a431e44ae57ad
at /home/pimeys/code/tiberius/src/client.rs:63
21: 0x55f6b028e432 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h59693abadb793728
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/future/mod.rs:78
22: 0x55f6b03bede2 - query::transactions_async_std::{{closure}}::hc7d85b6e3f838d65
at tests/query.rs:50
23: 0x55f6b0285312 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h1c531d3b875327d1
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/future/mod.rs:78
24: 0x55f6b018c94e - <async_std::task::builder::SupportTaskLocals<F> as core::future::future::Future>::poll::{{closure}}::h865bd302a785556b
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/builder.rs:199
25: 0x55f6b01e82c7 - async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::{{closure}}::ha6f4bab892b612bd
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/task_locals_wrapper.rs:60
26: 0x55f6b00c3781 - std::thread::local::LocalKey<T>::try_with::hb661b4d0c7a4f06d
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:263
27: 0x55f6b007d19e - std::thread::local::LocalKey<T>::with::h0c21588fbe2ca350
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:239
28: 0x55f6b01dffef - async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::h16226ade7ae65d25
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/task_locals_wrapper.rs:55
29: 0x55f6b018b28b - <async_std::task::builder::SupportTaskLocals<F> as core::future::future::Future>::poll::h7cd9e5be02a3bc58
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/builder.rs:197
30: 0x55f6b03b140f - <futures_lite::future::Or<F1,F2> as core::future::future::Future>::poll::h0bd82c5f22cb39ad
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-lite-1.8.0/src/future.rs:528
31: 0x55f6b02e8ac8 - async_executor::Executor::run::{{closure}}::hf7027d9be0e1c9d8
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-executor-1.3.0/src/lib.rs:214
32: 0x55f6b0294bb2 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h7f4740670e789cad
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/future/mod.rs:78
33: 0x55f6b02c9e8d - async_executor::LocalExecutor::run::{{closure}}::h3a163a88dc05d212
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-executor-1.3.0/src/lib.rs:393
34: 0x55f6b028a0b2 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h39ca6bd8d2b323b3
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/future/mod.rs:78
35: 0x55f6b0215a95 - async_io::driver::block_on::h517fd1baac558c2d
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-io-1.1.4/src/driver.rs:142
36: 0x55f6b039a373 - async_global_executor::reactor::block_on::ha8fa1557f5aca4b1
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-1.3.0/src/lib.rs:53
37: 0x55f6b02fa439 - async_global_executor::block_on::{{closure}}::ha076dcfa8bc56aa5
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-1.3.0/src/lib.rs:179
38: 0x55f6b00c6d45 - std::thread::local::LocalKey<T>::try_with::hc575102fb44c4620
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:263
39: 0x55f6b0091845 - std::thread::local::LocalKey<T>::with::headb35e05fa41fab
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:239
40: 0x55f6b02f7ee0 - async_global_executor::block_on::h13eb69a060da3402
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-global-executor-1.3.0/src/lib.rs:179
41: 0x55f6b01ab8ab - async_std::task::builder::Builder::blocking::{{closure}}::{{closure}}::h8ce56e42b4a651ce
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/builder.rs:171
42: 0x55f6b01e6f9a - async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::{{closure}}::h7e827a3b9f989c97
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/task_locals_wrapper.rs:60
43: 0x55f6b009cea1 - std::thread::local::LocalKey<T>::try_with::h2b7cd734ad2bfc51
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:263
44: 0x55f6b007e8c5 - std::thread::local::LocalKey<T>::with::h1b5335748a459c1e
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:239
45: 0x55f6b01e0cb5 - async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::h52e3c6d593cdc45f
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/task_locals_wrapper.rs:55
46: 0x55f6b01a8467 - async_std::task::builder::Builder::blocking::{{closure}}::hefc8a66421fbf332
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/builder.rs:168
47: 0x55f6b00d69cd - std::thread::local::LocalKey<T>::try_with::hfe7cb36aa29f4083
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:263
48: 0x55f6b0086eb5 - std::thread::local::LocalKey<T>::with::h6fb1471021bc304a
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/local.rs:239
49: 0x55f6b019b72c - async_std::task::builder::Builder::blocking::h6b201419829dfbb8
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/builder.rs:161
50: 0x55f6b0201b27 - async_std::task::block_on::block_on::h17621ea76d951fe8
at /home/pimeys/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.6.5/src/task/block_on.rs:33
51: 0x55f6b03283c2 - query::transactions_async_std::h891bf3802642b022
at tests/query.rs:50
52: 0x55f6b03be5a1 - query::transactions_async_std::{{closure}}::h17ead2905b73b6af
at tests/query.rs:50
53: 0x55f6b04e0e7e - core::ops::function::FnOnce::call_once::he8d8373c6c2f303b
at /home/pimeys/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:233
54: 0x55f6b0764bec - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h6633cb15d0d76942
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/boxed.rs:1081
55: 0x55f6b0764bec - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1e20120def172c5c
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:318
56: 0x55f6b0764bec - std::panicking::try::do_call::hcc2ec3bbb75e9316
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:348
57: 0x55f6b0764bec - std::panicking::try::h94eaebaaa7dd6f41
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:325
58: 0x55f6b0764bec - std::panic::catch_unwind::h151c07c08497cf8b
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:394
59: 0x55f6b0764bec - test::run_test_in_process::hd082de93b1922c89
at src/libtest/lib.rs:541
60: 0x55f6b0764bec - test::run_test::run_test_inner::{{closure}}::h22369c9424e5ab3a
at src/libtest/lib.rs:450
61: 0x55f6b073cb66 - std::sys_common::backtrace::__rust_begin_short_backtrace::h0660a89f67243e05
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/sys_common/backtrace.rs:130
62: 0x55f6b0741c15 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::h8bb2049509aa1add
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/thread/mod.rs:475
63: 0x55f6b0741c15 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb49f3484cb9c3dd3
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:318
64: 0x55f6b0741c15 - std::panicking::try::do_call::ha910a12d1577339b
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:348
65: 0x55f6b0741c15 - std::panicking::try::hd8b3d620360e55fa
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panicking.rs:325
66: 0x55f6b0741c15 - std::panic::catch_unwind::h475454730ea43154
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/panic.rs:394
67: 0x55f6b0741c15 - std::thread::Builder::spawn_unchecked::{{closure}}::h2407d9379d805151
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libstd/thread/mod.rs:474
68: 0x55f6b0741c15 - core::ops::function::FnOnce::call_once{{vtable.shim}}::haef9f772dfab225e
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/ops/function.rs:233
69: 0x55f6b0a0b1aa - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::hd2b3bc04af94a84f
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/boxed.rs:1081
70: 0x55f6b0a0b1aa - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h1044417e186e567a
at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/liballoc/boxed.rs:1081
71: 0x55f6b0a0b1aa - std::sys::unix::thread::Thread::new::thread_start::h276e6ca033938925
at src/libstd/sys/unix/thread.rs:87
72: 0x7fd8698f3609 - start_thread
at /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477
73: 0x7fd8697fd293 - __clone
74: 0x0 - <unknown>
The library internally already uses the socket2
crate. It would be cool if this functionality was also exposed in a convenient way as part of the public API. For example, I'd like to have an impl Async<socket2::Socket>
that provides methods like bind
etc.
serde_cbor is unmaintained
Details | |
---|---|
Status | unmaintained |
Package | serde_cbor |
Version | 0.11.2 |
URL | https://github.com/pyfisch/cbor |
Date | 2021-08-15 |
The serde_cbor
crate is unmaintained. The author has archived the github repository.
Alternatives proposed by the author:
See advisory page for additional details.
I am very fond of async-io and use it in many projects. However tokio users aren't very happy with the additional thread. Is there any way that we can wrap the tokio evented trait or whatever it's called these days and add a tokio feature flag?
I ran into the problem that rustix panics when using async-io inside a 32-bit Linux sandbox with no /proc
available. Turns out that the default feature use-libc-auxv
of rustix would avoid that. Is the feature intentionally disabled?
EPOLLERR
Error condition happened on the associated file descriptor.
This event is also reported for the write end of a pipe when
the read end has been closed.
epoll_wait(2) will always report for this event; it is not
necessary to set it in events when calling epoll_ctl().
I was reading the implementation behind read_with
and I was wondering - wouldn't the loop
in the method potentially cause the op
, such as l.accept()
to be called multiple times? Are all methods fed to read_with
meant to be idempotent?
/// # Examples
///
/// ```no_run
/// use async_io::Async;
/// use std::net::TcpListener;
///
/// # futures_lite::future::block_on(async {
/// let listener = Async::<TcpListener>::bind(([127, 0, 0, 1], 0))?;
///
/// // Accept a new client asynchronously.
/// let (stream, addr) = listener.read_with(|l| l.accept()).await?;
/// # std::io::Result::Ok(()) });
/// ```
pub async fn read_with<R>(&self, op: impl FnMut(&T) -> io::Result<R>) -> io::Result<R> {
let mut op = op;
loop {
match op(self.get_ref()) {
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {}
res => return res,
}
optimistic(self.readable()).await?;
}
}
I do notice that methods like accept
, when set to nonblocking
won't wait for a new TCP connection to be established:
#[test]
fn test_accept() {
let l = std::net::TcpListener::bind("127.0.0.1:8080").unwrap();
l.set_nonblocking(true).unwrap();
loop {
let res = l.accept();
match res {
Ok(_) => break,
Err(_) => {}
}
}
println!("Finished");
}
How about a default Timer::new()
constructor that makes a timer that never fires? Of course, that timer would be possible to update with set_at()
, set_after()
and so on.
I'm inclined to just add this constructor (and a Default
impl for Timer
) but am wondering if anyone has objections?
https://cirrus-ci.com/task/5088564308869120
---- tcp_connect stdout ----
thread 'tcp_connect' panicked at 'assertion failed: `(left == right)`
left: `TimedOut`,
right: `ConnectionRefused`', tests/async.rs:57:9
stack backtrace:
0: rust_begin_unwind
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:575:5
1: core::panicking::panic_fmt
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/panicking.rs:65:14
2: core::panicking::assert_failed_inner
3: core::panicking::assert_failed
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/panicking.rs:203:5
4: async::tcp_connect::{{closure}}
at ./tests/async.rs:57:9
5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
6: futures_lite::future::block_on::{{closure}}
at /.cargo/registry/src/github.com-1ecc6299db9ec823/futures-lite-1.12.0/src/future.rs:89:27
7: std::thread::local::LocalKey<T>::try_with
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/thread/local.rs:446:16
8: std::thread::local::LocalKey<T>::with
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/thread/local.rs:422:9
9: futures_lite::future::block_on
at /.cargo/registry/src/github.com-1ecc6299db9ec823/futures-lite-1.12.0/src/future.rs:79:5
10: async::tcp_connect
at ./tests/async.rs:38:5
11: async::tcp_connect::{{closure}}
at ./tests/async.rs:37:1
12: core::ops::function::FnOnce::call_once
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:251:5
13: core::ops::function::FnOnce::call_once
at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:251:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
failures:
tcp_connect
test result: FAILED. 12 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 75.10s
error: test failed, to rerun pass `--test async`
It should be replaced by an object that implements Wake
.
Currently, most methods async_io::Async
requires the type it wraps to implement AsRawFd
, or AsRawSocket
on Windows.
The standard library now supports the safe alternative AsFd
(or AsSocket
on Windows), and async-io has some support for this. However, it doesn't seem possible to create an Async<T>
for a type that implements AsFd
but not AsRawFd
.
I have types I'd love to use Async
with, and I'd like to drop the implementation of AsRawFd
for those types. (This would also eliminate unsafe
code.)
Please consider adding a constructor for Async<T>
where T: AsFd
.
They can improve robustness under certain conditions, as they avoid late failures
It appears that Async::<TcpListener>::accept
may have a bug on OS X where it can miss connection attempts. I was able to reduce this bug down to this:
#[test]
fn tcp_connect_bug() {
for i in 1.. {
println!("attempt {i}");
future::block_on(async {
let listener = Async::<TcpListener>::bind(([127, 0, 0, 1], 8081)).unwrap();
let addr = listener.get_ref().local_addr().unwrap();
let task_client = spawn(async move { Async::<TcpStream>::connect(addr).await });
let task_server = spawn(async move { listener.accept().await });
let stream_client = task_client.await.unwrap();
let stream_server = task_server.await.unwrap().0;
let stream_server = listener.accept().await.unwrap().0;
let stream_client = task_client.await.unwrap();
assert_eq!(
stream_client.get_ref().peer_addr().unwrap(),
stream_server.get_ref().local_addr().unwrap(),
);
assert_eq!(
stream_client.get_ref().peer_addr().unwrap(),
stream_server.get_ref().local_addr().unwrap(),
);
})
}
}
On my OS X laptop, it will succeeds for about approximately ~16000 loops before I get:
...
attempt 16277
thread 'tcp_connect_bug' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 60, kind: TimedOut, message: "Operation timed out" }', tests/async.rs:48:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test tcp_connect_bug ... FAILED
I don't get any time out running this on a Linux machine after ~100000ish cycles.
Right now, inside read_with()
/write_with()
we aways optimistically attempt a read/write operation. This can be costly in loops that select!
over I/O operations.
However, we know whether the resource is ready simply by looking at wakers. If there are wakers waiting for the I/O handle to become readable/writable, we can skip the optimistic attempt.
tokio
uses cooperative task yielding to improve tail latencies in the circumstance that a resource polled in a loop fails to yield, leading to starvation of other tasks. Something similar could be set up for async-io
as well.
The Reactor
already has a concept of "ticks", which are used to coordinate polling. My idea would be that each Source
keeps track of the tick which it last returned Poll::Pending
. If it compares tick
and this theoretical last_pending_tick
value and finds that the difference between the two is a large value (say, 10_000
, but this could use some tweaking), pooling the Source
automatically returns Pending
.
Any thoughts on this? This could be a global counter as well, or maybe a thread-local one.
I would be willing to implement this if the functionality is desired.
since rust doesn't support self referential structs there needs to be an alternative to async fn next(&self)
. Since self isn't owned there is no way to get it into a struct so that it can be polled.
Calloop recently found a bug where it's possible to deregister a source by trying to register it twice. See Smithay/calloop#153 for more information.
I don't think async-io
is affected by this bug, but we should add a test case to make sure.
Line 39 in 823d6f1
Related: smol-rs/async-process#27, smol-rs/polling#46
This 'regression' appears a while ago in criner
, and apparently went unnoticed when switching from tokio::time::Delay
to async_io::Timer
.
The picture above shows a criner
process doing nothing, consuming 10 to 15% of CPU for displaying 4 progress bars which update once per second. The expensive part of this used to be the drawing itself, but now it appears to be dominated by the condvar in parking
.
In smol
0.1.5 using smol::Timer
, this looked like that:
All that shows up is the drawing of the TUI.
Having had a look myself I am absolutely puzzled as to why that is - the code looks very similar.
Any hint at how to solve this would be greatly appreciated.
Thank you
criner
didn't change in the relevant portions of the code responsible for the timer invocations see utils.rs.I actually thought about proposing this while investigating #18 and now that async-io
is at v1.0.x I am not sure if a breaking change directly after v1.0 would be appreciated.
That said I believe it would be a good idea to move all the net related impls meaning impl Async<TcpListener>
, impl Async<TcpStream>
and so on to async-net
. That way we can remove the socket2
dependency from async-io
and it would reduce the overall code base. I already looked into this a bit and one obvious downside would be that the current doc tests have to be rewritten since they mostly 12/20 use net related types. Feel free to close this issue if you don't think this is a good idea or if I have simply missed something and it's not possible to implement it that way ๐
I was going though a libp2p tutorial and found that I couldn't reliably run libp2p-lookup against libp2p-relay_v2. The libp2p-lookup would work the first time I ran libp2p-relay_v2 after a reboot. But libp2p-lookup would fail if I stopped and restarted libp2p-relay_v2. Here is a discussion on libp2p repo. Initially I thought it was a performance problem as debug builds could fail but release builds worked. Also, I didn't see the problem on my desktop machine only on a very small Digital Ocean VM with 1CPU and 1GB ram. Also, recently I tested a tokio build of libp2p-relay_v2 and it worked fine.
By using RUST_LOG=trace
the "timeout" symptom would "always" fail and I started adding copious amounts of logging trying to narrow down the problem. To shorten the story, the problem appears to be that async-io can sometimes miss waker.wake
events and is resolved by making a one line change to async-io in Source::poll_ready.
I change:
state[dir].ticks = Some((Reactor::get().ticker(), state[dir].tick));
to:
state[dir].ticks = Some((state[dir].tick, 0)
My analysis of the problem is that using the current Reactor::ticker
value in ticks
is not always correct. It is my understanding that when placing a ticker value into Direction::ticks
there is a guarantee that the waker.wake call has completed and the associated task has been run. The problem is the increment of the ticker value is in ReactorLock::react
is typically running on a different CPU thread than the Source::poll_ready
(called by poll_readable
, poll_writeable
), which is saving the ticker value in ticks as show in the above code snippet.
Because the increment and use of ticker value are on different threads the required guarantee can not be full filled and I'm proposing the above fix which only uses the a current state[dir].tick
and not the ticker value. Another possible solution is to set state[dir].ticks = None
but the proposed solution is a "smaller" change so I've chosen it.
Of course I fully expect async-io experts may provide other solutions or even identify that problem as something else all together. But, at the moment, this solution does resolve the problem.
it seems that TcpStream
return by accept
is not in noblocking mode.
I would like to watch Mach ports for new messages using smol.
There are basically two ways to go about watching a Mach port. One, which would work on any Mach system, is to have the reactor use Mach port sets in place of current epoll/kqueue.
Another way, which only works on Darwin, is to use EVFILT_MACHPORT
with kqueue. This could then be exposed in Async
with a cfg
(e.g. Async::mach_port(...)
).
The source of async-std has the following snippet:
impl<'a> Stream for Incoming<'a> {
type Item = io::Result<TcpStream>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let future = self.0.accept();
pin_utils::pin_mut!(future);
let (socket, _) = futures_core::ready!(future.poll(cx))?;
Poll::Ready(Some(Ok(socket)))
}
}
In the latest release of async-io, the accept future was changed to release interest when the future is dropped, which breaks the above implementation. And as far as I can see, there's no poll_*
based alternative for waiting for read-readiness on the Async
type.
Originally reported to async-std as async-rs/async-std#888
WASM targets don't support Async
and probably won't until WASI reaches a more stable point. However, since parking
is usable on WASM thanks to atomics, it may be desirable to use the Timer
implementation.
My suggestion is to add two new features: io
and timers
. They compose as such:
io
disabled but timers
enabled, the Reactor
retains its timer processing capabilities but instead of calling to polling
, it calls parking
(which works with WASM) with the timeout resulting from process_timer_ops
.Reactor
no longer exists and block_on
is just an alias to the ones in futures_lite
.This would be a breaking change.
It would be great if Timer
was not just a Future
but also a Stream
so that it could act as a "ticker" delivering events at regular intervals.
Proposed API:
impl Timer {
fn interval(period: Duration) -> Timer;
fn interval_at(start: Instant, period: Duration) -> Timer;
fn set_interval(&mut self, period: Duration);
fn set_interval_at(&mut self, start: Instant, period: Duration);
}
impl Stream for Timer {
type Item = Instant;
}
Thoughts or suggestions?
With the new release of once_cell
1.15, the MSRV had been increased to 1.56 and the edition has been bumped to 2021. This means async-io
, which depends on once_cell
, no longer builds on 1.46:
$ cargo +1.46 check
warning: unused manifest key: package.rust-version
Updating crates.io index
Downloaded once_cell v1.15.0
error: failed to parse manifest at `/home/jtnunley/.cargo/registry/src/github.com-1ecc6299db9ec823/once_cell-1.15.0/Cargo.toml`
Caused by:
failed to parse the `edition` key
Caused by:
this version of Cargo is older than the `2021` edition, and only supports `2015` and `2018` editions.
I see two options:
once_cell
version to 1.14.x
. This would keep our MSRV at 1.46, but prevent us from receiving any new once_cell
updates.See also: matklad/once_cell#201
Since async-std switched to smol I'm unable to compile my project in the stable-gnu toolchain. It's working perfectly in stable-msvc as well as native linux compilation.
I'm getting the following error:
error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
And multiple missing functions like this:
In function `smol::reactor::sys::Reactor::new::h4fc4018e596093ae':
...\src\github.com-1ecc6299db9ec823\smol-0.1.18\src/reactor.rs:741: undefined reference to `epoll_create1'
For the functions epoll_create1
, epoll_ctl
, and epoll_wait
.
I'm using the default compiler that comes with installing the toolchain with rustup. Maybe I'm missing a library?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.