Description
Modifier keys don't really work like they should on MacOS. Maybe this is only an issue with MacOS Monterey, I don't think it is though. I can't really "describe" the bug in detail because I have no clue what's really going on. All I know is that using toggleKey
and then type
with a single letter really just inserts that letter as text and autopilot will completely ignore that modifier keys are ignored. However, I can compare autopilot's behaviour with cliclick's.
Reproduce
- Go to https://w3c.github.io/uievents/tools/key-event-viewer.html
- Run this:
deno repl --no-check --unstable
- Then enter this and press enter:
import AutoPilot from "https://deno.land/x/autopilot/mod.ts";
const sleep = async (milliseconds) => new Promise(resolve => setTimeout(resolve, milliseconds));
await sleep(5000);
await pilot.toggleKey("meta", true);
await pilot.toggleKey("shift", true);
await pilot.type("g");
await pilot.toggleKey("meta", false);
await pilot.toggleKey("shift", false);
- You now have five seconds to switch to your browser again and click on that input field.
- This is the result:
- Install cliclick and run this, then click on the input field on the website again:
cliclick w:5000 kd:cmd,shift t:g ku:cmd,shift
- This is the result:
Expected Behaviour
While modifier keys somehow work individually, they don't work like expected in most cases I experienced. For example, all hotkey combinations that include a modifier key and any ordinary letter, like the typical ⌘+C to copy, just don't work in my experience. To give you one more example, on MacOS, whenever an "open file" dialog is present, you can use ⌘+⇧+G to open a textbox allowing you to enter the path to a folder. Using cliclick for this just works, while autopilot doesn't, and instead just presses G.
Desktop
- OS: MacOS Montery 12.1 Beta (21C5045a)
- Browser Chrome
- Version 0.4.0
Additional context
Going through the issues of this repository, I assume the root of this issue going unnoticed is toggleKey
only allowing specific keys for some reason. Looks like up until version 0.4.0, hotkey stuff did work, by using toggleKey
. Note that I haven't found any evidence of the combination toggleKey(modifier, true); type(letter); toggleKey(modifier, false);
ever working. If one uses ordinary letters with toggleKey
today, everything crashes and rust panicks:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error("unknown variant `c`, expected one of `f1`, `f2`, `f3`, `f4`, `f5`, `f6`, `f7`, `f8`, `f9`, `f10`, `f11`, `f12`, `f13`, `f14`, `f15`, `f16`, `f17`, `f18`, `f19`, `f20`, `f21`, `f22`, `f23`, `f24`, `leftarrow`, `uparrow`, `rightarrow`, `downarrow`, `insert`, `delete`, `home`, `end`, `pageup`, `pagedown`, `tab`, `backspace`, `enter`, `escape`, `space`, `meta`, `alt`, `control`, `shift`, `capslock`", line: 1, column: 10)', src/lib.rs:122:1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
Backtrace with RUST_BACKTRACE=1
:
stack backtrace:
0: _rust_begin_unwind
1: core::panicking::panic_fmt
2: core::result::unwrap_failed
3: _toggle_key
4: _ffi_call_unix64
5: _ffi_call_int
6: deno_ffi::ffi_call
7: deno_core::ops_json::op_sync::{{closure}}
8: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
9: deno_core::ops::OpTable::route_op
10: <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn
11: __ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE
12: __ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE
13: __ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
fatal runtime error: failed to initiate panic, error 5
Backtrace with RUST_BACKTRACE=full
:
stack backtrace:
0: 0x1144675d1 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hec762383110a7685
1: 0x11447f3bb - core::fmt::write::h2d5ecb4b9764759c
2: 0x114465c0a - std::io::Write::write_fmt::h9d7d3ae333151289
3: 0x114468805 - std::panicking::default_hook::{{closure}}::h552de0233eed7dab
4: 0x1144683ef - std::panicking::default_hook::hf4e8e1e5a5c43b90
5: 0x114468ef0 - std::panicking::rust_panic_with_hook::h7c7e0153f3e14d6b
6: 0x11446898e - std::panicking::begin_panic_handler::{{closure}}::h3c7f7ffd2b05c635
7: 0x114467a47 - std::sys_common::backtrace::__rust_end_short_backtrace::h3bdf4f6c89eee6ea
8: 0x1144688fa - _rust_begin_unwind
9: 0x114489d6f - core::panicking::panic_fmt::h88a1b6fbb9084d2c
10: 0x114489e55 - core::result::unwrap_failed::h8abe20a396a5e82a
11: 0x114403a35 - _toggle_key
12: 0x1100571e2 - _ffi_call_unix64
13: 0x110056c35 - _ffi_call_int
14: 0x10f6eff40 - deno_ffi::ffi_call::h7b8d00a17ac9084e
15: 0x10f9613b3 - deno_core::ops_json::op_sync::{{closure}}::h2dff410c79672602
16: 0x10f67b81f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd8d69a3a2de8f014
17: 0x10f67074c - deno_core::ops::OpTable::route_op::h86ef2e6087baaab6
18: 0x10f682a8c - <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn::hfa3b6c5b5cdbaae7
19: 0x1101345d5 - __ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE
20: 0x110133bc1 - __ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE
21: 0x11013312e - __ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE
fatal runtime error: failed to initiate panic, error 5