stevepryde / thirtyfour Goto Github PK
View Code? Open in Web Editor NEWSelenium WebDriver client for Rust, for automated testing of websites
License: Other
Selenium WebDriver client for Rust, for automated testing of websites
License: Other
What do you think about adding this list of functions?
It useful to have these with wait_for_*
type functions. To not have an explicit sleep
. Which speed up general testing scripts. And make them more reliable.
Honestly I've spend a few days playing with this in order to implement a month ago. And it seems like not an trivial task at all ๐
I've also gone through the selenium
s libraries code base to check how they implemented it. I expected to see a simple js
script which does a check or some check by driver functions itself.
As far as I got lucky to understand they reuse the same javascript
files in all libraries (They call these files atoms
).
To sum up I post this issues just to hear from you what do you think about such functionality. And if you think it appropriate the best way to have it done.
An example of display property which is an equivalent of is_displayed
Hey!
Let me begin by saying this is a wonderful library and I'm having a great time developing with it.
I have just one question, could you please post an example using with_filter
from the ElementQuery interface?
I'm having a hard time getting my head around it ...
Cheers!
This is the api of selenium, hope this library can achieve this function: execute_cdp_cmd
This function is very useful and can be used to limit the operation of the webdriver, For example, the following example is:
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
"source": """
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined
})
"""
})
Thanks !
Hi,
I'm facing the following error while trying to start a new WebDriver session (WebDriver::new(selenium_server_location, &caps).await?;
) with a local standalone Chrome selenium docker container (docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome:85.0
):
Error: UnknownError(WebDriverErrorInfo { status: 500, error: "", value: WebDriverErrorValue { message: "Content-Type header does not indicate utf-8 encoded json: application/json", error: Some("unknown error"), stacktrace: Some(""), data: None } })
The selenium_server_location
is http://localhost:4444/wd/hub
.
The issue is not present if I run the chromedriver
locally with chromedriver --port=4444
.
I tried to supply RUST_BACKTRACE=1
but I didn't get any additional info about the error.
P.S: do not hesitate to ask for any kind of additional info ๐
Hi I am using this with geckodriver which is connected to my already running instance of firefox.
When I tried to run the window_handles()
function it failed with the following error and crashed firefox (but not geckodriver):
Error while executing action: Unknown error:
Status: 500
Additional info:
TypeError: browser.browsingContext is null
Error: unknown error
Stacktrace:
GeckoDriver.prototype.getIdForBrowser@chrome://marionette/content/driver.js:1398:15
get@chrome://marionette/content/driver.js:239:28
GeckoDriver.prototype.getWindowHandles@chrome://marionette/content/driver.js:1443:3
despatch@chrome://marionette/content/server.js:297:40
execute@chrome://marionette/content/server.js:267:16
onPacket/<@chrome://marionette/content/server.js:240:20
onPacket@chrome://marionette/content/server.js:241:9
_onJSONObjectReady/<@chrome://marionette/content/transport.js:504:2
When it comes to querying elements, the built-in WebDriver element selectors are good but often we need more advanced functionality, for example polling or filtering based on some predicate.
In my experience I have found it better to turn off implicit waits and implement polling separately outside of selenium. This provides far greater control over element queries, including powerful any/all filtering or polling multiple selectors in parallel and returning whichever element is found first (useful when CSS has changed between old/new versions of a website).
This also allows for different retry criteria. You might wish to retry a set number of times regardless of how long it takes, or you might only care about retrying within a set duration, or you might want some combination of the two (to avoid the scenario where a slow network means you never actually retried due to the first query taking too long).
One possibility is a functional approach, which should allow for a composable queries. Something like this (none of this has been tested):
fn poll_query<F>(timeout: Duration, interval: Duration, f: F) -> F
where
F: Fn() -> WebDriverResult<Vec<WebElement>>
{
return || {
let now = Instant::now();
while (now.elapsed() < timeout) {
match f() {
Ok(x) => return Ok(x),
Err(WebDriverError::NoSuchElement(_)) => break,
Err(e) => return Err(e)
}
sleep(interval);
}
// Always do one last find.
f()
}
}
let found = poll_query(Duration::new(10, 0), Duration::new(0, 200), || e.find_elements(By::Tag("sometag")))?;
Another possibility is a builder approach, which would allow something like this:
let found = ElementQuery(driver).poll(Duration::new(10, 0), Duration::new(0, 200)).find(By::Tag("sometag"))?;
In both cases you could reduce the code required by having default timeout/interval that can be configured/overridden as needed.
So that would shorten to something like:
Functional version:
let found = poll_query(|| e.find_elements(By::Tag("sometag")))?;
Builder version:
let found = ElementQuery(driver).poll().find(By::Tag("sometag"))?;
Both would allow further expansion using functions such as:
find_any(Vec<By>)
- return the first one that is non-empty - if nested below poll() this will execute both queries once per poll iteration.
with_retry(num_tries: u32)
- retry the query a specified number of times if not found, rather than using a fixed timeout.
poll_missing()
- poll until the query returns an empty vec.
Also since these return a Vec you can easily iterate and filter as needed as well. The filtering could even be applied at any stage of the pipeline (easier in the functional version).
Personally I prefer the functional version since it allows for easier composition and it's easier to add your own functions. There are some pros and cons to each pattern - and I'm not opposed to supporting both (the builder pattern would simply wrap the functional pattern).
All of this will go in a separate crate at least until it is deemed stable. That allows for more experimentation and rapid version bumps without affecting this crate, which by now should be looking at stabilization as much as possible.
Hi,
is there a way to check if a element on a page exists. Currently I'm using:
async fn html_contains_any_element_webdriver(driver: &WebDriver, url: &str, elements: Vec<&str>) -> Result<bool> {
driver.get(url).await?;
for element in elements.into_iter() {
let filtered_elements = driver.find_element(By::XPath(element)).await;
if filtered_elements.is_ok() {
return Ok(true);
}
}
Ok(false)
}
But this just returns after 30 seconds.
Why does the crate needs 30 seconds to check if the current html contains an given element? (30 seconds for each element)
T
Steps to reproduce:
(1) Run docker run --rm -d --network host --name selenium-server -v /dev/shm:/dev/shm selenium/standalone-chrome
(2) Run the example (async or sync)
Output:
Error: SessionNotCreated(WebDriverErrorInfo { status: 500, error: "", value: WebDriverErrorValue { message: "Unable to create new service: ChromeDriverService\nBuild info: version: \'3.141.59\', revision: \'e82be7d358\', time: \'2018-11-14T08:25:53\'\nSystem info: host: \'dsternlicht\', ip: \'172.16.10.252\', os.name: \'Linux\', os.arch: \'amd64\', os.version: \'5.4.15-arch1-1\', java.version: \'1.8.0_232\'\nDriver info: driver.version: unknown", error: Some("session not created"), stacktrace: Some("org.openqa.selenium.SessionNotCreatedException: Unable to create new service: ChromeDriverService\nBuild info: version: \'3.141.59\', revision: \'e82be7d358\', time: \'2018-11-14T08:25:53\'\nSystem info: host: \'dsternlicht\', ip: \'172.16.10.252\', os.name: \'Linux\', os.arch: \'amd64\', os.version: \'5.4.15-arch1-1\', java.version: \'1.8.0_232\'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.session.remote.ServicedSession$Factory.lambda$get$0(ServicedSession.java:135)\n\tat org.openqa.selenium.grid.session.remote.ServicedSession$Factory.apply(ServicedSession.java:152)\n\tat org.openqa.selenium.remote.server.ActiveSessionFactory.lambda$apply$12(ActiveSessionFactory.java:180)\n\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)\n\tat java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:440)\n\tat java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)\n\tat java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)\n\tat java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)\n\tat java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)\n\tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)\n\tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)\n\tat java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)\n\tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)\n\tat java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531)\n\tat org.openqa.selenium.remote.server.ActiveSessionFactory.apply(ActiveSessionFactory.java:183)\n\tat org.openqa.selenium.remote.server.NewSessionPipeline.lambda$null$2(NewSessionPipeline.java:66)\n\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)\n\tat java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)\n\tat java.util.Collections$2.tryAdvance(Collections.java:4719)\n\tat java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)\n\tat java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)\n\tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)\n\tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)\n\tat java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)\n\tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)\n\tat java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531)\n\tat org.openqa.selenium.remote.server.NewSessionPipeline.lambda$createNewSession$3(NewSessionPipeline.java:69)\n\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)\n\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)\n\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)\n\tat java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:175)\n\tat java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)\n\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)\n\tat java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)\n\tat java.util.stream.Streams$StreamBuilderImpl.tryAdvance(Streams.java:405)\n\tat java.util.stream.Streams$ConcatSpliterator.tryAdvance(Streams.java:728)\n\tat java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)\n\tat java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)\n\tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)\n\tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)\n\tat java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)\n\tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)\n\tat java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531)\n\tat org.openqa.selenium.remote.server.NewSessionPipeline.createNewSession(NewSessionPipeline.java:72)\n\tat org.openqa.selenium.remote.server.commandhandler.BeginSession.execute(BeginSession.java:65)\n\tat org.openqa.selenium.remote.server.WebDriverServlet.lambda$handle$0(WebDriverServlet.java:235)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.openqa.selenium.grid.session.remote.ServicedSession$Factory.lambda$get$0(ServicedSession.java:131)\n\t... 50 more\nCaused by: java.lang.RuntimeException: Unable to find a free port\n\tat org.openqa.selenium.net.PortProber.findFreePort(PortProber.java:67)\n\tat org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:351)\n\tat org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)\n\t... 55 more\n"), data: None } })
In GenericWebDriver::drop
implementation futures::executor::block_on
never returns if it run inside tokio
runtime.
Specifically if driver is dropped inside a tokio::task
.
Not sure if it's caused only by a usage of a different executor.
use thirtyfour::prelude::*;
let driver = WebDriver::new("http://localhost:4444", &DesiredCapabilities::firefox()).await?;
tokio::spawn(async move {
println!("START");
driver.close().await.unwrap();
println!("FINISH");
}).await;
You will see START
and FINISH
in output but it will never end the spawned task
I spend a good number of hours to locate the issue ๐
Please read this: #Firefox is set but using chrome
When the Selenium Endpoint does not return a valid JSON object but instead throws some kind of error (e.g. 502 Bad Gateway
) that error code and the response body is not part of the thrown error. Instead, it contains null
:
The WebDriver server returned an unrecognised response: Server returned unknown response: null
Here is the corresponding HTTP response captured with Wireshark:
HTTP/1.1 502 Bad Gateway
Content-Length: 148
Date: Fri, 07 May 2021 07:42:20 GMT
Content-Type: text/plain; charset=utf-8
Unable to forward request: error trying to connect: tcp connect error: Operation timed out (os error 110)
It would be neat if that error message and code would be propagated instead of discarded ๐
Hi Guys,
I found several scenerios where the geckdodriver becomes a state, where I can't create a new connection.
These cases I found:
In the cases after the application restarts you get these exception:
2021-04-23 18:31:30,333 INFO [app] Connect to WebDriver
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: SessionNotCreated(WebDriverErrorInfo { status: 500, error: "", value: WebDriverErrorValue { message: "Session is already started", error: Some("session not created"), stacktrace: Some(""), data: None } })', src\main.rs:20:80
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\item_tracker.exe ./` (exit code: 101)
T
hi all, who can give code example use Capabilities & ChromeOptions with thirtyfour same here:
https://chromedriver.chromium.org/capabilities
ex:
Ruby
caps = Selenium::WebDriver::Remote::Capabilities.chrome("chromeOptions" => {"args" => [ "--disable-web-security" ]})
driver = Selenium::WebDriver.for :remote, url: 'http://localhost:4444/wd/hub' desired_capabilities: caps
thank you so much.
WebDriverSession is not public, but it's a part of WebDriverCommands interface. This prohibits for from implementing their own WebDrivers.
Why do I need this in the first place? The GenericWebDriver doesn't accept existing client connection but always creates a new one. This with combination of HTTP/1 restrictions on chromedriver prohibits users from scaling for more than 5 parallel sessions.
So there are to solutions to this:
In both cases I would be happy to help
Hi Guys,
this lines causes an panic:
let mut filtered_needed_elements = FuturesUnordered::new();
for element in needed_elements.into_iter() {
filtered_needed_elements.push(driver.find_element(By::XPath(element)));
}
while let Some(filtered_needed_element) = filtered_needed_elements.next().await {
if filtered_needed_element.is_ok() {
return Ok(false);
}
}
This panic occurs somewhen after executing this lines.
Here the stacktrace:
thread 'tokio-runtime-worker' panicked at 'Failed to send response: Err(NoSuchElement(WebDriverErrorInfo { status: 404, error: "", value: WebDriverErrorValue { message: "Unable to locate element: //*[@id=\"buy-now-button\"]", error: Some("no such element"), stacktrace: Some("WebDriverError@chrome://marionette/content/error.js:181:5\nNoSuchElementError@chrome://marionette/content/error.js:393:5\nelement.find/</<@chrome://marionette/content/element.js:306:16\n"), data: None } }))', C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\thirtyfour-0.23.0\src\session.rs:47:30
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:493
1: core::panicking::panic_fmt
at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\core\src\panicking.rs:92
2: core::option::expect_none_failed
at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\core\src\option.rs:1268
3: core::result::Result<tuple<>, core::result::Result<serde_json::value::Value, thirtyfour::error::WebDriverError>>::expect<tuple<>,core::result::Result<serde_json::value::Value, thirtyfour::error::WebDriverError>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\result.rs:933
4: thirtyfour::session::session_runner::{{closure}}
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\thirtyfour-0.23.0\src\session.rs:47
5: core::future::from_generator::{{impl}}::poll<generator-0>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\future\mod.rs:80
6: tokio::runtime::task::core::{{impl}}::poll::{{closure}}<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\core.rs:173
7: tokio::loom::std::unsafe_cell::UnsafeCell<tokio::runtime::task::core::Stage<core::future::from_generator::GenFuture<generator-0>>>::with_mut<tokio::runtime::task::core::Stage<core::future::from_generator::GenFuture<generator-0>>,core::task::poll::Poll<tup
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\loom\std\unsafe_cell.rs:14
8: tokio::runtime::task::core::Core<core::future::from_generator::GenFuture<generator-0>, alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>::poll<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_pool::
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\core.rs:158
9: tokio::runtime::task::harness::{{impl}}::poll::{{closure}}<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\harness.rs:107
10: core::ops::function::FnOnce::call_once<closure-0,tuple<>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:227
11: std::panic::{{impl}}::call_once<core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>,closure-0>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:322
12: std::panicking::try::do_call<std::panic::AssertUnwindSafe<closure-0>,core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:379
13: thirtyfour::webdrivercommands::start_session::{{closure}}::_::{{impl}}::deserialize::{{impl}}::expecting
14: std::panicking::try<core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>,std::panic::AssertUnwindSafe<closure-0>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:343
15: std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure-0>,core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:396
16: tokio::runtime::task::harness::Harness<core::future::from_generator::GenFuture<generator-0>, alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>::poll<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\harness.rs:89
17: tokio::runtime::task::raw::poll<core::future::from_generator::GenFuture<generator-0>,alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\raw.rs:104
18: tokio::runtime::task::raw::RawTask::poll
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\raw.rs:66
19: tokio::runtime::task::Notified<alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>::run<alloc::sync::Arc<tokio::runtime::thread_pool::worker::Worker>>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\mod.rs:171
20: tokio::runtime::thread_pool::worker::{{impl}}::run_task::{{closure}}
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\thread_pool\worker.rs:370
21: tokio::coop::with_budget::{{closure}}<core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, tuple<>>,closure-0>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\coop.rs:121
22: std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>::try_with<core::cell::Cell<tokio::coop::Budget>,closure-0,core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, tuple<>>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\thread\local.rs:272
23: std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>::with<core::cell::Cell<tokio::coop::Budget>,closure-0,core::result::Result<alloc::boxed::Box<tokio::runtime::thread_pool::worker::Core, alloc::alloc::Global>, tuple<>>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\thread\local.rs:248
24: tokio::coop::with_budget
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\coop.rs:114
25: tokio::coop::budget
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\coop.rs:98
26: tokio::runtime::thread_pool::worker::Context::run_task
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\thread_pool\worker.rs:348
27: tokio::runtime::thread_pool::worker::Context::run
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\thread_pool\worker.rs:318
28: tokio::runtime::thread_pool::worker::run::{{closure}}
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\thread_pool\worker.rs:303
29: tokio::macros::scoped_tls::ScopedKey<tokio::runtime::thread_pool::worker::Context>::set<tokio::runtime::thread_pool::worker::Context,closure-0,tuple<>>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\macros\scoped_tls.rs:61
30: tokio::runtime::thread_pool::worker::run
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\thread_pool\worker.rs:300
31: tokio::runtime::thread_pool::worker::{{impl}}::launch::{{closure}}
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\thread_pool\worker.rs:279
32: tokio::runtime::blocking::task::{{impl}}::poll<closure-0,tuple<>>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\blocking\task.rs:42
33: tokio::runtime::task::core::{{impl}}::poll::{{closure}}<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\core.rs:173
34: tokio::loom::std::unsafe_cell::UnsafeCell<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<closure-0>>>::with_mut<tokio::runtime::task::core::Stage<tokio::runtime::blocking::task::BlockingTask<closure-0>>,core::task::poll::Po
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\loom\std\unsafe_cell.rs:14
35: tokio::runtime::task::core::Core<tokio::runtime::blocking::task::BlockingTask<closure-0>, tokio::runtime::blocking::schedule::NoopSchedule>::poll<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\core.rs:158
36: tokio::runtime::task::harness::{{impl}}::poll::{{closure}}<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\harness.rs:107
37: core::ops::function::FnOnce::call_once<closure-0,tuple<>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:227
38: std::panic::{{impl}}::call_once<core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>,closure-0>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:322
39: std::panicking::try::do_call<std::panic::AssertUnwindSafe<closure-0>,core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:379
40: tokio::time::driver::wheel::level::slot_for
41: std::panicking::try<core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>,std::panic::AssertUnwindSafe<closure-0>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:343
42: std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure-0>,core::task::poll::Poll<core::result::Result<tuple<>, tokio::runtime::task::error::JoinError>>>
at C:\Users\TiTan\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:396
43: tokio::runtime::task::harness::Harness<tokio::runtime::blocking::task::BlockingTask<closure-0>, tokio::runtime::blocking::schedule::NoopSchedule>::poll<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedul
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\harness.rs:89
44: tokio::runtime::task::raw::poll<tokio::runtime::blocking::task::BlockingTask<closure-0>,tokio::runtime::blocking::schedule::NoopSchedule>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\raw.rs:104
45: tokio::runtime::task::raw::RawTask::poll
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\raw.rs:66
46: tokio::runtime::task::Notified<tokio::runtime::blocking::schedule::NoopSchedule>::run<tokio::runtime::blocking::schedule::NoopSchedule>
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\task\mod.rs:171
47: tokio::runtime::blocking::pool::Inner::run
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\blocking\pool.rs:277
48: tokio::runtime::blocking::pool::{{impl}}::spawn_thread::{{closure}}
at C:\Users\TiTan\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-1.0.1\src\runtime\blocking\pool.rs:257
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
T
I have a nodejs chrome driver script that I would like to port to rust.
In project I have a packed CRX file that I would like to include into chrome before running chromedriver. Is that possible using thrityfour?
Hi, I am back to test your project again and I think that the API have changed a lot especially using a feature flag to use async-std, tokio, sync etc.
I keep having the project work at runtime because I am not sure how I can configure features flags to use tokio and async feature in Cargo.toml
[package]
name = "login"
version = "0.1.0"
authors = ["www.steadylearner.com/blog/search/Rust"]
edition = "2018"
[dependencies]
tokio = "0.2.21"
# https://doc.rust-lang.org/cargo/reference/features.html#the-features-section
# https://github.com/stevepryde/thirtyfour#feature-flags
# https://rust-random.github.io/book/features.html
[dependencies.thirtyfour]
version = "0.12.0"
default-features = false
features = ["tokio-runtime"]
It is currently working with $cago c with the main.rs but when I use $cargo run it shows the error below and I think that it is from Cargo.toml file.
Error: ReqwestError(reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) })
[Finished running. Exit status: 1]
Can you share the fully working example? I just need to visit that I can visit the reddit website with your project.
p.s It is difficult to what I need to do to make feature flags work correctly. Would you include them at separate folders or README.md at your project?
It will be better for beginners like myself to use your project better.
I want to commit the Reddit login example to your project so please help me to get through this.
Hi,
I have a form that I want to submit, which then prompts a file download with Content-Disposition: attachment
Is it possible to download this file using thirtyfour/chromedriver, to a specific directory, or somehow interact with the download request?
After
driver.set_page_load_timeout( Duration::new(5_u64, 0)).await?;
Sometimes:
[1619342394.318][SEVERE]: Timed out receiving message from renderer: 5.000
[1619342394.337][SEVERE]: Timed out receiving message from renderer: -0.019
[1619342394.359][SEVERE]: Timed out receiving message from renderer: -0.019
Is it a normal value -0.019 ?
Timeout with a negative value ?
Thank you for perfect crates!
Hello!
I'm creating a web scraper. I have a website that's hard to log into using plain POST, so I create a temporary Selenium instance, log in and copy the session cookies to be later reused by reqwest::Client
.
However, I ran into a problem - I have to access the name
field of the Cookie
struct, but it's private. I temporarily resorted to creating a MyCookie
struct and invoking unsafe { std::mem::transmute<Cookie, MyCookie>(instance) }
. Unfortunately this is risky, since Cookie
is not a #[repr(C)]
struct.
Please consider adding more getters to the struct, that would be immensely helpful.
What are the default durations for the WebDriver::Query timeout and polling? I'm working with a website with slow load times and I'd like to increase the explicit wait timeout. I know I can set this with .wait() on the query but I'm not sure what a reasonable default for the poll time is, as I've never had a need to manually set it before.
Hi, I want to select a drop-down item in the selector.
In Selenium with Python, I can do it by :
from selenium.webdriver.support.ui import Select
s1 = Select(driver.find_element_by_id('s1Id'))
s1.select_by_index(1)
s1.select_by_value("o2")
s1.select_by_visible_text("o3")
Is there any equivalent to do these things in ThirtyFour?
Hi @stevepryde what do you thing to make a public constant or exposed method for a default TimeoutConfiguration
?
It would be useful to be able to obtain a default one to be able to switch back after a usage of thirtyfour_query
and set_implicit_wait_timeout
thirtyfour/src/webdrivercommands.rs
Lines 78 to 83 in 83c91ce
I am trying to build a small web scaper while learning rust. However, I am unable to connect to chromedriver.
Arch Linux
rustc 1.44.0-nightly
ChromeBrowser Version: 83.0.4103.61
Chromedriver Version: 83.0.4103.39
Here is my original code:
let caps = DesiredCapabilities::chrome();
let driver = WebDriver::new("http://localhost:4444", &caps)
.await
.expect("could not connect to webdriver");
This results in a invalid argument: cannot parse capability error the Rust DEBUG logs:
thread 'main' panicked at 'could not connect to webdriver: InvalidArgument(WebDriverErrorInfo { status: 400, error: "", value: WebDriverErrorValue { message: "invalid argument: cannot parse capability: browserVersion\nfrom invalid argument: cannot be empty", error: Some("invalid argument"), stacktrace: Some("#0 0x5646c0b6c579 <unknown>\n"), data: None } })', src/main.rs
And the chromedriver logs:
[INFO]: RESPONSE InitSession ERROR invalid argument: cannot parse capability:
browserVersion from invalid argument: cannot be empty
[DEBUG]: Log type 'driver' lost 1 entries on destruction
[INFO]: COMMAND InitSession {
"capabilities": {
"alwaysMatch": {
"browserName": "chrome",
"browserVersion": "",
"platformName": "ANY"
},
"firstMatch": [ {
} ]
},
"desiredCapabilities": {
"browserName": "chrome",
"platform": "ANY",
"version": ""
}
}
[INFO]: RESPONSE InitSession ERROR invalid argument: cannot parse capability:
browserVersion from invalid argument: cannot be empty
[DEBUG]: Log type 'driver' lost 1 entries on destruction
So I tried adding the browserVersion manually:
let mut caps = DesiredCapabilities::chrome();
caps.set_version("83.0.4103.61")
.expect("failed to set browserVersion for webdriver");
let driver = WebDriver::new("http://localhost:4444", &caps)
.await
.expect("could not connect to webdriver");
But I get a invalid argument: cannot be empty error in Rust DEBUG logs:
thread 'main' panicked at 'could not connect to webdriver: InvalidArgument(WebDriverErrorInfo { status: 400, error: "", value: WebDriverErrorValue { message: "invalid argument: cannot parse capability: browserVersion\nfrom invalid argument: cannot be empty", error: Some("invalid argument"), stacktrace: Some("#0 0x5646c0b6c579 <unknown>\n"), data: None } })', src/main.rs
And a "No matching capabilities found" error in the chromedriver logs:
[INFO]: RESPONSE InitSession ERROR session not created: No matching capabilities found
[DEBUG]: Log type 'driver' lost 1 entries on destruction
[INFO]: COMMAND InitSession {
"capabilities": {
"alwaysMatch": {
"browserName": "chrome",
"browserVersion": "83.0.4103.61",
"platformName": "ANY"
},
"firstMatch": [ {
} ]
},
"desiredCapabilities": {
"browserName": "chrome",
"platform": "ANY",
"version": "83.0.4103.61"
}
}
[INFO]: RESPONSE InitSession ERROR session not created: No matching capabilities found
[DEBUG]: Log type 'driver' lost 1 entries on destruction
What am I doing wrong here?
From my perspective DesiredCapabilities::default()
isn't able to be any useful. It might be a good candidate for being deleted?
'tests::test_ignore_url' panicked at 'called `Option::unwrap()` on a `None` value', /home/zhiburt/.cargo/registry/src/github.com-1ecc6299db9ec823/thirtyfour-0.15.1/src/common/capabilities/desiredcapabilities.rs:34:19
Even though the official Selenium Grid does not support HTTP/2 just yet, there is a distinct possibility that it will eventually do so. Additionally, there may be other grid solutions out there that do support it already (shameless plug ๐).
Especially when running multiple tests in parallel across multiple browser sessions, using a single HTTP/2 connection to the Grid with request multiplexing can help reduce network load and latency. Thus I'd really like to see this feature.
Regarding the implementation, it should be somewhat straightforward. We need a way to share the Reqwest Client structure between multiple WebDrivers and call the http2_prior_knowledge
function on the Client to force HTTP/2 (it does not appear to negotiate it on its own).
Currently .get_attribute()
returns a "null"
if the attribute doesn't exist.
I think it should return a None
insted of "null"
If someone had a <div data-some="null">
, how would that someone detect the attribute data-some
is defined.
.get_attribute()
function to return a Option<String>
(BREAKING CHANGE), .get_attribute_opt()
that returns a Option<String>
,Hey @stevepryde are these lifetime discrepancies are intentional?
WebElement
pub async fn find_element(&self, by: By<'a>) -> WebDriverResult<WebElement<'a>>;
WebDriver
async fn find_element<'a>(&'a self, by: By<'_>) -> WebDriverResult<WebElement<'a>>;
What I am trying to say is that why we bound to 'a
lifetime in a WebElement
?
*Kind of bump on the issue with this ๐ฅ
First of all thanks for your work on this project, it's been a pleasure to use it. The async version is really handy as I need to find elements in batch and doing them concurrently has been a huge win.
I was hoping I could do a standalone musl build out of it, but that didn't work. The build is looking for openssl-sys
. I assumed using the following feature flag would disable native-tls
.
thirtyfour = { version = "0.23.0", features = [ "tokio-runtime", "reqwest-rustls-tls" ] }
However when I checked the dep tree, it shows the following
โ โโโ reqwest v0.11.3
โ โ โโโ base64 v0.13.0
โ โ โโโ bytes v1.0.1
โ โ โโโ encoding_rs v0.8.28 (*)
โ โ โโโ futures-core v0.3.14
โ โ โโโ futures-util v0.3.14 (*)
โ โ โโโ http v0.2.4 (*)
โ โ โโโ http-body v0.4.1 (*)
โ โ โโโ hyper v0.14.5 (*)
โ โ โโโ hyper-rustls v0.22.1
โ โ โ โโโ futures-util v0.3.14 (*)
โ โ โ โโโ hyper v0.14.5 (*)
โ โ โ โโโ log v0.4.14 (*)
โ โ โ โโโ rustls v0.19.1 (*)
โ โ โ โโโ tokio v1.5.0 (*)
โ โ โ โโโ tokio-rustls v0.22.0
โ โ โ โ โโโ rustls v0.19.1 (*)
โ โ โ โ โโโ tokio v1.5.0 (*)
โ โ โ โ โโโ webpki v0.21.4 (*)
โ โ โ โโโ webpki v0.21.4 (*)
โ โ โโโ hyper-tls v0.5.0
โ โ โ โโโ bytes v1.0.1
โ โ โ โโโ hyper v0.14.5 (*)
โ โ โ โโโ native-tls v0.2.7
โ โ โ โ โโโ log v0.4.14 (*)
โ โ โ โ โโโ openssl v0.10.33
โ โ โ โ โ โโโ bitflags v1.2.1
โ โ โ โ โ โโโ cfg-if v1.0.0
โ โ โ โ โ โโโ foreign-types v0.3.2
โ โ โ โ โ โ โโโ foreign-types-shared v0.1.1
โ โ โ โ โ โโโ libc v0.2.93
โ โ โ โ โ โโโ once_cell v1.7.2
โ โ โ โ โ โโโ openssl-sys v0.9.61
โ โ โ โ โ โโโ libc v0.2.93
โ โ โ โ โ [build-dependencies]
โ โ โ โ โ โโโ autocfg v1.0.1
โ โ โ โ โ โโโ cc v1.0.67
โ โ โ โ โ โโโ pkg-config v0.3.19
โ โ โ โ โโโ openssl-probe v0.1.2
โ โ โ โ โโโ openssl-sys v0.9.61 (*)
โ โ โ โโโ tokio v1.5.0 (*)
โ โ โ โโโ tokio-native-tls v0.3.0
โ โ โ โโโ native-tls v0.2.7 (*)
โ โ โ โโโ tokio v1.5.0 (*)
Anyway to turn it off completely please? Thanks
Hi Guys,
I'm trying to trigger a click on a tag.
click() on form elements works.
But on elements it doesn't work. Is there a different method for such a purpose?
T
Hello @stevepryde I was trying to set a custom User-Agent for geckodriver as I understand to set it we should do the following in python
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", "whatever you want")
But as I see there's no such option currently
thirtyfour/src/common/capabilities/firefox.rs
Lines 126 to 139 in dd8830d
I could imagine that there's a bigger list of such settings so might a custom parameter not a bad idea?
PS: I didn't test setting of preference
Hey @stevepryde,
so I encounter if you're running in your environment chromedriver but provide thirtyfour with firefox Capabilities Webdriver::new
function never returns.
You can run this test to repeat it.
Hello,
this error: error sending request for url (http://localhost:6565/session): error trying to connect: tcp connect error: Connection refused (os error 111)
is returned by the WebClient new function when I call it.
How to reproduce:
./path/to/geckodriver --port=6565
WebDriver::new("http://localhost:6565/", my_capabilities).await?;
Implicitly waiting before calling ::new() doesn't work either.
I'm working on a project where I manage between 3 and 7 WebDrivers at any given time. They are really cluttering up my desktop while they're running. Is the a way to run multiple unique WebDrivers in the same window just on different tabs?
Most of the time when I perform an operation in a new tab, I simply need to open a new tab, do some stuff, then close it and return to the original. A convenience method or trait on WebDriver to do this would go a long way to decluttering some of my code. I quickly wrote up the simple function version of what I'm talking about.
use std::future::Future;
use thirtyfour::prelude::*;
use tokio;
pub async fn in_new_tab<F, Fut, T>(driver: &WebDriver, f: F) -> WebDriverResult<T>
where
F: FnOnce() -> Fut,
Fut: Future<Output = WebDriverResult<T>>,
{
// Opening Tab ----------------------------------------
let handle = driver.current_window_handle().await?;
driver
.execute_script(r#"window.open("about:blank", target="_blank");"#)
.await?;
let handles = driver.window_handles().await?;
driver.switch_to().window(&handles[1]).await?;
// ----------------------------------------------------
let result = f().await;
// Closing Tab --------------------------------------------
driver.execute_script(r#"window.close();"#).await?;
driver.switch_to().window(&handle).await?;
// --------------------------------------------------------
result
}
#[tokio::main]
async fn main() -> WebDriverResult<()> {
let caps = DesiredCapabilities::firefox();
let driver = WebDriver::new(&format!("http://localhost:{}/wd/hub", "4444"), &caps).await?;
driver.get("https://google.com").await?;
in_new_tab(&driver, || {
async {
driver.get("https://wikipedia.com").await?;
Ok(())
}
}).await?;
driver.get("https://youtube.com").await?;
driver.quit().await?;
Ok(())
}
I started to write an async_trait to implement on WebDriver but I ran into issues with the closure not being Send. What do you think about something similar to this becoming part of the WebDriverCommands trait? So the we could have something like
driver.in_new_tab(|| { async { Ok(()) } }).await?;
or for the nightly folks with async closures
driver.in_new_tab(async || { Ok(()) }).await?;
We've able to execute cdp command in thirtyfour
, but I've not found a way to listen CDP event.
Sometimes, I hope to use fetch.enable
to catch all network request in a page. When page send a network request, a requestPaused
event is issued, and I should capture this event and send fulfillRequest
or failRequest
.
I'm trying to setup some integration tests with actix-web and thirtyfour but I can't get it to work. I've tried both the async and sync version.
#[cfg(test)]
mod tests {
use thirtyfour::sync::prelude::*;
use crate::routes::routes;
use actix_web::{test, App};
#[actix_rt::test]
async fn test_index_get() -> WebDriverResult<()> {
let mut app = test::init_service(App::new().configure(routes)).await;
let req = test::TestRequest::get().uri("/").to_request();
let resp = test::call_service(&mut app, req).await;
assert!(resp.status().is_success());
let caps = DesiredCapabilities::chrome();
let driver = WebDriver::new("http://localhost:4444", &caps)?;
Ok(())
}
}
thread 'tests::e2e::tests::test_index_get' panicked at 'Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.', /home/oskar/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/runtime/blocking/shutdown.rs:49:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Panic in Arbiter thread.
#[cfg(test)]
mod tests {
use thirtyfour::prelude::*;
use crate::routes::routes;
use actix_web::{test, App};
#[actix_rt::test]
async fn test_index_get() -> WebDriverResult<()> {
let mut app = test::init_service(App::new().wrap(NormalizeSlashes).configure(routes)).await;
let req = test::TestRequest::get().uri("/").to_request();
let resp = test::call_service(&mut app, req).await;
assert!(resp.status().is_success());
let caps = DesiredCapabilities::chrome();
let driver = WebDriver::new("http://localhost:4444", &caps).await?;
Ok(())
}
}
Error: JsonError(Error("invalid type: null, expected struct ConnectionResp", line: 0, column: 0))
thread 'tests::e2e::tests::test_index_get' panicked at 'assertion failed: `(left == right)`
left: `1`,
right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', /home/oskar/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/macros.rs:16:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I was testing with async API of this crate with my React frontend at www.steadylener.com
What I want is very simple
The code I used is similar to this.
use thirtyfour::error::WebDriverResult;
use thirtyfour::{
By,
DesiredCapabilities,
WebDriver,
Keys,
TypingData,
};
use tokio;
#[tokio::main]
async fn main() -> WebDriverResult<()> {
let caps = DesiredCapabilities::chrome();
let driver = WebDriver::new("http://localhost:4444/wd/hub", &caps).await?;
let target = "https://www.steadylearner.com";
println!("Wait before you play with {}", &target);
// Navigate to https://www.steadylearner.com.
driver.get(target).await?;
println!("{:#?}", &driver.title().await?); // Should be "Steadylearner Home"
let search_input = driver.find_element(By::Id("searchInput")).await?;
println!("{}", &search_input.get_attribute("value").await?); // ""
// The problem happens with the code below
search_input.send_keys("Rust").await?;
search_input.send_keys(Keys::Enter).await?;
println!("{:#?}", &driver.title().await?); // "Posts for Rust"
Ok(())
}
But, after a long wait, it shows this error.
Error: ElementNotInteractable(WebDriverErrorInfo { status: 400, error: "", value: WebDriverErrorValue { message: "element not interactable\n (Session info: chrome=79.0.3945.117)", error: Some("element not interactable"), stacktrace: Some("#0 0x5566c2005479 <unknown>\n"), data: None } })
I don't use Chrome browser, but it is installed in my machine and used it to follow the example easily.
My doubts are
I want to test Selenium with React or other single page apps. But, I haven't used it for a long time. If I missed something, please let me know.
I wanted to migrate my fantoccini
project to thirtyfour
since it had find_element timeouts. But when was trying to migrate I realized that I need a method that doesn't wait for the element and returns the current element even if it doesn't exist/hasn't loaded. I know I can set the timeout but it became annoying when i had to set the timeout to 0 and then reset it back. And bacause i didn't always have a handle to WebDriver
.
find_element()
and find_elements()
that doesn't wait and returns the current state (AKA error that no element was found).wait_find_element()
and find_element()
but that would break code compatibility,element_exist()
method, or with a different name like has_element()
, is_present()
, that returns a boolfind_optional_element()
that returns a Optionfind_present_element()
that returns a Result.NOTE OF course all of them would be wrapped with a WebdriverResult.
Please if you could find a solution because this is the only problem stopping me from migration.
It looks like the async-std-runtime
is not working properly because the header is not being configured as in the tokio-runtime version.
Hi and thanks for this wonderful library.
Could you please comment on whether or not it's possible to interact with a dropdown combobox? And if so, how?
Thanks.
Hey there! This is really about Async/Await in general but as I'm encountering the issue in the context of Selenium I thought you might have a good answer. I'm processing hrefs from a list of links scraped from the page, and I want to await every call to get the href attribute, but I can't do async in a map.
My code:
// Pull the urls from the href attribute of each link and filter out urls that have already been processed
let new_order_urls = new_order_links.iter()
.map(|link| { link.get_attribute("href").await })
.map(|href_result_option| {
if let Ok(Some(url)) = href_result_option {
return url;
}
// Gets filtered out of the list, can't crash on error or no href, will need to try to get the url on the next go round
"error_url".to_string()
})
.filter(|url| !enforced_orders.contains(url)).collect();
This of course gives the error
error[E0728]: `await` is only allowed inside `async` functions and blocks
--> src\unassigned_email_queue.rs:68:27
|
68 | .map(|link| { link.get_attribute("href").await })
| ------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ only allowed inside `async` functions and blocks
| |
| this is not `async`
What is the best way to call and await get_attribute for each link in the list?
Hello, is there any way to not open a new session but reuse an existing one ? I haven't seen a way to do that in the doc.
Thanks for the great project, best regards.
I haven't tried compiling this to WASM but I will assume it doesn't ๐
I might sound weird wanting to run a browser automation tool from a browser environment but I have a use case of a plug-in system where plugins are JS or Rust compiled to WASM running in a WebWorker, this plug-ins can run in the browser and the server with the help of Deno. The browser APIs limit me to use fetch
or WebSocket
based libraries so I was looking for Webdriver rust libraries that would use reqwest as HTTP client which supports compiling to browser WASM, I used fantoccini a bit and was planning to work on a fork that used reqwest but I'm glad to see Thirtyfour exists and the API might even be nicer.
Setting such option will cause a error
cops.set_proxy(thirtyfour::Proxy::Manual {
ftp_proxy: None,
http_proxy: None,
ssl_proxy: None,
socks_proxy: Some(proxy.clone()),
socks_password: None,
socks_username: None,
no_proxy: None,
})?;
Unable to create WebDriver session:
Status: 500
Additional info:
InvalidArgumentError: Expected "socksVersion" to be a positive integer, got [object Undefined] undefined
Error: session not created
Stacktrace:
WebDriverError@chrome://marionette/content/error.js:175:5
InvalidArgumentError@chrome://marionette/content/error.js:304:5
assert.that/<@chrome://marionette/content/assert.js:479:13
assert.integer@chrome://marionette/content/assert.js:325:48
assert.positiveInteger@chrome://marionette/content/assert.js:343:10
fromJSON@chrome://marionette/content/capabilities.js:333:35
match_@chrome://marionette/content/capabilities.js:539:21
fromJSON@chrome://marionette/content/capabilities.js:518:25
GeckoDriver.prototype.newSession@chrome://marionette/content/driver.js:800:38
despatch@chrome://marionette/content/server.js:305:40
execute@chrome://marionette/content/server.js:275:16
onPacket/<@chrome://marionette/content/server.js:248:20
onPacket@chrome://marionette/content/server.js:249:9
_onJSONObjectReady/<@chrome://marionette/content/transport.js:501:20
As I understand currently socksVersion
field is missed
https://www.w3.org/TR/webdriver1/#proxy
Geckodriver
Thirtyfour does not build with "async-std-runtime" feature enabled.
/thirtyfour-0.15.1/src/http_async/surf_async.rs:40:31
|
40 | request = request.body_json(&x)?;
| ^^^^^^^^^ method not found in `surf::RequestBuilder`
I was testing the async api with a runtime support.
And a very strange behaviour appeared, the code bellow won't move out at the end of the block_on closure.
The issue has been tested with 0.8 and 0.9 version of thirtyfour, tokio 0.2.19 and using the selenium docker command given in the readme.
use thirtyfour::prelude::*;
use tokio;
fn main() -> WebDriverResult<()> {
let mut runtime = tokio::runtime::Builder::new()
.basic_scheduler()
.enable_all()
.build().unwrap();
let result : WebDriverResult<()> = runtime.block_on(async {
let caps = DesiredCapabilities::chrome();
let driver = WebDriver::new("http://localhost:4444/wd/hub", &caps).await?;
println!("{:?}", driver);
Ok(())
});
println!("End");
result
}
After some investigation, the problem seems to come from the function drop (webdriver.rs:159).
fn drop(&mut self) {
if self.quit_on_drop && !(*self.session_id).is_empty() {
if let Err(e) = block_on(self.quit()) {
error!("Failed to close session: {:?}", e);
}
}
}
For some unknown reason, the block_on(self.quit()) loops without returning.
I tried to explicitly call driver.quit() before the end of the closure, but it didn't solve the problem.
Thanks
The wait method on query is documented as taking core::time::Duration. Can I / should I use the tokio::time::Duration here, and what are the effects in terms of yielding a thread vs. blocking (I would like to avoid blocking). Sorry I'm still wrapping my head around async w/Tokio.
First, thanks for a great project.
So I have an Actix Web project project that uses rust-tls https://github.com/contor-systems/contor
I'm trying to write the integration tests with thirtyfour. https://github.com/contor-systems/contor/tree/master/tests
We're both using Reqwest accept I use the feature "rust-tls". I add thirtyfour as a dev-dependency.
Cargo doesn't have a way to disable dev dependencies during a build in my CI/CD. So I get an openssl error.
My first reaction was to look for a rust-tls feature on thirtyfour.
I can try some work arounds, but just thought I'd let you know.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.