tesaguri / twitter-stream-rs Goto Github PK
View Code? Open in Web Editor NEWA Rust library for listening on Twitter Streaming API.
License: MIT License
A Rust library for listening on Twitter Streaming API.
License: MIT License
The library provides TwitterJsonStream
type which lets the user get raw JSON messages:
extern crate json;
// ...
let fut = stream.for_each(|json| {
let msg = json::parse(&json).map_err(Error::custom)?;
if msg["id"].is_number() // Tweet
{
// ...
}
Ok(())
});
In the above example, the user doesn't need TwitterStream
, message types like StreamMessage
or even serde
-related dependencies.
In such cases, an option to disable those types would significantly reduce compile time.
When using TwitterStream::follow
with a token with Read Only permissions I get a 403 error.
This is my code:
let token = Token::from_parts(
config.api_key,
config.api_secret,
config.access_key,
config.access_secret,
);
let mut stream = TwitterStream::follow(&[11230702 as u64], &token).try_flatten_stream();
while let Some(json) = stream.next().await {
if let Err(why) = json {
error!("Failed to get Twitter stream: {:?}", why);
panic!();
}
if let Ok(StreamMessage::Tweet(tweet)) =
serde_json::from_str(&json.unwrap())
{
println!("{}", tweet.text);
}
}
And this log:
452653Z TRACE hyper::client::pool: checkout waiting for idle connection: ("https", stream.twitter.com)
452856Z TRACE hyper::client::connect::http: Http::connect; scheme=Some("https"), host=Some("stream.twitter.com"), port=None
453220Z DEBUG hyper::client::connect::dns: resolving host="stream.twitter.com"
716896Z DEBUG hyper::client::connect::http: connecting to 104.244.42.131:443
717236Z TRACE mio::poll: registering event source with poller: token=Token(1), interests=READABLE | WRITABLE
740092Z DEBUG hyper::client::connect::http: connected to 104.244.42.131:443
767531Z TRACE hyper::client::conn: client handshake Http1
767782Z TRACE hyper::client::client: handshake complete, spawning background dispatcher task
768020Z TRACE want: signal: Want
768093Z TRACE want: signal found waiting giver, notifying
768168Z TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Init, writing: Init, keep_alive: Busy }
768351Z TRACE want: poll_want: taker wants!
768453Z TRACE hyper::client::pool: checkout dropped for ("https", stream.twitter.com)
768988Z TRACE encode_headers: hyper::proto::h1::role: Client::encode method=POST, body=Some(Known(15))
769187Z TRACE hyper::proto::h1::io: buffer.flatten self.len=414 buf.len=15
769431Z DEBUG hyper::proto::h1::io: flushed 429 bytes
769483Z TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Init, writing: KeepAlive, keep_alive: Busy }
895101Z TRACE hyper::proto::h1::conn: Conn::read_head
895714Z TRACE hyper::proto::h1::io: received 1137 bytes
895998Z TRACE parse_headers: hyper::proto::h1::role: Response.parse bytes=1137
896185Z TRACE parse_headers: hyper::proto::h1::role: Response.parse Complete(777)
896513Z DEBUG hyper::proto::h1::io: parsed 10 headers
896619Z DEBUG hyper::proto::h1::conn: incoming body is content-length (360 bytes)
896744Z TRACE hyper::proto::h1::decode: decode; state=Length(360)
896929Z DEBUG hyper::proto::h1::conn: incoming body completed
897035Z TRACE hyper::proto::h1::conn: maybe_notify; read_from_io blocked
897396Z TRACE want: signal: Want
897464Z TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Init, writing: Init, keep_alive: Idle }
897545Z TRACE want: signal: Want
897639Z TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Init, writing: Init, keep_alive: Idle }
897944Z TRACE hyper::client::pool: put; add idle connection for ("https", stream.twitter.com)
898094Z DEBUG hyper::client::pool: pooling idle connection for ("https", stream.twitter.com)
898341Z ERROR sidekick::twitter: Failed to get Twitter stream: Http(403)
What am I doing wrong?
A lot of projects have already moved to updated serde and updated their crates. There is such situation that each crate is capable of working with serde 1.0 but your crate does not (0.2.0 still needs old serde). Could you release new version of the crate quicker please? At the very least release 0.3
with only serde changes and continue work on the crate for the 0.4 version.
error[E0277]: the trait bound `twitter_stream::TwitterStream: futures_core::stream::Stream` is not satisfied
--> src/main.rs:14:10
|
14 | .try_flatten_stream()
| ^^^^^^^^^^^^^^^^^^ the trait `futures_core::stream::Stream` is not implemented for `twitter_stream::TwitterStream`
|
= note: required because of the requirements on the impl of `futures_core::stream::TryStream` for `twitter_stream::TwitterStream`
error[E0599]: no method named `try_for_each` found for type `futures_util::future::try_future::try_flatten_stream::TryFlattenStream<twitter_stream::FutureTwitterStream>` in the current scope
--> src/main.rs:15:10
|
15 | .try_for_each(|json| {
| ^^^^^^^^^^^^ method not found in `futures_util::future::try_future::try_flatten_stream::TryFlattenStream<twitter_stream::FutureTwitterStream>`
|
= note: the method `try_for_each` exists but the following trait bounds were not satisfied:
`&futures_util::future::try_future::try_flatten_stream::TryFlattenStream<twitter_stream::FutureTwitterStream> : futures_util::stream::try_stream::TryStreamExt`
`&mut futures_util::future::try_future::try_flatten_stream::TryFlattenStream<twitter_stream::FutureTwitterStream> : futures_util::stream::try_stream::TryStreamExt`
`futures_util::future::try_future::try_flatten_stream::TryFlattenStream<twitter_stream::FutureTwitterStream> : futures_util::stream::try_stream::TryStreamExt`
error: aborting due to 2 previous errors
I'm unable to build the example, the compiler just complains about a trait from futures not being implemented. I'm using 'futures = "0.3.1"', are you using a different package by any change? Would be great if the example also showed exactly the dependencies needed. I had to install a very specific version of tokio as it was not resolved automatically..
Compiling serde_derive v0.8.23
error: attribute must only have one argument
--> /Users/l/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_derive-0.8.23/src/lib.rs:6:3
|
6 | #[proc_macro_derive(Serialize, attributes(serde))]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: attribute must only have one argument
--> /Users/l/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_derive-0.8.23/src/lib.rs:14:3
|
14 | #[proc_macro_derive(Deserialize, attributes(serde))]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: the `proc-macro` crate type is experimental
|
= help: add #![feature(proc_macro)] to the crate attributes to enable
error: the `#[proc_macro_derive]` attribute is an experimental feature (see issue #35900)
--> /Users/l/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_derive-0.8.23/src/lib.rs:6:1
|
6 | #[proc_macro_derive(Serialize, attributes(serde))]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: the `#[proc_macro_derive]` attribute is an experimental feature (see issue #35900)
--> /Users/l/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_derive-0.8.23/src/lib.rs:14:1
|
14 | #[proc_macro_derive(Deserialize, attributes(serde))]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: aborting due to 2 previous errors
Build failed, waiting for other jobs to finish...
error: failed to compile `tw v0.1.0 (file:///Users/l/Desktop/tw)`, intermediate artifacts can be found at `/Users/l/Desktop/tw/target`
Caused by:
Can you provide a function that enables this library to send tweets with pictures? Thanks.
We are looking for an example function to stop the streaming, any guidance or suggestion?
I have found interesting thing: I can't compile my crate which uses twitter-stream
anymore. According to what contributors of ring
crate say it is impossible to link to more than one ring
crate at once but you do.
Please, do something to make your crate use one version of ring
crate at a time.
Is it okay to run multiple instances of stream
in different threads with same credentials? I miss tweets sometimes. I use same credentials for the both streams in both threads.
I have just reposted a tweet and it's text is:
text: "RT @QuakeRussia: Quake Champions BETA countdown April 27: The next QC Beta will start at 16:00 CEST (10:00AM EST) Read More https://t.co/OT…",
However in the same structure if we try to see entities::retweeted_status
it's text is:
text: "Quake Champions BETA countdown April 27: The next QC Beta will start at 16:00 CEST (10:00AM EST) Read More https://t.co/OTcq6H6sXU",
The full object looks like this:
Now that hyper is introducing a Tokio-based async API, we will be able to (and have to) integrate Tokio into our library. I'm currently working on the integration on the tokio
branch:
https://github.com/d12i/twitter-stream-rs/tree/tokio
Here is the task list:
TwitterStreamBuilder
take a Handle
(5c53e2f)hyper::Body
in a streaming style (51248ad)flate2::read
can no longer be used directly since hyper::Body
does not impl Read
)Core
(hyperium/hyper#1002)When the Twitter API returns an error, it provides an error code and a message, but currently, our library simply returns an HTTP status code in case of a Twitter error. (Edit 2: Oh, the error code is not returned either.)
Also, the (Edit: Streaming API does not return these headers.)X-Rate-Limit-*
headers are ignored.
We should provide users with these kind of information like tweetust
and egg-mode
crates do.
error[E0599]: no function or associated item named `track` found for struct `twitter_stream::TwitterStream<_>` in the current scope
--> src/main.rs:8:20
|
8 | TwitterStream::track("@Twitter", &token)
| ^^^^^ function or associated item not found in `twitter_stream::TwitterStream<_>`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0599`.
I am trying the example code, I gather consumer keys from
and used it client_identifier_ and client_secret then for token and token_secret I used
and when I ran the code I am getting error below
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Http(404)', src/main.rs:41:10
stack backtrace:
0: 0x55ae7ff6c00a - std::backtrace_rs::backtrace::libunwind::trace::ha271a8a7e1f3d4ef
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: 0x55ae7ff6c00a - std::backtrace_rs::backtrace::trace_unsynchronized::h85739da0352c791a
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x55ae7ff6c00a - std::sys_common::backtrace::_print_fmt::hbc6ebcfb2910b329
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:65:5
3: 0x55ae7ff6c00a - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::he1c117e52d53614f
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:44:22
4: 0x55ae7ff8db6e - core::fmt::write::h25eb51b9526b8e0c
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/fmt/mod.rs:1213:17
5: 0x55ae7ff675a5 - std::io::Write::write_fmt::ha9edec5fb1621933
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/io/mod.rs:1682:15
6: 0x55ae7ff6bdd5 - std::sys_common::backtrace::_print::hf8657cd429fc3452
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:47:5
7: 0x55ae7ff6bdd5 - std::sys_common::backtrace::print::h41b9b18ed86f86bd
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:34:9
8: 0x55ae7ff6d5bf - std::panicking::default_hook::{{closure}}::h22a91871f4454152
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:267:22
9: 0x55ae7ff6d2fb - std::panicking::default_hook::h21ddc36de0cd4ae7
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:286:9
10: 0x55ae7ff6dcc9 - std::panicking::rust_panic_with_hook::h5059419d6d59b3d0
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:688:13
11: 0x55ae7ff6da69 - std::panicking::begin_panic_handler::{{closure}}::h0f383c291cd78343
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:579:13
12: 0x55ae7ff6c4bc - std::sys_common::backtrace::__rust_end_short_backtrace::h70ab22f2ad318cdd
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:137:18
13: 0x55ae7ff6d772 - rust_begin_unwind
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:575:5
14: 0x55ae7f926c23 - core::panicking::panic_fmt::hd1d46bcde3c61d72
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/panicking.rs:64:14
15: 0x55ae7f9270d3 - core::result::unwrap_failed::h456a23f68607268c
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/result.rs:1790:5
16: 0x55ae7f9c1709 - core::result::Result<T,E>::unwrap::h60f365746df74acf
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/result.rs:1112:23
17: 0x55ae7f9e5f39 - rust_feed_hub::main::{{closure}}::h7cb33536a194969a
at /home/oz-mint/projects/rust_feed_hub/src/main.rs:34:5
18: 0x55ae7f9e8b44 - tokio::runtime::park::CachedParkThread::block_on::{{closure}}::h9a9aa76d43984d7f
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/park.rs:283:63
19: 0x55ae7f9e8966 - tokio::runtime::coop::with_budget::h529e3c1343fc9bff
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/coop.rs:107:5
20: 0x55ae7f9e8966 - tokio::runtime::coop::budget::h773b879e87692ef4
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/coop.rs:73:5
21: 0x55ae7f9e8966 - tokio::runtime::park::CachedParkThread::block_on::hc48e4a7ed26d8522
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/park.rs:283:31
22: 0x55ae7f9d1c45 - tokio::runtime::context::BlockingRegionGuard::block_on::hf71e2e32f29b9748
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/context.rs:315:13
23: 0x55ae7f9bb225 - tokio::runtime::scheduler::multi_thread::MultiThread::block_on::hb6e001defe81f7a4
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/scheduler/multi_thread/mod.rs:66:9
24: 0x55ae7f9473ba - tokio::runtime::runtime::Runtime::block_on::h8659d9886e07120c
at /home/oz-mint/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/runtime.rs:304:45
25: 0x55ae7f94ce93 - rust_feed_hub::main::ha30909ffa2a6746a
at /home/oz-mint/projects/rust_feed_hub/src/main.rs:52:5
26: 0x55ae7f968a9b - core::ops::function::FnOnce::call_once::hf22ce111b44ea710
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:250:5
27: 0x55ae7f9589ee - std::sys_common::backtrace::__rust_begin_short_backtrace::haa82a813abc3fa89
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:121:18
28: 0x55ae7f9cfc21 - std::rt::lang_start::{{closure}}::ha69d3f6d2317a371
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:166:18
29: 0x55ae7ff639ec - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h203afb3af230319a
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:287:13
30: 0x55ae7ff639ec - std::panicking::try::do_call::hf68e87013b70f3c5
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
31: 0x55ae7ff639ec - std::panicking::try::h040ea8f298390ba2
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
32: 0x55ae7ff639ec - std::panic::catch_unwind::h1e17b198887a05fa
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
33: 0x55ae7ff639ec - std::rt::lang_start_internal::{{closure}}::hfb902d8927e51b86
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:148:48
34: 0x55ae7ff639ec - std::panicking::try::do_call::h354e6eb41f2e7d42
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
35: 0x55ae7ff639ec - std::panicking::try::h4a39749cd018228c
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
36: 0x55ae7ff639ec - std::panic::catch_unwind::h30bce83b8de61cca
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
37: 0x55ae7ff639ec - std::rt::lang_start_internal::h8f7e70b1a2558118
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:148:20
38: 0x55ae7f9cfbfa - std::rt::lang_start::h8142dc36d9175bf6
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/rt.rs:165:17
39: 0x55ae7f94cfbe - main
40: 0x7f7f3a48ad90 - __libc_start_call_main
at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
41: 0x7f7f3a48ae40 - __libc_start_main_impl
at ./csu/../csu/libc-start.c:392:3
42: 0x55ae7f9272b5 - _start
43: 0x0 - <unknown>
Process finished with exit code 101
I tried to change places of tokens and secrets but still 404, what am I missing ?
Thanks
twitter_stream
.Please take a look at https://crates.io/crates/chrono, the version you depend was yanked, change it to 0.4 asap please.
Twitter is going to abandon its User streams (and Site stream) API on June 19, 2018, and TwitterStreamBuilder::user
and TwitterStream::user
methods will no longer work. So I will remove these methods after then.
The POST statuses/filter and GET statuses/sample APIs will (I think) still work, so the library will continue to be maintained.
I am using this library and I wish to filter by location boxes, but I'm not sure where to start. An example of each type of filter, or some way to connect the dots for users between the APIs and this library would be nice!
extern crate twitter_stream;
extern crate ini;
use twitter_stream::{Token, TwitterStreamBuilder};
use twitter_stream::rt::{self, Future, Stream};
use ini::Ini;
fn main() {
let conf = Ini::load_from_file("config.ini").expect("Unable to read config.ini");
let bounding = conf.section(Some("geo_bounding")).expect("No geo_bounding section found in config");
let bounds = vec![];
for bound in &["west", "south", "east", "north"] {
bounds.push(
bounding
.get(bound.to_owned()).expect(&format!("Unable to find value for '{}' in config", bound))
.parse::<f64>() .expect(&format!("Unable to parse value for '{}' as f64", bound))
);
}
let bounds = ((bounds[0], bounds[1]), (bounds[2], bounds[3]));
let future = TwitterStreamBuilder::filter(Token::new("consumer_key", "consumer_secret", "access_token", "access_secret",))
.locations(bounds)
.listen()
.unwrap()
.flatten_stream()
.for_each(|json| {
println!("{}", json);
Ok(())
})
.map_err(|e| println!("error: {}", e));
rt::run(future);
}
This is my best shot, but I get an error that I haven't been able to resolve:
26 | .locations(bounds)
| ^^^^^^^^^ the trait `std::convert::From<((f64, f64), (f64, f64))>` is not implemented for `std::option::Option<&[((f64, f64), (f64, f64))]>`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.