parroty / extwitter Goto Github PK
View Code? Open in Web Editor NEWTwitter client library for elixir.
License: MIT License
Twitter client library for elixir.
License: MIT License
Hi,
I'm creating a sort of Twitter client using this library and i'm wondering if you had any plans to support multiples configurations (per process, for example) ? The ETS table currently used is global to a node.
I've quickly implemented a "local" configuration mode using process dictionary and the application env instead of ETS in my fork, but I wanted to know your plans before continuing working on it.
Looks like they moved to a new format: https://developer.twitter.com/en/docs/direct-messages/sending-and-receiving/guides/direct-message-migration
I was looking into making the changes, I can make a WIP PR, but I think I'll nee some help.
Deprecated endpoint New migration alternative POST direct_messages/new POST direct_messages/events/new GET direct_messages/show GET direct_messages/events/show GET direct_messages GET direct_messages/events/list GET direct_messages/sent GET direct_messages/events/list POST direct_messages/destroy DELETE direct_messages/events/destroy
Summary
- Message is defined in JSON POST body
- Content-type header must be set to application/json
- JSON body is not included in the generation of the OAuth signature.
What was the original design decision to raise
errors (e.g. in ExTwitter.Api.Base
) instead of returning an error tuple? As ExTwitter users we are forced to try ... rescue
code instead of just pattern matching against a more-standard {:ok, [...]}
or {:error, reason}
type return?
Just curious.
Without setting the environment variables $TWITTER_ACCESS_TOKEN
etc, run iex -S mix
and call
ExTwitter.stream_sample |> Enum.to_list
(Or set the environment variables to invalid values, or invalidate your credentials using the web interface, etc.)
The call fails immediately, logging or returning an auth error
The call does nothing for several seconds, then eventually times out without displaying an error.
Poison version gets in conflict with another applications.
This is my output in phoenix umbrella application:
Failed to use "poison" (version 3.1.0) because
/home/virviil/dev/elixir/market_face_umbrella/apps/twsubscriber
/mix.exs requires ~> 3.0
/home/virviil/dev/elixir/market_face_umbrella/deps/ueberauth_tw
itter/mix.exs requires ~> 1.3 or ~> 2.0 or ~> 3.0 or ~> 4.0
extwitter (version 0.8.3) requires ~> 2.0
guardian (version 0.14.4) requires >= 1.3.0
phoenix (version 1.3.0-rc.2) requires ~> 2.2 or ~> 3.0
mix.lock specifies 3.1.0
** (Mix) Hex dependency resolution failed, relax the version requ
irements of your dependencies or unlock them (by using mix deps.u
pdate or mix deps.unlock). If you are unable to resolve the confl
icts you can try overriding with {:dependency, "~> 1.0", override
: true}
I'm sure, that putting everywhere override: true
tuple is not the best way to solve this problem.
The same issue here - we are still in discussion what to do
I want to get the twitter streaming infinitely.
But if next message doesn't arrive in specific time, it will be interrupted by timeout.
Can I get the twitter streaming without timeout?
For example, I want to use following like:
ExTwitter.stream_filter(parameter, :infinite)
Twitter's API allows you to receive metadata for search results.
Using ExTwitter.search
I am able to search for tweets. As an example, I can search for 120 tweets about pizza near Portland with:
def search(topic, count, radius) do
options = [
count: count,
lang: "en",
geocode: "45.5231,-122.6765,#{radius}mi",
result_type: "recent"
]
ExTwitter.search("#{topic}", options)
|> IO.inspect
end
Running this in console, we can see that the amount of tweets returned is 100 with a = MyModule.search("pizza", 120, 500) |> Enum.count
No where in this list is any search metadata that includes a next_results
token for me to obtain the next twenty tweets. In the Twitter API, we should have meta data that might look like this
"search_metadata": {
"max_id": 250126199840518145,
"since_id": 24012619984051000,
"refresh_url": "?since_id=250126199840518145&q=%23freebandnames&result_type=mixed&include_entities=1",
"next_results": "?max_id=249279667666817023&q=%23freebandnames&count=4&include_entities=1&result_type=mixed",
"count": 4,
"completed_in": 0.035,
"since_id_str": "24012619984051000",
"query": "%23freebandnames",
"max_id_str": "250126199840518145"
}
My suspicion is that when the results are parsed, we are excluding this metadata.. I've forked this library and tested searching without parsing, and I have access to this metadata.
Is there currently a way we can parse the json to include the search_metadata
? If not, how can I contribute? I would love to have a feature that allows me to page through my data, because right now I am locked to only getting 100 results when I may need thousands.
I think adding the statuses/lookup endpoint would be a great addition to the library. It lets you pass in a comma separated string of ids and get the associated tweets.
Perfect for the use case when you want to collect stats from the tweets. E.g., favorites, retweets, and mentions. Instead of fetching the individual tweets you can get them all in one single call.
See https://developer.twitter.com/en/docs/tweets/post-and-engage/api-reference/get-statuses-lookup
Right now, if I perform ExTwitter.search("lol", count: 5, search_metadata: true)
, I get the following metadata payload:
%{
completed_in: 0.013,
count: 5,
max_id: 1113508101062262784,
max_id_str: "1113508101062262784",
next_results: "?max_id=1113508099283750911&q=lol&count=5&include_entities=1",
query: "lol",
refresh_url: "?since_id=1113508101062262784&q=lol&include_entities=1",
since_id: 0,
since_id_str: "0"
}
Notice the difference between the max_id
and the next_results
max_id param. The thing listed as max_id
is actually the latest tweet, as evidenced by its inclusion in the refresh_url
field as the since_id
. Grabbing the max_id out of next_results
allows pagination to work properly: using the raw max_id
field leads one to infinitely loop the same page.
When I get a chance I would be happy to PR a fix for this as well.
I am trying your code with
Tweetmap = ExTwitter.search("elixir-lang", [count: 10]) |>
Enum.map(fn(tweet) -> tweet.text end)
But the list never seems to go beyond 5 items. Not sure why.
Having a dependency on erlang-oauth creates some confusion and inconvenience for people using this library. For example: erlangpack/erlang-oauth#25 and erlangpack/erlang-oauth#29
Now that Elixir is more established it makes sense to replace erlang-oauth with an Elixir OAuth library such as oauther. This will make it easier for users of the library to specify their dependencies and make use of extwitter in their projects.
First off, let me say - I love this library :) But I have an issue... (I think just a warning in newer Elixir?)
This is related to the elixir_friends application. I'm on Elixir 1.1.1 at present, and I get the following error when starting up on the latest extwitter:
You have configured application :ex_twitter in your config/config.exs
but the application is not available.
This usually means one of:
1. You have not added the application as a dependency in a mix.exs file.
2. You are configuring an application that does not really exist.
Please ensure :ex_twitter exists or remove the configuration.
This is because the application is named :extwitter
but the configuration key being used is :ex_twitter
. I suppose the fix is to change it to use the key :extwitter
for the configuration. I don't believe it's presently breaking anything, but I figured it was worth noting. I'd be glad to send a PR to fix this if you'd like.
Thanks again!
Long story short, the Twitter API has this documented-in-one-place thing called tweet_mode
where you get the full_text
instead of the text
. If you pass metadata from a search
to a search_next_results
, it does not pass on the tweet_mode=extended
param. Even if I fudge it and manually add the &tweet_mode=extended
to next_results
on the metadata before passing it in, all tweets fetched by search_next_results
in this fashion will only return text
and not full_text
.
Since you can pass max_id
from next_results as an opt to plain search
in itself to get the next page of results, it seems prudent to perhaps rely on this and encourage the manual merging of opts, rather than using next_results
which is incomplete. (Alternatively, it can also just auto-merge the max_id
into the existing opts as a default).
If this isn't objectionable (I might be missing something about next_results
), then I'd be very happy to PR this. Let me know!
To reproduce:
ExTwitter.user_lookup([783214, 10230812, 507309896])
Result:
** (UnicodeConversionError) invalid code point 10230812
(elixir) lib/list.ex:727: List.to_string/1
(elixir) lib/enum.ex:1755: Enum."-join/2-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1755: Enum.join/2
(extwitter) lib/extwitter/parser.ex:119: anonymous fn/1 in ExTwitter.Parser.parse_batch_user_lookup_params/1
(elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2
(extwitter) lib/extwitter/api/users.ex:32: ExTwitter.API.Users.user_lookup/2
"tweet.retweeted" does not return true even if retweeted.
Following are how to reproduce it.
I expect "tweet.retweeted" to be true in the latter case.
/Users/niku/projects/extwitter% git show --oneline | head -1
84d5835 Requires elixir v1.0.0 for catching up to deprecations
/Users/niku/projects/extwitter% iex -S mix
Erlang/OTP 17 [erts-6.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Interactive Elixir (1.0.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ExTwitter.configure(
...(1)> consumer_key: System.get_env("TWITTER_CONSUMER_KEY"),
...(1)> consumer_secret: System.get_env("TWITTER_CONSUMER_SECRET"),
...(1)> access_token: System.get_env("TWITTER_ACCESS_TOKEN"),
...(1)> access_token_secret: System.get_env("TWITTER_ACCESS_SECRET")
...(1)> )
:ok
iex(2)> filter = [follow: "8571452"] # your twitter ID
[follow: "8571452"]
iex(3)> pid = spawn(fn ->
...(3)> stream = ExTwitter.stream_filter(filter)
...(3)> for tweet <- stream do
...(3)> IO.puts tweet.retweeted
...(3)> IO.puts tweet.text
...(3)> IO.puts inspect tweet
...(3)> IO.puts "----"
...(3)> end
...(3)> end)
#PID<0.139.0>
# And I tweet and re-tweet.
# https://twitter.com/niku_name/status/515108584942600192
# https://twitter.com/consadole/status/514946914282795008
false
def
%ExTwitter.Model.Tweet{contributors: nil, coordinates: nil, created_at: "Thu Sep 25 12:00:31 +0000 2014", entities: %{"hashtags" => [], "symbols" => [], "trends" => [], "urls" => [], "user_mentions" => []}, favorite_count: 0, favorited: false, geo: nil, id: 515108584942600192, id_str: "515108584942600192", in_reply_to_screen_name: nil, in_reply_to_status_id: nil, in_reply_to_status_id_str: nil, in_reply_to_user_id: nil, in_reply_to_user_id_str: nil, lang: "und", place: nil, retweet_count: 0, retweeted: false, source: "<a href=\"http://niku.name/\" rel=\"nofollow\">(´・肉・`)<どやっ</a>", text: "def", truncated: false, user: %ExTwitter.Model.User{contributors_enabled: false, created_at: "Sat Sep 01 02:34:50 +0000 2007", default_profile: false, default_profile_image: false, description: "北海道と沖縄と東京に長く住んでいます.Ruby,バイク,コンサドーレ札幌,格闘技が好きです.emacs 派.Elixir はじめました", entities: nil, favourites_count: 111, follow_request_sent: nil, followers_count: 2467, following: nil, friends_count: 2491, geo_enabled: true, id: 8571452, id_str: "8571452", is_translation_enabled: nil, is_translator: false, lang: "ja", listed_count: 175, location: "Sapporo, Hokkaido", name: "ヽ(´・肉・`)ノ", notifications: nil, profile_background_color: "FCFAF4", profile_background_image_url: "http://abs.twimg.com/images/themes/theme1/bg.png", profile_background_image_url_https: "https://abs.twimg.com/images/themes/theme1/bg.png", profile_background_tile: false, profile_image_url: "http://pbs.twimg.com/profile_images/1144575949/icon_normal.png", profile_image_url_https: "https://pbs.twimg.com/profile_images/1144575949/icon_normal.png", profile_link_color: "FF8C00", profile_sidebar_border_color: "FCFAF4", ...}}
----
false
RT @consadole: 【道スポ】コンサが「都倉チケット」 試合後ピッチで交流も 来月26日湘南戦10枚限定販売 http://t.co/hgcp9VWeuK #consadole
%ExTwitter.Model.Tweet{contributors: nil, coordinates: nil, created_at: "Thu Sep 25 12:00:52 +0000 2014", entities: %{"hashtags" => [%{"indices" => 'S]', "text" => "consadole"}], "symbols" => [], "trends" => [], "urls" => [%{"display_url" => "hokkaido-np.co.jp/news/consadole…", "expanded_url" => "http://www.hokkaido-np.co.jp/news/consadole/564721.html", "indices" => '<R', "url" => "http://t.co/hgcp9VWeuK"}], "user_mentions" => [%{"id" => 18762059, "id_str" => "18762059", "indices" => [3, 13], "name" => "コンサドーレ札幌非公式bot", "screen_name" => "consadole"}]}, favorite_count: 0, favorited: false, geo: nil, id: 515108673857662977, id_str: "515108673857662977", in_reply_to_screen_name: nil, in_reply_to_status_id: nil, in_reply_to_status_id_str: nil, in_reply_to_user_id: nil, in_reply_to_user_id_str: nil, lang: "ja", place: nil, retweet_count: 0, retweeted: false, source: "<a href=\"http://niku.name/\" rel=\"nofollow\">(´・肉・`)<どやっ</a>", text: "RT @consadole: 【道スポ】コンサが「都倉チケット」 試合後ピッチで交流も 来月26日湘南戦10枚限定販売 http://t.co/hgcp9VWeuK #consadole", truncated: false, user: %ExTwitter.Model.User{contributors_enabled: false, created_at: "Sat Sep 01 02:34:50 +0000 2007", default_profile: false, default_profile_image: false, description: "北海道と沖縄と東京に長く住んでいます.Ruby,バイク,コンサドーレ札幌,格闘技が好きです.emacs 派.Elixir はじめました", entities: nil, favourites_count: 111, follow_request_sent: nil, followers_count: 2467, following: nil, friends_count: 2491, geo_enabled: true, id: 8571452, id_str: "8571452", is_translation_enabled: nil, is_translator: false, lang: "ja", listed_count: 175, location: "Sapporo, Hokkaido", name: "ヽ(´・肉・`)ノ", notifications: nil, profile_background_color: "FCFAF4", profile_background_image_url: "http://abs.twimg.com/images/themes/theme1/bg.png", profile_background_image_url_https: "https://abs.twimg.com/images/themes/theme1/bg.png", profile_background_tile: false, profile_image_url: "http://pbs.twimg.com/profile_images/1144575949/icon_normal.png", profile_image_url_https: "https://pbs.twimg.com/profile_images/1144575949/icon_normal.png", profile_link_color: "FF8C00", profile_sidebar_border_color: "FCFAF4", ...}}
----
Placeholder for revisiting #56 for returning tuples instead of raising error.
Hi guys,
I'm writing to enquire about suitable architectural decisions one can take with using ExTwitter within a Supervisor in an OTP application?
In a prototype project, I spawn a child task running ExTwitter within a supervisor module.
However, I noticed that ExTwitter would timeout but I'm unable and don't know how to trap these timeout signals to restart the stream again.
Is there a way to do this within a supervisor module, maybe monitoring the pid of the stream or is this an anti-pattern? I've just started building OTP apps and still finding my way around. Thanks for your help in anyway !
As specified here: https://elixirforum.com/t/genstage-and-twitter-stream/1435/7?u=axelson
ExTwitter should not discard unknown messages from the process inbox because it has the potential to screw up genstage/flow
Had a dependency conflict with timex in this package and another one. From the changelog Drop Timex.Date.now() call in favor of native Erlang (#43).
and grep of the project it doesn't seem like timex is used anymore. Could it be removed if thats the case?
Hi there!
First of all, many thanks for this library.
I'm looking to use the newly-released ecto_timex
1.x in my project but it requires timex
2.x. extwitter
is locked to ~> 1.0.0
so I have a conflict.
Looking through extwitter
's usage of timex
, I noticed that its only usage is to calculate the seconds since epoch in ExTwitter.API.Base.parse_error/2
. The API has changed for timex
2.x so the solution isn't as simple as easing the version requirement.
Will submit a PR shortly with my proposed solution.
This is possibly due to a meck incompatibility.
Example:
1) test gets Twitter sample stream (ExTwitterStreamTest)
test/extwitter_stream_test.exs:30
** (ErlangError) Erlang error: {:compile_forms, {:error, [{[], [{:none, :compile, {:crash, :sys_core_fold, {{:case_clause, {:EXIT, {:function_clause, [{:sys_core_fold, :module, [[{:attribute, 1, :file, {'lib/extwitter/oauth.ex', 1}}, {:attribute, 1, :module, ExTwitter.OAuth_meck_original}, {:attribute, 1, :compile, [:no_auto_import]}, {:attribute, 1, :export, [__info__: 1, oauth_get: 7, oauth_post: 7, request: 7, request_async: 7, send_httpc_request: 3]}, {:attribute, 1, :spec, {{:__info__, 1}, [{:type, 1, :fun, [{:type, 1, :product, [{:type, 1, ...}]}, {:type, 1, :any, []}]}]}}, {:function, 0, :__info__, 1, [{:clause, 0, [{:atom, 0, :module}], [], [{:atom, 0, ExTwitter.OAuth}]}, {:clause, 0, [{:atom, 0, :functions}], [], [{:cons, 0, {:tuple, 0, ...}, {:cons, ...}}]}, {:clause, 0, [{:atom, 0, :macros}], [], [nil: 0]}, {:clause, 0, [{:match, 0, {:var, 0, ...}, {:atom, ...}}], [], [{:call, 0, {...}, ...}]}, {:clause, 0, [{:match, 0, {:var, ...}, {...}}], [], [{:call, 0, ...}]}, {:clause, 0, [{:match, 0, {...}, ...}], [], [{:call, ...}]}, {:clause, 0, [{:atom, 0, ...}], [], [nil: 0]}]}, {:function, 54, :get_signed_params, 7, [{:clause, 54, [{:var, 54, :_method@1}, {:var, 54, :_url@1}, {:var, 54, :_params@1}, {:var, 54, :_consumer_key@1}, {:var, 54, :_consumer_secret@1}, {:var, 54, ...}, {:var, ...}], [], [{:match, 55, {:var, 55, ...}, {:call, ...}}, {:call, 61, {:remote, ...}, [...]}]}]}, {:function, 34, :oauth_get, 7, [{:clause, 34, [{:var, 34, :_url@1}, {:var, 34, :_params@1}, {:var, 34, :_consumer_key@1}, {:var, 34, :_consumer_secret@1}, {:var, 34, ...}, {:var, ...}, {...}], [], [{:match, 35, {:var, ...}, {...}}, {:match, 37, {...}, ...}, {:match, 38, ...}, {:call, ...}]}]}, {:function, 42, :oauth_post, 7, [{:clause, 42, [{:var, 42, :_url@1}, {:var, 42, :_params@1}, {:var, 42, :_consumer_key@1}, {:var, 42, ...}, {:var, ...}, {...}, ...], [], [{:match, 43, {...}, ...}, {:match, 45, ...}, {:match, ...}, {...}]}]}, {:function, 68, :proxy_option, 0, [{:clause, 68, [], [], [{:call, 69, ...}]}]}, {:function, 9, :request, 7, [{:clause, 9, [{:atom, 0, :get}, {:var, 9, ...}, {:var, ...}, {...}, ...], [], [{:call, ...}]}, {:clause, 16, [{:atom, 0, ...}, {:var, ...}, {...}, ...], [], [{...}]}]}, {:function, 23, :request_async, 7, [{:clause, 23, [{:atom, 0, ...}, {:var, ...}, {...}, ...], [], [{...}]}, {:clause, 30, [{:atom, ...}, {...}, ...], [], [...]}]}, {:function, 50, :send_httpc_request, 3, [{:clause, 50, [{:var, ...}, {...}, ...], [], [...]}]}, {:function, 64, :stream_option, 0, [{:clause, 64, [], [], ...}]}], [:binary, :return_errors, :debug_info, :dialyzer, :no_spawn_compiler_process, :from_core, :no_auto_import]], [file: 'sys_core_fold.erl', line: 109]}, {:compile, :"-select_passes/2-anonymous-2-", 3, [file: 'compile.erl', line: 578]}, {:compile, :"-internal_comp/5-anonymous-1-", 3, [file: 'compile.erl', line: 342]}, {:compile, :fold_comp, 4, [file: 'compile.erl', line: 369]}, {:compile, :internal_comp, 5, [file: 'compile.erl', line: 353]}, {:compile, :"-do_compile/2-anonymous-0-", 2, [file: 'compile.erl', line: 177]}, {:meck_code, :compile_and_load_forms, 2, [file: '/Users/me/extwitter/deps/meck/src/meck_code.erl', line: 71]}, {:meck_proc, :backup_original, 3, [file: '/Users/me/extwitter/deps/meck/src/meck_proc.erl', line: 363]}, {:meck_proc, :init, 1, [file: '/Users/me/extwitter/deps/meck/src/meck_proc.erl', line: 206]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 365]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 333]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]}}}, [{:compile, :"-select_passes/2-anonymous-2-", 3, [file: 'compile.erl', line: 578]}, {:compile, :"-internal_comp/5-anonymous-1-", 3, [file: 'compile.erl', line: 342]}, {:compile, :fold_comp, 4, [file: 'compile.erl', line: 369]}, {:compile, :internal_comp, 5, [file: 'compile.erl', line: 353]}, {:compile, :"-do_compile/2-anonymous-0-", 2, [file: 'compile.erl', line: 177]}, {:meck_code, :compile_and_load_forms, 2, [file: '/Users/me/extwitter/deps/meck/src/meck_code.erl', line: 71]}, {:meck_proc, :backup_original, 3, [file: '/Users/me/extwitter/deps/meck/src/meck_proc.erl', line: 363]}, {:meck_proc, :init, 1, [file: '/Users/me/extwitter/deps/meck/src/meck_proc.erl', line: 206]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 365]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 333]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]}}}]}], []}}
code: test_with_mock "gets Twitter sample stream", ExTwitter.OAuth,
stacktrace:
/Users/me/extwitter/deps/meck/src/meck_proc.erl:96: :meck_proc.start(ExTwitter.OAuth, [])
test/extwitter_stream_test.exs:30: anonymous fn/2 in ExTwitterStreamTest."test gets Twitter sample stream"/1
(elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3
test/extwitter_stream_test.exs:30: (test)
Hey! Is it possible to add support for users/profile_banner
please?
15:24:34.648 [error] Error in process <0.101.0> with exit value: {undef,[{oauth,get,["https://stream.twitter.com/1.1/statuses/sample.json",[{<<5 bytes>>,<<8 bytes>>}],{"API","SECRET",hmac_sha1},"TOKEN....
And thats basically all I get, here is my code:
def start_bot() do
stream = ExTwitter.stream_sample(track: "word")
for message <- stream do
case message do
tweet = %ExTwitter.Model.Tweet{} ->
IO.puts "TWEET"
IO.inspect message
stall_warning = %ExTwitter.Model.StallWarning{} ->
IO.puts "STALL"
IO.inspect message
_ ->
IO.inspect message
end
end
end
Which I run in a mix task
I'm currently working on a project where I need to use the streaming APIs of twitter.
I added the dep to mix.exs and the tokens to the config.exs.
Still I am not able to use the API. I also double and triple checked the tokes and also regenerated them
on twitters side.
➜ iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.4.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ExTwitter.configure()
[consumer_key: "xxx",
consumer_secret: "xxx",
access_token: "xxx",
access_token_secrect: "xxx"]
iex(2)> ExTwitter.search("test")
** (ExTwitter.Error) Could not authenticate you.
(extwitter) lib/extwitter/api/base.ex:86: ExTwitter.API.Base.parse_error/2
(extwitter) lib/extwitter/api/search.ex:10: ExTwitter.API.Search.search/2
It would be handy when this library could also support the OAuth dance. I'm thinking along the lines of this:
ExTwitter.API.Authorize.request_token/0
-- calls Twitter to get a request tokenExTwitter.API.Authorize.authorize_url/2
(with oauth_token
and redirect_url
parameters) -- returns a url to which the user should be redirected; this is the Twitter logon / authorize page.ExTwitter.API.Authorize.access_token/3
(with verifier
, request_token
, request_secret
parameters) -- Verify that the request token was authorized and swap it for an access token.Hi, I was looking into the user streams and noticed that streams are being replaced by the Account Activity API. (They provide this migration guide.)
I was just wondering if you had this on your radar, and if this is something you plan to support or if its outside the scope of ExTwitter.
Thanks!
Adam
follow/1
API return Model of ExTwitter.Model.User
which has following: false
.
But, the Twitter Account succeed to follow. 😓
Right now when you use ExTwitter.Parser.parse_request_params/1
with a list as the value of the keyword list you'll get an ArgumentError or UnicodeConversionError, this happened when I was trying to use the Streaming module with the :follow
option and multiple ids. (This probably won't matter too much after August since the API changes Twitter is going to implement...) example:
This is the current implementation.
def parse_request_params(options) do
Enum.map(options, fn({k, v}) -> {to_string(k), to_string(v)} end)
end
Which can cause the above errors when used as:
iex(9)> Enum.map([follow: [12389212, 995_719_891_624_673_280]], fn {k, v} -> {to_string(k), to_string(v)} end)
** (ArgumentError) cannot convert the given list to a string.
To be converted to a string, a list must contain only:
* strings
* integers representing Unicode codepoints
* or a list containing one of these three elements
Please check the given list or call inspect/1 to get the list representation, got:
[12389212, 995719891624673280]
(elixir) lib/list.ex:821: List.to_string/1
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(stdlib) erl_eval.erl:878: :erl_eval.expr_list/6
(stdlib) erl_eval.erl:236: :erl_eval.expr/5
(elixir) lib/enum.ex:1294: Enum."-map/2-lists^map/1-0-"/2
iex(9)> Enum.map([follow: [12389212, 8369812]], fn {k, v} -> {to_string(k), to_string(v)} end)
** (UnicodeConversionError) invalid code point 12389212
(elixir) lib/list.ex:839: List.to_string/1
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(stdlib) erl_eval.erl:878: :erl_eval.expr_list/6
(stdlib) erl_eval.erl:236: :erl_eval.expr/5
(elixir) lib/enum.ex:1294: Enum."-map/2-lists^map/1-0-"/2
When a TCP/IP hickup occurs, the streaming API stops and does no longer receive any messages. The HTTP client returns {:error, :socket_closed_remotely}
to the controlling process but this message is not processed correctly by the receive_next_tweet
function; in this case, it should recover and re-create the HTTP connection; possibly with an exponential backoff.
This is easy to reproduce by creating a stream process and, while the stream is running, disconnecting the ethernet cable and plugging it in after 30 seconds.
Hey there, I noticed that the twitter streaming API can return two separate portions of tweets in a given message. I was receiving errors using the streaming API due to JSON parse errors. I took a look and saw it was trying to parse two JSON documents concatenated together.
[error] Error returned, stopping stream ({{:invalid, "{", 8142}, "{\"created_at\":\"Sat Oct 13 23:00:18 +0000 2018\"...
Twitter's API Doc's:
note that Tweets and other streamed messages will not necessarily fall on HTTP chunk boundaries – be sure to use the delimiter defined above to determine activity boundaries when reassembling the stream.
I took a shot and getting this to work and it seemed to solve the problem I was facing: #104
The latest change in stream_filter/2
causes a syntax error as follows:
== Compilation error on file lib/tweet_streamer/twitter_client.ex ==
** (SyntaxError) lib/tweet_streamer/twitter_client.ex:17: syntax error before: infinity
I tried to debug it but can't work out why the typespecs are failing?
I can see the field coming in, but whatever I do I cannot access it with ExTwitter.
Any pointers?
This library's been great so far - only hiccup until now.
Tested on Elixir 1.8.1.
$ mix docs
...
** (RuntimeError) module ExTwitter.API.Auth was not compiled with flag --docs
(ex_doc) lib/ex_doc/retriever.ex:152: ExDoc.Retriever.export_docs?/1
(ex_doc) lib/ex_doc/retriever.ex:136: ExDoc.Retriever.get_module/2
(elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
(ex_doc) lib/ex_doc/retriever.ex:116: ExDoc.Retriever.docs_from_modules/2
(ex_doc) lib/ex_doc.ex:101: ExDoc.generate_docs/3
(ex_doc) lib/mix/tasks/docs.ex:159: anonymous fn/6 in Mix.Tasks.Docs.run/3
(elixir) lib/enum.ex:1940: Enum."-reduce/3-lists^foldl/2-0-"/3
(ex_doc) lib/mix/tasks/docs.ex:158: Mix.Tasks.Docs.run/3
Caller can pass a number to follower_ids function in lib/extwitter/api/friends_and_followers.ex, and the number will be passed to the twitter API as a user_id parameter, while anything else is passed to the API as a screen_name parameter.
Currently the parameter is always treated as a screen name.
A simple change using guards and pattern matching. I have a working fix, and will submit a PR in a bit.
Hi! Thanks for building this library. It's awesome; we're using it over at: https://github.com/hashrocket/tilex
We're getting the following warnings at compile:
warning: Kernel.to_char_list/1 is deprecated, use Kernel.to_charlist/1
lib/extwitter/oauth.ex:38
warning: Kernel.to_char_list/1 is deprecated, use Kernel.to_charlist/1
lib/extwitter/oauth.ex:46
warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
lib/extwitter/proxy.ex:28
warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
lib/extwitter/proxy.ex:38
warning: String.to_char_list/1 is deprecated, use String.to_charlist/1
lib/extwitter/proxy.ex:39
I'd love try fixing them, but I'm not sure how to set up the application and get the tests passing. Thank you!
Hi!
I'm getting this error from time to time
** (EXIT from #PID<0.177.0>) an exception was raised:
** (Poison.SyntaxError) Unexpected end of input
lib/poison/parser.ex:54: Poison.Parser.parse!/2
lib/poison.ex:83: Poison.decode!/2
(extwitter) lib/extwitter/api/base.ex:60: ExTwitter.API.Base.parse_result/1
(extwitter) lib/extwitter/api/search.ex:10: ExTwitter.API.Search.search/2
(hashtag_graph) lib/graph/graph.ex:84: HashtagGraph.Graph.tweets/1
(hashtag_graph) lib/graph/graph.ex:102: HashtagGraph.Graph.get_vertex/1
(elixir) lib/enum.ex:1233: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1768: Enum."-map/2-anonymous-3-"/3
This may be related to #52
By inspecting my own code, can this error be a case of exceeding the API limit?
Thanks 😄
Please update poison dependency to somewhat ~> 2.0.0. its causing issues with other dependencies in even the release latest version 0.7.1.
Hello
Where can I configure a proxy server?
Thanks in advance
Matteo
http://www.redaelli.org/matteo/
Provide a way to control the stream.
http://stackoverflow.com/questions/25173666/how-do-i-interupt-an-infinite-stream
I'm trying to get the URL of a tweet after tweeting an update with the ExTwitter.update/1
function. Looking at the twitter update status example, there is a URL to the tweet under the entities field.
But, doing ExTwitter.update("update sample")
returns an ExTwitter.Model.Tweet
with the entities field all empty:
#ExTwitter.Model.Entities<
hashtags: [],
media: [],
polls: [],
symbols: [],
urls: [],
user_mentions: [],
...
>
How am I supposed to get the URL of the tweet after using ExTwitter.update/1
if the entities fields are empty?
Right now I'm just appending the id_str
field from the tweet to this url: https://twitter.com/i/web/status/{id_str} but not sure if this is conventional.
Disclaimer: I'm using {:poison, "~> 4.0", override: true}
in mix.exs since ExTwitter is set to 3.0.
Hey there, I'm trying out this repo to write my first elixir code and so far I've ran into some problems...
I cloned the repo and ran:
mix deps.get
mix test
All went fine
mix vcr.delete --all
mix test
All hell broke loose:
33 tests, 27 failures
All of them with:
27) test lookup status (ExTwitterTest)
test/extwitter_test.exs:213
** (ArgumentError) argument error
stacktrace:
src/jsx_parser.erl:171: :jsx_parser.object/4
lib/exvcr/json.ex:10: ExVCR.JSON.save/2
test/extwitter_test.exs:217
This might be an exvcr bug, but since both projects are yours you might know better.
I've spent some time trying to debug this but things started falling apart once I started trying to alter and rebuild code on the modules on the deps directory :( So much for a first night on elixir haha
ps: I'm running on Windows, I don't know if that's important on the elixir/erlang ecosystem
As far as I can tell, the current streaming implementation does not seem to receive status messages as documented here:
https://dev.twitter.com/streaming/overview/messages-types
(In particular, I am interested in being able to see limit and stall warnings, but for other people the status deletions are probably particularly important because failing to respect them can result in Twitter killing your dev account.)
Am I just missing something obvious?
In the decodes the keys of the result are always cast to an atom. Because of the limitations on atoms and because atoms never get garbage collected this creates an attack vector for denial of service attacks since there is no way to determine the Twitter API is safe.
https://github.com/devinus/poison#parser
Since most JSON data is already parsed to structs, maybe the data should be parsed with with the as:
argument which parses the data to structs through Poison.
This however does require quite an extensive refactor since parsing is most often done after decoding the data.
Hello,
I am using this project to learn about elixir.
Right now I am fetching the master branch due to problems in JSON with 0.3.0 solved in master.
Would be lovely to have a version fixed, just in case 👍
Thanks
First of all thanks for this great library. I tried deploying a phoenix app that uses Extwitter with Docker and I keep getting this error:
01:05:20.529 [error] Error in process #PID<0.1154.0> on node :"[email protected]" with exit value:
{:undef,
[{OAuther, :credentials,
[[consumer_key: "MYCONSUMERKEY",
consumer_secret: "MYSECRET",
token: "MYTOKEN",
token_secret: "MYTOKENSECRET"]], []},
{ExTwitter.OAuth, :get_signed_params, 7,
[file: 'lib/extwitter/oauth.ex', line: 55]},
{ExTwitter.OAuth, :oauth_post, 7, [file: 'lib/extwitter/oauth.ex', line: 43]},
{ExTwitter.API.Streaming, :"-spawn_async_request/1-fun-0-", 2,
[file: 'lib/extwitter/api/streaming.ex', line: 64]}]}
The app runs locally in production mode just fine. The correct credentials are in there, I just replaced them.
Any hint on what is going wrong is greatly appreciated! Thanks!
On
Line 875 in e77527d
verify_credentials
call, is that expect?Hi,
I noticed that the ExTwitter parser does a custom struct for the "user" element in a tweet:
defmodule ExTwitter.Parser do
def parse_tweet(object) do
tweet = struct(ExTwitter.Model.Tweet, object)
user = parse_user(tweet.user)
%{tweet | user: user}
end
# ...
def parse_user(object) do
struct(ExTwitter.Model.User, object)
end
Currently, the user struct seems to be out of sync with the JSON being returned. As a result, there are differences between the JSON encoded ExTwitter.Model.Tweet (left) and the original JSON tweet (right):
Specifically, the user element in the tweet is missing the translator_type
field and has extra fields that are always null
because they are not present in the incoming tweet:
"status": null,
"email": null,
"withheld_scope": null,
"entities": null,
"is_translation_enabled": null,
"withheld_in_countries": null,
"show_all_inline_media": null,
Would it be possible to update the User struct to remove this mismatch? Or perhaps just eliminate the special handling for the "user" element?
Happy to help with this.
I am able to successfully stream from twitter when I spawn a new process
def stream(term) do
spawn(fn ->
stream = ExTwitter.stream_filter(track: term, receive_messages: true)
for message <- stream do
IO.puts message.text
end
end)
end
I can call this function and stream terms like, "pizza" or "cats". All is well. However, when I stream a very talked about term, like "trump", I am able to stream for about 5 seconds until I receive an error.
16:34:18.968 [error] Process #PID<0.156.0> raised an exception
** (KeyError) key :text not found in: %ExTwitter.Model.Limit{track: 4}
(sentient) lib/sentient.ex:26: anonymous fn/2 in Sentient.stream/1
(elixir) lib/enum.ex:1777: anonymous fn/3 in Enum.reduce/3
(elixir) lib/stream.ex:1259: Stream.do_resource/5
(elixir) lib/enum.ex:1776: Enum.reduce/3
(sentient) lib/sentient.ex:25: anonymous fn/1 in Sentient.stream/1
From what I can tell, a message
came in and this message
does not have a property of text
. Do some results from ExTwitter.stream_filter()
not return the struct %ExTwitter.Model.Tweet{}
? How can I account for the various types of results in a stream?
If this question is not appropriate here, I can move it to stack overflow. I'm asking it here because I cannot find an answer to this through the docs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.