sparkapi / spark_api Goto Github PK

After talking with Josh. It sounds like the default standard is that all resources should support POST requests using a single Hash instead of an array.

This is just to remind me to look into that, since there's the potential of getting rid of all that pluralization logic crap.

He mentioned some of the older resources might not support that, but at least I can look into it and get a list or something. All new and future resources should support it though from what it sounds like.

https://github.com/sparkapi/spark_api/blob/master/lib/spark_api/models/finders.rb#L33

Could we just implement that to do the same thing as find except pass _limit: 1? That might make for some speedier requests in some cases.

Edit: also I can add this to my fork if it makes sense so we can go ahead and start using it.

I have a requirement, in which I need to update listing cart name only. But when I tried with this code. It seemed that name is not getting updated.

cart = SparkApi::Models::ListingCart.find(cart_id)
cart.Name = options[:name]
cart.save

Some companies will only use gems with a certain license.
The canonical and easy way to check is via the gemspec,

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Even for projects that already specify a license, including a license in your gemspec is a good practice, since it is easily
discoverable there without having to check the readme or for a license file. For example, it is the field that rubygems.org uses to display a gem's license.

For example, there is a License Finder gem to help companies ensure all gems they use
meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough
issue that even Bundler now generates gems with a default 'MIT' license.

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), github has created a license picker tool.

In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :).

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue and let me know. In either case, I'll follow up. Thanks!

p.s. I've written a blog post about this project

I am seeing references to https://developers.sparkapi.com and https://developers.sparkplatform.com in this project, however both of these endpoints appear to no longer be functioning.

Currently I have a demo user, and while I can authenticate and get a token back for this user, I am unable to query for any data. Is there any way to get these developer endpoints working with demo accounts, or is there another way I should be going about this?

It is for RESO Web API calls, though (via nextlink). Example:

SparkApi.client.get "/listings?_skiptoken="

returns results but not the SkipToken

https://github.com/sparkapi/spark_api/blob/master/lib/spark_api/connection.rb#L15

We will have to add this as a setting, or turn this off on non production environments so we don't get errors on dev and such.

API requests include a Request-Id header that can be useful for debugging. We should include that at the debug level in SparkApi::Request#request

https://github.com/ifightcrime/spark_api/blob/master/lib/spark_api/models/finders.rb#L14

I was wondering if that can just be removed since it throws a not implemented exception anyway. Is the idea that that can be overridden by a model?

Add VCR gem so we can start writing tests with some real data. Particularly GET requests. I'm not sure how/if we want to implement live requesting for operations that modify data through the api (PUT/POST/DELETE). Either way, it would be good to get some real data in there that we can test against.

Found my crud and old parallel implementation: https://github.com/wlmcewen/spark_api/tree/RUBYAPI-6

If you use it, the Response class here should be integrated into the newer ApiResponse class. I'm sure there are plenty of issues, but it gets the job done.

When I upgraded our mobile app to the current spark_api gem I started getting warnings when running our rspec tests.

Example:
/home/bhines/mobile/vendor/bundle/ruby/1.8/gems/spark_api-1.2.0/lib/spark_api/models/dirty.rb:48: warning: Object#id will be deprecated; use Object#object_id

When working with a token that needs a refresh. The following error will arise unless I set the oauth2_provider.code attribute to a non nil value:

E, [2012-08-09T09:14:24.863684 #27221] ERROR -- : Authentication failed or server is sending us expired tokens, nothing we can do here.
/usr/local/rvm/gems/ree-1.8.7-2011.03/gems/spark_api-1.1.0/lib/spark_api/faraday_middleware.rb:49:in on_complete': The token you provided is invalid. A new authorization is required. (SparkApi::PermissionDenied) from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/faraday-0.8.1/lib/faraday/response.rb:9:incall'
from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/faraday-0.8.1/lib/faraday/response.rb:63:in on_complete' from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/faraday-0.8.1/lib/faraday/response.rb:8:incall'
from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/faraday-0.8.1/lib/faraday/connection.rb:226:in run_request' from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/faraday-0.8.1/lib/faraday/connection.rb:87:inget'
from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/spark_api-1.1.0/lib/spark_api/authentication/oauth2.rb:50:in send' from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/spark_api-1.1.0/lib/spark_api/authentication/oauth2.rb:50:inrequest'
from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/spark_api-1.1.0/lib/spark_api/request.rb:72:in request' from /usr/local/rvm/gems/ree-1.8.7-2011.03/gems/spark_api-1.1.0/lib/spark_api/request.rb:15:inget'

We are developing an app using your gem (version 1.3.10).
The flow is that users will authenticate via oauth against the spart platform. Which yields us a user token.
However we're confused about how to allow many user accounts to work simultaneously.

As far as I understand your gem does not allow to handle multiple concurrent requests using different user accounts. This is because of static implementation of connection (https://github.com/sparkapi/spark_api/blob/master/lib/spark_api.rb#L34) that is used from models (https://github.com/sparkapi/spark_api/blob/master/lib/spark_api/models/base.rb#L48)

Am I correct? If so do you have any suggestions or future plans to implement this?

It looks like we're sending "D" => {} with requests even if there are no other attributes or data being sent.

I have never seen refreshing an oauth token actually work. Tokens expire and require the agent to enter credentials to get fresh tokens. This is bad for us because we want to get fresh listing data on behalf of agents automatically in the background, without their direct involvement. We're hoping that the use of refresh tokens can do this.

I've got sample code now that actually shows the refresh process failing. The root of the issue seems to be when the gem calls to create a session to https://sparkapi.com/v1/oauth2/grant with the param "grant_type" set to "refresh_token". The response is {"error"=>"invalid_grant", "error_description"=>"The access grant you supplied is invalid"}.

Attached is a ruby script using the latest gem v1.4.14 that shows the issue. It's based on script/combined_flow_example.rb, but with the added twist of forcing the tokens to expire so that we can attempt a refresh.

The best way to run this is from irb by pasting in one SECTION at a time. This allows you to see the debug logging. Recognize that you need to paste your key, secret, and callback on lines 8-10, and a valid client.oauth2_provider.code in on line 27.

Let me know if I'm not understanding how refresh tokens work, or if there's a better path through the code that does cause it to work.

Log output for me showing the exact error:
D, [2017-07-06T12:15:07.895235 #54093] DEBUG -- : [oauth2] setup SparkApi::Authentication::OAuth2Impl::GrantTypeCode
D, [2017-07-06T12:15:07.895297 #54093] DEBUG -- : [oauth2] Refresh oauth session.
D, [2017-07-06T12:15:07.895330 #54093] DEBUG -- : [oauth2] Refreshing authentication to https://sparkapi.com/v1/oauth2/grant using [cbg4cv1goko1sahh9u4szzmkf]
D, [2017-07-06T12:15:07.895399 #54093] DEBUG -- : [oauth2] create_session to https://sparkapi.com/v1/oauth2/grant params {"redirect_uri":"https://cloudcma.dev/spark/callback","client_id":"xxx","client_secret":"yyy","grant_type":"refresh_token","refresh_token":"cbg4cv1goko1sahh9u4szzmkf"}
D, [2017-07-06T12:15:08.425257 #54093] DEBUG -- : [oauth2] Response Body: {"error"=>"invalid_grant", "error_description"=>"The access grant you supplied is invalid"}

spark_refresh.rb.txt

Hi,

I am new to spark api and I am just trying to generate all listings in a new rails app.

Here are my steps:

• Add to gemfile
  gem 'spark_api'

• bundle install

• Create a listings controller
  class ListingsController < ApplicationController
def index
@listings = Listing.find(:all)
end
end

• Create a view
  <h2>Listings</h2>
<ul>
<% @listings.each do |listing| %>
<li><%= listing.ListingKey %></li>
<% end %>
</ul>

• Create settings.yml in config/spark_api/settings.yml
  development:
api_key: 'my_api_key'
api_secret: 'my_secret_key'

So what am I missing here? Thank you for your response!

Add something like http://apidock.com/rails/HashWithIndifferentAccess/symbolize_keys! to clean the input parameters.

This is my final query which becomes like,

{:_select=>"Id,ListingKey,ListPrice,BathsTotal,BedsTotal,LotSizeArea,StreetNumber,StreetDirPrefix,StreetName,StreetSuffix,StreetDirSuffix,StreetAdditionalInfo,City,PostalCode,YearBuilt,WaterFrontYN,DaysOnMarket,ListingId,ListingPrefix,PhotosCount,OnMarketDate,MlsStatus,BuildingAreaTotal,Latitude,Longitude,CustomFields", :_expand=>"PrimaryPhoto", :_limit=>10, :_pagination=>1, :_filter=>"(MlsStatus Eq 'Closed' OR MlsStatus Eq 'Expired') AND (PropertyType Eq 'A')", :_orderby=>"-ListPrice", :_page=>1}

If I fire this query, I get this error

SparkApi::ClientError (The requested operation took too long to perform. If running a search, try simplifying or adding additional restrictions to your _filter. If ordering, try including the _orderby criteria in your _filter parameter.)

I also added order_by and page criteria in the filter, still it gives me that error.

It works great on other status like Active, Active Contingent, Pending and Temp Off Market.

Occassionally, the API (or more likely: the load balancer) will return anon-JSON response and we're handling that poorly:

MultiJson::ParseError lexical error: invalid char in json text.
                                       <h1>This website is under heavy
                     (right here) ------^
  
  /home/httpd/mls_search/prod/shared/bundle/ruby/2.3.0/gems/multi_json-1.12.1/lib/multi_json/adapters/yajl.rb:11:in `parse'
  /home/httpd/mls_search/prod/shared/bundle/ruby/2.3.0/gems/multi_json-1.12.1/lib/multi_json/adapters/yajl.rb:11:in `load'
  /home/httpd/mls_search/prod/shared/bundle/ruby/2.3.0/gems/multi_json-1.12.1/lib/multi_json/adapter.rb:21:in `load'
  /home/httpd/mls_search/prod/shared/bundle/ruby/2.3.0/gems/multi_json-1.12.1/lib/multi_json.rb:122:in `load'
  /home/httpd/mls_search/prod/shared/bundle/ruby/2.3.0/gems/spark_api-1.4.13/lib/spark_api/faraday_middleware.rb:20:in `on_complete'

I want to create a blank listing cart without providing its name and listing ids.
Is it possible to do that?

The OAuth2 examples have about 4-5 steps which easily lead to developers running into expired tokens and other authentication failures. Clean up the examples so that they can get up and running with OAuth2 faster, and with better logging about what is going on behind the scenes.

Add warning statements to the command line oauth2 provider to advise that this provider is not intended for production rails use.

I encountered the same issue as this fellow after running 'gem install spark_api' and running the 'spark_api' script. When I tried to issue a request, it failed with the same error. Am running 1.9.3 under rvm in a fresh gemset.

http://tom.meinlschmidt.org/2011/11/01/cucumber-yajl-troubles-osx-lion/

A suggestion in the comments is to upgrade yajl ruby which is now two minor version numbers ahead of the one in the gemspec.

If I am able I may be able to fork and test a fix but for now I rebuilt the yajl.bundle as suggested after modifying the files in place.

SparkApi::Models::Base should correctly override respond_to?.
As we can see in documentation to Object#respond_to? it accepts two parameters. But current SparkApi::Models::Base#respond_to? accepts only one.

This implementation produces many warnings with ruby 2.1.1 like

warning: SparkApi::Models::Photo#respond_to?(:_dump) is old fashion which takes only one parameter

This involves:

1. allowing access_tokens to be loaded from configuration
2. creating an oauth provider file to automatically activate a new session with that access_token.

I'm receiving sporadic timeouts when running from the Interactive Console and HTTP Interface. Any particular reason?

Faraday::Error::TimeoutError: execution expired
    from /usr/lib/ruby/2.1.0/net/http.rb:879:in `initialize'
    from /usr/lib/ruby/2.1.0/net/http.rb:879:in `open'
    from /usr/lib/ruby/2.1.0/net/http.rb:879:in `block in connect'
    from /usr/lib/ruby/2.1.0/timeout.rb:91:in `block in timeout'
    from /usr/lib/ruby/2.1.0/timeout.rb:101:in `call'
    from /usr/lib/ruby/2.1.0/timeout.rb:101:in `timeout'
    from /usr/lib/ruby/2.1.0/net/http.rb:878:in `connect'
    from /usr/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
    from /usr/lib/ruby/2.1.0/net/http.rb:852:in `start'
    from /usr/lib/ruby/2.1.0/net/http.rb:1369:in `request'
    from /var/lib/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:75:in `perform_request'
    from /var/lib/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/adapter/net_http.rb:38:in `call'
    from /var/lib/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/response.rb:8:in `call'
    from /var/lib/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:253:in `run_request'
    from /var/lib/gems/2.1.0/gems/faraday-0.8.9/lib/faraday/connection.rb:118:in `post'
    from /var/lib/gems/2.1.0/gems/spark_api-1.3.16/lib/spark_api/authentication/api_auth.rb:23:in `authenticate'
    from /var/lib/gems/2.1.0/gems/spark_api-1.3.16/lib/spark_api/authentication.rb:24:in `authenticate'
    from /var/lib/gems/2.1.0/gems/spark_api-1.3.16/lib/spark_api/request.rb:61:in `request'
    from /var/lib/gems/2.1.0/gems/spark_api-1.3.16/lib/spark_api/request.rb:15:in `get'
    from (irb):1
    from /usr/bin/irb:11:in `<main>'

The protected write_attribute method of SparkApi::Models::Base checks whether the value being set for an attribute is already set to the assigned value. This causes inconsistent behavior when the attribute is new and the assigned value is nil because attributes is a hash and returns nil when a key does not exist.

l = SparkApi::Models::Listing.new
l.attributes # {}
l.City       # NoMethodError: undefined method `include?' for nil:NilClass
l.State = 'ND'
l.City = nil
l.State      # "ND"
l.City       # NoMethodError: undefined method `include?' for nil:NilClass
l.attributes # {"State"=>"ND"}
l.attributes['City'] = nil
l.City       # nil
l.attributes # {"State"=>"ND", "City"=>nil}
l.changes    #{"State"=>[nil, "ND"]}

This can be partially worked around by writing to the attributes hash directly. However, as illustrated above, that isn't a full-featured approach with respect to the SparkApi::Models::Dirty module at least.

On second thought, l.changes isn't a great example of the workaround not being full-featured. However, the general point remains that this is inconsistent behavior.

Not sure if the test is bad, or the code just yet, but it seems that we have an off by one error when calculating times here: https://github.com/sparkapi/spark_api/blob/master/lib/spark_api/models/subresource.rb#L17