sul-dlss-deprecated / dor-fetcher-service Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 1.0 43.57 MB

Provides harvesting capabilities for druids in SDR

Ruby 95.19% HTML 4.81%

application infrastructure rails-api

dor-fetcher-service's People

Contributors

Watchers

Forkers

atz

dor-fetcher-service's Issues

adapt to new solr index (dor-services 5.x)

See config/solr_terms.yml:
https://github.com/sul-dlss/dor-fetcher-service/blob/purl-fetcher/config/solr_terms.yml

Note: purl-fetcher branch is the development tip, 6+ months ahead of master.

DOR Fetcher collection call does not return members

DOR Fetcher is getting no results from a membership query for either collections or APOs, e.g.,

https://dorfetcher-prod.stanford.edu/collections/druid:sj775xm6965
https://dorfetcher-prod.stanford.edu/collections/druid:wr005wn5739

The log in production is is showing field names from the old index (e.g., is_member_of_collection_s):

I, [2016-06-30T10:26:27.436466 #7778]  INFO -- : Started GET "/collections/druid:sj775xm6965" for 171.66.28.61 at 2016-06-30 10:26:27 -0700
I, [2016-06-30T10:26:27.440305 #7778]  INFO -- : Processing by CollectionsController#show as JSON
I, [2016-06-30T10:26:27.440482 #7778]  INFO -- :   Parameters: {"id"=>"druid:sj775xm6965"}
I, [2016-06-30T10:26:27.519939 #7778]  INFO -- : Request from 171.66.28.61 to /collections/druid:sj775xm6965 at 2016-06-30 10:26:27 -0700
I, [2016-06-30T10:26:27.520165 #7778]  INFO -- : Solr query: {:q=>"(is_member_of_collection_s:\"info:fedora/druid:sj775xm6965\" OR id:\"druid:sj775xm6965\") AND published_dt:[\"1970-01-01T00:00:00Z\" TO \"9999-12-31T23:59:59Z\"]", :wt=>:json, :fl=>"id AND published_dt AND identityMetadata_objectType_t AND title_t AND public_dc_title_t AND catkey_id_t", :rows=>100000000}

I, [2016-06-30T11:02:21.526456 #25187]  INFO -- : Started GET "/apos/druid:wr005wn5739" for 171.67.45.15 at 2016-06-30 11:02:21 -0700
I, [2016-06-30T11:02:21.527683 #25187]  INFO -- : Processing by AposController#show as JSON
I, [2016-06-30T11:02:21.527752 #25187]  INFO -- :   Parameters: {"id"=>"druid:wr005wn5739"}
I, [2016-06-30T11:02:21.564482 #25187]  INFO -- : Request from 171.67.45.15 to /apos/druid:wr005wn5739 at 2016-06-30 11:02:21 -0700
I, [2016-06-30T11:02:21.564779 #25187]  INFO -- : Solr query: {:q=>"(is_governed_by_s:\"info:fedora/druid:wr005wn5739\" OR id:\"druid:wr005wn5739\") AND published_dt:[\"1970-01-01T00:00:00Z\" TO \"9999-12-31T23:59:59Z\"]", :wt=>:json, :fl=>"id AND published_dt AND identityMetadata_objectType_t AND title_t AND public_dc_title_t AND catkey_id_t", :rows=>100000000}

The correct field names are mostly correct in the solr_terms.yaml in github

  apo_field:         "is_governed_by_ssim"
  collection_field:  "is_member_of_ssim"
  fedora_type_field: "objectType_ssim"
  last_changed:      "published_dttsim"
  title_field:       "title_tesim"
  title_field_alt:   "public_dc_title_tesim"
  catkey_field:      "catkey_id_ssim"

except that "is_member_of_ssim" should be "is_member_of_colleciton_ssim"

spec_helper method find_druid_in_array is trivial AND incorrect

See:
07c81e9#diff-93830fa29d616f7c87903d08b5b1b29aR221

The conditional actually performs assignment here.

Ruby's Array already has .find. Use it and remove this helper kludge crap.

Document how to run purl_fetcher branch as dor-fetcher

Provide configuration for both uses.

Use shared_configs

Move server configuration management to the shared_configs model.

Merge purl-fetcher branch

Some 75 commits are outstanding and we want to avoid any complex divergence.

Tag master branch with current position in case we need to come back to it,
PR (should be fast forwardable),
review and merge

There may be downstream considerations to deployment, like allowing us to consolidate 2 existing instances into one.

determine_latest_date behavior bogus

Description says "If no timeframe provided in the params, it is just the latest date", which makes sense, but then the actual code just returns nil:

https://github.com/sul-dlss/dor-fetcher-service/blob/purl-fetcher/lib/fetcher.rb#L309

Upgrade to rails 5.2

method responds to xml, calls render :json

https://github.com/sul-dlss/dor-fetcher-service/blob/purl-fetcher/app/controllers/application_controller.rb#L31

Isn't this mistaken?

    respond_to do |format|
      format.json {render :json => result.to_json}
      format.xml {render :json => result.to_xml(:root => 'results')}
    end

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.