Take action on error logs to improve data quality in KG about rcgraph

[lobodemonte]

After looking at Failures for Step 2, sources of failures seem to come from:

1. Title differs on special character (quotes, hyphens, etc
2. Title with encoding issue or cut-off special character
3. API Source not used (e.g. RePEc, SSRN)

[lobodemonte]

I reran the failed titles (step_2) (763 total) with other API's and found that by adding 3 more APIs we could decrease the number of failed lookups by 25%, but that leaves a huge chunk of titles with possible typing/encoding-related errors
CrossRef: (found/total)
33/763
CrossRef + PubMed:
38/763
CrossRef + DataCite:
109/763
Core:
95/763
CrossRef + DataCite + Core:
195/763

[ceteri]

Thank you @lobodemonte -
Sounds like the improved title matching (less exact matches) may help here.

[ceteri]

Nice work! That's super-helpful to guide how we leverage the APIs in the workflow.

For the typing/encoding-related errors, is it mostly the case that the titles we have are incorrect?

[lobodemonte]

Hi @ceteri,

Yes, the rest seem to be either some minor difference with the actual title and what we have in our data partitions, there's also a small portion of titles that won't have a doi (e.g. RePEc publications)