Document Type Support

[wearpants]

Adds preliminary support for a documentType field for crossref & plos harvesters

[coveralls]

Coverage increased (+0.2%) to 93.642% when pulling 536cd97 on wearpants:feature/document-type into 0528257 on CenterForOpenScience:develop.

[wearpants]

Pull request with initial support for a documentType field for crossref and plos. See SHARE-294 in Jira. /cc @chrisseto It supports the following values: "article", "abstract", "dataset", "book", "book-chapter", "dissertation", "correction", "preprint", "source-code", "clinical-trial", "reference-entry", "monograph"

To update production, we'll need to renormalize after merging: invoke migrate renormalize -s 'crossref,plos' --no-dry. I estimate this will take about 23 machine hours - may be able to get some speedup by using more celery tasks, but not 100% sure where the bottleneck is. Be sure to merge branch on all harvester machines before running this migration.

@alexschiller if you'd like to start adding frontend support, checkout the feature/document-type branch from my fork: https://github.com/wearpants/scrapi/ ; You can either renormalize an existing database or re-run the crossref & plos harvesters to generate new data. The OSF web server should pass through documentType field in share_search API results.

[chrisseto]

You might have to add $oneOf here as most of the time this field will be None

[wearpants]

Yup sure. Way to do that is adding null http://spacetelescope.github.io/understanding-json-schema/reference/generic.html

[coveralls]

Coverage increased (+0.2%) to 93.642% when pulling f0941bd on wearpants:feature/document-type into 0528257 on CenterForOpenScience:develop.