Feature/lastharvest#509
Open
MerlinZhang wants to merge 7 commits intoCenterForOpenScience:developfrom
Open
Conversation
Included LastHarvest model to keep track of most recently harvested metadata for each scrapi source
Added new serializer for LastHarvest model
Added new url to display database information for most recent harvests, sorted by source
Created view to display most recent harvest information as a list of json objects
Added method last_harvested() to update LastHarvest database when more recent metadata is harvested Included method call in process_normalized to only store recent updates when the metadata is normalized
| @@ -0,0 +1,29 @@ | |||
| # from api.webview.models import Document | |||
There was a problem hiding this comment.
This file needs to be removed from the git repository.
|
Can you add tests to this feature? We normally would require tests to be implemented for every new feature. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New Feature
Keeps track of when sources were most recently harvested.
A new database model called LastHarvest keeps records of each source and the most recent date when it was harvested. The database is updated every time a new document from a more recent date is normalized.
Code updates include a new view and url to view the database information for most recently updated sources.