Feature: Python script and module to check dataset readiness for data preservation

[astrochun]

Summary

With multiple datasets now published, we now need to ensure that the curated copies are identical to those that are published. This requires the following steps:

• A Preserve class as part of a sub-pckage called `preserve
  • Save a JSON file of the published metadata from the Figshare API in METADATA
  • Perform checksum verification to ensure the files on the curation server is identical. If not, provide warnings of differences
  • If checksum is incorrect, retrieve correct file and save in DATA (This is partially done, but still needs work and requires testing)
  • Generate symbolic links in DATA from ORIGINAL_DATA if files are unchanged and not present in DATA. See: Enhancement: Symbolic links to ORIGINAL_DATA for DATA when publishing #192
  • Delete all hidden files (e.g., .DS_Store, ._*)
  • Delete old README.txt files (CL: My recollection is that this was tested with a couple of old README.txt files)
  • To avoid overwrite, change the file permissions to read-only (This will be done in ReBACH in 6.Archived)
• A python script called preserve_checks that takes two main inputs/parameters: --article-id and --version-no
• Add option to do full run with update. By default, it would be "dry run". This would allow for the flexibility to check the script while not retrieving/updating published dataset or creating new metadata files for record.

Objectives

Proposal

Testing notes

Additional notes

Implemented in: TBD