Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] download zip #882

Merged
merged 28 commits into from
Aug 22, 2023

Conversation

danibene
Copy link
Collaborator

Description

This PR aims to allow for zip files to be automatically downloaded and extracted, as a first step towards automating the downloading and formatting of various datasets (see Tam-Pham/HRVStructure#2 (comment))

Proposed Changes

I added a download_zip() function to a database.py file (was thinking that this could be where the refactored functions for downloading individual databases could be moved to within NeuroKit), and I used this function for the fantasia and mit arrhythmia databases.

Let me know if you want me to change the naming (wasn't sure about database.py), and also if you have any thoughts on whether there should be automated tests for this (not sure if downloading entire datasets is a good idea with GitHub Actions)

Checklist

  • I have read the CONTRIBUTING file.
  • My PR is targetted at the dev branch (and not towards the master branch).

import zipfile
from pathlib import Path

def download_zip(url, destination_path):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we do this somewhat more "generic", like something like download_from_url() and then there is an unzip=True argument that triggers, if the content is detected to be a zipped file, automatically unzips it?

So that we could use that function for both zipped un non-zipped stuff? (although I'm not sure if there would be such a need)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I just realized you meant downloading any files from a url and not just a zip. Yeah sure I can do that too (though I feel like then we should still have download_zip() that calls download_from_url(), since there are zip-specific functionalities in the function like removing the original zip file), but I have to go now, can get back to it in a few days probably

@codecov-commenter
Copy link

codecov-commenter commented Aug 11, 2023

Codecov Report

Patch coverage: 22.50% and project coverage change: -0.12% ⚠️

Comparison is base (ed5e999) 55.13% compared to head (4f704ae) 55.01%.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #882      +/-   ##
==========================================
- Coverage   55.13%   55.01%   -0.12%     
==========================================
  Files         298      299       +1     
  Lines       13956    13993      +37     
==========================================
+ Hits         7694     7698       +4     
- Misses       6262     6295      +33     
Files Changed Coverage Δ
neurokit2/data/database.py 18.42% <18.42%> (ø)
neurokit2/data/__init__.py 100.00% <100.00%> (ø)
neurokit2/data/data.py 56.25% <100.00%> (ø)

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DominiqueMakowski
Copy link
Member

Looks good to me, should we merge?

@danibene danibene merged commit 899ffa7 into neuropsychology:dev Aug 22, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants