Add s3 provider #1009
Open
Add s3 provider #1009
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dolfandringa
commented
May 19, 2022
| "Content-Disposition", | ||
| "attachment; filename={};".format(filename.replace(",", "_")), | ||
| ) | ||
| if self._downloaded_path == path and self._downloadable_data is not None: |
There was a problem hiding this comment.
I wanted to prevent re-fetching the file from s3. Maybe there is a beter way to cache the boto3 s3 transfer using memcached somehow. Any suggestions are welcome.
SylvainCorlay
force-pushed
the
add_s3_provider
branch
from
ca1b1c6
to
db0a04f
Compare
… subprocess.POpen, so we can't mock anything.
SylvainCorlay
force-pushed
the
add_s3_provider
branch
from
db0a04f
to
25e9afa
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This addresses #630 . It allows downloading notebooks from s3 buckets through the S3 uri (
s3://bucket/path/to/key). If the file/bucket is public, this should work straight away. But if it is a private bucket, it can be configured by setting the appropriate environmental variables in the docker container or providing the Shared credentials file to the same user running NBViewer.I attempted to write a test too, but since the test server is started with
subprocess.POpenwe can't mock boto3. So that test has been commented for now. If there is a way to start the server without using subprocess, that would allow mocking, I could uncomment (and expand) that test.