Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement an early-decrypting check #5171

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

kdm9
Copy link

@kdm9 kdm9 commented May 3, 2020

Hello,

(NB: this is definitely a draft)

This is my quick attempt at implementing the early-decrypting behaviour asked for in #5170. I'm not sure if it behaves exactly as I wish, as i haven't trusted it to run on a real repo until it passes a review in case i've screwed something up. The tests that relate to borg check or archive checking seem to pass fine, but there are a couple of tests failing for reasons I can't easily fathom (hoping these failures go away in CI).

Cheers,
Kevin

verify_data=False, save_space=False):
"""Perform a set of checks on 'repository'
"""A checker of repostiory archives
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

Comment on lines -1462 to +1471
def verify_data(self):
def do_verify_data(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you rename this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we now have a boolean flag we need to save to self:

def __init__(self, ..., verify_data=False):
    ...
    self.verify_data = verify_data

Comment on lines +335 to 342
checker = ArchiveChecker(repository, repair=args.repair, archive=args.location.archive, first=args.first,
last=args.last, sort_by=args.sort_by or 'ts', glob=args.glob_archives,
verify_data=args.verify_data, save_space=args.save_space)
except Exception as exc:
self.print_error(str(exc))
return EXIT_WARNING
if not args.archives_only:
if not repository.check(repair=args.repair, save_space=args.save_space, max_duration=args.max_duration):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the potential fundamental problem with first accessing the repo inside ArchiveChecker.__init__ before doing repository.check is that we do not know whether we are working on valid data.

The repo check / repair makes sure that the fundamental data is ok (in memory and in repair mode also on disk).
The archives check / repair builds upon that (and assumes that lower level structures are ok).

So, guess there needs to be pretty much review before touching repo data before the repo has been checked.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, and that's what I feared. Is there some way to verify only the chunks that contain the metadata needed to decrypt in ArchiveChecker.__init__(), so we know that the crypto & manifest & whatever other metadata is kosher, but don't spend 5 hours doing all of repo/data/*

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, a "miniature version" of the borg repo check, that only makes sure that 1 chunk is ok (which could be the manifest (id 0) or any other chunk). And then detect crypto based on that chunk.

that might work, needs checking...

Comment on lines 1412 to +1413
self.init_chunks()
self.key = self.identify_key(repository)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guess these 2 need checking.

do these have any unwanted consequences when run before the repo check has happened?

@ThomasWaldmann
Copy link
Member

Hmm, what are we doing with this PR?

@kdm9
Copy link
Author

kdm9 commented Sep 24, 2020

Hmm, what are we doing with this PR?

@ThomasWaldmann I don't think I have the confidence to verify that my changes here have no terrible side effects, so this PR has somewhat stalled. As far as I can tell it's OK, but I don't have anywhere near the knowledge of Borg's internals I'd need to be confident about that. I guess I'd need someone with good knowledge of Borg's innards to check my change and see that I'm not doing anything stupid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants