Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: Fix rsync of base rpmdb with sqlite #2436

Merged
merged 1 commit into from
Jan 9, 2021

Conversation

cgwalters
Copy link
Member

@cgwalters cgwalters commented Jan 8, 2021

Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime). It's really ostree's mtime canonicalization
that's unfortunate here.

Anyways, we obviously don't care about performance here so use
-I to disable that rsync check.

(Also remove the mkdir -p since it's not necessary since a long time)

Closes: #2435

Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime).  It's really ostree's mtime canonicalization
that's unfortunate here.

Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.

(Also remove the `mkdir -p` since it's not necessary since a long time)

Closes: coreos#2435
@cgwalters
Copy link
Member Author

(Also we should really rewrite this in Rust and stick it in rpm-ostree testutils since that'd also lay the basic groundwork for #442 )

@jamescassell
Copy link

I'd really like to see ostree preserve timestamps like RPMs do, and with btrfs/xfs reflinks, it should be possible without increasing storage drastically.

Copy link
Contributor

@lucab lucab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, lucab

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cgwalters
Copy link
Member Author

cgwalters commented Jan 9, 2021

I'd really like to see ostree preserve timestamps like RPMs do, and with btrfs/xfs reflinks, it should be possible without increasing storage drastically.

Yes, moved that to ostreedev/ostree#2259
(The reason we haven't done that yet is because it'd require touching 30% of core ostree codepaths; it's not a full new "backend" but it's a lot of additional new logic - and we can't just toss our current hardlink backend because there are a ton of ostree users using plain ext4 as well as older xfs, etc.)

@wmanley
Copy link

wmanley commented Feb 18, 2021

It's really ostree's mtime canonicalization that's unfortunate here.

One option, that might sound a little crazy, but could solve issues is to change the canonicalization. Instead of setting it to 1970-01-01T00:00:00Z we could encode the SHA of the file in the timestamp. If you took the top 28 bits (7 hex chars) the timestamp would still be in the 70s so obviously artificial. We could even squeeze another 7 hex chars in the nanosecond field if we fancied.

Tools like rsync could then continue to use (name, size, date) to identify whether something's changed. You still wouldn't be able to use the timestamp to determine orderings of modifications a la make, but it might help other use cases.

@cgwalters
Copy link
Member Author

That's an interesting idea! I think it might fix about half the timestamp issues? Could be confusing to some tools if the timestamp goes backwards, but OTOH agree that it'd fix this rsync case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Switch to sqlite db breaks some tests
6 participants