status: Wrong password silently ignored / misleading output (HTTP remote)

[sqeph]

Bug Report

Description

When using a HTTP remote with basic authentication and password prompt, dvc status --cloud does not recognize a failed authentication attempt (wrong/missing password) and outputs misleading information. Instead of an error message, DVC lists all my local files as if they do not exist on remote (regardless of whether they were already pushed or not).

In fact, I'm experiencing similar behavior when using dvc pull and, for imported data, dvc update. If the password is incorrect, DVC more or less ignores it and tells me that everything is up-to-date. This is rather problematic in a collaborative setting, since it's easy to mistype a password and miss out on important changes, etc.

Reproduce

Prerequisite: A HTTP remote with basic authentication and ask_password = true

1. dvc add image.jpg
2. dvc status --cloud (correct password)

        new:                image.jpg

3. dvc push (correct password)
4. dvc status --cloud (correct password)

Cache and remote 'myremote' are in sync.

5. dvc status --cloud (wrong password or no password)

        new:                image.jpg

Expected

An error message about the failed authentication attempt.

Environment information

Output of dvc doctor:

$ dvc doctor

DVC version: 2.9.3 (pip)
---------------------------------
Platform: Python 3.9.5 on Linux-5.11.0-49-generic-x86_64-with-glibc2.33
Supports:
	webhdfs (fsspec = 2021.11.1),
	http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.6),
	https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.6),
	ssh (sshfs = 2021.11.2)
Cache types: reflink, hardlink, symlink
Cache directory: btrfs on /dev/mapper/home
Caches: local
Remotes: https
Workspace directory: btrfs on /dev/mapper/home
Repo: dvc, git

Additional Information (if any):

We are using a Nexus system as our HTTP remote. With correct credentials, dvc status, dvc pull, dvc push, etc, works as expected. I don't know how DVC communicates with the HTTP storage, but if I try to fetch a file directly from the Nexus system via cURL and provide wrong credentials, the response is a 401 and so it seems this is something DVC should be able to pick up and react to.

Please let me know if anything is unclear or if you need additional information.

[dberenbaum]

@efiop It would be great to be able to look into this one as part of updates to the status command.

[efiop]

@dberenbaum Seems like this is more of an httpfs issue with this particular server. Normally it should indeed error out. I think this was discussed elsewhere, but I can't find any links yet. Lets keep this open for now in case someone else is running into this.

[sqeph]

For what it is worth, I did a bit of investigation and it seems that dvc status -c ends up checking the existence of the various files through fsspec's HTTPFileSystem._exists() (https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/http.py#L297). Here the response status code is indeed 401 (if I fail to provide correct credentials), but this simply results in _exists() returning False... indistinguishable from, say, a 404. This would be my best guess as to why DVC lists the files as non-existent...

[jnareb]

The same problem also shows itself when trying to do dvc pull on https:// remote when lacking authentication credentials. What is even more confusing is that the error seems to be in dvc checkout, not in dvc fetch.

[efiop]

For the record: ^^^ https://discord.com/channels/485586884165107732/485596304961962003/1059218805106159687

checkout error is normal, the issue here is the lack of error in fetch, because we seem to take the code from dagshub as FileNotFoundError.