shutil.move creates copy if rename fails with errors unrelated to EXDEV

[ptim0626]

Bug report

(originally came across in https://gist.github.com/devdanzin/3198710e3c0128fda5e0a7b4e0768e5f 12/13)

If I mv a/b somewhere/ with the permission of a/ set to 500/r-x------, I will get:

mv: rename a/b to somewhere/b: Permission denied

and no somewhere/b is created. However if I use shutil.move:

import shutil
shutil.move("a/b", "somewhere")

It raises with PermissionError which is also correct but somewhere/b is created. The correct behaviour should not create somewhere/b, as seen in the implementation of mv. If a/b is a very large file, the current behaviour of shutil.move will copy it first, which takes a long time, then raises with PermissionError and results in a duplicated copy of a very large file as somewhere/b, which is both bad to the file system and confusing to the user, also not matching what mv does.

The copy-then-remove fallback, after failure of os.rename captured by https://github.com/python/cpython/blob/main/Lib/shutil.py#L946, should be performed only if the error from os.rename is EXDEV (failure because of cross filesystem), as suggested in its docstring and also coreutils do_move. Other failure such as permission issue, disk error, file name too long etc., should not trigger the copy-then-remove fallback, but they are all currently captured by OSError.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS, Linux

Linked PRs

• gh-158666: Avoid copy if shutil.move fails other than EXDEV #158667