Inconsistent exceptions raised by get() and put() depending on file size

[stinos]

The DataObjectManager.put method forwards to parallel_put depending on certain conditions but because of the way it handles errors (starting at https://github.com/irods/python-irodsclient/blob/main/irods/manager/data_object_manager.py#L369) the exceptions raised are not necessarily the same as for put without parallel transfer. Moreover depending on what goes wrong, parallel_put raises a differet exception type than put even though from the user's point of view the situation is exactly the same. This makes it rather non-trivial to handle specific exceptions.

For instance when trying to upload data to a path for which there's no collection yet I'm expecting an irods.exception.CAT_UNKNOWN_COLLECTION which I can then handle properly. But with default settings, for a large file put calls into parallel_put and ultimately leads to a RuntimeError with message 'parallel_put failed' being raised.
Note that even if put would not catch exceptions but just raise whatevever parallel_put raises it would still not result in an irods.exception.CAT_UNKNOWN_COLLECTION in this case, but an irods.exception.CollectionDoesNotExist because that is what parallel_put produces.

Similar issues exist for get() / parallel_get(). Didn't check if there are other places where this could also happen.

[d-w-moore]

The DataObjectManager.put method forwards to parallel_put depending on certain conditions but because of the way it handles errors (starting at https://github.com/irods/python-irodsclient/blob/main/irods/manager/data_object_manager.py#L369) the exceptions raised are not necessarily the same as for put without parallel transfer. Moreover depending on what goes wrong, parallel_put raises a differet exception type than put even though from the user's point of view the situation is exactly the same. This makes it rather non-trivial to handle specific exceptions.

For instance when trying to upload data to a path for which there's no collection yet I'm expecting an irods.exception.CAT_UNKNOWN_COLLECTION which I can then handle properly. But with default settings, for a large file put calls into parallel_put and ultimately leads to a RuntimeError with message 'parallel_put failed' being raised. Note that even if put would not catch exceptions but just raise whatevever parallel_put raises it would still not result in an irods.exception.CAT_UNKNOWN_COLLECTION in this case, but an irods.exception.CollectionDoesNotExist because that is what parallel_put produces.

Similar issues exist for get() / parallel_get(). Didn't check if there are other places where this could also happen.

Certainly we can do better than RuntimeError. However, I doubt the exceptions raised will ever match exactly to the counterpart error types generated by an iput involving the same files and objects, ie. as with iput myfile $(ipwd)/does_not_exist/myfile generating a CAT_UNKNOWN_COLLECTION. This is for the simple reason that, at best, the PRC's buffered and parallel put operations are not like a true PUT as performed by iput. Different APIs are actually followed for the iput vs. the Python client.

It's possible however that we might encapsulate the exceptions that are now being generated inside a new exception_object type, perhaps called PutException (or as the case may be, GetException where the exception_object.__cause__ indicates the underlying error source.