NSFS | Object Operations Performance Improvements

[romayalon]

Environment info

• NooBaa Version: Master
• Platform: NC

This issue will gather NSFS object operations optimizations -

Small objects performance

Actual behavior

1. As for today, on POSIX file systems, users experience low performance when writing small objects due to rename() syscall taking a lot of time -

2022-01-24 13:56:57.560361 [PID-14/TID-21] [L0] FS::FSWorker::Execute: WARNING Rename _old_path=/nsfs/fs1/cb6/.noobaa-nsfs_61b11a909954d0002a5d14fd/uploads/4a0c95ec-30ad-4161-8fa6-150f800e851e _new_path=/nsfs/fs1/cb6/586_64KB  took too long: 881.188 ms

We solved this issue on GPFS filesystems by using libgpfs.
As mentioned on #6858, a POSIX optimization would be to try using link() instead of rename() when the target file is missing, and if there is a race then fallback to rename. That should allow reduce the rename overhead from most of the POSIX calls that do not involve replacing an existing target object.

Expected behavior

1. Writing of small object performance optimized for POSIX as well.

Steps to reproduce

1. See on the original issue - Small object write performance very slow #6858