[Filestore] write-back cache in filestore-vhost

[qkrorlqr]

If some other client tries to open the same file for RDWR as well, the tablet needs to notify the first client so that it would disable writeback caching, and only after that allow RDWR opening by another client.

fsync should cause cache flush of course

this issue will help with rsync-like access patterns - we noticed that rsync sometimes uses lots of tiny (1KiB) requests for writing large files - those files are usually opened by only one client, we can easily buffer those writes in filestore-vhost and send them to the storage in large chunks and with proper parallelism

the cache can be stored in a file over tmpfs (so physically - in the memory of the host where the guest is running)

TRingBuffer:

• Implement in-place allocation - issue-1751: TFileRingBuffer - in-place allocation #3895
• Support size >= 4GB (ui32 -> ui64) - issue-1751: TFileRingBuffer supports > 4GB capacity; do not zero the entire slackspace; fully utilize capacity #3734; issue-1751: [Filestore] WriteBackCache should support >=4GB capacity #4546
• Add IsCorrupted flag; prohibit adding new elements when the flag is set - issue-1751: TFileRingBuffer: add IsCorrupted flag; prohibit adding new elements when the flag is set #3816
• Resolve inconsistency in logic between PushBack, PopFront, Visit - issue-1751: TFileRingBuffer: resolve inconsistency in logic between PushBack, PopFront, Visit + deduplicate code #3817

TWriteBackCache:

• Refactoring - issue-1751: TWriteBackCache: refactoring - move implementation details to a separate header (v2) #3978
• Replace SessionSequencer with ReadWriteRangeLock and change synchronization point - issue-1751: TWriteBackCache: replace SessionSequencer with ReadWriteRangeLock and change synchronization point #4064
• Limit the amount and size of WriteData operations generated by Flush - issue-1751: TWriteBackCache - limit the amount and size of WriteData operations generated by Flush #4093
• Get rid of double copy (multiple places) - issue-1751: TWriteBackCache::ReadData: support buffer offsetting, get rid of reallocation #3925
• Store request buffer only once - issue-1751: TWriteBackCache: reference WriteData buffer directly in the ring buffer #3926
• Execute simultaneously only one Flush operation per handle - issue-1751: TWriteBackCache - improve flush logic #4059
• Copy request buffer to cache outside lock - issue-1751: TWriteBackCache - read/write requests from/to TFileRingBuffer outside lock sections #3928
• Reduce MaxTime for WriteData requests - issue-1751: TWriteBackCache - limit the amount and size of WriteData operations generated by Flush #4093
• Flush performance optimization - issue-1751: TWriteBackCache: replace SessionSequencer with ReadWriteRangeLock and change synchronization point #4064
• ReadData performance optimization - issue-1751: Optimize CalculateDataPartsToRead in TWriteBackCache using disjoint interval tree #4135
• Make write-back cache associated with an inode instead of a handle - [Filestore] Make write-back cache associated with an inode instead of a handle  #4294
• Avoid allocation in read when there are no overlapping cached fragments - issue-1751: [Filestore] Get rid of memory allocation in ReadData when there is no cached data #4562
• Get rid of copy by referencing data directly in the cache - issue-1751: [Filestore] Get rid of copy in TWriteBackCache in Flush by using iovecs #4572

Metrics and logging:

• issue-4154: [Filestore] Report metrics from TWriteBackCache via TWriteBackCacheStats #4397
• issue-1751: [Filestore] Introduce logging to TWriteBackCache #4558
• issue-4154: [Filestore] Introduce metrics for server write-back cache and check them in tests #4373

Stability:

• Do not fail when WriteBackCacheCapacity changes - issue-1751: TFileRingBuffer: do not fail when the requested capacity differs from the existing capacity #3957
• Some requests may not be flushed automatically - issue-4503: [Filestore] Automatic flush should be triggered in WriteBackCache without waiting for the completion of the previous FlushAll #4505

Unit test:

• Data race fixes:
• • WriteBackCache: fix race condition in TWriteBackCacheTest.ShouldFlushAutomatically  #3869
• • Resolve data race in unit test TWriteBackCacheTest::ShouldRetryFlushOnFailure #4181
• • Resolve data race in unit test TWriteBackCacheTest::ShouldFlushAutomatically #4185
• • [FileStore] Increment flush interval for WriteBackCache to 1s in unit tests #3934
• Broken tests:
• • issue-1751: [Filestore] TWriteBackCache: fix TestShouldReadAfterWriteConcurrently test #4545

Concerns:

• [Filestore] write-back cache in filestore-vhost #1751 (comment)

[budevg]

Fuse already supports writeback cache on the client side. I will add option to pass this flag during fuse session initialization