arc eviction altgough zfs_arc_min is not yet reached - metadata eviction although zfs_arc_meta_min is not yet reached

[kalleb64]

Hello,
afaik ARC eviction is done under memory pressure.
Shouldn't zfs_arc_min and zfs_arc_meta_min be some kind of guaranteed lower limits for ARC buffers ?
I my configuration (with no memory pressure at all) eviction prevents metadata even from reaching zfs_arc_meta_min.
So I had a look into the arc.c code (2.1.16):

if (arc_evict_type(arc_mru) == ARC_BUFC_METADATA &&
	    ameta > arc_meta_min) {
		bytes = arc_evict_impl(arc_mru, 0, target, ARC_BUFC_METADATA);
		total_evicted += bytes;

		/*
		 * If we couldn't evict our target number of bytes from
		 * metadata, we try to get the rest from data.
		 */
		target -= bytes;

		total_evicted +=
		    arc_evict_impl(arc_mru, 0, target, ARC_BUFC_DATA);
	} else {
		bytes = arc_evict_impl(arc_mru, 0, target, ARC_BUFC_DATA);
		total_evicted += bytes;

		/*
		 * If we couldn't evict our target number of bytes from
		 * data, we try to get the rest from metadata.
		 */
		target -= bytes;

		total_evicted +=
		    arc_evict_impl(arc_mru, 0, target, ARC_BUFC_METADATA);
	}

The else part will be executed when ameta <= arc_meta_min and potentially evict metadata (last line of the code).
So this is why arc_meta_min seems to be ignored.

The same error (?) is made for the mfu buffer a few lines further down - with an additional comment flaw:

		 /*
		 * If we couldn't evict our target number of bytes from
		 * *data*, we try to get the rest from *data*.
		 */

Generally speaking, I think that eviction should be completely disabled as long as zfs_arc_min is not reached. I did not find any guard for this in arc_evict_impl(). This function relies on argument bytes but there is no guarantee that the numerous calling functions properly set this value.

[aheid]

I don't think the code you're refering to is the issue.

The ARC splits the cache between MRU and MFU (which is technically not MFU but that's another discussion). Assuming it can't grow in overall size, then if it wants to grow the MFU, it has to reduce the MRU. However, it only maintains a single MRU size target.

Thus when it wants to reduce the MRU and no user data is eligible for eviction, it "has" to evict metadata, hence the code you posted.

Now, I'm not sure if zfs_arc_meta_min is supposed to be a guarantee. If that is the intention, it seems the real issue is that the size of the MRU, arc_p, is allowed to shrink without considering if doing so would violate zfs_arc_meta_min. However, as far as I can see, the ARC doesn't maintain enough information to make this determination, as it only keeps a global metadata size rather than one for each of the MRU and MFU lists.