swiotlb: Reinstate page-alignment for mappings >= PAGE_SIZE

jira LE-1907
cve CVE-2024-35814
Rebuild_History Non-Buildable kernel-4.18.0-553.16.1.el8_10
commit-author Will Deacon <[email protected]>
commit 14cebf6
Empty-Commit: Cherry-Pick Conflicts during history rebuild.
Will be included in final tarball splat. Ref for failed cherry-pick at:
ciq/ciq_backports/kernel-4.18.0-553.16.1.el8_10/14cebf68.failed

For swiotlb allocations >= PAGE_SIZE, the slab search historically
adjusted the stride to avoid checking unaligned slots. This had the
side-effect of aligning large mapping requests to PAGE_SIZE, but that
was broken by 0eee5ae ("swiotlb: fix slot alignment checks").

Since this alignment could be relied upon drivers, reinstate PAGE_SIZE
alignment for swiotlb mappings >= PAGE_SIZE.

	Reported-by: Michael Kelley <[email protected]>
	Signed-off-by: Will Deacon <[email protected]>
	Reviewed-by: Robin Murphy <[email protected]>
	Reviewed-by: Petr Tesarik <[email protected]>
	Tested-by: Nicolin Chen <[email protected]>
	Tested-by: Michael Kelley <[email protected]>
	Signed-off-by: Christoph Hellwig <[email protected]>
(cherry picked from commit 14cebf6)
	Signed-off-by: Jonathan Maple <[email protected]>

# Conflicts:
#	kernel/dma/swiotlb.c

Author: PlaidCat

ciq/ciq_backports/kernel-4.18.0-553.16.1.el8_10/14cebf68.failed

@@ -0,0 +1,113 @@
+swiotlb: Reinstate page-alignment for mappings >= PAGE_SIZE
+
+jira LE-1907
+cve CVE-2024-35814
+Rebuild_History Non-Buildable kernel-4.18.0-553.16.1.el8_10
+commit-author Will Deacon <[email protected]>
+commit 14cebf689a78e8a1c041138af221ef6eac6bc7da
+Empty-Commit: Cherry-Pick Conflicts during history rebuild.
+Will be included in final tarball splat. Ref for failed cherry-pick at:
+ciq/ciq_backports/kernel-4.18.0-553.16.1.el8_10/14cebf68.failed
+
+For swiotlb allocations >= PAGE_SIZE, the slab search historically
+adjusted the stride to avoid checking unaligned slots. This had the
+side-effect of aligning large mapping requests to PAGE_SIZE, but that
+was broken by 0eee5ae10256 ("swiotlb: fix slot alignment checks").
+
+Since this alignment could be relied upon drivers, reinstate PAGE_SIZE
+alignment for swiotlb mappings >= PAGE_SIZE.
+
+	Reported-by: Michael Kelley <[email protected]>
+	Signed-off-by: Will Deacon <[email protected]>
+	Reviewed-by: Robin Murphy <[email protected]>
+	Reviewed-by: Petr Tesarik <[email protected]>
+	Tested-by: Nicolin Chen <[email protected]>
+	Tested-by: Michael Kelley <[email protected]>
+	Signed-off-by: Christoph Hellwig <[email protected]>
+(cherry picked from commit 14cebf689a78e8a1c041138af221ef6eac6bc7da)
+	Signed-off-by: Jonathan Maple <[email protected]>
+
+# Conflicts:
+#	kernel/dma/swiotlb.c
+diff --cc kernel/dma/swiotlb.c
+index c0e227dcb45e,86fe172b5958..000000000000
+--- a/kernel/dma/swiotlb.c
++++ b/kernel/dma/swiotlb.c
+@@@ -611,36 -1012,50 +611,66 @@@ static int swiotlb_do_find_slots(struc
+  	unsigned int slot_index;
+  
+  	BUG_ON(!nslots);
+++<<<<<<< HEAD
+ +	BUG_ON(area_index >= mem->nareas);
+ +
+ +	/*
+ +	 * For allocations of PAGE_SIZE or larger only look for page aligned
+ +	 * allocations.
+ +	 */
+ +	if (alloc_size >= PAGE_SIZE)
+ +		iotlb_align_mask |= ~PAGE_MASK;
+ +	iotlb_align_mask &= ~(IO_TLB_SIZE - 1);
+ +
+ +	/*
+ +	 * For mappings with an alignment requirement don't bother looping to
+ +	 * unaligned slots once we found an aligned one.
+ +	 */
+ +	stride = (iotlb_align_mask >> IO_TLB_SHIFT) + 1;
+ +
+++=======
++ 	BUG_ON(area_index >= pool->nareas);
++ 
++ 	/*
++ 	 * Historically, swiotlb allocations >= PAGE_SIZE were guaranteed to be
++ 	 * page-aligned in the absence of any other alignment requirements.
++ 	 * 'alloc_align_mask' was later introduced to specify the alignment
++ 	 * explicitly, however this is passed as zero for streaming mappings
++ 	 * and so we preserve the old behaviour there in case any drivers are
++ 	 * relying on it.
++ 	 */
++ 	if (!alloc_align_mask && !iotlb_align_mask && alloc_size >= PAGE_SIZE)
++ 		alloc_align_mask = PAGE_SIZE - 1;
++ 
++ 	/*
++ 	 * Ensure that the allocation is at least slot-aligned and update
++ 	 * 'iotlb_align_mask' to ignore bits that will be preserved when
++ 	 * offsetting into the allocation.
++ 	 */
++ 	alloc_align_mask |= (IO_TLB_SIZE - 1);
++ 	iotlb_align_mask &= ~alloc_align_mask;
++ 
++ 	/*
++ 	 * For mappings with an alignment requirement don't bother looping to
++ 	 * unaligned slots once we found an aligned one.
++ 	 */
++ 	stride = get_max_slots(max(alloc_align_mask, iotlb_align_mask));
++ 
+++>>>>>>> 14cebf689a78 (swiotlb: Reinstate page-alignment for mappings >= PAGE_SIZE)
+  	spin_lock_irqsave(&area->lock, flags);
+ -	if (unlikely(nslots > pool->area_nslabs - area->used))
+ +	if (unlikely(nslots > mem->area_nslabs - area->used))
+  		goto not_found;
+  
+ -	slot_base = area_index * pool->area_nslabs;
+ +	slot_base = area_index * mem->area_nslabs;
+  	index = area->index;
+  
+ -	for (slots_checked = 0; slots_checked < pool->area_nslabs; ) {
+ -		phys_addr_t tlb_addr;
+ -
+ +	for (slots_checked = 0; slots_checked < mem->area_nslabs; ) {
+  		slot_index = slot_base + index;
+ -		tlb_addr = slot_addr(tbl_dma_addr, slot_index);
+  
+ -		if ((tlb_addr & alloc_align_mask) ||
+ -		    (orig_addr && (tlb_addr & iotlb_align_mask) !=
+ -				  (orig_addr & iotlb_align_mask))) {
+ -			index = wrap_area_index(pool, index + 1);
+ +		if (orig_addr &&
+ +		    (slot_addr(tbl_dma_addr, slot_index) &
+ +		     iotlb_align_mask) != (orig_addr & iotlb_align_mask)) {
+ +			index = wrap_area_index(mem, index + 1);
+  			slots_checked++;
+  			continue;
+  		}
+* Unmerged path kernel/dma/swiotlb.c