[SPARK-54860][INFRA] Followup of the revert to set the permission correctly

### What changes were proposed in this pull request?

This is a followup of 63f1f96 to clean revert.

### Why are the changes needed?

Followup of the revert. The revert wasn't clean.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

N/A

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #54213 from HyukjinKwon/follow-up-permission.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>

Author: HyukjinKwon

.github/workflows/labeler.yml

@@ -32,7 +32,6 @@ jobs:
     runs-on: ubuntu-latest
     permissions:
       contents: read
-      pull-requests: write
     steps:
     - uses: actions/labeler@v5
       with: