GH-38792: [C++][Gandiva] Optimize cache #3912

Job	Run time
check-labels / Check labels	26s
JNI macOS	0s
ODBC	0s
ARM64 Windows 11 MSVC	0s
matrix.title	0s
JNI ${{ matrix.platform.runs-on }} ${{ matrix.platform.arch }}	0s
report-extra-cpp	0s
	26s

Provide feedback