ci: Fix integration test by avoid writing to read-only test directory (#675)

ci: Fix integration test by copying test dataset to writable directory

In the AWS infra, our test directory is not writable. HF datasets
unfortunately will always attempt to write a lock file even if the test
data exists in the directory. This avoids that by copying the test
datasets to a writable temp directory on the container.

In other repos, sometimes mock the datasets.builder.FileLock object to
also fix this problem. But it seems that the eval integration tests are
called in a different process. So, it's not feasible to mock that object
during the test. Fortunately, the test dataset is easy to copy over.

Successful run:

https://github.com/NVIDIA-NeMo/Evaluator/actions/runs/21506727996/job/61964385541

---------

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Author: chtruong814

tests/integration_tests/Launch_Integration_Tests.sh

@@ -17,8 +17,12 @@ export CUDA_VISIBLE_DEVICES="0"
 export HF_HOME="/home/TestData/HF_HOME"
 export HF_DATASETS_OFFLINE="1"
 export TRANSFORMERS_OFFLINE="1"
-export HF_DATASETS_CACHE="${HF_HOME}/datasets"
+export HF_DATASETS_CACHE="/tmp/datasets"
 
+mkdir /tmp/datasets
+cp -r ${HF_HOME}/datasets/gsm8k /tmp/datasets
+cp -r ${HF_HOME}/datasets/google___if_eval /tmp/datasets
+ls -alh /tmp/datasets
 mkdir -p /checkpoints && \
 ln -s /home/TestData/nemo2_ckpt/llama-3_2-1b-instruct_v2.0 /checkpoints/llama-3_2-1b-instruct_v2.0
 
@@ -34,4 +38,4 @@ coverage run \
     -o log_cli_level=INFO \
     -m "not pleasefixme" \
    tests/integration_tests
-coverage combine -q
+coverage combine -q