Remove unnecessary `GEM_PATH` modification #275

Job	Run time
cargo test	53s
make (check, --enable-yjit, --yjit)	6m 15s
make (check, --enable-yjit=dev, --yjit-call-threshold=1 --yjit-verify-ctx --yjit-code-gc, -T --yj...	20m 45s
YJIT macOS Arm64 result	3s
	27m 56s

Provide feedback