Remove unnecessary `GEM_PATH` modification #664

Job	Run time
make (check, ubuntu-22.04)	9m 40s
make (check, i686)	10m 54s
make (check, --disable-yjit)	6m 53s
make (check, cppflags=-DVM_CHECK_MODE)	8m 35s
make (check, --enable-shared --enable-load-relative)	7m 51s
make (test-bundler-parallel, 50)	29m 0s
make (check, ubuntu-24.04)	9m 11s
make (test-bundled-gems)	16m 11s
Ubuntu result	2s
	1h 38m 17s

Provide feedback