How to bring in Candidate URLs from the test server

Bring in selected collections:

1. Log into the test server (bastion)

2. From the bastion, RDP into the following server 10.51.14.135

3. On the server, open up Anaconda Powershell Prompt (miniconda3) and navigate to E:\sde-indexing-helper\config_generation

4. Find python file called get_csv_for_all_sources.py and, in the list called COLLECTIONS_TO_UPLOAD, add a list of collection config folders.

5. Run python get_csv_for_all_sources.py

6. This will upload zipped json files to the S3 bucket here.

Bring in everything on the test server:

Follow steps, 1, 2, 3 from above.

4. Run python export_whole_index.py

5. This will upload one large zipped json file to the S3 bucket here.

If you see something in the pipeline that can be optimized, be the change you wish to see in the world. :D