Add integration tests for the remote inference feature (#1073) (#1109)

* Add remote inference integration tests for OpenAI and Cohere



* Change edits model test



* Change loop number in edits test



* Change edits model test again



* Fix spotless



* Refactoring



* Fix typo



* Remove env vars



* Removing unnecessary comments



* Rebase with 2.x and remove set up for master key



* Remove print line



* Add throttling check



* Minor change for assert statement



* Test changes



* Uncomment delete connector test



---------




(cherry picked from commit 3e56ad5)

Signed-off-by: Ryan Bogan <[email protected]>
Signed-off-by: Peter Zhu <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>
Co-authored-by: Ryan Bogan <[email protected]>
Co-authored-by: Peter Zhu <[email protected]>

Author: 3 people

.github/workflows/CI-workflow.yml

@@ -35,7 +35,6 @@ jobs:
           role-to-assume: ${{ secrets.ML_ROLE }}
           aws-region: us-west-2
 
-      # ml-commons
       - name: Checkout MLCommons
         uses: actions/checkout@v2