Merge pull request #4870 from linuxfoundation/unicron-v2-test-coverage-api-fixes-and-dev-claims

Unicron v2 test coverage api fixes and dev claims

Author: lukaszgryglicki

.github/copilot-instructions.md

@@ -0,0 +1,85 @@
+# GitHub Copilot Project Instructions
+
+You are assisting in this repository. The following rules are **mandatory** and override any defaults.
+
+---
+
+## General Behavior
+
+1. **Read full files, not fragments**
+
+   - When you open a file, always consider the **entire file** contents, not just the visible or edited region.
+   - Before refactoring or implementing features, scan all relevant files fully (including related modules) to understand existing patterns, types, and architecture.
+   - Do not assume behavior from partial context if you can open and read the whole file instead.
+   - Read files from other GitHub repositories as needed.
+   - Read issues/pull requests from remote repositories as needed.
+
+2. **No artificial time/phasing limits**
+
+   - Do **not** shorten work or split it into artificial “phases” just to finish quickly.
+   - Always attempt to provide the **most complete, fully implemented solution** you reasonably can in a single pass:
+     - Implement full feature logic instead of stubs.
+     - Add tests when appropriate.
+     - Update related documentation/configs only if clearly required.
+   - Avoid responses like “I’ll do the rest later” or “this is just a partial implementation” unless explicitly requested.
+
+3. **Never leave partially implemented code**
+
+   - Do **not** leave functions, classes, or modules partially implemented.
+   - Always provide a complete implementation for any code you modify or add.
+   - If you cannot fully implement something due to missing context, dependencies, or constraints, clearly state what is missing and provide a best-effort implementation based on available information.
+   - In such cases ask for missing context and wait for my response before proceeding further.
+
+---
+
+## Testing & Validation
+
+5. **Always test changes**
+
+   - Whenever you modify code, **run tests** relevant to the change to ensure correctness.
+   - If you add or update tests:
+     - Execute the new or modified tests to confirm they pass.
+     - Ensure coverage reflects the newly added logic.
+   - When presenting results:
+     - Output plain test results or shell-style logs.
+     - **Do not generate fancy Markdown summaries**, reports, tables, or “change overview” documents. outputting this in console is OK.
+     - A simple textual summary or shell-like output is sufficient (e.g., which tests passed/failed).
+
+---
+
+## Git / Commits
+
+3. **Do NOT generate commits**
+
+   - Never run Git commands that create commits, amend commits, or push changes.
+   - Do **not** auto-generate commit messages or suggest commands like:
+     - `git commit ...`
+     - `git commit -m "..."`
+     - `git commit --amend`
+     - `git push`
+   - I will create commits myself; they must be signed (`-S`) and signed-off (`-s`), so leave all commit creation to me.
+
+   - It is acceptable to:
+     - Modify files in the working tree.
+     - Describe *what* should be committed, but not *how* to commit.
+
+---
+
+## File / Directory Access
+
+4. **Repository file access and permissions**
+
+   - You may read any file and directory within this repository and other repositories as needed.
+   - **Never ask for additional read permissions** for items inside this repo; assume full access.
+   - Locate relevant code by searching the repo, exploring related modules, tests, and configs. Fetch data from remote if needed.
+   - If something is outside this repository entirely, its also OK to read it.
+
+---
+
+## Style of Assistance
+
+- Follow the repository’s existing style, patterns, conventions, and architecture.
+- Avoid unnecessary churn or unrelated refactoring.
+- Explain non-obvious design decisions briefly in comments when helpful.
+- When uncertain, produce a **best-effort, concrete implementation** using the available code rather than asking unnecessary questions.
+

.gitignore

@@ -262,3 +262,7 @@ body.json
 
 cla-backend-go/signatures_timestamp_backfill
 cla-backend-go/backfill-fallback-commands-cla*.sh
+cla-backend/python-api.log
+cla-backend/python-api.err
+cla-backend-go/golang-api.err
+cla-backend-go/golang-api.log

cla-backend/cla/auth.py

@@ -11,9 +11,6 @@
 
 import cla
 
-# LG: for local environment override
-# os.environ["AUTH0_USERNAME_CLAIM"] = os.getenv("AUTH0_USERNAME_CLAIM_CLI", os.environ["AUTH0_USERNAME_CLAIM"])
-
 auth0_base_url = os.environ.get('AUTH0_DOMAIN', '')
 auth0_username_claim = os.environ.get('AUTH0_USERNAME_CLAIM', '')
 algorithms = [os.environ.get('AUTH0_ALGORITHM', '')]
@@ -123,9 +120,16 @@ def authenticate_user(headers):
 
         username = payload.get(auth0_username_claim)
         if username is None:
-            # LG: to have more info
-            # raise AuthError(f"username not found in {auth0_username_claim}")
-            raise AuthError('username claim not found')
+            alt_claim = os.environ.get('AUTH0_USERNAME_CLAIM_CLI', '')
+            if alt_claim != '':
+                cla.log.warning(f"username claim not found in {auth0_username_claim}, trying {alt_claim}")
+                username = payload.get(alt_claim)
+                if username is None:
+                    cla.log.error(f"username claim not found in alternate source {alt_claim}")
+                    raise AuthError('username claim not found')
+            else:
+                cla.log.error(f"username claim not found in {auth0_username_claim}")
+                raise AuthError('username claim not found')
 
         auth_user = AuthUser(payload)
 

cla-backend/cla/controllers/project.py

@@ -338,7 +338,7 @@ def get_project_companies(project_id):
     company_ids = list(set([signature.get_signature_reference_id() for signature in signatures]))
     company = Company()
     all_companies = [comp.to_dict() for comp in company.all(company_ids)]
-    all_companies = sorted(all_companies, key=lambda i: i['company_name'].casefold())
+    all_companies = sorted(all_companies, key=lambda i: (i.get('company_name') or '').casefold())
 
     return all_companies
 

cla-backend/cla/models/github_models.py

@@ -354,7 +354,7 @@ def oauth2_redirect(self, state, code, request):  # pylint: disable=too-many-arg
             # Eventually handle user-from-session API callback
             try:
                 state_data = json.loads(base64.urlsafe_b64decode(state.encode()).decode())
-            except (ValueError, json.JSONDecodeError, binascii.Error):
+            except (ValueError, json.JSONDecodeError, binascii.Error) as err:
                 cla.log.warning(f"{fn} - failed to decode state: {state}, error: {err}")
                 raise falcon.HTTPBadRequest("Invalid OAuth2 state", state)
             state_token = state_data["csrf"]

cla-backend/cla/routes.py

@@ -1356,7 +1356,7 @@ def check_and_prepare_employee_signature(
     "/signed/individual/{installation_id}/{github_repository_id}/{change_request_id}", versions=2,
 )
 def post_individual_signed(
-        body,
+        request,
         installation_id: hug.types.number,
         github_repository_id: hug.types.number,
         change_request_id: hug.types.number,
@@ -1369,7 +1369,7 @@ def post_individual_signed(
 
     Callback URL from signing service upon ICLA signature.
     """
-    content = body.read()
+    content = request.bounded_stream.read()
     return cla.controllers.signing.post_individual_signed(
         content, installation_id, github_repository_id, change_request_id
     )
@@ -1378,7 +1378,7 @@ def post_individual_signed(
     "/signed/gitlab/individual/{user_id}/{organization_id}/{gitlab_repository_id}/{merge_request_id}", versions=2,
 )
 def post_individual_signed_gitlab(
-        body,
+        request,
         user_id: hug.types.uuid,
         organization_id: hug.types.text,
         gitlab_repository_id: hug.types.number,
@@ -1389,25 +1389,25 @@ def post_individual_signed_gitlab(
 
     Callback URL from signing service upon ICLA signature for a Gitlab user.
     """
-    content = body.read()
+    content = request.bounded_stream.read()
     return cla.controllers.signing.post_individual_signed_gitlab(
         content, user_id, organization_id, gitlab_repository_id, merge_request_id
     )
 
 
 @hug.post("/signed/gerrit/individual/{user_id}", versions=2)
-def post_individual_signed_gerrit(body, user_id: hug.types.uuid):
+def post_individual_signed_gerrit(request, user_id: hug.types.uuid):
     """
-    POST: /signed/gerritindividual/{user_id}
+    POST: /signed/gerrit/individual/{user_id}
 
     Callback URL from signing service upon ICLA signature for a Gerrit user.
     """
-    content = body.read()
+    content = request.bounded_stream.read()
     return cla.controllers.signing.post_individual_signed_gerrit(content, user_id)
 
 
 @hug.post("/signed/corporate/{project_id}/{company_id}", versions=2)
-def post_corporate_signed(body, project_id: hug.types.uuid, company_id: hug.types.uuid):
+def post_corporate_signed(request, project_id: hug.types.uuid, company_id: hug.types.uuid):
     """
     POST: /signed/corporate/{project_id}/{company_id}
 
@@ -1416,7 +1416,7 @@ def post_corporate_signed(body, project_id: hug.types.uuid, company_id: hug.type
 
     Callback URL from signing service upon CCLA signature.
     """
-    content = body.read()
+    content = request.bounded_stream.read()
     return cla.controllers.signing.post_corporate_signed(content, str(project_id), str(company_id))
 
 

setenv.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-rm -rf /tmp/aws
+rm -rf /tmp/.aws
 cp -R ~/.aws /tmp/.aws
 
 dev_arn="$(cat ./product-contractors-role.dev.secret)"

tests/functional/utils/run-single-test.sh

@@ -23,6 +23,7 @@ then
     echo "Usage: $0 <test-file-name-without-extension> [test-name-regexp]"
     echo "Example (v4 APIs groups): V=4 $0 cla-group, cla-manager, company, docs, events, foundation, github-organizations, github-repositories, githubActivity, gitlab-organizations, gitlab-repositories, health, metrics, projects, signatures, version"
     echo "Example (v3 APIs groups): V=3 $0 cla-manager, docs, gerrits, github-organizations, health, project, template, version, company, events, github, github-repositories, organization, signatures, users"
+    echo "Example (v2 APIs groups): V=2 $0 company, events, gerrit, github, health, project, repository, signatures, user"
     exit 1
   fi
 fi

utils/run_go_api_server.sh

@@ -1,2 +1,2 @@
 #!/bin/bash
-make build-linux && PORT=5001 AUTH0_USERNAME_CLAIM_CLI='http://lfx.dev/claims/username' AUTH0_EMAIL_CLAIM_CLI='http://lfx.dev/claims/email' AUTH0_NAME_CLAIM_CLI='http://lfx.dev/claims/username' ./bin/cla
+cd .. && source setenv.sh && cd cla-backend-go && make build-linux && PORT=5001 AUTH0_USERNAME_CLAIM_CLI='http://lfx.dev/claims/username' AUTH0_EMAIL_CLAIM_CLI='http://lfx.dev/claims/email' AUTH0_NAME_CLAIM_CLI='http://lfx.dev/claims/username' ./bin/cla 1>golang-api.log 2>golang-api.err

utils/run_py_api_server.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+cd .. && source setenv.sh && cd cla-backend && AUTH0_USERNAME_CLAIM_CLI='http://lfx.dev/claims/username' AUTH0_EMAIL_CLAIM_CLI='http://lfx.dev/claims/email' AUTH0_NAME_CLAIM_CLI='http://lfx.dev/claims/username' yarn serve:ext 1>python-api.log 2>python-api.err
+# cd .. && source setenv.sh && cd cla-backend && yarn serve:ext 1>python-api.log 2>python-api.err