Skip to content

Conversation

@ANJAN672
Copy link

Fixes #3459

Description
When the repository path does not exist (e.g., repo not cloned yet), the scc tool outputs a plain-text error instead of JSON, causing json.loads() to fail.

This fix adds a path existence check before running scc. If the path doesn't exist, the task logs a warning and returns gracefully instead of failing.

Notes for Reviewers

  • Tested with mock: function returns gracefully when path is missing
  • All unit tests pass

Signed commits

  • Yes, I signed my commits.

Fixes chaoss#3459

Signed-off-by: ANJAN672 <rockyanjan672@gmail.com>
Signed-off-by: ANJAN672 <rockyanjan672@gmail.com>
@iGufrankhan
Copy link
Contributor

@ANJAN672
This is a good change — it definitely prevents SCC from crashing when the repo path is missing.
One suggestion, if you have time:
could you help verify why this is happening in the first place?
My guess is that there might be a path mismatch between where the repo is cloned and where SCC looks for it.
Maybe we can add a few log lines:
in the place where the repo path is created (clone step)
and in the place where SCC uses the path
Then we can compare those two paths in the logs and see if they really match.
That should tell us whether it’s a clone failure, ordering issue, or path mismatch.

path used in this file https://github.com/chaoss/augur/blob/main/augur/tasks/git/scc_value_tasks/core.py
and path is created in this file https://github.com/chaoss/augur/blob/main/augur/tasks/git/util/facade_worker/facade_worker/repofetch.py
also check the
function like
1.get_repo_by_repo_git
2.get_absolute_repo_path
if we able to find the exact error why this happen and able to fiix it this best for this project in longer term context

@ANJAN672
Copy link
Author

Sure @iGufrankhan

@shlokgilda
Copy link
Collaborator

Thanks for the contribution @ANJAN672.

That's a good suggestion @iGufrankhan . Can you make this a draft PR @ANJAN672 while you debug the issue? Maybe we can continue the discussion in the linked issue.

@ANJAN672 ANJAN672 marked this pull request as draft January 17, 2026 10:54
@ANJAN672
Copy link
Author

@iGufrankhan , @shlokgilda
I traced through the code paths you mentioned:

Path used in SCC (scc_value_tasks/core.py:17):

  • Calls get_absolute_repo_path() which builds: {repo_base_dir}{repo_id}-{repo_path}/{repo_name}

Path created in Facade (repofetch.py:127-157):

  • Creates directory with mkdir()
  • Then runs git clone (can take 5-30+ min for large repos)

Root Cause: Race Condition

The problem is timing between tasks. When the Facade task runs:

  1. It updates repo_path and repo_name in the database immediately
  2. Then starts git clone which takes a long time

Meanwhile, the SCC task sees the database record and tries to run, but the clone isn't finished yet. So os.path.exists(path) returns FALSE because the files haven't been downloaded.

Proposed Solution

My current fix handles the symptom (skip gracefully if path missing). To address the root cause, I can also add a check for collection_status.facade_status = 'Success' before running SCC - this ensures the clone is actually complete.

Let me know if you'd like me to implement that additional check!

@ANJAN672

This comment was marked as spam.

@MoralCode
Copy link
Contributor

this sounds like it may be some kind of task sequencing issue. it sounds like the clone task and the SCC analysis task are set up in a group, or some other kind of celery structure that runs in parallel. which, as you point out, seems to cause a race condition depending on the speed of the clone.

I know facade has some extra status values to indicate the progress of the clone. Success will not work though as that is the status used for when every step is completed, including all the analysis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

process_scc_value_metrics seems to have trouble processing the json output

4 participants