Skip to content

Add missing TestWorkflowSecurity and TestEdgeCases classes to iteration status emails workflow tests#177

Open
Copilot wants to merge 12 commits intomainfrom
copilot/sub-pr-171-another-one
Open

Add missing TestWorkflowSecurity and TestEdgeCases classes to iteration status emails workflow tests#177
Copilot wants to merge 12 commits intomainfrom
copilot/sub-pr-171-another-one

Conversation

Copy link
Contributor

Copilot AI commented Jan 25, 2026

  • Understand the feedback: rename TestSecurityBestPractices to TestWorkflowSecurity and add TestEdgeCases class
  • Explore the repository structure and understand the consistency requirements
  • Rename TestSecurityBestPractices class to TestWorkflowSecurity
  • Add TestEdgeCases class with appropriate test methods
  • Run tests to validate changes (39 tests passing)
  • Address code review feedback for consistency
  • Run final code review and security checks
  • Reply to review comment
  • Improve test_workflow_handles_missing_dashboard to verify parse step checks for dashboard file and has failure handler
  • Fix test_yaml_is_parseable to handle YAML 'on' key parsing as True
  • Address code review feedback: fix True key check and simplify file check pattern
  • Improve test robustness: use step id for reliable matching, more specific assertions
  • Fix formatting: add blank line between test methods
  • Add jobs fixture and simplify test_no_duplicate_job_names to match pattern in other workflow tests
  • Remove trailing whitespace from blank lines
  • Change test_no_duplicate_job_names to use workflow_content['jobs'].keys() instead of jobs fixture

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.


Summary by cubic

Renames TestSecurityBestPractices to TestWorkflowSecurity and adds a TestEdgeCases suite to cover YAML and workflow edge cases, per review on #171. No behavior changes; this aligns the refactor with feedback to unblock #171.

Written for commit 625eb2c. Summary will update on new commits.

Copilot AI and others added 2 commits January 25, 2026 02:08
…EdgeCases class

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
…_missing_dashboard

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
Copilot AI changed the title [WIP] WIP Address feedback from review on big refactor PR Add missing TestWorkflowSecurity and TestEdgeCases classes to iteration status emails workflow tests Jan 25, 2026
Copilot AI requested a review from JaclynCodes January 25, 2026 02:11
@JaclynCodes JaclynCodes marked this pull request as ready for review January 25, 2026 02:14
Copilot AI review requested due to automatic review settings January 25, 2026 02:14
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 25, 2026

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the iteration status emails workflow test module to match the common workflow test structure expected by cross-file consistency tests.

Changes:

  • Renamed TestSecurityBestPractices to TestWorkflowSecurity for naming consistency across workflow test modules.
  • Added a TestEdgeCases class with additional YAML/workflow edge-case checks.

JaclynCodes and others added 2 commits January 24, 2026 21:18
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Misfit <218383634+JaclynCodes@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Misfit <218383634+JaclynCodes@users.noreply.github.com>
@JaclynCodes
Copy link
Owner

@copilot apply changes based on the comments in this thread🚀

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="tests/workflows/test_iteration_status_emails_workflow.py">

<violation number="1" location="tests/workflows/test_iteration_status_emails_workflow.py:393">
P2: The required key check treats only the literal "on" key as valid, but YAML may parse it as True (noted elsewhere in this file). This will cause false failures on valid workflows that use the boolean key.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Copilot AI and others added 3 commits January 25, 2026 02:21
…error handling

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
…k pattern

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
…pecific assertions

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
JaclynCodes

This comment was marked as duplicate.

…sts and remove trailing whitespace

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 25, 2026

@copilot apply changes based on the comments in this thread

Applied all changes from review #3703102668:

  1. Added jobs fixture matching other workflow tests
  2. Simplified test_no_duplicate_job_names to use jobs fixture (consistent with test_static_workflow.py)
  3. Removed trailing whitespace from blank lines

All 39 tests pass. Changes in commit 9a9acd3.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="tests/workflows/test_iteration_status_emails_workflow.py">

<violation number="1" location="tests/workflows/test_iteration_status_emails_workflow.py:359">
P2: This test no longer detects duplicate job names because it checks uniqueness on a dict that has already discarded duplicates. Use a loader that raises on duplicate keys so the test still fails when the YAML contains duplicate job entries.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

assert leading_spaces % 2 == 0, \
f"Line {i} has indentation that is not a multiple of 2 spaces"

def test_no_duplicate_job_names(self, jobs):
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This test no longer detects duplicate job names because it checks uniqueness on a dict that has already discarded duplicates. Use a loader that raises on duplicate keys so the test still fails when the YAML contains duplicate job entries.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/workflows/test_iteration_status_emails_workflow.py, line 359:

<comment>This test no longer detects duplicate job names because it checks uniqueness on a dict that has already discarded duplicates. Use a loader that raises on duplicate keys so the test still fails when the YAML contains duplicate job entries.</comment>

<file context>
@@ -347,26 +355,11 @@ def test_consistent_indentation(self, workflow_raw):
-        # Parsing with UniqueKeyLoader will raise if any duplicate keys are present in the YAML.
-        yaml.load(workflow_raw, Loader=UniqueKeyLoader)
+
+    def test_no_duplicate_job_names(self, jobs):
+        """Test that there are no duplicate job names."""
+        job_names = list(jobs.keys())
</file context>
Fix with Cubic

@JaclynCodes
Copy link
Owner

@copilot apply changes based on the comments in this thread

…ys() as requested

Co-authored-by: JaclynCodes <218383634+JaclynCodes@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 25, 2026

@copilot apply changes based on the comments in this thread🚀

All changes from review thread #3703096165 have been applied. The implementation now uses workflow_content['jobs'].keys() as clarified in comment 2724788844. Commit 625eb2c.

@copilot apply changes based on the comments in this thread

Applied the requested change to use workflow_content['jobs'].keys() instead of the jobs fixture. All 39 tests pass. Commit 625eb2c.

@copilot Check if these issues are valid — if so, understand the root cause of each and fix them.

...

The issue was clarified in comment 2724788844. Changed the implementation to use workflow_content['jobs'].keys() as requested. All tests pass. Commit 625eb2c.

This comment has been minimized.

Copy link
Owner

@JaclynCodes JaclynCodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀🚀

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

Comment on lines +364 to +371
def test_no_duplicate_step_ids(self, workflow_content):
"""Test that step IDs are unique within each job."""
jobs = workflow_content.get('jobs', {})
for job_name, job_config in jobs.items():
steps = job_config.get('steps', [])
step_ids = [s.get('id') for s in steps if 'id' in s]
assert len(step_ids) == len(set(step_ids)), \
f"Duplicate step IDs in job '{job_name}'"
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_no_duplicate_step_ids assumes every step is a dict; if the YAML contains an empty step (None) or a non-mapping value, if 'id' in s will raise. Consider asserting isinstance(step, dict) (or filtering to dicts) before inspecting keys so the test fails cleanly on malformed steps.

Copilot uses AI. Check for mistakes.
Comment on lines +57 to +62
@pytest.fixture(scope='module')
def jobs(workflow_content):
"""
Module-scoped fixture for jobs configuration.
"""
return workflow_content.get('jobs', {})
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new jobs fixture is not referenced anywhere in this module. Either remove it to avoid dead test code, or switch the edge-case tests to accept jobs and use it (matching the pattern in other workflow test modules).

Copilot uses AI. Check for mistakes.

def test_no_duplicate_job_names(self, workflow_content):
"""Test that there are no duplicate job names."""
job_names = list(workflow_content['jobs'].keys())
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_no_duplicate_job_names indexes workflow_content['jobs'], which will raise a KeyError (test error) if the YAML is missing the jobs key. Prefer asserting jobs exists first (or using the jobs fixture / workflow_content.get('jobs')) so the test fails with a clear assertion message instead of erroring.

Suggested change
job_names = list(workflow_content['jobs'].keys())
jobs = workflow_content.get('jobs')
assert jobs is not None, "Workflow must define jobs"
job_names = list(jobs.keys())

Copilot uses AI. Check for mistakes.
missing = required_keys.difference(workflow_content.keys())
assert not missing, f"Workflow is missing required top-level keys: {', '.join(sorted(missing))}"
# Verify trigger configuration exists (can be 'on' or True)
has_triggers = 'on' in workflow_content or True in workflow_content.keys()
Copy link

Copilot AI Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has_triggers = 'on' in workflow_content or True in workflow_content.keys() can produce false positives because True == 1 in Python (so a numeric key 1 would satisfy it). Use the same approach as the other trigger tests (workflow_content.get('on') or workflow_content.get(True)) or check boolean keys with identity (e.g., any(k is True for k in keys)).

Suggested change
has_triggers = 'on' in workflow_content or True in workflow_content.keys()
has_triggers = (
'on' in workflow_content
or any(k is True for k in workflow_content.keys())
)

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

<img src="https://octodex.github.com/images/supportcat.png" align="right" height="100px" />

2 participants