Skip to content

Implement Spector integration tests for 25 scenario groups#29

Draft
Copilot wants to merge 4 commits intomainfrom
copilot/implement-spector-tests-scenario-groups-again
Draft

Implement Spector integration tests for 25 scenario groups#29
Copilot wants to merge 4 commits intomainfrom
copilot/implement-spector-tests-scenario-groups-again

Conversation

Copy link

Copilot AI commented Mar 5, 2026

Adds missing Azure Modular integration test coverage for 25 scenario groups across authentication, ARM resources, encoding, payload, parameters, and service patterns.

New test entries in cadl-ranch-list.js + generated clients

8 new paths added to azureModularTsps:

  • azure/client-generator-core/client-default-value
  • azure/resource-manager/multi-service, multi-service-older-versions, multi-service-shared-models
  • service/multi-service
  • authentication/noauth/union
  • parameters/query
  • response/status-code-range

New spec files (8)

File Scenarios
azureClientGeneratorCoreClientDefaultValue.spec.ts getHeaderParameter, getOperationParameter, getPathParameter, putModelProperty
azureArmMultiService.spec.ts VMs + Disks: createOrUpdate, get
azureArmMultiServiceOlderVersions.spec.ts VMs + Disks (older API versions): createOrUpdate, get
azureArmMultiServiceSharedModels.spec.ts VMs + StorageAccounts (shared models): createOrUpdate, get
serviceMultiService.spec.ts ServiceA Foo + ServiceB Bar: test
authNoauthUnion.spec.ts validNoAuth, validToken
parametersQuery.spec.ts Parameters_Query_Constant post
responseStatusCodeRange.spec.ts errorResponseStatusCode404, errorResponseStatusCodeInRange

Updated existing spec files (5)

  • azureCorePage.spec.ts — adds withRelativeNextLink
  • azureArmOperationTemplates.spec.ts — adds exportArray (LRO returning ExportResult[])
  • resiliencySrvDrivenMain.spec.ts — adds breakTheGlass: uses the old v1-compiled client's raw HTTP pipeline to call a v2-only endpoint (add-operation), bypassing the typed client
  • payloadXml.spec.ts — adds ModelWithDatetimeValue, ModelWithEnumValue, XmlErrorValue
  • azureArmResources.spec.ts — un-skips TopLevel.update (PATCH → 200, LRO resolves immediately)

Known skips / emitter limitations

  • Azure_ClientGenerator_Core_Override_GroupParameters (group) — emitter bug #3540
  • Azure_ResourceManager_CommonProperties_Error.createForUserDefinedError — emitter bug #2965
  • Encode_Duration_Header/Query floatSecondsLargerUnit / floatMillisecondsLargerUnit — remain it.skip
  • Payload_MultiPart_FormData_HttpParts_NonString_float — entire multipart file commented out (serialization bug)
  • Streaming_Jsonl_Basic (send + receive) — emitter emits __PLACEHOLDER_o51__ for the JSONL stream parameter type; spec not created pending emitter fix
Original prompt

This section details on the original issue you should resolve

<issue_title>[Copilot] Implement spector tests for 25 scenario groups</issue_title>
<issue_description>## Task: Implement Spector Integration Tests
Use the spector-test-implementer skill to implement test files for the following not-implemented scenario groups.
Not-implemented scenario groups:

  • Azure_ClientGenerator_Core_ClientDefaultValue (4 scenarios: getHeaderParameter, getOperationParameter, getPathParameter, putModelProperty)
  • Azure_ClientGenerator_Core_Override_GroupParameters (1 scenarios: group)
  • Azure_Core_Page (2 scenarios: withParameterizedNextLink, withRelativeNextLink)
  • Azure_ResourceManager_CommonProperties_Error (1 scenarios: createForUserDefinedError)
  • Azure_ResourceManager_MultiServiceOlderVersions_ComputeDisk_Disks (2 scenarios: createOrUpdate, get)
  • Azure_ResourceManager_MultiServiceOlderVersions_Compute_VirtualMachines (2 scenarios: createOrUpdate, get)
  • Azure_ResourceManager_MultiServiceSharedModels_Compute_VirtualMachines (2 scenarios: createOrUpdate, get)
  • Azure_ResourceManager_MultiServiceSharedModels_Storage_StorageAccounts (2 scenarios: createOrUpdate, get)
  • Azure_ResourceManager_MultiService_ComputeDisk_Disks (2 scenarios: createOrUpdate, get)
  • Azure_ResourceManager_MultiService_Compute_VirtualMachines (2 scenarios: createOrUpdate, get)
  • Azure_ResourceManager_OperationTemplates_Lro (1 scenarios: exportArray)
  • Azure_ResourceManager_Resources_TopLevel (1 scenarios: update)
  • Resiliency_ServiceDriven (1 scenarios: breakTheGlass)
  • Service_MultiService_ServiceA_Foo (1 scenarios: test)
  • Service_MultiService_ServiceB_Bar (1 scenarios: test)
  • Authentication_Noauth_Union (2 scenarios: validNoAuth, validToken)
  • Encode_Duration_Header (2 scenarios: floatMillisecondsLargerUnit, floatSecondsLargerUnit)
  • Encode_Duration_Query (2 scenarios: floatMillisecondsLargerUnit, floatSecondsLargerUnit)
  • Parameters_Query_Constant (1 scenarios: post)
  • Payload_MultiPart_FormData_HttpParts_NonString (1 scenarios: float)
  • Payload_Xml_ModelWithDatetimeValue (2 scenarios: get, put)
  • Payload_Xml_ModelWithEnumValue (2 scenarios: get, put)
  • Payload_Xml_XmlErrorValue (1 scenarios: get)
  • Response_StatusCodeRange (2 scenarios: errorResponseStatusCode404, errorResponseStatusCodeInRange)
  • Streaming_Jsonl_Basic (2 scenarios: receive, send)
    Instructions:
  1. For each scenario group above, check if a .spec.ts file already exists in packages/typespec-ts/test/azureModularIntegration/ — skip if it does
  2. For remaining groups, follow the spector-test-implementer workflow to implement the tests
  3. Report which tests passed and which failed
    PR description format:
    The PR description must use this format for each section:

New spec files

For each newly created .spec.ts file, use this structure:

  • Scenario_Group_NamescenarioGroupName.spec.ts
    • scenarioName1
    • scenarioName2
    • scenarioName3 (reason for failure)

Updated existing spec files

Same structure as above, but for spec files that already existed and were updated with new scenarios.

Known skips / emitter limitations

List any scenarios that were skipped due to emitter bugs or limitations, with the reason.

Results should be commented on issue #17.</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits March 5, 2026 09:29
Co-authored-by: JialinHuang803 <139532647+JialinHuang803@users.noreply.github.com>
…comments

Co-authored-by: JialinHuang803 <139532647+JialinHuang803@users.noreply.github.com>
Co-authored-by: JialinHuang803 <139532647+JialinHuang803@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement Spector integration tests for scenario groups Implement Spector integration tests for 25 scenario groups Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Copilot] Implement spector tests for 25 scenario groups

2 participants