(V3) Verifying a contract with a missing content field is always successful when it should error #1103
Description
Have you read the Contributing Guidelines on issues?
- I have read the Contributing Guidelines on issues.
Prerequisites
- I'm using the latest version of
pact-python. - I have read the console error message carefully (if applicable).
Description
I have two pacts, one is incorrectly setting the content thus:
"contents": {
"Payload": "exists"
}
The other is correctly setting the content thus:
"contents": {
"content": {
"Payload": "exists"
},
"contentType": "application/json",
"encoded": false
}
When verifying with content that does not match, I expect both pacts to fail to verify. However, the first pact with the incorrect json verifies successfully and there is no error or warning that anything is wrong:
Verifying a pact between my-consumer and my-provider
This works as expected when verifying my-provider (0s loading, 8ms verification)
Given When reading a message from the SQS queue
generates a message which
includes metadata
"contentType" with value "application/json" (OK)
has a matching body (FAILED)
Verifying a pact between other-consumer and my-provider
This is silently ignored while verifying my-provider (0s loading, 6ms verification)
Given When reading a message from the SQS queue
generates a message which
has a matching body (OK)
I'll see if I can get approval to share a producible demo repo, but for now the pacts are:
pact-missing-contents.json
{
"consumer": {
"name": "other-consumer"
},
"provider": {
"name": "my-provider"
},
"interactions": [
{
"_id": "1",
"type": "Asynchronous/Messages",
"description": "This is silently ignored while verifying my-provider",
"providerStates": [
{
"name": "When reading a message from the SQS queue"
}
],
"contents": {
"Payload": "exists"
}
}
],
"metadata": {
"pactRust": {
"ffi": "0.4.22",
"models": "1.2.3"
},
"pactSpecification": {
"version": "4.0"
}
}
}
pact-with-contents.json
{
"consumer": {
"name": "my-consumer"
},
"provider": {
"name": "my-provider"
},
"interactions": [
{
"_id": "2",
"type": "Asynchronous/Messages",
"description": "This works as expected when verifying my-provider",
"providerStates": [
{
"name": "When reading a message from the SQS queue"
}
],
"contents": {
"content": {
"Payload": "exists"
},
"contentType": "application/json",
"encoded": false
}
}
],
"metadata": {
"pactRust": {
"ffi": "0.4.22",
"models": "1.2.3"
},
"pactSpecification": {
"version": "4.0"
}
}
}
And here's an idea of running the verification. I actually use a whole pytest setup with fixtures, but for brevity this ought to be able to reproduce the issue. Again, hopefully I'll be able to share a repo soon.
verify.py
def run_message_handler(name: str, metadata: dict[str, Any] | None) -> Message: """This should fail to match due to missing Payload field for **both** pacts""" return Message(contents=b"{}", content_type="application/json", metadata=None)verifier = Verifier("my-provider")
pact_dir = Path(file).parent / "pacts"
verifier = verifier.add_source(pact_dir).set_error_on_empty_pact()
verifier.message_handler(run_message_handler).verify()
*caveat to "I'm using the latest version": I'm using the latest version available on pypi, version 2.3.1 (as opposed to 2.3.2 available via github releases)
Reproducible demo
No response
Steps to reproduce
step 1: copy the two pact jsons and place them in separate files in a pacts/ directory
step 2: verify the pacts using pact v3, see verify.py for an example
Expected behavior
it should fail with an error due to missing "content" and/or "contentType" but it does not.
You can also enable logging (before step 3) and confirm there is no warning or error logged.
Actual behavior
"other-consumer" pact (pact-missing-contents.json) is verified successfully.
"my-consumer" pact (pact-with-contents.json) fails to verify (due to the mismatched content, as expected).
Your environment
- Public source code: no
- Is this a consumer or provider issue? Do you have information about the other side?: provider
- Pact Python version used:
pip list | grep pact: pact-python 2.3.1 - Information about your Pact broker (version, hosted where, pactflow, ...): N/A
- Operating system and version (e.g. Ubuntu 20.04.2 LTS, macOS Ventura): Debian GNU/Linux 12 (bookworm)
Self-service
- I'd be willing to fix this bug myself.
Metadata
Metadata
Assignees
Type
Projects
Status