stepfunctions test_state doesn't returns correct response - lambda state

[royassis]

Describe the bug

import boto3

client = boto3.client('stepfunctions', region_name="[region]")

extra={'mock': {'result': '{"Payload": "{\\"statusCode\\": 200, \\"body\\": [{\\"sample_name\\": \\"unigen_GSM3954947\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954947.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954950\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954950.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954951\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954951.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954952\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954952.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954954\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954954.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954955\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954955.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954956\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954956.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954957\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954957.csv\\"}, {\\"sample_name\\": \\"unigen_GSM3954958\\", \\"sample_csv_path\\": \\"/Unigen/unigen_GSM3954958.csv\\"}]}", "ExecutedVersion": "239"}'},
 'context': '{"Execution": {"Id": "String", "Input": {"ann": {"annotation_level": "sample", "annotation_file": null, "annokey": null, "obscolumn": null, "geo_accession": "GSE134520"}, "test_data": null, "concurrency": 5, "dataset_name": "GSE134520_Homo_sapiens", "raw": true, "organism": "Homo_sapiens", "technology": "single-cell", "additional_h5ad2prod_params": {"mongodataset": "GSE134520_Homo_sapiens", "skip_annotations": false}, "mem": {"general": 4096, "scrublet": 65536, "infercnv": 4096, "cellranger": 65536}, "user_email": "sandrav@cgen.com", "pipeline_invoker": {"oid": "50e32390-100c-4bd7-8891-e9f19eeb3762", "username": "sergeyn@cgen.com"}, "force": false, "run": {"infercnv": false}, "celltypist": {"tissue_model": null}, "params": {"preprocessing": {"mt_threshold": 20, "n_genes_by_counts_threshold": 7000}, "scrublet": {"doublet_rate": 0.2, "input_threshold": 100}}, "fastq_input_directory": "", "working_directory": "", "reference": {"cellranger": "", "gtf": "/Rnd/data/db/mygene/gencode.v46.cgen.filtered.gtf", "rsem": "/Rnd/data/db/mygene/GgenHumanV46RsemRef", "star": "/Rnd/data/db/mygene/GgenHumanV46BulkRef"}, "cellranger": {"nosecondary": true, "create_bam": true, "chemistry": "auto"}, "jdi": {"unigen-bulk_processing": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen-bulk_processing:74", "cell_ranger_jd": "arn:aws:batch:[region]:[accountnum]:job-definition/cell_ranger_jd:133", "unigen_celltypist": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen_celltypist:131", "unigen_gene_symbols": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen_gene_symbols:102", "unigen_infercnv_step": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen_infercnv_step:109", "unigen-merge": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen-merge:113", "unigen-scrublet": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen-scrublet:144", "unigen-cellranger_cleanup": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen-cellranger_cleanup:74", "unigen-h5ad_2_prod": "arn:aws:batch:[region]:[accountnum]:job-definition/unigen-h5ad_2_prod:145"}}, "Name": "String", "RoleArn": "String", "StartTime": "2025-12-14T18:00:00Z", "RedriveCount": 12, "RedriveTime": "2025-12-14T18:00:00Z"}, "State": {"EnteredTime": "2025-12-14T18:00:00Z", "Name": "String", "RetryCount": 12}, "StateMachine": {"Id": "String", "Name": "String"}}'}

response = client.test_state(
	definition='{"Type": "Task", "Resource": "arn:aws:states:::lambda:invoke", "Retry": [{"ErrorEquals": ["Lambda.ServiceException", "Lambda.AWSLambdaException", "Lambda.SdkClientException", "Lambda.TooManyRequestsException"], "IntervalSeconds": 1, "MaxAttempts": 3, "BackoffRate": 2}], "Next": "Map - Samples", "QueryLanguage": "JSONata", "Arguments": {"Payload": {"input_directory": "{% $states.input.fastq_input_directory %}", "results_location": "{% $workfolder %}", "type": "single-cell"}, "FunctionName": "arn:aws:lambda:[region]:[accountnum]:function:unigen_get_sample_locations:prod"}, "Output": "{%  $merge([$states.input,{\'samples\':$states.result.Payload.body}])  %}"}',
	input='{"ann":{"annotation_level":"sample","annotation_file":null,"annokey":null,"obscolumn":null,"geo_accession":"GSE134520"},"test_data":null,"concurrency":5,"dataset_name":"GSE134520_Homo_sapiens","raw":true,"organism":"Homo_sapiens","technology":"single-cell","additional_h5ad2prod_params":{"mongodataset":"GSE134520_Homo_sapiens","skip_annotations":false},"mem":{"general":4096,"scrublet":65536,"infercnv":4096,"cellranger":65536},"user_email":"sandrav@cgen.com","pipeline_invoker":{"oid":"50e32390-100c-4bd7-8891-e9f19eeb3762","username":"sergeyn@cgen.com"},"force":false,"run":{"infercnv":false},"celltypist":{"tissue_model":null},"params":{"preprocessing":{"mt_threshold":20,"n_genes_by_counts_threshold":7000},"scrublet":{"doublet_rate":0.2,"input_threshold":100}},"fastq_input_directory":"","working_directory":"","reference":{"cellranger":"","gtf":"/Rnd/data/db/mygene/gencode.v46.cgen.filtered.gtf","rsem":"/Rnd/data/db/mygene/GgenHumanV46RsemRef","star":"/Rnd/data/db/mygene/GgenHumanV46BulkRef"},"cellranger":{"nosecondary":true,"create_bam":true,"chemistry":"auto"},"jdi":{"unigen-bulk_processing":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen-bulk_processing:74","cell_ranger_jd":"arn:aws:batch:[region]:[accountnum]:job-definition/cell_ranger_jd:133","unigen_celltypist":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen_celltypist:131","unigen_gene_symbols":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen_gene_symbols:102","unigen_infercnv_step":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen_infercnv_step:109","unigen-merge":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen-merge:113","unigen-scrublet":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen-scrublet:144","unigen-cellranger_cleanup":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen-cellranger_cleanup:74","unigen-h5ad_2_prod":"arn:aws:batch:[region]:[accountnum]:job-definition/unigen-h5ad_2_prod:145"}}',
	variables= '{"workfolder": "asd"}',
	**extra
)

assert "samples" in response["output"]

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Result of step should be the same as in a real step function invocation, i.e.
"samples" key in response["output"]

Current Behavior

"samples" key not in response["output"]

Reproduction Steps

see above

Possible Solution

No response

Additional Information/Context

No response

SDK version used

'1.42.9'

Environment details (OS name and version, etc.)

uname -a --------- Linux ip-172-31-39-52 6.8.0-1043-aws #45~22.04.1-Ubuntu SMP Wed Nov 12 16:16:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

[royassis]

After some investigation I see that 'Output': "{% {'samples':$states.result.Payload} %}" returns data,
and that states.result.Payload is a string instead of an object, it should return an object like in a real world response.

This is related to #4683

[adev-code]

Hello @royassis, thanks for reaching out. I have tried to replicate your code above and the output did not have "samples" . I have reached out to the Step Functions service team, if the output should have "samples". I will update as soon as there are updates from the team.

Internal Ref: P358738069