Acceptance test failure "When scaling by memoryused it should scale out ..." #2434
Description
Subject of the issue
go_app isn't reaching the set memory to allow the acceptance tests to pass.
Test case:
When using:
node_memory_limit: 256
[2023-12-08 15:27:55.63 (UTC)]> cf bind-service autoscaler-1-dynamic-policy-69d05a732f89ac88 autoscaler-1-service-2fbfe96f7de3c61f -c {"instance_min_count":1,"instance_max_count":2,"scaling_rules":[{"metric_type":"memoryused","breach_duration_secs":60,"threshold":200,"operator":">=","cool_down_secs":60,"adjustment":"+1"},{"metric_type":"memoryused","breach_duration_secs":60,"threshold":115,"operator":"<","cool_down_secs":60,"adjustment":"-1"}]}
Binding service instance autoscaler-1-service-2fbfe96f7de3c61f to app autoscaler-1-dynamic-policy-69d05a732f89ac88 in org ASATS-1-ORG-9f4f58e8ad4197bf / space ASATS-1-SPACE-fd4611c5a432f27d as ASATS-1-USER-ccbb9c446b3aff56...
OK
...
[2023-12-08 15:28:02.02 (UTC)]> curl -H Expect: -s https://autoscaler-1-dynamic-policy-69d05a732f89ac88.dev02.dev.cloudpipelineapps.digital/memory/200/5 -H X-Cf-App-Instance: 00855bde-2eaf-44cb-a6b7-53158e1ad9c4:0 -f --connect-timeout 5 --max-time 10 --retry 5 --retry-delay 0 --retry-max-time 15
{"memoryMiB":200,"minutes":5}
cf autoscaling-metrics autoscaler-1-dynamic-policy-69d05a732f89ac88 memoryused
Retrieving aggregated memoryused metrics for app autoscaler-1-dynamic-policy-69d05a732f89ac88...
Metrics Name Value Timestamp
memoryused 198MB 2023-12-08T15:32:49Z
memoryused 198MB 2023-12-08T15:32:09Z
memoryused 198MB 2023-12-08T15:31:29Z
memoryused 198MB 2023-12-08T15:30:49Z
memoryused 198MB 2023-12-08T15:30:09Z
memoryused 198MB 2023-12-08T15:29:29Z
memoryused 198MB 2023-12-08T15:28:49Z
Memory does not reach 200MB to trigger scale up
When using:
node_memory_limit: 128
[2023-12-08 15:34:41.51 (UTC)]> curl -H Expect: -s https://autoscaler-1-dynamic-policy-dcec3f630328ab50.dev02.dev.cloudpipelineapps.digital/memory/100/5 -H X-Cf-App-Instance: 27403589-5b59-4ce3-be47-6b8ba7c654f5:0 -f --connect-timeout 5 --max-time 10 --retry 5 --retry-delay 0 --retry-max-time 15
{"memoryMiB":100,"minutes":5}
...
[2023-12-08 15:34:32.15 (UTC)]> cf bind-service autoscaler-1-dynamic-policy-dcec3f630328ab50 autoscaler-1-service-1d3e70a74f9240e6 -c {"instance_min_count":1,"instance_max_count":2,"scaling_rules":[{"metric_type":"memoryused","breach_duration_secs":60,"threshold":100,"operator":">=","cool_down_secs":60,"adjustment":"+1"},{"metric_type":"memoryused","breach_duration_secs":60,"threshold":70,"operator":"<","cool_down_secs":60,"adjustment":"-1"}]}
Binding service instance autoscaler-1-service-1d3e70a74f9240e6 to app autoscaler-1-dynamic-policy-dcec3f630328ab50 in org ASATS-1-ORG-040deec86b0e6ce3 / space ASATS-1-SPACE-d80dc2be2e82029d as ASATS-1-USER-6ac21c8e731fa950...
OK
cf autoscaling-metrics autoscaler-1-dynamic-policy-dcec3f630328ab50 memoryused
Retrieving aggregated memoryused metrics for app autoscaler-1-dynamic-policy-dcec3f630328ab50...
Metrics Name Value Timestamp
memoryused 94MB 2023-12-08T15:38:09Z
memoryused 94MB 2023-12-08T15:37:29Z
memoryused 94MB 2023-12-08T15:36:49Z
memoryused 94MB 2023-12-08T15:36:09Z
memoryused 94MB 2023-12-08T15:35:29Z
Memory does not reach 100MB to trigger scale up
I don't know if this is a problem with the go_app or the metrics being reported.
Your environment
Version: v11.4.0
Database: postgres 13
Release: is using app-autoscaler-deployment.yml?
With custom operations
Acceptance tests: https://github.com/alphagov/paas-cf/blob/BC-app-autoscaler-base-manifest-switch/concourse/tasks/app-autoscaler-acceptance-tests-run.yml#L73
Steps to reproduce
Run the acceptance tests
Expected behaviour
Test passes.
Actual behaviour
Test fails