ERROR: relation "binding" does not exist

[itessential]

Subject of the issue

Golangapiserver can't execute query even if proper DB exists (but with different name - which is mentioned properly in the manifest):

data { error ERROR: relation "binding" does not exist (SQLSTATE 42P01) instanceId 58dfae44-9e58-43b5-8581-1a4839381e1c query SELECT binding_id FROM binding WHERE service_instance_id=$1 session 3 }

Your environment

• Version of app-autoscaler-release: 15.8.0
• DB: Postgres
• How did you deploy the release?
  • Is your manifest based on app-autoscaler.yml - YES
  • BOSH release

Steps to reproduce

Specify binding database with other name (autoscaler_binding in my example).

Expected behaviour

Query should be executed successfully.

Actual behaviour

Binding for service can not be created due to the error.

[geigerj0]

@itessential could you please share a redacted version of your deployment manifest?

[itessential]

manifest.yml

Here it is.

[geigerj0]

For historical reasons, the Autoscaler supports configuring multiple databases. The original idea was that each microservice could have its own database. Consequently, each microservice was designed to be configured with potentially multiple database connections. In practice, this setup proved cumbersome, and everyone has since standardized on configuring the Autoscaler with a single database. Additionally, in earlier versions, servicebroker and api were two separate microservices, but they were later merged into a single component called golangapiserver.

The issue you’re seeing is most likely caused by the following:

• The binding table is created via this servicebroker migration file:

  • app-autoscaler-release/src/autoscaler/api/db/servicebroker.db.changelog.yaml

    Lines 54 to 55 in c2f15b7

    	- createTable:
    	tableName: binding

• The BPM pre-start script of golangapiserver runs servicebroker migration also targeting the policy_db — which was originally dedicated to the old api microservice:

  • app-autoscaler-release/jobs/golangapiserver/templates/bpm-pre-start.erb

    Line 9 in c2f15b7

    database = p_arr('autoscaler.policy_db.databases').find { |database| database['tag'] == 'policydb' or database['tag'] == 'default' }

  • app-autoscaler-release/jobs/golangapiserver/templates/bpm-pre-start.erb

    Line 22 in c2f15b7

    DBNAME='<%= database['name'] %>'

  • app-autoscaler-release/jobs/golangapiserver/templates/bpm-pre-start.erb

    Line 47 in c2f15b7

    JDBCDBURL="jdbc:mysql://$HOST:$PORT/$DBNAME?autoReconnect=true"

  • app-autoscaler-release/jobs/golangapiserver/templates/bpm-pre-start.erb

    Lines 65 to 66 in c2f15b7

    	retry java -cp "$DB_DIR/target/lib/*:$API_DIR" liquibase.integration.commandline.Main --url "$JDBCDBURL" --username=$USER --password=$PASSWORD \
    	--driver=org.postgresql.Driver --changeLogFile=servicebroker.db.changelog.yaml update

Quick workaround:
Use the “goto approach” — configure all database connections to point to the same database. This aligns with how the Autoscaler is typically deployed today.

Long-term fix:
If you’d like to improve this behavior, feel free to open a PR. Unfortunately, we’re not able to address this issue in the foreseeable future.

[itessential]

OK, thanks.

Do you know maybe why my autoscaler_policy database (policy_db) is empty then?

[geigerj0]

You could check the Liquibase logs (perhaps via the BPM pre-start script logs of the golangapiserver — though I’m not entirely sure how to access them) to see what changes are being applied to your database during the migration runs.

[itessential]

Thank you for your help! Moving all of these to one DB solved the issue.

[geigerj0]

Reopening the issue since the root cause of this issue hasn't been solved yet, see #3967 (comment)