agent crashing while cycling sender_allocation created #682
Description
Upgraded from 1.8.0 to 1.9.1 on agent and 1.4.2 to 1.5.0 on service.
Agent is cycling through sender allocation creations and then crashing, excerpt of the crash log below
indexer-tap-agent | 2025-04-07T17:27:07.302303Z INFO indexer_tap_agent::agent::sender_allocation: SenderAllocation created!, sender: 0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467, allocation_id: 0x43Fd71685a125AE89DC45b5B30ef908B26F7333E
indexer-tap-agent | at crates/tap-agent/src/agent/sender_allocation.rs:284
indexer-tap-agent | in ractor::actor::Actor with id: "0.755", name: "0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467:0x43Fd71685a125AE89DC45b5B30ef908B26F7333E"
indexer-tap-agent | in ractor::actor::Actor with id: "0.1", name: "legacy:0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467"
indexer-tap-agent | in ractor::actor::Actor with id: "0.0"
indexer-tap-agent |
indexer-tap-agent | 2025-04-07T17:27:07.370081Z INFO indexer_tap_agent::agent::sender_allocation: SenderAllocation created!, sender: 0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467, allocation_id: 0x5EbDCbf4E826FE9b59d8896C0f0AA47de8D6C024
indexer-tap-agent | at crates/tap-agent/src/agent/sender_allocation.rs:284
indexer-tap-agent | in ractor::actor::Actor with id: "0.760", name: "0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467:0x5EbDCbf4E826FE9b59d8896C0f0AA47de8D6C024"
indexer-tap-agent | in ractor::actor::Actor with id: "0.1", name: "legacy:0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467"
indexer-tap-agent | in ractor::actor::Actor with id: "0.0"
indexer-tap-agent |
indexer-tap-agent | 2025-04-07T17:27:07.370115Z INFO indexer_tap_agent::agent::sender_account: SenderAccount created!, sender: 0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467
indexer-tap-agent | at crates/tap-agent/src/agent/sender_account.rs:948
indexer-tap-agent | in ractor::actor::Actor with id: "0.1", name: "legacy:0xDDE4cfFd3D9052A9cb618fC05a1Cd02be1f2F467"
indexer-tap-agent | in ractor::actor::Actor with id: "0.0"
indexer-tap-agent |
indexer-tap-agent | thread 'main' panicked at crates/tap-agent/src/agent/sender_accounts_manager.rs:654:10:
indexer-tap-agent | should be able to fetch pending receipts from the database: Database(PgDatabaseError { severity: Error, code: "42P01", message: "relation \"tap_horizon_receipts\" does not exist", detail: None, hint: None, position: Some(Original(118)), where: None, schema: None, table: None, column: None, data_type: None, constraint: None, file: Some("parse_relation.c"), line: Some(1449), routine: Some("parserOpenTable") })
indexer-tap-agent | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
indexer-tap-agent | thread 'main' panicked at /root/crates/tap-agent/src/agent.rs:194:10:
indexer-tap-agent | Failed to start sender accounts manager actor.: StartupFailed("should be able to fetch pending receipts from the database: Database(PgDatabaseError { severity: Error, code: \"42P01\", message: \"relation \\\"tap_horizon_receipts\\\" does not exist\", detail: None, hint: None, position: Some(Original(118)), where: None, schema: None, table: None, column: None, data_type: None, constraint: None, file: Some(\"parse_relation.c\"), line: Some(1449), routine: Some(\"parserOpenTable\") })")