Skip to content

docs: warn about firestore+conversations#250

Open
KnorpelSenf wants to merge 1 commit intomainfrom
firestore-warning
Open

docs: warn about firestore+conversations#250
KnorpelSenf wants to merge 1 commit intomainfrom
firestore-warning

Conversation

@KnorpelSenf
Copy link
Copy Markdown
Member

@petalvlad
Copy link
Copy Markdown

Workaround

In conversations v2, anything returned from conversation.external(...) becomes part of the persisted state. With hydrate() in play, message/ctx objects may carry functions (e.g. helpers around api.forwardMessage). Firestore refuses to encode these and also dislikes some nested entities — leading to errors like:

  • Cannot encode value: (chat_id, other, signal) => api.forwardMessage(...)
  • 3 INVALID_ARGUMENT: Property state contains an invalid nested entity
  • Value for argument "data" is not a valid Firestore document. Input is not a plain JavaScript object.

So, the idea is to persist the entire conversation state as a single string field (state) to keep Firestore happy, while still preserving object graphs:

  1. Use flatted to handle cycles/shared refs.

  2. Add a custom replacer/reviver:

    • BigInt"bigint:<digits>"
    • Date"timestamp:<ms>"
    • Functions/Symbol/undefined are dropped.

This avoids invalid nested entities and not a plain object errors, and keeps the state reversible.


Code

Adapter

const { stringify, parse } = require("flatted");

const BIGINT_PREFIX = "bigint:";
const TIMESTAMP_PREFIX = "timestamp:";

function replacer(key, value) {
  if (typeof value === "function" || typeof value === "symbol" || value === undefined) {
    return undefined;
  }
  if (typeof value === "bigint") {
    return BIGINT_PREFIX + value.toString();
  }    
  if (value instanceof Date) {
    return TIMESTAMP_PREFIX + value.getTime();
  }
  return value;
}

function reviver(key, value) {
  if (typeof value === "string" && value.startsWith(BIGINT_PREFIX)) {
    try { 
      return BigInt(value.slice(BIGINT_PREFIX.length)); 
    } 
    catch { }
  }
  if (typeof value === "string" && value.startsWith(TIMESTAMP_PREFIX)) {
    try { 
      return new Date(Number(value.slice(TIMESTAMP_PREFIX.length)));
    } 
    catch { }
  }
  return value;
}

function firestoreAdapter(collection) {
  return {
    async read(key) {
      const snapshot = await collection.doc(key).get();
      try {
        return parse(snapshot.data()?.state, reviver);
      } 
      catch {
        return undefined;
      }
    },
    async write(key, value) {
      const state = stringify(value, replacer);
      await collection.doc(key).set({ state }, { merge: false });
    },
    async delete(key) {
      await collection.doc(key).delete();
    },
  };
}

module.exports = {
  firestoreAdapter
};

Wiring

const { Composer, session } = require("grammy");
const { conversations, createConversation } = require("@grammyjs/conversations");
const { hydrate } = require("@grammyjs/hydrate");
const { getFirestore } = require("firebase-admin/firestore");
const admin = require("firebase-admin");
const { firestoreAdapter } = require("../utils/adapter");
const conversation = require("./conversations/conversation");

admin.initializeApp();

const composer = new Composer();
composer.use(session({ 
  initial: () => ({ user: null })
}));
composer.use(hydrate());
composer.use(conversations({
  storage: firestoreAdapter(getFirestore().collection("conversations")),
  plugins: [hydrate()]
}));

composer.use(createConversation(conversation, "conversation"));

Example

Here’s what the serialized conversation state looks like in Firestore (5.87 KB):

[{"version":"1","state":"2"},[0,0],{"bybitDepositAddress":"3"},["4"],{"interrupts":"5","replay":"6"},[6],{"send":"7","receive":"8"},["9","10","11","12","13","14","15"],["16","17","18","19","20","21"],{"payload":"22"},{"payload":"23"},{"payload":"23"},{"payload":"23"},{"payload":"23"},{"payload":"24"},{"payload":"25"},{"send":0,"returnValue":"26"},{"send":1,"returnValue":"27"},{"send":2,"returnValue":"28"},{"send":3,"returnValue":"29"},{"send":4,"returnValue":"30"},{"send":5,"returnValue":"31"},"wait","external","editMessageText","callback",{"update_id":9957752,"callback_query":"32"},{"ok":true},{"ok":true,"ret":"33"},{"ok":true},{"ok":true,"ret":"34"},{"ok":true,"res":"35"},{"id":"36","from":"37","message":"38","chat_instance":"39","data":"40"},{"id":"41","user_id":"42","connection_failure_message":null,"bybit_account":"43","connected_at":"44"},{"ok":true,"data":"45"},{"ok":true,"result":"46"},"1326540972706916017",{"id":308859388,"is_bot":false,"first_name":"47","username":"48","language_code":"49","is_premium":true},{"message_id":710,"from":"50","chat":"51","date":1757090907,"edit_date":1757092930,"text":"52","reply_markup":"53"},"6490534562294901250","bybit_deposit_address","lBL2ha0AjfWgUeR24nfw","Zaeh2g1o9oRZfZH3rrJq",{"authorization_code":"54","created_at":"55","access_token":"56","refresh_token":"57","api_key":"58","api_secret":"59","uta":1,"kyc_level":"60","remote_id":20412326,"kyc_region":"61","status":"62","subaccount":"63"},"2025-09-04T06:33:05.507Z",["64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81"],{"message_id":710,"from":"82","chat":"83","date":1757090907,"edit_date":1757092932,"text":"84","reply_markup":"85"},"Александр","apetropavlovsky","ru",{"id":7582470148,"is_bot":true,"first_name":"86","username":"87"},{"id":308859388,"first_name":"47","username":"48","type":"88"},"Для подключения стратегии на балансе вашего сабаккаунта должно быть как минимум 300 USDT. Пополнить баланс сабаккаунта можно переводом с основного аккаунта, но на основном аккаунте тоже недостаточно средств. Сперва пополните его основной аккаунт.",{"inline_keyboard":"89"},"6CsaB51TqCkflkLQwfghYahsu","2025-09-04T06:33:02.580Z","eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NTcwNTM5ODIsIkNsaWVudElEIjoiM0V0aXB1amR1Vlo0IiwiR3JhbnRNZW1iZXJJRCI6MjA0MTIzMjYsIkFwcHJvdmVkU2NvcGUiOlsib3BlbmFwaSJdLCJOb25jZSI6ImpralFpOWpRc04ifQ.WMQAcLVqGeXmPnR_g2Dw0i2ouappkA1dZXnvWFb7p0U","eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3NTk1NTk1ODIsIkNsaWVudElEIjoiM0V0aXB1amR1Vlo0IiwiR3JhbnRNZW1iZXJJRCI6MjA0MTIzMjYsIkFwcHJvdmVkU2NvcGUiOlsib3BlbmFwaSJdLCJOb25jZSI6IlNPN0NXUHB3NU8ifQ.ZfVdM-XGWsrfexqgBLh9_xDNrvjCZiUGAyAtSaC2vek","FWJ9piuU3ZaZ00u6pd","GkFvtckH7VReSoBRSMai8uzJluZnJZu9lNPi","LEVEL_1","RUS","SUCCESSFUL",{"remote_id":505352331,"created_at":"90","username":"91","api_key":"92","api_secret":"93","expired_at":"94","status":"62"},{"chain":"95","chainType":"96"},{"chain":"97","chainType":"98"},{"chain":"99","chainType":"100"},{"chain":"101","chainType":"102"},{"chain":"103","chainType":"103"},{"chain":"104","chainType":"104"},{"chain":"105","chainType":"105"},{"chain":"106","chainType":"107"},{"chain":"108","chainType":"109"},{"chain":"110","chainType":"110"},{"chain":"111","chainType":"112"},{"chain":"113","chainType":"114"},{"chain":"115","chainType":"116"},{"chain":"117","chainType":"118"},{"chain":"119","chainType":"119"},{"chain":"120","chainType":"120"},{"chain":"121","chainType":"121"},{"chain":"122","chainType":"123"},{"id":7582470148,"is_bot":true,"first_name":"86","username":"87"},{"id":308859388,"first_name":"47","username":"48","type":"88"},"Выберите сеть для USDT",{"inline_keyboard":"124"},"Earnie Trading","earnie_trade_bot","private",["125","126","127"],"2025-09-04T06:33:04.260Z","ERNlBL1756967583","j1BY9G5brA2iMy5Qvo","fLSGmROuN8PGzgcGI0qkNM7nXcADP1q76EeF","2025-12-04T06:33:04Z","APTOS","Aptos","ARBI","Arbitrum One","BERA","Berachain","BSC","BNB Smart Chain","CAVAX","CELO","CORN","ETH","Ethereum","HYPEREVM","HyperEVM","KAVAEVM","KLAY","KAIA","MANTLE","Mantle Network","MATIC","Polygon PoS","OP","OP Mainnet","SOL","TON","TRX","ZKSYNC","zkSync Lite",["128","129","130","131","132","133","134","135","136","137","138"],["139"],["140"],["141"],["142","143"],["144","145"],["146","147"],["148","149"],["150","151"],["152","153"],["154","155"],["156","157"],["158","159"],["160"],["161"],{"text":"162","callback_data":"40"},{"text":"163","callback_data":"164"},{"text":"165","callback_data":"166"},{"text":"167","callback_data":"95"},{"text":"168","callback_data":"97"},{"text":"169","callback_data":"99"},{"text":"170","callback_data":"101"},{"text":"171","callback_data":"103"},{"text":"172","callback_data":"104"},{"text":"173","callback_data":"105"},{"text":"174","callback_data":"106"},{"text":"175","callback_data":"108"},{"text":"176","callback_data":"110"},{"text":"177","callback_data":"111"},{"text":"178","callback_data":"113"},{"text":"179","callback_data":"115"},{"text":"180","callback_data":"117"},{"text":"181","callback_data":"119"},{"text":"182","callback_data":"120"},{"text":"183","callback_data":"121"},{"text":"184","callback_data":"122"},{"text":"185","callback_data":"164"},{"text":"186","callback_data":"187"},"Адрес пополнения основного аккаунта","Я пополнил основной аккаунт","start","Обратиться в поддержку","support_menu","APTOS (Aptos)","ARBI (Arbitrum One)","BERA (Berachain)","BSC (BNB Smart Chain)","CAVAX (CAVAX)","CELO (CELO)","CORN (CORN)","ETH (Ethereum)","HYPEREVM (HyperEVM)","KAVAEVM (KAVAEVM)","KLAY (KAIA)","MANTLE (Mantle Network)","MATIC (Polygon PoS)","OP (OP Mainnet)","SOL (SOL)","TON (TON)","TRX (TRX)","ZKSYNC (zkSync Lite)","Назад","Главное меню","menu"]

Notes

  • I chose flatted over a custom decycle because it looks battle-tested, preserves cycles/shared references with a simple stringify/parse pair, keeps the payload JSON-friendly for Firestore, and avoids the maintenance burden of a hand-rolled solution.
  • Firestore document size still applies (~1 MB). Keep in mind that according to this workaround the whole conversation state is stored in a single field state.
  • BigInt/Date round-trip works via the prefixed markers BIGINT_PREFIX and TIMESTAMP_PREFIX.

Status / Help wanted

⚠️ Testing status: this adapter is lightly tested. I built it to get my conversations working with storage in a webhook setup and verified it only against my use case. There’s likely room for edge cases and improvements.

I’d really appreciate reviews, test cases, and fixes from anyone interested. Happy to iterate on this and adjust the approach if maintainers prefer a different direction. Also, big thanks to @KnorpelSenf for the initial idea to use JSON.stringify.

@KnorpelSenf KnorpelSenf requested a review from Satont September 5, 2025 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants