Using multi-sample/semantic-entropy for topic categorization (and for summarization?)

[jucor]

Dear Jigsaw team

If you are concerned about hallucinations in the topic categorization, I have just summarized various ideas that we have at polis to reduce them by exploiting compute at test-time with multi-sampling and/or semantic entropy: compdemocracy/polis#1880 .

You are already exploiting the stochasticity of the LLM with the rejection/retry scheme when the LLM gives an empty answer, in

sensemaking-tools/src/tasks/categorization.ts

Lines 54 to 89 in 9c86f3b

	for (let attempts = 1; attempts <= MAX_RETRIES; attempts++) {
	// convert JSON to string representation that will be sent to the model
	const uncategorizedCommentsForModel: string[] = uncategorized.map((comment) =>
	JSON.stringify({ id: comment.id, text: comment.text })
	);
	const outputSchema: TSchema = Type.Array(
	includeSubtopics ? SubtopicCategorizedComment : TopicCategorizedComment
	);
	const newCategorized: CommentRecord[] = (await model.generateData(
	getPrompt(instructions, uncategorizedCommentsForModel, additionalInstructions),
	outputSchema
	)) as CommentRecord[];
	const newProcessedComments = processCategorizedComments(
	newCategorized,
	inputComments,
	uncategorized,
	includeSubtopics,
	topics
	);
	categorized = categorized.concat(newProcessedComments.commentRecords);
	uncategorized = newProcessedComments.uncategorizedComments;
	if (uncategorized.length === 0) {
	break; // All comments categorized successfully
	}
	if (attempts < MAX_RETRIES) {
	console.warn(
	`Expected all ${uncategorizedCommentsForModel.length} comments to be categorized, but ${uncategorized.length} are not categorized properly. Retrying in ${RETRY_DELAY_MS / 1000} seconds...`
	);
	await new Promise((resolve) => setTimeout(resolve, RETRY_DELAY_MS));
	} else {
	categorized = categorized.concat(assignDefaultCategory(uncategorized, includeSubtopics));
	}
	}

Multi-sampling / Semantic entropy would go one step further by systematically calling multiple times, not just upon failure [the two ideas can also be nested]. You would have a for loop not to MAX_RETRIES but to N_SAMPLES (say, 5 or 10, ideally parallelized), then a processing step.

Yes it will take longer, thus a trade-off with #11 :) but hopefully that parallelization will more than compensate. Cost-wise, it does multiply the number of calls to the LLM over all the comments, thus the total cost, which might or might not be an issue depending on the current typical cost, and if topics are "good enough" right now.

[jucor]

Follow-up: if you're interested in looking how to use similar test-time semantic entropy for summarization, I've written here how that could work: compdemocracy/polis#1881 . It's a bit more complicated, but the core idea is the same.

[dborkan]

Hi and thank you for the suggestions and link to the detailed write up! We're currently considering something similar for the text summary, based on recent DeepMind work. We haven't considered this yet for topic categorization but this could also be an option. As you mention there is a trade-off between the number of LLM calls and latency/cost, so if we pursue this it will likely be configurable by the caller

[jucor]

Thanks for the answer @dborkan :) Makes absolutely sense to have this configurable!