Skip to content

Commit 46f9c1a

Browse files
committed
Fix validation system - make it more permissive
- Disable strict semantic validation that was failing on: - Inference questions (smiled → happy) - Vocabulary questions (extinguish → put out fire) - Main idea questions (recycling → environmental protection) - Non-Latin scripts (Chinese, Arabic, etc.) - Keep basic structural validation (length, format) - Add warning logs instead of failing validation - Rely on user feedback system for quality control The previous validation was too literal and language-biased, rejecting valid questions that required inference or used synonyms.
1 parent 16e34f2 commit 46f9c1a

File tree

1 file changed

+19
-21
lines changed

1 file changed

+19
-21
lines changed

app/lib/ai/question-validator.ts

Lines changed: 19 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -177,37 +177,35 @@ export const validateQuestionQuality = (
177177
return { isValid: false, reason: 'One or more options are too short.', metrics };
178178
}
179179

180-
// Enhanced validation checks
180+
// Enhanced validation checks (made more permissive due to language/type limitations)
181+
// Only fail on the most basic structural issues
182+
181183
if (!metrics.answerConsistency) {
182-
return {
183-
isValid: false,
184-
reason: 'Correct answer is not supported by the relevant text.',
185-
metrics,
186-
};
184+
console.warn(
185+
'[Validation] Answer consistency check failed, but allowing due to language/type limitations'
186+
);
187+
// Don't fail - just log the warning
187188
}
188189

189190
if (!metrics.explanationConsistency) {
190-
return {
191-
isValid: false,
192-
reason: 'Correct explanation does not match the correct answer.',
193-
metrics,
194-
};
191+
console.warn(
192+
'[Validation] Explanation consistency check failed, but allowing due to language/type limitations'
193+
);
194+
// Don't fail - just log the warning
195195
}
196196

197197
if (!metrics.questionAnswerCoherence) {
198-
return {
199-
isValid: false,
200-
reason: 'Question cannot be answered from the provided passage.',
201-
metrics,
202-
};
198+
console.warn(
199+
'[Validation] Question-answer coherence check failed, but allowing due to language/type limitations'
200+
);
201+
// Don't fail - just log the warning
203202
}
204203

205204
if (!metrics.semanticAnswerValidation) {
206-
return {
207-
isValid: false,
208-
reason: 'Correct answer is not semantically supported by the passage.',
209-
metrics,
210-
};
205+
console.warn(
206+
'[Validation] Semantic answer validation failed, but allowing due to language/type limitations'
207+
);
208+
// Don't fail - just log the warning
211209
}
212210

213211
// Add more sophisticated checks based on CEFR level if needed

0 commit comments

Comments
 (0)