diff --git a/README.md b/README.md
index 8867c394..7c773f21 100644
--- a/README.md
+++ b/README.md
@@ -36,6 +36,15 @@
+
+ If you like Dingo, please give us a ⭐ on GitHub!
+
+
+
+
+
+
+
# Introduction
Dingo is a data quality evaluation tool that helps you automatically detect data quality issues in your datasets. Dingo provides a variety of built-in rules and model evaluation methods, and also supports custom evaluation methods. Dingo supports commonly used text datasets and multimodal datasets, including pre-training datasets, fine-tuning datasets, and evaluation datasets. In addition, Dingo supports multiple usage methods, including local CLI and SDK, making it easy to integrate into various evaluation platforms, such as [OpenCompass](https://github.com/open-compass/opencompass).
@@ -183,6 +192,7 @@ Our evaluation system includes:
- **Classification Metrics**: Topic categorization and content classification
- **Multimodality Assessment Metrics**: Image classification and relevance evaluation
- **Rule-Based Quality Metrics**: Automated quality checks using heuristic rules for effectiveness and similarity detection
+- **Factuality Assessment Metrics**: Two-stage factuality evaluation based on GPT-5 System Card
- etc
Most metrics are backed by academic sources to ensure objectivity and scientific rigor.
@@ -217,6 +227,12 @@ For detailed guidance on using Dingo's hallucination detection capabilities, inc
📖 **[View Hallucination Detection Guide →](docs/hallucination_guide.md)**
+### Factuality Assessment
+
+For comprehensive guidance on using Dingo's two-stage factuality evaluation system:
+
+📖 **[View Factuality Assessment Guide →](docs/factcheck_guide.md)**
+
# Rule Groups
Dingo provides pre-configured rule groups for different types of datasets:
diff --git a/README_ja.md b/README_ja.md
index 96e2b179..29eab120 100644
--- a/README_ja.md
+++ b/README_ja.md
@@ -33,6 +33,14 @@
👋 DiscordとWeChatでご参加ください
+
+ このプロジェクトが役に立ったら、GitHubで⭐を付けてください!
+
+
+
+
+
+
# はじめに
@@ -178,6 +186,7 @@ Dingoはルールベースおよびプロンプトベースの評価メトリク
- **分類メトリクス**: トピック分類とコンテンツ分類
- **マルチモーダル評価メトリクス**: 画像分類と関連性評価
- **ルールベース品質メトリクス**: ヒューリスティックルールによる効果性と類似性検出を用いた自動品質チェック
+- **事実性評価メトリクス**: GPT-5 System Cardに基づく二段階事実性評価
- など
大部分のメトリクスは学術的なソースによって支持されており、客観性と科学的厳密性を保証しています。
@@ -212,6 +221,12 @@ HHEM-2.1-Openローカル推論とLLMベース評価を含む、Dingoの幻覚
📖 **[幻覚検出ガイドを見る →](docs/hallucination_guide.md)**
+### 事実性評価
+
+Dingoの二段階事実性評価システムの使用に関する詳細なガイダンス:
+
+📖 **[事実性評価ガイドを見る →](docs/factcheck_guide.md)**
+
# ルールグループ
Dingoは異なるタイプのデータセット用に事前設定されたルールグループを提供します:
diff --git a/README_zh-CN.md b/README_zh-CN.md
index deaf4cca..484a24ab 100644
--- a/README_zh-CN.md
+++ b/README_zh-CN.md
@@ -29,6 +29,14 @@
👋 加入我们 Discord 和 微信
+
+ 如果觉得有帮助,欢迎在 GitHub 上点个 ⭐ 支持!
+
+
+
+
+
+
@@ -179,6 +187,7 @@ Dingo通过基于规则和基于提示的评估指标提供全面的数据质量
- **分类指标**:主题分类和内容分类
- **多模态评估指标**:图像分类和相关性评估
- **基于规则的质量指标**:使用启发式规则进行效果性和相似性检测的自动化质量检查
+- **事实性评估指标**:基于 GPT-5 System Card 的两阶段事实性评估
- 等等
大部分指标都由学术来源支持,以确保客观性和科学严谨性。
@@ -213,6 +222,12 @@ input_data = {
📖 **[查看幻觉检测指南 →](docs/hallucination_guide.md)**
+### 事实性评估
+
+有关使用Dingo两阶段事实性评估系统的详细指导:
+
+📖 **[查看事实性评估指南 →](docs/factcheck_guide.md)**
+
# 规则组
Dingo为不同类型的数据集提供预配置的规则组:
diff --git a/app_gradio/app.py b/app_gradio/app.py
index 680bde77..6520bb17 100644
--- a/app_gradio/app.py
+++ b/app_gradio/app.py
@@ -48,20 +48,26 @@ def dingo_demo(
try:
input_data = {
- "dataset": dataset_source,
- "data_format": data_format,
"input_path": final_input_path,
"output_path": "" if dataset_source == 'hugging_face' else os.path.dirname(final_input_path),
- "save_data": True,
- "save_raw": True,
-
- "max_workers": max_workers,
- "batch_size": batch_size,
-
- "column_content": column_content,
- "custom_config": {
+ "dataset": {
+ "source": dataset_source,
+ "format": data_format,
+ "field": {
+ "content": column_content
+ }
+ },
+ "executor": {
"rule_list": rule_list,
"prompt_list": prompt_list,
+ "result_save": {
+ "bad": True,
+ "raw": True
+ },
+ "max_workers": max_workers,
+ "batch_size": batch_size,
+ },
+ "evaluator": {
"llm_config": {
scene_list: {
"model": model,
@@ -72,11 +78,11 @@ def dingo_demo(
}
}
if column_id:
- input_data['column_id'] = column_id
+ input_data['dataset']['field']['id'] = column_id
if column_prompt:
- input_data['column_prompt'] = column_prompt
+ input_data['dataset']['field']['prompt'] = column_prompt
if column_image:
- input_data['column_image'] = column_image
+ input_data['dataset']['field']['image'] = column_image
# print(input_data)
# exit(0)
diff --git a/app_gradio/header.html b/app_gradio/header.html
index 78f11d45..b0800b05 100644
--- a/app_gradio/header.html
+++ b/app_gradio/header.html
@@ -67,7 +67,7 @@
color: #fafafa;
opacity: 0.8;
">
- Dingo: A Comprehensive Data Quality Evaluation Tool.
+ Dingo: A Comprehensive AI Data Quality Evaluation Tool.