Skip to content

Commit 3340d64

Browse files
shijinpjlabshijinpjlab
authored
Merge pull request #230 from Kylie-dot-s/dev
feat: Add resume quality detection with complete three-layer architecture
2 parents 22dd0a2 + f0a3555 commit 3340d64

File tree

3 files changed

+694
-0
lines changed

3 files changed

+694
-0
lines changed

‎dingo/model/llm/llm_resume_quality.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
import json
2+
3+
from dingo.model import Model
4+
from dingo.model.llm.base_openai import BaseOpenAI
5+
from dingo.model.modelres import ModelRes
6+
from dingo.model.prompt.prompt_resume_quality import PromptResumeQualityZh
7+
from dingo.model.response.response_class import ResponseScoreTypeNameReason
8+
from dingo.utils import log
9+
from dingo.utils.exception import ConvertJsonError
10+
11+
12+
@Model.llm_register("LLMResumeQuality")
13+
class LLMResumeQuality(BaseOpenAI):
14+
"""LLM-based resume quality evaluation."""
15+
16+
prompt = PromptResumeQualityZh
17+
18+
@classmethod
19+
def process_response(cls, response: str) -> ModelRes:
20+
log.info(response)
21+
22+
# Clean response format
23+
if response.startswith("```json"):
24+
response = response[7:]
25+
if response.startswith("```"):
26+
response = response[3:]
27+
if response.endswith("```"):
28+
response = response[:-3]
29+
30+
try:
31+
response_json = json.loads(response)
32+
except json.JSONDecodeError:
33+
raise ConvertJsonError(f"Convert to JSON format failed: {response}")
34+
35+
# Validate response using Pydantic model
36+
response_model = ResponseScoreTypeNameReason(**response_json)
37+
38+
result = ModelRes()
39+
40+
# Check if resume is good quality
41+
if response_model.type == "Good" and response_model.score == 1:
42+
result.error_status = False
43+
result.type = "QUALITY_GOOD"
44+
result.name = "ResumeQualityGood"
45+
result.reason = [response_model.reason]
46+
else:
47+
# Resume has quality issues
48+
result.error_status = True
49+
50+
# Map issue type to metric type
51+
type_mapping = {
52+
"Privacy": "RESUME_QUALITY_BAD_PRIVACY",
53+
"Contact": "RESUME_QUALITY_BAD_CONTACT",
54+
"Format": "RESUME_QUALITY_BAD_FORMAT",
55+
"Structure": "RESUME_QUALITY_BAD_STRUCTURE",
56+
"Professionalism": "RESUME_QUALITY_BAD_PROFESSIONALISM",
57+
"Date": "RESUME_QUALITY_BAD_DATE",
58+
"Completeness": "RESUME_QUALITY_BAD_COMPLETENESS"
59+
}
60+
61+
result.type = type_mapping.get(response_model.type, "RESUME_QUALITY_BAD")
62+
result.name = response_model.name
63+
result.reason = [response_model.reason]
64+
65+
return result

‎dingo/model/prompt/prompt_resume_quality.py‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
from dingo.model.model import Model
2+
from dingo.model.prompt.base import BasePrompt
3+
4+
5+
@Model.prompt_register("RESUME_QUALITY_ZH", [], ['LLMResumeQuality'])
6+
class PromptResumeQualityZh(BasePrompt):
7+
"""Chinese prompt for resume quality evaluation."""
8+
9+
_metric_info = {
10+
"category": "Resume Quality Assessment Metrics",
11+
"metric_name": "PromptResumeQualityZh",
12+
"description": "Comprehensive resume quality evaluation covering privacy, contact, format, structure, professionalism, date, and completeness issues",
13+
"paper_title": "N/A",
14+
"paper_url": "",
15+
"paper_authors": "Dingo Team",
16+
"evaluation_results": ""
17+
}
18+
19+
content = """
20+
# Role
21+
You are an expert in resume quality evaluation.
22+
23+
# Background
24+
The resume is submitted by job seekers for employment opportunities. Your task is to evaluate the quality of the resume based on professional standards.
25+
26+
# Goals
27+
Your primary objective is to assess the quality of this resume. If the resume meets any of the following negative criteria, it will be considered as having quality issues.
28+
29+
# Criteria
30+
1. Privacy
31+
1.1 Error_ID_Card: The resume contains Chinese ID card numbers (18 digits), which is a serious privacy leak.
32+
1.2 Error_Detailed_Address: The resume contains detailed address information (province, city, district, street, building number), which may leak privacy.
33+
34+
2. Contact
35+
2.1 Error_Email_Missing: The resume does not contain a valid email address.
36+
2.2 Error_Phone_Missing: The resume does not contain a valid phone number.
37+
2.3 Error_Phone_Format_Error: The phone number format is incorrect or invalid.
38+
39+
3. Format
40+
3.1 Error_Excessive_Whitespace: The resume contains excessive consecutive spaces (3 or more spaces).
41+
3.2 Error_Markdown_Syntax_Error: The resume has Markdown syntax errors (e.g., too many # symbols, excessive * or _).
42+
43+
4. Structure
44+
4.1 Error_Name_Missing: The resume does not have a clear name or heading in the first section.
45+
4.2 Error_Section_Missing: The resume is missing required sections such as education or work experience.
46+
4.3 Error_Heading_Level_Error: The resume has inconsistent or incorrect heading hierarchy.
47+
48+
5. Professionalism
49+
5.1 Error_Emoji_Usage: The resume contains emoji characters, which reduces professionalism.
50+
5.2 Error_Informal_Language: The resume uses informal or colloquial expressions (e.g., "搞定", "牛逼", "厉害").
51+
5.3 Error_Typo: The resume contains obvious typos or grammatical errors.
52+
53+
6. Date
54+
6.1 Error_Date_Format_Inconsistent: The resume uses inconsistent date formats (e.g., mixing "2020.01" and "2021-03").
55+
6.2 Error_Date_Logic_Error: The resume has date logic errors (e.g., graduation date earlier than enrollment date, end date earlier than start date).
56+
57+
7. Completeness
58+
7.1 Error_Education_Missing: The resume does not contain education background information.
59+
7.2 Error_Experience_Missing: The resume does not contain work experience or project experience information.
60+
61+
# Workflow
62+
1. Carefully read and understand the provided resume content, evaluate the quality based on the negative criteria above.
63+
2. Assign a type to the resume.
64+
- If the resume does not hit any negative criteria above, type must only be 'Good'.
65+
- Otherwise, type must only be one of the list ['Privacy', 'Contact', 'Format', 'Structure', 'Professionalism', 'Date', 'Completeness'].
66+
3. Assign a name to the resume.
67+
- If type is 'Good', name must only be 'None'.
68+
- If type is 'Privacy', name must only be one of ['Error_ID_Card', 'Error_Detailed_Address'].
69+
- If type is 'Contact', name must only be one of ['Error_Email_Missing', 'Error_Phone_Missing', 'Error_Phone_Format_Error'].
70+
- If type is 'Format', name must only be one of ['Error_Excessive_Whitespace', 'Error_Markdown_Syntax_Error'].
71+
- If type is 'Structure', name must only be one of ['Error_Name_Missing', 'Error_Section_Missing', 'Error_Heading_Level_Error'].
72+
- If type is 'Professionalism', name must only be one of ['Error_Emoji_Usage', 'Error_Informal_Language', 'Error_Typo'].
73+
- If type is 'Date', name must only be one of ['Error_Date_Format_Inconsistent', 'Error_Date_Logic_Error'].
74+
- If type is 'Completeness', name must only be one of ['Error_Education_Missing', 'Error_Experience_Missing'].
75+
4. Assign a score to the resume according to the type. If the type is 'Good', score is 1, otherwise the score is 0.
76+
5. Provide a clear reason for the evaluation.
77+
6. Return the results in JSON format: {"score": 0/1, "type": "", "name": "", "reason": ""}.
78+
79+
# Warning
80+
Please remember to output only a JSON format data, without any additional content.
81+
82+
# Input content
83+
"""
84+
85+
86+
@Model.prompt_register("RESUME_QUALITY_EN", [], ['LLMResumeQuality'])
87+
class PromptResumeQualityEn(BasePrompt):
88+
"""English prompt for resume quality evaluation."""
89+
90+
_metric_info = {
91+
"category": "Resume Quality Assessment Metrics",
92+
"metric_name": "PromptResumeQualityEn",
93+
"description": "Comprehensive resume quality evaluation covering privacy, contact, format, structure, professionalism, date, and completeness issues",
94+
"paper_title": "N/A",
95+
"paper_url": "",
96+
"paper_authors": "Dingo Team",
97+
"evaluation_results": ""
98+
}
99+
100+
content = """
101+
# Role
102+
You are an expert in resume quality evaluation.
103+
104+
# Background
105+
The resume is submitted by job seekers for employment opportunities. Your task is to evaluate the quality of the resume based on professional standards.
106+
107+
# Goals
108+
Your primary objective is to assess the quality of this resume. If the resume meets any of the following negative criteria, it will be considered as having quality issues.
109+
110+
# Criteria
111+
1. Privacy
112+
1.1 Error_ID_Card: The resume contains ID card numbers or social security numbers, which is a serious privacy leak.
113+
1.2 Error_Detailed_Address: The resume contains detailed address information (street, building number, apartment), which may leak privacy.
114+
115+
2. Contact
116+
2.1 Error_Email_Missing: The resume does not contain a valid email address.
117+
2.2 Error_Phone_Missing: The resume does not contain a valid phone number.
118+
2.3 Error_Phone_Format_Error: The phone number format is incorrect or invalid.
119+
120+
3. Format
121+
3.1 Error_Excessive_Whitespace: The resume contains excessive consecutive spaces (3 or more spaces).
122+
3.2 Error_Markdown_Syntax_Error: The resume has Markdown syntax errors (e.g., too many # symbols, excessive * or _).
123+
124+
4. Structure
125+
4.1 Error_Name_Missing: The resume does not have a clear name or heading in the first section.
126+
4.2 Error_Section_Missing: The resume is missing required sections such as education or work experience.
127+
4.3 Error_Heading_Level_Error: The resume has inconsistent or incorrect heading hierarchy.
128+
129+
5. Professionalism
130+
5.1 Error_Emoji_Usage: The resume contains emoji characters, which reduces professionalism.
131+
5.2 Error_Informal_Language: The resume uses informal or colloquial expressions.
132+
5.3 Error_Typo: The resume contains obvious typos or grammatical errors.
133+
134+
6. Date
135+
6.1 Error_Date_Format_Inconsistent: The resume uses inconsistent date formats (e.g., mixing "2020.01" and "2021-03").
136+
6.2 Error_Date_Logic_Error: The resume has date logic errors (e.g., graduation date earlier than enrollment date, end date earlier than start date).
137+
138+
7. Completeness
139+
7.1 Error_Education_Missing: The resume does not contain education background information.
140+
7.2 Error_Experience_Missing: The resume does not contain work experience or project experience information.
141+
142+
# Workflow
143+
1. Carefully read and understand the provided resume content, evaluate the quality based on the negative criteria above.
144+
2. Assign a type to the resume.
145+
- If the resume does not hit any negative criteria above, type must only be 'Good'.
146+
- Otherwise, type must only be one of the list ['Privacy', 'Contact', 'Format', 'Structure', 'Professionalism', 'Date', 'Completeness'].
147+
3. Assign a name to the resume.
148+
- If type is 'Good', name must only be 'None'.
149+
- If type is 'Privacy', name must only be one of ['Error_ID_Card', 'Error_Detailed_Address'].
150+
- If type is 'Contact', name must only be one of ['Error_Email_Missing', 'Error_Phone_Missing', 'Error_Phone_Format_Error'].
151+
- If type is 'Format', name must only be one of ['Error_Excessive_Whitespace', 'Error_Markdown_Syntax_Error'].
152+
- If type is 'Structure', name must only be one of ['Error_Name_Missing', 'Error_Section_Missing', 'Error_Heading_Level_Error'].
153+
- If type is 'Professionalism', name must only be one of ['Error_Emoji_Usage', 'Error_Informal_Language', 'Error_Typo'].
154+
- If type is 'Date', name must only be one of ['Error_Date_Format_Inconsistent', 'Error_Date_Logic_Error'].
155+
- If type is 'Completeness', name must only be one of ['Error_Education_Missing', 'Error_Experience_Missing'].
156+
4. Assign a score to the resume according to the type. If the type is 'Good', score is 1, otherwise the score is 0.
157+
5. Provide a clear reason for the evaluation.
158+
6. Return the results in JSON format: {"score": 0/1, "type": "", "name": "", "reason": ""}.
159+
160+
# Warning
161+
Please remember to output only a JSON format data, without any additional content.
162+
163+
# Input content
164+
"""

0 commit comments

Comments
 (0)