Skip to content

Commit 6b569a2

Browse files
jon-belljon-bell
committed
GPT5 docs are better than no docs (?)
1 parent c1e2f68 commit 6b569a2

File tree

2 files changed

+309
-5
lines changed

2 files changed

+309
-5
lines changed

‎docs/staff/assignments/rubrics.md‎

Lines changed: 308 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,315 @@
11
---
22
title: Creating and Managing Rubrics
33
---
4+
### Rubrics: Instructor Guide
45

5-
## Overview
6+
This guide explains how rubrics drive the hand‑grading experience and how scores are computed. It also includes copy‑pasteable YAML examples for each configuration option.
67

7-
## Creating and Managing Rubrics
8+
## High‑level overview
9+
- **Rubric**: Named collection of parts used for a specific review round (self‑review, grading‑review, meta‑grading‑review). The grading review is the main review that students will see.
10+
- **Part**: Logical section within a rubric (e.g., Code Quality, Functionality). Contains criteria.
11+
- **Criteria**: A scoring rule block with:
12+
- `is_additive`: If true, checks add points up to `total_points`. If false, checks deduct points from `total_points`.
13+
- `total_points`: Max points for this criteria. It is not possible for a grader to assign more points to nested checks, even if you allow them to select that many points.
14+
- `min_checks_per_submission` / `max_checks_per_submission`: How many checks must/can be applied for the criteria.
15+
- One or more checks.
16+
- **Check**: The atom of feedback and scoring. A check can be global or an annotation applied to a file line or to an artifact. Checks can have:
17+
- `points` or selectable `data.options` with distinct labels/points
18+
- `is_annotation` + `annotation_target` (`file` or `artifact`), optional `file`/`artifact` to link
19+
- `is_required`: Must be applied by graders
20+
- `is_comment_required`: Comment is required when applying
21+
- `max_annotations`: Cap on times a check may be applied as annotations
22+
- `student_visibility`: `always` | `if_applied` | `if_released` | `never`
823

9-
## Using Rubrics
24+
Where this shows up in the hand‑grader:
25+
- The right‑hand panel is the rubric sidebar. Global checks render as radio/checkbox options. Annotation checks are applied from the code view or artifact cards by right‑clicking a line or selecting an artifact.
26+
- Each applied check creates a comment entry tied to a check with points and optional text.
27+
- For each criteria, the sidebar shows a running subtotal.
1028

11-
## Example: A Simple Rubric
29+
How points are computed:
30+
- Per criteria:
31+
- If `is_additive = true`: criteria score = sum(applied check points) up to `total_points`.
32+
- If `is_additive = false`: criteria score = `total_points - sum(applied deduction points)` (floored at 0).
33+
- The rubric total is the sum across criteria.
34+
35+
Note on options vs base points:
36+
- If a check has `data.options`, the selected option’s `points` replaces the check’s base `points` when applied.
37+
- Options must be 2+ (single options are disallowed by the editor and schema).
38+
39+
References across rubrics:
40+
- Checks can reference checks from other rubrics to surface related feedback when grading. This is managed inline in the sidebar (preview mode).
41+
42+
## YAML schema quick reference
43+
These examples conform to `public/RubricSchema.json`.
44+
45+
### Minimal rubric
46+
```yaml
47+
name: Sample Rubric
48+
parts:
49+
- name: Code Quality
50+
criteria:
51+
- name: Style and Clarity
52+
total_points: 10
53+
is_additive: false
54+
checks:
55+
- name: Missing Javadocs
56+
is_annotation: false
57+
is_required: false
58+
is_comment_required: false
59+
points: 2
60+
- name: Poor variable naming
61+
is_annotation: true
62+
annotation_target: file
63+
is_required: false
64+
is_comment_required: true
65+
max_annotations: 3
66+
points: 1
67+
```
68+
69+
### Additive vs subtractive criteria
70+
```yaml
71+
name: Additive vs Subtractive
72+
parts:
73+
- name: Functionality
74+
criteria:
75+
- name: Passing tests (additive)
76+
total_points: 20
77+
is_additive: true
78+
checks:
79+
- name: Public API works
80+
is_annotation: false
81+
is_required: false
82+
is_comment_required: false
83+
points: 5
84+
- name: Edge cases handled
85+
is_annotation: false
86+
is_required: false
87+
is_comment_required: false
88+
points: 5
89+
- name: Style deductions (subtractive)
90+
total_points: 10
91+
is_additive: false
92+
checks:
93+
- name: Magic numbers
94+
is_annotation: true
95+
annotation_target: file
96+
is_required: false
97+
is_comment_required: false
98+
points: 1
99+
- name: Redundant code
100+
is_annotation: true
101+
annotation_target: file
102+
is_required: false
103+
is_comment_required: true
104+
points: 2
105+
```
106+
107+
### Check with options (multiple choice)
108+
```yaml
109+
name: Options Example
110+
parts:
111+
- name: API Correctness
112+
criteria:
113+
- name: HTTP response quality
114+
total_points: 10
115+
is_additive: true
116+
checks:
117+
- name: Response completeness
118+
is_annotation: false
119+
is_required: true
120+
is_comment_required: false
121+
points: 0
122+
data:
123+
options:
124+
- label: Complete and correct
125+
points: 5
126+
- label: Mostly complete
127+
points: 3
128+
- label: Incomplete
129+
points: 1
130+
```
131+
132+
### Annotations (file vs artifact) and limits
133+
```yaml
134+
name: Annotation Targets
135+
parts:
136+
- name: Docs and Reports
137+
criteria:
138+
- name: README quality (deductions)
139+
total_points: 5
140+
is_additive: false
141+
checks:
142+
- name: Missing section
143+
is_annotation: true
144+
annotation_target: artifact
145+
artifact: README.md
146+
is_comment_required: true
147+
is_required: false
148+
max_annotations: 2
149+
points: 1
150+
- name: Code comments
151+
total_points: 5
152+
is_additive: false
153+
checks:
154+
- name: Unclear comment
155+
is_annotation: true
156+
annotation_target: file
157+
file: src/Main.java
158+
points: 1
159+
```
160+
161+
### Required checks and required comments
162+
```yaml
163+
name: Requireds
164+
parts:
165+
- name: Process
166+
criteria:
167+
- name: Submission hygiene
168+
total_points: 5
169+
is_additive: true
170+
checks:
171+
- name: Compiles
172+
is_annotation: false
173+
is_required: true # graders must apply
174+
is_comment_required: false
175+
points: 5
176+
- name: Explain deviation
177+
is_annotation: false
178+
is_required: false
179+
is_comment_required: true # applying this check requires a comment
180+
points: 0
181+
```
182+
183+
### Min/Max checks per criteria
184+
```yaml
185+
name: Min/Max Checks
186+
parts:
187+
- name: Code Review
188+
criteria:
189+
- name: Choose exactly one pattern
190+
total_points: 5
191+
is_additive: true
192+
min_checks_per_submission: 1
193+
max_checks_per_submission: 1
194+
checks:
195+
- name: Builder pattern used
196+
is_annotation: false
197+
points: 5
198+
- name: Strategy pattern used
199+
is_annotation: false
200+
points: 5
201+
- name: Choose up to two strengths
202+
total_points: 4
203+
is_additive: true
204+
max_checks_per_submission: 2
205+
checks:
206+
- name: Test readability
207+
is_annotation: false
208+
points: 2
209+
- name: Modular design
210+
is_annotation: false
211+
points: 2
212+
```
213+
214+
### Student visibility
215+
```yaml
216+
name: Visibility
217+
parts:
218+
- name: Feedback
219+
criteria:
220+
- name: Public notes
221+
total_points: 0
222+
is_additive: true
223+
checks:
224+
- name: General praise
225+
is_annotation: false
226+
points: 0
227+
student_visibility: always
228+
- name: Internal notes
229+
total_points: 0
230+
is_additive: true
231+
checks:
232+
- name: For staff only
233+
is_annotation: false
234+
points: 0
235+
student_visibility: never
236+
- name: Released only
237+
total_points: 0
238+
is_additive: true
239+
checks:
240+
- name: Visible when released
241+
is_annotation: false
242+
points: 0
243+
student_visibility: if_released
244+
- name: Only if applied
245+
total_points: 0
246+
is_additive: true
247+
checks:
248+
- name: Shown when applied
249+
is_annotation: false
250+
points: 0
251+
student_visibility: if_applied
252+
```
253+
254+
## How this renders in the hand‑grader
255+
- Global checks appear as radio buttons (when `max_checks_per_submission = 1`) or checkboxes (otherwise).
256+
- Checks with `data.options` render a choice list. The selected option’s label is prefixed to the comment.
257+
- Annotation checks are applied via right‑click on a code line or by selecting an artifact.
258+
- For each criteria, the sidebar shows:
259+
- Additive: `earned / total_points`
260+
- Subtractive: `remaining / total_points`
261+
262+
## Notes and tips
263+
- Keep criteria focused and keep check names short; longer explanation should go in `description`.
264+
- Use `max_annotations` to prevent over‑counting nitpicks.
265+
- Prefer options when the same check has graded tiers (e.g., Complete/Partial/Incomplete).
266+
- Use `student_visibility` to separate internal notes from student‑facing feedback.
267+
268+
## Field reference: required vs optional (with defaults)
269+
270+
- Rubric
271+
- Mandatory: `name`, `parts`
272+
- Optional: `description`
273+
274+
- Part
275+
- Mandatory: `name`, `criteria`
276+
- Optional: `id`, `description`, `data`
277+
278+
- Criteria
279+
- Mandatory: `name`, `checks`
280+
- Optional: `id`, `description`, `data`, `is_additive` (default: `false`), `total_points` (default: `0`), `min_checks_per_submission`, `max_checks_per_submission`
281+
282+
- Check
283+
- Mandatory: `name`, `is_annotation` (boolean), `is_required` (boolean), `is_comment_required` (boolean), `points` (number; use `0` when the check relies on `data.options`)
284+
- Optional: `id`, `description`, `file`, `artifact`, `annotation_target` (`file` | `artifact`; default behavior in UI is `file` when omitted for annotations), `max_annotations`, `data` (see below), `student_visibility` (default: `always`)
285+
286+
- Check `data.options` (for multiple-choice checks)
287+
- Mandatory per option: `label`, `points`
288+
- Optional per option: `description`
289+
- Notes: Must define at least two options (single-option checks are rejected by the editor). When options are present, the selected option’s `points` replace the base `points` for that check when applied.
290+
291+
Defaults and behaviors used by the grader UI
292+
- Criteria without `is_additive` are treated as subtractive (deduction) criteria.
293+
- Criteria without `total_points` default to `0` (i.e., no contribution unless checks add points in additive mode).
294+
- Check `student_visibility` defaults to `always`.
295+
- Annotation `annotation_target` defaults to `file` in the UI if omitted.
296+
- `min_checks_per_submission` and `max_checks_per_submission` are optional; when not set, graders are not constrained by count.
297+
298+
## Check references (cross‑rubric context)
299+
300+
What it is
301+
- A check on one rubric can reference a check on another rubric. During grading, any applied feedback (comments/points) from the referenced check(s) is shown inline under the current check as “Related Feedback from Other Reviews”. This gives graders context from, e.g., self‑review or prior review rounds.
302+
303+
Scoring and visibility
304+
- References are informational only. They do not contribute points to the current rubric’s criteria and do not alter the score computation.
305+
- Referenced feedback is surfaced to graders in grading mode. Student visibility continues to follow each original check’s own `student_visibility` and release state within its source review.
306+
307+
How to configure
308+
- Navigate to the rubric preview/editor for an assignment.
309+
- For the check you want to augment, click “Add Reference”.
310+
- Search/select a check from other rubrics (the current rubric’s checks are excluded).
311+
- Save. The relationship is stored so that, when grading, the referenced feedback appears under the referencing check.
312+
313+
Good use cases
314+
- Show a student’s self‑review evidence next to the corresponding grading check.
315+
- Pull in meta‑grading notes when doing final pass reviews.

‎src/pages/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Markdown page example
2+
title: Pawtograder Documentation Overview
33
---
44

55
# Pawtograder Documentation

0 commit comments

Comments
 (0)