Skip to content

Constraint and Or Graders #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
T-Brick wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Constraint and Or Graders #2

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a837b8a
Constraint and Or Graders
T-Brick Oct 26, 2021
6d3c78f
renaming OrGrader variables
T-Brick Oct 26, 2021
2f8ffa5
description changes
T-Brick Oct 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions src/ConstraintGrader.fun
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
functor ConstraintGrader (structure Constraint : GRADER
Copy link
Member

@HarrisonGrodin HarrisonGrodin Oct 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity - what's the benefit of having structure Constraint : GRADER rather than, say, constraint : bool (or unit -> bool)?

Copy link
Author

@T-Brick T-Brick Oct 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how you would do this with just bool or a unit->bool?

My idea was that it simply runs the Constraint grader and if it passes that then it would run the normal grader (rather than running the normal grader and having a TA manually grade the code). I'm unsure of any other clean way to use the existing infrastructure to run student code.

Copy link
Member

@HarrisonGrodin HarrisonGrodin Oct 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gotcha. I was thinking you could use Result.evaluate, but I guess you'd have to rewrite some boilerplate (run a list of tests, etc.).

structure Standard : GRADER
val threshold : Rational.t
val message : string) :> GRADER =
struct
structure Rubric =
struct
val description = "ConstraintGrader: " ^ Constraint.Rubric.description
^ " and " ^ Standard.Rubric.description
Copy link
Member

@HarrisonGrodin HarrisonGrodin Oct 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels a bit odd, since description is a student-facing. Instinctively, I would imagine val description = Standard.Rubric.description would be reasonable (a la functor Preamble) if constraint : bool, but not sure if that makes sense.

Copy link
Author

@T-Brick T-Brick Oct 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where exactly is this student facing?

Oh when its in the prod grader is one example maybe? It doesn't seem like it shows if you are using the grader directly.

Copy link
Member

@HarrisonGrodin HarrisonGrodin Oct 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's expected that the student can see description. As an edge case, the 150 infrastructure doesn't show the description at the top level (since graders are per-problem anyway so descriptions wouldn't be very useful), but other grader combinators can show it.

In retrospect, the more graders we write, I think this might be a design flaw: I suspect the right thing to do would actually be make val weights in ProdGraderN be (string * int) * (string * int) * ..., where the strings are descriptions. Then, graders wouldn't need descriptions (because they just "exist"), but combining graders would require that you specify a description for each component.


datatype t = Violation
| Result of Standard.Rubric.t

val toString = fn Violation => message
| Result r => Standard.Rubric.toString r

val score = fn Violation => Rational.zero
| Result r => Standard.Rubric.score r
end

val process = fn () =>
let
val score = Constraint.Rubric.score (Constraint.process ())
in
case Rational.compare (threshold, score) of
GREATER => Rubric.Violation
| _ => Rubric.Result (Standard.process ())
end
end
30 changes: 30 additions & 0 deletions src/OrGrader.fun
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
functor OrGrader (structure Grader1 : GRADER
structure Grader2 : GRADER) :> GRADER =
struct
structure Rubric =
struct
val description = "ConstraintGrader: " ^ Constraint.Rubric.description
^ " and " ^ Standard.Rubric.description

datatype t = Result1 of Grader1.Rubric.t
| Result2 of Grader2.Rubric.t

val toString = fn Result1 r => Grader1.Rubric.toString r
| Result2 r => Grader2.Rubric.toString r

val score = fn Result1 r => Grader1.Rubric.score r
| Result2 r => Grader2.Rubric.score r
end

val process = fn () =>
let
val process1 = Grader1.process ()
val score1 = Grader1.Rubric.score process1
val process2 = Grader2.process ()
val score2 = Grader2.Rubric.score process2
in
case Rational.compare (score1, score2) of
LESS => Rubric.Result2 process2
| _ => Rubric.Result1 process1
end
end
6 changes: 6 additions & 0 deletions src/sources.cm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ Library
functor EquivGraderBucketList
functor EquivGraderList

functor ConstraintGrader
functor OrGrader

functor Preamble

structure FormatUtil
Expand Down Expand Up @@ -49,6 +52,9 @@ is
EquivGraderBucket.fun
EquivGrader.fun

ConstraintGrader.fun
OrGrader.fun

Preamble.fun

FormatUtil.sml