Skip to content

Commit ce53b14

Browse files
PCain02PCain02
authored
Merge pull request #3 from GatorEducator/execexam
Add content for ExecExam slides
2 parents 9f491db + cbb5f0b commit ce53b14

File tree

4 files changed

+250
-0
lines changed

4 files changed

+250
-0
lines changed

‎conference-presentations/slides/execexam/ExecExam_-_Logo_-_300.png‎

400 KB
Loading

‎conference-presentations/slides/execexam/index.qmd‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
---
2+
title: "ExecExam: A Tool to Facilitate Effective Executable Examinations in Python"
3+
date: "2025-05-15"
4+
date-format: long
5+
author: Pallas-Athena Cain, Hemani Alaparthi, and Gregory Kapfhammer
6+
format:
7+
revealjs:
8+
theme: default
9+
slide-number: false
10+
incremental: false
11+
code-fold: true
12+
code-tools: true
13+
code-link: true
14+
history: false
15+
scrollable: true
16+
transition: slide
17+
highlight-style: github
18+
footer: "PyCon Education Summit 2025"
19+
css: styles.css
20+
---
21+
22+
## What is an Execuatable Examination?
23+
24+
::: {.fragment style="margin-top: -0.15em; font-size: 0.8em;"}
25+
26+
**Assessing real programming ability**
27+
28+
- A student writes, modifies, and runs code to solve a real problem
29+
- Graded via automated tests (e.g. Pytest)
30+
- Unlike static exams, this tests:
31+
- Programming logic
32+
- Debugging ability
33+
- Tool use (editor, terminal, Git)
34+
35+
🎯 Like a take-home project — but precise, consistent, and scalable
36+
37+
:::
38+
39+
## The Problem
40+
41+
**Why do we need better assessments?**
42+
43+
- Manual grading is slow and inconsistent
44+
- Students often don’t know why their code fails
45+
- Feedback is shallow or missing altogether
46+
47+
🚫 “Test failed” isn’t enough.
48+
49+
## What is ExecExam?
50+
51+
**Scalable, feedback-rich assessment tool**
52+
53+
- Runs Pytest on student code
54+
- Unlike Pytest, continues past the first failure
55+
- Explains why a test failed
56+
- Suggests how to fix it
57+
- Utilizes LLMs for even better feedback
58+
59+
![](ExecExam_-_Logo_-_300.png)
60+
61+
##
62+
63+
![](terminal.png)
64+
65+
## Key Features
66+
67+
**Why use ExecExam?**
68+
69+
- 🧪 Full Pytest coverage, Streamlines assessment
70+
- 💻 Can be run throughout the student coding process
71+
- 🧠 AI-powered failure analysis
72+
- ⚙️ GitHub integration, easy CI/CD
73+
- 🔁 Enhance the learning experience by offering actionable insights throughout the coding process
74+
75+
## For Instructors
76+
77+
**How to adopt it**
78+
79+
- Design scaffolded coding tasks
80+
- Write tests
81+
- Deploy with GitHub Actions or CLI
82+
- Grade fairly, at scale
83+
84+
🧰 Lightweight setup—just Python, Git, and your test cases
85+
86+
## Future Work
87+
88+
::: {.fragment style="margin-top: -0.15em; font-size: 0.7em;"}
89+
90+
**What’s next for ExecExam?**
91+
92+
- 📊 Analytics & Instructor Features
93+
- Store test outcomes and feedback over time
94+
- Visualize student debugging and improvement paths
95+
- Log LLM interactions to evaluate effectiveness
96+
- Hold out hidden test cases for instructor-only grading
97+
98+
- 🧠 Adaptive Feedback Loops
99+
- Tailor feedback complexity to student performance
100+
- Allow students to rate LLM feedback
101+
102+
:::
103+
104+
## Key Takeaways
105+
106+
::: {.fragment .fade .tight-boxed-content style="margin-top: -0.15em; font-size: 0.7em;"}
107+
108+
**Better exam grading, better learning**
109+
110+
- Helps students debug and learn
111+
- Saves teachers time
112+
- More fair + consistent grading
113+
- Scales to large classes
114+
115+
:::
116+
117+
::: {.fragment .fade .tight-boxed-content style="margin-top: -0.15em; font-size: 0.6em;"}
118+
119+
**🚀 Try ExecExam: Let’s build smarter CS assessments**
120+
121+
- 🔗 GitHub Repository: [https://github.com/GatorEducator/execexam](https://github.com/GatorEducator/execexam)
122+
- 💻 PyPI: [https://pypi.org/project/execexam/](https://pypi.org/project/execexam/)
123+
- 🤝 Reach out or contribute
124+
- Pallas-Athena Cain: **cain01@allegheny.edu**
125+
- Gregory M. Kapfhammer: **gkapfhammer@allegheny.edu**
126+
127+
🦚 Consider a birds of a feather session about automated grading
128+
129+
:::
130+
131+
## References
132+
133+
::: {.fragment .tight-boxed-content style="margin-top: -0.15em; font-size: 0.5em;"}
134+
135+
- João Paulo Barros, Luís Estevens, Rui Dias, Rui Pais, and Elisabete Soeiro. 2003. Using lab exams to ensure programming practice in an introductory programming course. SIGCSE Bull. 35, 3 (September 2003), 16–20. https://doi.org/10.1145/961290.961519
136+
- Jonathan Corley, Ana Stanescu, Lewis Baumstark, and Michael C. Orsega. 2020. Paper Or IDE? The Impact of Exam Format on Student Performance in a CS1 Course. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE '20). Association for Computing Machinery, New York, NY, USA, 706–712. https://doi.org/10.1145/3328778.3366857
137+
- Scott Grissom, Laurie Murphy, Renée McCauley, and Sue Fitzgerald. 2016. Paper vs. Computer-based Exams: A Study of Errors in Recursive Binary Tree Algorithms. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE '16). Association for Computing Machinery, New York, NY, USA, 6–11. https://doi.org/10.1145/2839509.2844587
138+
- Sevkli, Z. 2024. Assessing the Impact of Open-Resource Access on Student Performance in Computer-Based Examinations. In Proceedings of the 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://doi.org/10.18260/1-2--46619
139+
- Vesa Lappalainen, Antti-Jussi Lakanen, and Harri Högmander. 2016. Paper-based vs computer-based exams in CS1. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research (Koli Calling '16). Association for Computing Machinery, New York, NY, USA, 172–173. https://doi.org/10.1145/2999541.2999565
140+
141+
:::

‎conference-presentations/slides/execexam/styles.css‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
/* Custom styles for the conference presentations website */
2+
3+
/* General styles */
4+
body {
5+
font-family: 'Inter', sans-serif;
6+
line-height: 1.6;
7+
background-color: #fff8f0;
8+
color: #333;
9+
}
10+
11+
/* Code blocks */
12+
pre {
13+
border-radius: 8px;
14+
padding: 1em;
15+
background-color: #ffe6cc;
16+
color: #333;
17+
border: 1px solid #ff9f5c;
18+
}
19+
20+
code {
21+
font-family: 'Ubuntu Mono', monospace;
22+
color: #d35400;
23+
}
24+
25+
/* Navigation */
26+
.navbar {
27+
background-color: #ff9f5c;
28+
color: #fff;
29+
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
30+
}
31+
32+
.navbar a {
33+
color: #fff;
34+
text-decoration: none;
35+
}
36+
37+
.navbar a:hover {
38+
text-decoration: underline;
39+
}
40+
41+
/* Links */
42+
a {
43+
color: #ff6f3c;
44+
text-decoration: none;
45+
}
46+
47+
a:hover {
48+
text-decoration: underline;
49+
}
50+
51+
/* Footer */
52+
.footer {
53+
border-top: 1px solid #ff9f5c;
54+
padding: 1em 0;
55+
background-color: #fff8f0;
56+
color: #333;
57+
}
58+
59+
/* Presentation specific styles */
60+
.reveal .slides {
61+
text-align: left;
62+
}
63+
64+
.reveal h1, .reveal h2, .reveal h3 {
65+
color: #ff6f3c;
66+
}
67+
68+
/* Presentation specific styles */
69+
.reveal h1 {
70+
font-size: 2em;
71+
color: #ff6f3c;
72+
}
73+
74+
.reveal h2 {
75+
font-size: 1.5em;
76+
color: #ff6f3c;
77+
}
78+
79+
.reveal h3 {
80+
font-size: 1.25em;
81+
color: #ff6f3c;
82+
}
83+
84+
.reveal .slide-number {
85+
font-size: 0.8em;
86+
color: #ff9f5c;
87+
}
88+
89+
.reveal .quarto-title-block .quarto-title-authors .quarto-title-author .quarto-title-author-name {
90+
font-size: 0.8em;
91+
color: #333;
92+
}
93+
94+
/* Custom callout boxes */
95+
.callout {
96+
border-left: 4px solid #ff6f3c;
97+
padding: 1em;
98+
margin: 1em 0;
99+
background-color: #fff8f0;
100+
border-radius: 4px;
101+
color: #333;
102+
}
103+
104+
/* Code highlighting */
105+
.hljs {
106+
background: #ffe6cc;
107+
border-radius: 4px;
108+
color: #333;
109+
}

‎conference-presentations/slides/execexam/terminal.png‎

242 KB
Loading

0 commit comments

Comments
 (0)