Nbgrader development in 2024: can we do more?

[rkdarst]

Hi everyone,

[This is a survey to see about further nbgrader development: who is interested? Shall we try to do more? Even if you aren't interested, can you say three cool things you have done with nbgrader lately and three recent problems?]

We (Aalto University CS department) have been thinking about trying to push forward with more nbgrader development, at least a little bit. Clearly, there is some development in people's different branches (we've got some), but upstream dev is a bit slow. We understand that and are wondering what we can do about it. Is anyone interested in taking risks and trying to push things further?

We think one problem is that few of us are long-time developers in nbgrader and don't want to risk breaking things for others, or know what might break, or take responsibility if it does, and things mostly work anyway. Thus, development is very slow. What about accepting that things might get unstable and merge pull requests with less threshold? We could declare [0.9?] the stable version recommended for non-experts, and then make the main branch more unstable, where we work rapidly and learn as we go. By working faster, we learn more, build more community, and hopefully can fix things faster and eventually get to where we can .

(the null option: continue as it is, basically, and development is slow)

Does anyone have stats on global nbgrader usage? Should we make a survey to see what's going on in the world?

We (Aalto University CS department) have people hired who could work on this. We can't promise things forever in the future, but have at least one person who can watch over, test things, and try to merge things. Of course, it needs a big community and talking to be able to manage this.

What do you all think?

Could anyone interested in Nbgrader reply with your interests?

• Who would want to contribute improvements?
• What are three cool things you have done with nbgrader lately?
• What are three problems you have had lately?
• Would you want to have an online meet-up?

This is just one of our ideas, if others have an idea for faster development, we can help with that, too.

I propose a starting point for those interested: can you tell three cool things they have done with nbgrader lately, and three problems?

Richard on behalf of my team

cc people with recent nbgrader development: @brichet @danielmaitre @shreve @jeflem @ykazakov @lahwaacz @UBC-DSCI @perllaghu @trevorcampbell @tuncbkose @tmetzl
Or who replied to the last maintainer team issue #1637: @BertR @nthiery @willingc @damianavila @choldgraf @ocordes @jgwerner @jhamrick @SylvainCorlay @jtpio

[jeflem]

Hi,

I strongly support pushing forward development. Although I'm not a software developer I'll try to put my two cents in.

What are three cool things you have done with nbgrader lately?

• We (two colleagues and me) wrote some LTI related code to connect Nbgrader (and JupyterHub) to our learning management system and assembled the Ananke Jupyter Distribution to make things more accessible to academic teaching. We got some state funding for this and (hopefully) get some more funding soon. In other words, we have a developer at our university for doing Jupyter/Nbgrader related things. With our LTI code courses are created automatically, students get enroled automatically to the correct course, and grades are sent back to the LMS. Maybe we could move some of the code into Nbgrader's code in future.
• I've got an Nbgrader notebook with advanced feedback demos. Often I hear people saying that tasks written for Nbgrader aren't complex enough (at my university we have a quite strong commercial e-assessment tool nbgrader has to compete with). Some more complex demo tasks should make their way to Nbgrader docs to demonstrate Nbgrader's power.
• Colleagues and me try to use Nbgrader for math teaching. That's something that requires some more attention/doc. Maybe one could develop a Python module with functions for evaluating/comparing mathematical expressions in future.

What are three problems you have had lately?

• Docs are a bit confusing and incomplete.
• Combining Nbgrader and JupyterHub in a stable and usable way is quite complex.
• Course ids have to be URL safe (cf. PR 1797).

Would you want to have an online meet-up?
Don't know. Writen messages are more persistent...

Jens

[nthiery]

Hi!

I am on vacations, so quick answer for now.

Thank you so much @rkdarst for initiating the discussion. By large, I am in line with all proposed items.

Four cool things happening here these days:
- integration gitlab - nbgrader, making gitlab play the role of an exchange for nbgrader (though not as a plugin)
with Travo
- exploring interoperability with CorrectExam for a rich efficient manual grading environment
- contributing (through @brichet) a way to make the formgrader use the nbgrader configuration from a given directory. Thereby simplifying, e.g., "multiple courses on a single jupyterhub".
- jupylates: prototype of jupyter-based tool for repeated exercising. More on this when it will reach some more maturity.

Weak points of nbgrader that are itching us these days (not new though)
- modularity (and interoperability)
- robust auto grading
- manual grading interface
- development speed

Man power:
- We have hired one person here (@cmarmo) for >= 1.5 year that helps us with the development of jupyter-related tools for teaching. She can devote a bit of time to nbgrader development. Main barrier to contribution: getting familiar with the codebase, especially on the UI side.
- Currently applying for funds; e.g. for paying further contributions by QuantStack like @brichet's recent ones; but don't hold your breath.

Suggestion of action:

• online tutorial "how to do development on the nbgrader UI"

@jeflem: looking forward learning more about jupyter-nbgrader-lti integration

Cheers,
Nicolas

[perllaghu]

Yes, we are still active users of NBGrader.

Sadly @BertR moved back to mainland Europe, and is no longer part of our team - hence the lack of input from him of late.

We are also suffering a shortage of developer time to help improve the code [which I'm vocal about at work], however we are definitely "would if we could" camp.

One if the big challenges here is ensuring that what we add to the core product is generally useful rather that I need - a case in point would be LTI integration [I know about this one, as I've been playing with it in other arenas]: I could write an integration that connects Moodle to named assignments - which may well solve my the use-case, however misses a few issues:

• Canvas, Blackboard, and Brightspace all identify users in different ways, and
• not all services connect direct to notebooks/jupyterhub

The grade passback is definitely something we [in Noteable] are going to be looking at, and I will definitely be encouraging something that does not send us down a "fork of nbgrader" route!!

[and I should really try to make more time to spend attention here.... helping & supporting, if nothing else - that's on me]

[perllaghu]

... and some things I've been wanting to improve, which I thing are generally useful:

1. Change the value of the #submissions column to read the number of <submissions in the exchange> / <submissions collected> (if you have a class of 120 students, and only 80 have submitted, you may not want to collect)
2. A student may submit multiple times, and their assignment page may show each one, and students fetch feedback into timestamped folders.... however the link to "view feedback" is always the last feedback - which results in incorrect mapping.

I can already see challenges here, when systems use different exchange plugins.

[shreve]

Hi All 👋🏻

Thanks for opening this discussion. I've been wanting to contribute more, but haven't felt aligned with the greater direction of the project.

At the @umsi-mads program, we use nbgrader, but not the entire thing. We have a partnership with Coursera and use nbgrader in a cloud context where the exchange just doesn't make sense. I think this cloud use case is worth considering because it's used in-house at Coursera and Vocareum and probably many others, but that might be worthy of an entire fork. I can expand on this if anyone's interested.

My cool thing is that to work in this cloud context, I built my own grader that runs as a lambda (still private, sorry) that orchestrates NbGraderAPI under the hood.

No recent problems, but in the past couple weeks, I've gotten 2 different requests for help in accessing the contents of a Markdown cell from an autograder cell and the solution is really clunky, so there might be something there.

I'm more than happy to contribute upstream on nbgrader or anything down through ipython, but I don't have much in the way of direction at the moment.

[perllaghu]

Can I confirm that you've essentially written an external exchange service [the instructors release to some external store, and students pull from the external store] - like ngshare (other tools are available)

I think it would be interesting to collect & collate these examples

[we use nbexchange .... there must be more out there - there's plenty people use nbgrader at scale]

[lahwaacz]

Hi,

thank you for starting this discussion, I'm pretty new to nbgrader but I would like to get involved and help with future development.

Instead of the three cool things we have done with nbgrader lately, I will describe our use case how we started with nbgrader at our university. So this will be nothing new for nbgrader, but it is cool for us 😉
We started in June 2023, which is the time when Jupyter was migrating to Notebook 7 and Lab 4. Our JupyterHub instance is running on Arch Linux, which already had the latest versions, so integrating nbgrader before the start of the semester in September was very challenging as each issue might be a problem either in our configuration or due to nbgrader incompatibility with Notebook 7 or Lab 4. Eventually, we successfully deployed nbgrader for a Basics of Programming course – an introductory course for first-year students, this semester first taught in Python. We had about 350 students, 7 lecturers and graded 11 assignments (about 2500 submissions) during the semester. Without nbgrader, this would not be possible to deal with, so big thanks for this open source project! 🎉

The major problems we faced during the semester are:

• Duplicating cells results in autograding failure because of duplicate IDs, and confusing error message #1083
• Various issues that break autograding - Student's solution with input() passes "Validate", but fails on "Autograde" #1844, Formgrader says There was an error generating feedback of 'some-assignment'. with no details #1845, Validate button shows incomprehensible SyntaxError when the solution trips the timeout limit #1846
• Various issues that break other parts of the workflow (e.g. feedback generation) - I still need to analyze the cases properly and submit issues, that is on me...
• Ability for students to see assignment duedate #943
• The nbgrader/formgrader UI is not intuitive and has serious usability glitches.

In 2024, I would like to get involved in nbgrader development and work primarily on fixing and testing these issues. In the end, I should even get some funding from my university... I think that I have sufficient skills for hacking on a Python/JavaScript project, but as others mentioned, it is difficult to get started so I would like to participate in any meetup or hacking session.

Finally, I think this is a good place to mention a few things that I think should be addressed to improve the maintainability of nbgrader in the future:

• New issues are not labeled and there are only obsolete milestones. It would be nice to organize the issues, assign priority and provide a clear roadmap of planned features and enhancements (at least for 2024).
• In general, pull requests don't have any assignees or reviewers assigned and suffer from the same labeling problem (though on much smaller scope).
• I think that the main reason behind slow development is that nobody wants to (or has the time to) test new things during their academic terms, which leads to long delays before review and then before reaction to review. This is unavoidable given that different countries have different breaks during the year and even different number of terms. I don't know what is the best solution, but I would definitely appreciate faster development cycles at least for bug fixing.
• As a newcomer, can I ask about the people and institutions that oversee the development of nbgrader? E.g. who can provide review and accept pull requests, who can decide the direction of larger changes, etc.? Even among the people cc'd in this discussion I'm not sure about their role in the project.

[rkdarst]

Answering my own questions: We run nbgrader as a service for computer science (more like the "data science" part), and also for others in our university. It's running in Kubernetes but uses nbgrader also, and is well-integrated to the rest of our university IT systems, which gives us many nice features but also more complexity. We support courses with a significant fraction of grades in the department. Not every course (perhaps not even many) use autograding because of the difficulties in doing it well.

Three cool things:

• We run nbgrader with in kubernetes with a NFS exchange dir - letting us scale well. NFS also gives us proper home directories with persistant data for every user, accessible across all nodes. (NFS is managed by someone else)
• We have a secure autograding server running the kernels in Singularity/Apptainer. Right now this has to run by us, but we have someone who has a prototype of way to do it in Kubernetes without admin help.
• Course management: profile list in Kubespawner, user IDs (managed from the university system and aligning with it, so that the data is also available on other university systems).

Three problems:

• There are plenty of UI improvements.
• Exporting grades, managing identities, and connecting to grading platforms (export works OK and we have a format that can be imported to our course platform, but could be a bit better integrated and have less manual work. Our system has some internal problems, for example users can submit assignments for courses they aren't enrolled in, which breaks the grade import)
• Some teachers have reported that the manual grading UI not convenient

Many of these are specific to our setup. Overall, teachers really appreciate nbgrader but also understand its limits.

[perllaghu]

We have a secure autograding server running the kernels in ...

This is an interesting concept, which I remember floating around before: external autograding services.

[currently, we do it by keeping the autograding process in the instructors notebook-server, but have the files in a separate file-store (to bypass quota issues... and, oh boy... did we need to enforce quota issues: it doesn't take long for an enthusiastic power-user to squirrel away gigabytes of data & sample notebook-documents!]

[rkdarst]

What we will do for now is: update our own repo's main branch (https://github.com/AaltoSciComp/nbgrader/) with pull requests from jupyter/nbgrader. The goal is to see what fits together and make something that can be used to test all the latest changes together, including by others. Installation would probably be: pip install https://github.com/AaltoSciComp/nbgrader/archive/refs/heads/main.zip With us is currently @AlirezaT99 and @Gehock.

The idea is this would be more up to date code that fixes bugs, is synced with jupyter/nbgrader:main, and eventually becomes jupyter/nbgrader:main. However... it would be much better to work directly in jupyter/nbgrader:main, so @brichet, I'd say it's up to you: let us know what we should do. If we can somehow help keep upstream updated faster, then by all means we should work there and not try to make alternative main branches. Some of the pull requests we have submitted have been in our production for a while now, someone will comment on them separately.

Any thoughts on what we should do next? Or how people can help? For example we deploy PRs in our production and tell if they work?

[perllaghu]

I think one of the problems is we don't have many people who can actual action PRs [quite a few have moved onto other companies/services/jobs]

..... but yes, trying to get together some form of co-ordinated management would be good

[brichet]

Thanks @rkdarst for opening this discussion.

I agree that there is a lack of maintenance/developments in nbgrader. I'm probably the only one who's been merging PRs for a year. But unfortunately I can't devote too much time to it at the moment.

@brichet, I'd say it's up to you: let us know what we should do.

I don't think it's up to me to decide on the future of nbgrader. As an open source project, it belongs to anyone who wants to get involved, and of course mainly to the users (of which I'm not one).

What about accepting that things might get unstable and merge pull requests with less threshold?

The project is indeed missing some developers involved in reviewing/merging, but I don't think we should merge changes without review (if this is what you mean there).
This could add confusion in the codebase (which can already be difficult to understand), and make it more difficult to maintain in future.
Additionally it can bring some breaking changes that will be observed later and difficult to revert.

Of course, I advocate keeping the upstream updated with fixes and improvements.
Pushing PRs towards this goal is very important, but reviewing other people's PRs is also very important. I think most recent PRs with multiple reviewers have been merged, so it's really useful to have multiple feedbacks.

A minimal way to review PRs can also be to provide feedback that they don't break anything (and works) in other production deployment.

[rkdarst]

The project is indeed missing some developers involved in reviewing/merging, but I don't think we should merge changes without review (if this is what you mean there).
This could add confusion in the codebase (which can already be difficult to understand), and make it more difficult to maintain in future.
Additionally it can bring some breaking changes that will be observed later and difficult to revert.

I don't mean to merge entirely without review: there should be some balance, for example for big things, if at least someone has run it in production, that's a good sign.

But the second two sentences are exactly what I see as the problem: first off, things can become confusing if no one has the overall view. Second, almost any probably is probably depended on by someone (relevant xkcd of course...), and even then one solution may be wrong for someone else. Does anyone have a solution to this?

This is what prompted me to make my proposal: it seems we won't have anyone to make the hard decisions. Best (I think) we can do (in the near term) is be more active in accepting things, if

• Overall it seems to be reasonable code quality, has tests
• The submitter can explain the possible downsides for which it might break a workflow
• If it's complex, someone says it's worked in production (they should say length and type of production)
• There aren't many commenters saying it breaks their use case.

... and hope that by doing the above, we can nurture some people that do have the big-picture perspective in the future.

[brichet]

@rkdarst I've edited your first comment to add missing people who worked on nbgrader last year or were involved in the maintainer discussion.

[SylvainCorlay]

Hello everyone,

Many thanks to @rkdarst for starting this conversation. Seeing so much engagement in this conversation is really inspiring!

The work of @brichet at @QuantStack (which was partly funded by @nthiery and @pierrepo) was first focused on porting NbGrader to JupyterLab. Indeed, this was a critical requirements in advancing the Notebook 7 plan outlined in JEP #79. Indeed, the JEP included a commitment to make the most popular classic notebook extensions compatible with JupyterLab - with an explicit mention of nbgrader. However, it's evident that the funding allocated to the project falls short of providing the necessary attention it requires.

Despite the fact that the problem of grading notebook as received much less attention that the rest of the Jupyter stack, I am convinced that the issue addressed by nbgrader is important. Clearly, the amount of engagement in this thread shows that there is significant interest in the tool. Several people have invested considerable effort in adapting the current version to their specific use cases. Perhaps, we ought to adopt a more ambitious stance on this matter.

For this reason, last fall, we submitted a letter of intent to the CZI to secure funding for nbgrader's development. Unfortunately, it didn't progress to the stage of submitting a full application despite our efforts. The proposal included a significant overhaul of the UI, and work on consolidating the backend. I have included the body of the letter of intent below, which had to adhere to a strict 500-word limit:

With its ability to seamlessly blend narrative and executable code, Jupyter has become an invaluable tool in educational contexts and has been credited for facilitating computational thinking among students.
Many universities have adopted JupyterHub, enabling teachers to give assignments to their students in the form of notebooks. However, the current evaluation system revolving around the nbgrader project, needs a substantial overhaul.

Consolidation of the automated grading workflow
Nbgrader, unfortunately, exhibits vulnerabilities when handling notebooks that have been inadvertently altered by students, rendering the grading system ineffective. A common failure arises when students, in their attempt to copy and paste an exercise statement, carry over cell metadata along with the content, mislabeling a solution cell as a question cell. Another common issue occurs when a student changes the kernel associated with a notebook. To fix such situations, teachers resort to editing the metadata of student notebooks, which becomes intractable for large classes.
While it properly handles the case of code raising an exception, nbgrader doesn't recover from kernels crashing altogether. This is particularly inconvenient with the C++ kernel, where student code is more likely to cause segfaults than in Python. The situation is made worse by the fact that nbgrader only allows teachers to manually override grades if the automatic grading step was successful.

A modernized user interface
The nbgrader user interface still partly relies on the original codebase designed for the classic Jupyter notebook in 2015. Notably, the UI is challenging in scenarios where instructors need to adjust grading or feedback for a specific exercise across multiple students, especially when reassessment of earlier evaluations is necessary.
To tackle these limitations, we are embarking on a comprehensive overhaul of the nbgrader user interface addressing the pain points associated with the current version. The future UI will introduce an innovative "grid" grading tool, empowering teachers to simultaneously review and manage responses from multiple students and to perform batch operations.
Additionally, the new user interface will feature a tool for visualizing grade distributions for each exercise within a class, facilitating the instructors' ability to gain insights into the class's performance. Finally, the upgraded nbgrader will be better integrated with learning management systems.

A better story for large-scale deployments
Nbgrader relies strongly on the hypothesis that the JupyterHub deployment runs on a single machine. We will make nbgrader more agnostic to the deployment type of Jupyterhub. We will also enhance the configurability of nbgrader to handle the use case of multiple classes requiring individual configurations.

Partnering with Université Paris Saclay
For this project we will collaborate with Paris Saclay, a university that operates a large-scale deployment of JupyterHub. Multiple large courses with hundreds of students are taught each year, such as "Introduction to Data Sciences", "Numerical Methods", and "Imperative programming in C++" (which uses the C++ Jupyter kernel).
The new interface will first be reviewed by our partner at the university. It will consist of a modern JupyterLab extension, built up from JupyterLab UI components, providing a native feel to regular Jupyter users.

I think that it would be interesting to push this kind of proposal forward with another funding source.

[trevorcampbell]

Thanks for kicking off this discussion @rkdarst !

Context / Usage of NbGrader: I teach a large undergraduate data science course (1,000+ students per semester). We use NbGrader to autograde Jupyter notebook assignments. We use NbGrader only for its auto/manual grading functionality (via Formgrader), and feedback form generation. We don't use anything related to the exchange; I have written other software that handles distributing assignments, collecting them, monitoring TA grading, uploading grades to Canvas, etc. From the students' perspective, they just click a link on Canvas, and they're taken to a browser tab with a Jupyter notebook where they can work. When they're done working, they simply save the notebook and quit. In the background, we're using LTI Authentication into a JupyterHub we run in our department, which spawns a Docker container for the student to work in. At assignment deadlines, we take a ZFS snapshot of our student directories and use those for grading later on.

Cool thing: We have quite a few assignments in our course, each with quite a few questions. When we originally designed the course, we manually wrote test code for all the questions, which amounts to thousands of lines of code to maintain and debug.

I recently pushed some changes to NbGrader #1817 (also see the new documentation item here) that give instructors the ability to automatically generate test code for objects they want to test. If there's an object named foo in a notebook that you want to check, you simply add something like

# AUTOTEST foo

in the source/ notebook, the release/ notebook will contain code that checks various properties of foo for correctness. The test code is generated based on the type of foo using a template file that you specify in advance which determines how objects of each type are to be tested. It's much easier to maintain our assignment collection now -- when we make changes, we don't need to adjust our test code -- only the set of objects we want to test.

Problems: We share a lot of the same problems as mentioned by @lahwaacz in #1862 (comment) . Mostly related to students accidentally breaking notebooks, opaque autograding error messages, etc. But overall I find NbGrader isn't the main source of issues in my class any more -- it mostly does what it's supposed to do.

Development: I don't have a tonne of extra capacity for more development here at this point --- I've got other software projects that are in a much worse state that need my attention first ;-)

A random thought: I know that a lot of my colleagues are now using PrairieLearn. I don't know much about it, but I wonder if there are features/ideas there that we could map over to NbGrader.

[tmetzl]

Hello everyone,

Thank you @rkdarst for kickstarting the discussion!

Since 2019, we have been using nbgrader for exams and assignments at Hochschule Bonn-Rhein-Sieg. If anyone is interested, we presented parts of our work at JupyterCon 2023. You can find the presentation here.

What have we done with nbgrader?
We have developed a variety of add-ons and extensions for nbgrader (currently, we are still stuck at notebook <7 since we have not yet found the time to port everything to JupyterLab extensions):

• e2xgrader, an add-on for nbgrader, which introduces new cell types (multiple choice, drawio diagrams, ...) and includes a student mode that activates certain extensions to guide students through an assignment (visual indicators for where solutions should be placed, a restricted interface that prevents students from removing nbgrader cells or changing the cell type). We also have several grading-related features, such as horizontal grading that allows graders to navigate through submissions on a solution cell basis using IDs or annotating student solutions by drawing on them. Additionally, we have functionality for adding new autograders on a cell-type basis via the nbgrader_config.
• e2xauthoring, an add-on for simplifying the authoring process of assignments. Here, we author tasks as single Jupyter notebooks containing related questions and have a process for creating assignments by selecting which tasks should be included, along with a template that defines the overall structure of an assignment. These are all version-controlled using Git.
• e2xgradingtools, a small package designed to make testing of student solutions easier and to catch errors such as when a variable is not defined or a function does not have a return statement, etc. These tests produce a report which is parsed by the autograder of e2xgrader to determine the number of points a student is awarded.

What are points were we see room for improvement?

• A plugin system for nbgrader, similar to JupyterLab and Jupyter Server extensions, would be great. While this is already implemented with the exchange, there are many other parts of nbgrader one might want to modify. Different users of nbgrader have different needs, and the ability to write a simple plugin instead of having to create a pull request and get changes approved by all stakeholders could potentially accelerate the development of an ecosystem of useful nbgrader plugins.
• Lack of structure of assignments. Right now each cell has a single role and an id, but for example a test cell has no connection to the solution cell it should test.
• An official method for safe autograding would be great. I know there is an approach involving a remote kernel in a Docker container, but perhaps we could explore local autograding with restricted permissions.

Cheers,
Tim

[nthiery]

Hi @tmetzl,

For information: I'll possibly spend a good chunk of July at the Hausdorff institute in Bonn. Maybe that can
give us the occasion to work together.

[tmetzl]

That would be great.

[GabrielaVives]

Hello everyone,

I'm really happy that this discussion is happening, and that many people are contributing with their experiences using NBgrader. In November 2023 with the help of @SylvainCorlay and @brichet, I started doing some UX research and design on this subject, so I would be happy to contribute further, gather inputs and work with anyone here interested in improving the UX and UI of this tool.

I have a UX background and not a technical one, so it's also important for me to gather technical constraints and priorities so that I can take them into account when making design decisions.

For the moment I have

• created UX architecture maps of how the current UI is articulated
• detailed a few of the Teacher user journeys that seem to be the most important ones
• created wireframes to illustrate these user journeys with UX improvements to the current UI

I would like to organise or participate in meetings with users and developers to work together on this. I think that the first step would be to meet with all of you who use NBgrader (and/or contribute to its development, but that's a plus), share the state of the UX work and review the user journeys together to be sure I haven't missed any important pieces of the way you use, and need to use this tool.

So feel free to reach out if you are interested.

[GabrielaVives]

Hello All,

would some of you be interested in participating in a UX/project planning online workshop next week?
The idea would be to write down and prioritise the usability issues you currently encounter with NBgrader.
Depending on how many attendees there are and the time it takes, we can also brainstorm and prioritise solutions.

Please put a thumbs up if interested, and I will suggest time slots that could suit all time zones.

[tmetzl]

@GabrielaVives I would also like to participate if possible. Preferably Monday, Tuesday or Friday. But if it does not work I'm also okay with sitting this one out.

[GabrielaVives]

How about this Tuesday (19th) or next Tuesday (26th) from 2 to 4pm paris time?

[perllaghu]

Both are good for me...

[shreve]

I'm in US/Eastern (currently UTC-04:00). I want to loosely say I can attend those days+times. I'm not really a morning person 😅

[GabrielaVives]

Ok well let's plan this for Tuesday the 26th, to let everyone time to see this and block their calendar. I'll follow this with an invite to the meeting.

[tomoya-saito]

Sorry for the very late reply.
At the Yamaguchi University, in japan, we use JupyterHub and nbgrader for many classes span multiple departments.
We faced several issues when adopting the JupyterHub and nbgrader.

First, permission setting of root directory for file exchange directories is "777".
So, All user can see other students' submission, and users can delete submissions of other classes, or other students.
Then, we fixed JupyterHub's and nbgrader's codes in order to use "755" permission.
But, this fix prevents users from using the "quickstart" command.

Second, we fixed nbgrader's codes in order to user "JST" (Asia/Tokyo) timezone.
We think that the nbgrader should support multiple time zones.

Third, it is difficult for many classes to use the system at the same time.
In order to add group and course information to "jupyterhub_config.py", administrators need to stop JupyterHub system.
If we want your students to use the JupyterHub and nbgrader from home, nbgrader's ability to launch the "formgrader" service for each course could be a security hole.
We use the nbgrader for classes in all departments of the university, the system cannot be stopped.
Moreover, modification of “jupyterhub_config.py” to fix of course/participant information is a big burden for administrators.
So, we fixed JupyterHub's and nbgrader's codes so that many classes can freely use the nbgrader without requiring administrators or teachers to modify "jupyterhub_config.py".
Based on the information obtained from the LMS, our system creates the exchange directories and sets permissions automatically, and course settings are adopt to user's own Jupyter server (Docker container).

Regards,

[jeflem]

Thank you very much @tomoya-saito, sounds interesting but, as you mentioned, quite complex.

Me and my group applied for some funding to further develop Ananke and to contribute to related projects (jhub, nbgrader,...). Unfortunately, request wasn't successful. So there's little time for working on things at the moment.

Our long-term plan is to move all the LTI/LMS things in Ananke to JHub to have a general LTI interface/API within JHub. Then extend nbgrader to use this API (and connect another tool to it, too). Maybe we should start some discussion on how such a more general LTI API could/should look like.

[perllaghu]

Yep - a new thread for this would be good...... Noteable only uses LTI for access..... however our's is an even more complicated service :)

[tomoya-saito]

Thank you, @jeflem

A complex mechanism is needed to solve this permission problem while ensuring user compatibility, and adding to it the concept of "free use of the system by any subject, any user at any time."
The overall solution is in the field of LTI APIs development and integration of multiple systems, so it cannot be resolved as a single "nbgrader" issue.

If it's just a "777" permission issue, it might be resolved within the scope of the "nbgrader" issue.
However, if only this issue is solved, user convenience will decrease.

Regards,

[perllaghu]

Gentles..... can we either move the discussion of the exchange and it's mechanism to a new [not talking about UI stuff] topic, or agree it's an issue that needs looking at [not in a discussion about UI stuff]

[tmetzl]

I agree we should move this discussion but I quickly wanted to add something that might be of interest.

@tomoya-saito @jeflem
I think our exchange might address some of these problems. See the docs here:
We have personalized directories per student where they only have read and write permissions on that. The base package still relies on notebook 6 (we have a bunch of nbextensions for students and graders), but the exchange should work for newer versions.
For course setup we also have something here.

[GabrielaVives]

Hi Everyone,

Here is the Jitsi link to the UX workshop happening tomorrow Tuesday 26th from 2pm to 4pm Paris time.

And the calendar event if you wish to add it to your calendar.
If you would like to be invited by email you can send me your address on Gitter.

See you tomorrow!

[perllaghu]

Many thanks.... See you tomorrow!

[perllaghu]

Dang-it..... did I miss it?