Literate package development using Julia and Pluto.jl

[andreasKroepelin]

Literate package development using Julia and Pluto.jl

I propose a workshop in which we look into how to use Pluto notebooks to develop Julia packages in a way that tightly integrates implementation and exploration of scientific code.

Motivation

Software development in a scientific context is often initiated by a phase of "playing around" with the research subject at hand and trying out different approaches to solving a problem in an explorative way. For this, notebooks (such as Jupyter) provide a powerful tool to combine pieces of code with explanatory text (often stylable using some markup language) and outputs such as visualisations. They therefore allow for quick computational feedback and rapid iteration cycles. If done right, such a notebook can also read as conveniently as a chapter in a textbook, guiding the reader through the possibly intricate details of the implementation.

Often, scientific code is not meant to solve the one particular instance of a problem that is typically found in a notebook, however. Say, you begin your research by analysing one specific data set but later you want to create a general implementation of your solution that others can use as a package. In this case, you would normally have to extract parts of the notebook and create a separate entity of code that then becomes the package. When the exploration continues in the notebook later, this portation has to happen again and again.

As a consequence, it can be very beneficial if the notebook environment allows for the notebook to be "normal" code that can be executed without the environment. For Jupyter notebooks and their JSON based format, this is rather complicated. The aforementioned Pluto notebooks, however, are stored as regular Julia code can therefore actually be used for package development: The developer has to maintain a single entity of code that serves as the place for exploration, for tests, and for a reusable implementation. Based on ideas of Donald Knuth, this concept is sometimes called literate programming.

Plan for the workshop

This workshop will have roughly the following agenda:

• brief crashcourse on Julia (depending on the prior knowledge of the audience)
• package management in Julia and the typical structure of a package
• introducing Pluto notebooks:
  • writing code
  • adding text
  • visualising results
  • adding interactivity (PlutoUI.jl)
  • exporting static HTML
• combining different files/notebooks using PlutoLinks.jl and PlutoDevMacros.jl
• integrating interactive tests using PlutoTest.jl
• developing an example package using what we learned

[HeidiSeibold]

Thanks @andreasKroepelin

How much time do you think you'll need for this?

Is this intended as a workshop, where you are the trainer and people learn from you? Or is this something you would like to learn with others?

[HeidiSeibold]

I added a label to this break-out. Can you check if you feel it is appropriate and change it if not? Let me know if you have any questions.

[andreasKroepelin]

So if we are talking about multiples of 90 minutes here, then I would suggest either 90 or 180 minutes, but rather 90 minutes. It is indended as a workshop where the audience can learn the mentioned subjects from me. The label "learning" feels appropriate.

[HeidiSeibold]

Excellent thanks 👍

So what I need from you to be able to make a decision on this breakout are the following infos:

Who could be interested in collaborating on this?

(feel free to tag them with their GitHub username if they have one)

How much time do you need for this?

90 minutes

Abstract

(Can be short)

[HeidiSeibold]

Hi @andreasKroepelin can you please respond by Tuesday morning, thanks 😃

[andreasKroepelin]

Collaboration

@trahflow is a potential collaborator for this workshop.

Duration

90 minutes

Abstract

Writing code to interactively explore a research problem and writing code to develop a reusable package are often two very different things, leading to a lot of rewriting and potentially diverging code bases. In this workshop, we will explore how to use Pluto.jl notebooks and the Julia programming language to make the notebook be the package implementation. This allows us to have exploration, explanation, and implementation in one place, facilitating very readable and maintainable code.

[pancetta]

Hi! I just added the "Accepted" label to this BOS. Welcome on board!
https://un-derse23.sciencesconf.org/program

[BeastyBlacksmith]

Hi @andreasKroepelin, since we both do julia related sessions ( mine is #3 ), what would you think of I am joining yours as a helper and vice versa?

Best,
Simon

[andreasKroepelin]

Sounds like a great idea! Let us discuss details some day.

[pancetta]

Hi all, the unconference is only 3 weeks away now! On day 1 there will be a breakout blitz where all session organizers should advertise their sessions. 1 minute, 1 slide, let people know what you intend to do. Please prepare this slide in advance and add it right here (PDF please), by September 20.

[andreasKroepelin]

Here you can find the overview slide.