design.qmd

---
title: "Design"
description: "Expanded explanations and descriptions of the principles, practices, tools, methods, and decisions we have for work across DP-Next."
---

## Principles

Part of the aim for DP-Next, and the purpose of Work Package 1, is to
produce scientific results that follow open scientific and reproducible
practices. In addition to this aim, Work Package 1 also aims to help
produce better research in less time and resources. Aligning with these
overall aims and expanding on the
[guiding principles](strategy.qmd#guiding-principles) in the Strategy
page, we want to eventually follow these principles:

- Incorporate explicit feedback loops in how we collaborate
- Actively and continually seek out and identify ways we can improve how
  we work
- Start with what we can do now and improve iteratively and
  incrementally
- Make work visible (at least to the group), including tasks to do,
  project management
- Incorporate regular upskilling activities, such as "pair-working"
- Prefer written communication over verbal communication to ensure more
  people can participate
- Use open source software as much as possible and is reasonable
- Have meetings sparingly and only when extensive, concurrent
  discussions and brainstorming are needed

## Practices, tools, and methods

And to achieve these principles, we also eventually (not all at once)
want to work towards a point where we:

- Use GitHub or just Git to manage and track all files
- Use R or Python for doing data analysis
- Use Quarto Markdown for writing documentation
- Use Quarto with R or Python code to write reproducible scientific
  output
- Use Quarto to create websites of our work
- Use FAIR principles when managing data
- Use Seedcase software to structure and manage data
- Use Zenodo to upload protocols (before submitting to other outlets),
  analysis code, and preprints
- Use GitHub Project boards to follow a
  [Kanban](https://en.wikipedia.org/wiki/Kanban_board)-style approach to
  project management
- Use Discord for informal communication or for coordinating across
  groups
- Use Discord for video meetings, including for upskilling activities
- Use Teamup, a calendar tool, for scheduling meetings and events
  relevant to all groups
- Use GitHub Projects and Issues to create, discuss, and manage tasks
  and issues

## Decisions

To be explicit about the reasons for our choices of tools, methods, and
approaches that apply to the DP-Next project, we have documented them in
the [Decisions](decisions/index.qmd) section. This website will not
document choices for things that are specific to a single work package,
for instance, the type of machine to measure a specific biological
marker. Instead, we will focus on the decisions that are relevant to the
entire project or to multiple work packages.

## Design patterns

To help simplify how we work, it's useful to follow commonly used
patterns in project development and collaboration. In our case, the
design patterns we will use are:

### One Git or GitHub repository per "product"

The pattern of "one repo, one product" is a widely used pattern in many
types of projects. There are many tools, templates, and workflows that
are built around this pattern. For instance, it is easier to ensure a
scientific document is reproducible if it follows this pattern. So
administratively and collaboratively, it is easier to use a pattern like
this, since it is a standardized way of structuring work. For DP-Next, a
"product" includes scientific output such as papers, but also includes
protocols, documentation on data, standard operating procedures, and
guides (like this one).

Some servers don't allow internet or easy access to outside resources
like GitHub. But all servers have Git installed. In those cases, each
scientific output is one Git repository.

### Prefer to revolve work around teams and "products"

Linking to the "one repo, one product" pattern, it's also useful to
revolve work around teams. From the
[Team Topology](https://www.teamtopologies.com/) book, the "the means of
delivery of a product" is the team. What that means is that teams should
be assigned work, not individuals, as teams are what effectively create
and build products (especially knowledge work products). So by designing
work on a product with a focus on a team and teamwork, we can make use
of all the established and proven ways to structuring teams and
organising how people work within a team to more effectively and
efficiently produce a product.

How do we define team then? As per the (excellent) Team Topology book, a
team is a small group of people (of no more than 8-9 people) who work
closely and cohesively together on a shared product. A team usually has
these "signs":

- Requires a high level of communication and coordination of tasks
  between members.
- Is usually composed of people with different, though complementary,
  skills.
- Is responsible for the entire lifecycle of a product.
- Has a high level of autonomy in deciding how they do and structure
  their work.
- Has specific roles within the team.
- Has a lead who is responsible for curating and coordinating tasks and
  ensuring the environment and organisation enables members to work
  effectively.

With this in mind, the pattern of "one team, one product" is an
effective and widely used pattern to structure work around products.
This way, it makes it clear who is responsible for what. In DP-Next, we
use this pattern for most of our main, larger products. However, many of
the smaller products are not suitable for using a team-based approach,
for example, a PhD student's paper product for their thesis. In those
cases, the individual assigned the product is responsible for it, with
support and review from other (potentially explicitly assigned) people.

### Repository naming convention

When creating repositories for DP-Next, we decided on following a naming
convention that includes the work package number and the type of output
that follows a descriptive pattern of a higher-level category to a
lower-level category.

For Work Package 3, we use this convention, in order:

1. `wp3-` to indicate the work package number.
2. `d-` and `a-` to indicate the output is either "data" (`d-`) or
   "analysis" (`a-`).
3. `clin-` and `quest` for the clinical and questionnaire data,
   respectively. Without any later sub-outputs, this indicates the main
   output is the "data package" itself.
4. `protocol`, `dmp`, `sop`, and `dap` for the protocol, data management
   plan, standard operating procedure, and data analysis plan,
   respectively.

Based on that convention, we have the following repositories for Work
Package 3 (at least for the `d-` outputs):

- `wp3-d-clin`
- `wp3-d-clin-protocol`
- `wp3-d-clin-dmp`
- `wp3-d-clin-sop`
- `wp3-d-clin-dap`
- `wp3-d-quest`
- `wp3-d-quest-protocol`
- `wp3-d-quest-dmp`
- `wp3-d-quest-sop`
- `wp3-d-quest-dap`