Skip to content

Commit a3ecefe

Browse files
committed
Remove hierarchy of pages
1 parent d5c5e53 commit a3ecefe

33 files changed

+215
-356
lines changed
Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,26 @@
11
---
2-
title: Run evaluation
3-
parent: Prepare a submission
2+
title: About
43
layout: default
54
nav_order: 1
65
---
76

8-
# Run evaluation
9-
{: .no_toc }
7+
# About
108

9+
{: .no_toc }
1110

1211
## Table of Contents
12+
1313
{: .no_toc .text-delta }
1414

1515
1. TOC
16-
{:toc}
16+
{:toc}
17+
18+
## Motivation
19+
20+
## Behaviors
21+
22+
### Self-touch
23+
24+
### Hand regard
25+
26+
## MIMo
Lines changed: 53 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,95 @@
11
---
2-
title: Scoring system
2+
title: Competition guidelines
33
layout: default
4-
parent: Competition guidelines
5-
nav_order : 2
4+
nav_order: 5
65
---
76

7+
# Competition guidelines
88

9-
# Scoring
109
{: .no_toc }
1110

1211
## Table of Contents
12+
1313
{: .no_toc .text-delta }
1414

1515
1. TOC
16-
{:toc}
16+
{:toc}
17+
18+
**---**
19+
20+
## Judges
21+
22+
The evaluations will be conducted by a committee of experts in the field of robotics, machine learning, and developmental psychology, invited by the organizers of the competition. The review will be a single-blind process. The experts will [score](#scoring) the submissions and be encouraged to provide constructive feedback to the authors of the submissions. The final decisions on the winners will be made on the basis of the aggregated scores of all the experts.
23+
24+
25+
26+
## Scoring
1727

1828
Each submission will be scored out of 10 points, based on four criteria, described in detail below. These criteria reflect both the scientific goals of the competition and the broader aims of the IEEE ICDL community. They are intended to reward not only performance but also developmental relevance, interpretability, and methodological soundness.
1929

20-
A minimum score of 1 point in each of the first three criteria (likeness, achievement, plausibility) will be required to win the competition.
30+
A minimum score of 1 point in each of the first three criteria (likeness, achievement, plausibility) will be required to win the competition.
2131

2232
**Note**: The evaluation criteria are subject to change as the competition progresses. We will update this page with the final criteria as soon as they are finalized.
2333

24-
## Likeness of the generated examples (3 points)
34+
### Likeness of the generated examples (3 points)
2535

26-
The first criterion focuses on how closely the learned behaviors match the expected behaviors. The judges will evaluate videos and logs from 10 episodes with random initial conditions. The evaluation will consider qualitative features, including body trajectories, variability across trials, and resemblance to real infants.
36+
The first criterion focuses on how closely the learned behaviors match the expected behaviors. The judges will evaluate videos and logs from 10 episodes with random initial conditions. The evaluation will consider qualitative features, including body trajectories, variability across trials, and resemblance to real infants.
2737

2838
The likeness score will be subjective for each judge. However, authors are encouraged to take a look at the [example videos](../about/behaviors.md) to get a sense of what the expected behaviors should look like.
2939

30-
## Achievement of the target behavior (3 points)
40+
### Achievement of the target behavior (3 points)
3141

3242
Submissions will be evaluated on how well they replicate the learning of the target behavior. The evaluation will be made on the basis of the training logs generated during the training process and any results reported by the authors in the extended abstract.
3343

3444
Running the [evaluation code](../submission/evaluation) will return a preliminary score based on:
45+
3546
- the fraction of body parts touched by each hand for the self-touch task,
47+
3648
- the fraction of timesteps with either hand in the field of view of each eye for the hand-regard task.
3749

38-
This preliminary score is only provided as a refernce for the authors to assess their model's performance, and *will not necessarily reflect the final score given by the judges*.
50+
This preliminary score is only provided as a refernce for the authors to assess their model's performance, and *will not necessarily reflect the final score given by the judges*.
3951

4052
Incomplete or inconsistent results that nonetheless show promise will be rewarded in spite of the lack of completeness. This criterion is intended to encourage models that are both effective and relevant to the task at hand.
4153

42-
## Plausibility of the learning mechanism (3 points)
54+
### Plausibility of the learning mechanism (3 points)
4355

44-
One of the central aims of BabyBench is to promote the exploration of learning processes that are not only effective but also cognitively and developmentally plausible. This criterion rewards submissions that are inspired by mechanisms such as curiosity, intrinsic motivation, predictive coding, or self-supervised representation learning.
56+
One of the central aims of BabyBench is to promote the exploration of learning processes that are not only effective but also cognitively and developmentally plausible. This criterion rewards submissions that are inspired by mechanisms such as curiosity, intrinsic motivation, predictive coding, or self-supervised representation learning.
4557

4658
The plausibility score will be based on the description of the method provided in the extended abstract and potentially on the logs generated during the training process. The judges will be encouraged to favor models that reflect general principles from developmental psychology or neuroscience, or that offer interpretable insights into learning dynamics beyond the specific task at hand.
4759

48-
## Computational efficiency (1 bonus point)
60+
### Computational efficiency (1 bonus point)
4961

5062
Although not a primary evaluation criterion, computational efficiency will be considered as a potential bonus point, particularly in the case of ties. The bonus point may be awarded to models that achieve strong results while remaining lightweight in terms of:
63+
5164
- total training time,
65+
5266
- simulation speed,
67+
5368
- memory usage,
69+
5470
- or hardware demands.
5571

56-
The goal of this criterion is to encourage solutions that are elegant and accessible, without requiring extensive computational resources. This criterion will be adjusted based on the number of participants and the complexity of each task. We encourage submissions that rely on plausible learning mechanisms that can be run in a reasonable amount of time and memory.
72+
The goal of this criterion is to encourage solutions that are elegant and accessible, without requiring extensive computational resources. This criterion will be adjusted based on the number of participants and the complexity of each task. We encourage submissions that rely on plausible learning mechanisms that can be run in a reasonable amount of time and memory.
73+
74+
75+
76+
## Prizes
77+
78+
The winning team of the BabyBench Competition will receive a **€150 prize, generously sponsored by the IEEE Computational Intelligence Society**. In addition to the monetary award, the winning team will be invited to **present their submission during the IEEE ICDL conference**, offering an opportunity to share their work with a broad interdisciplinary audience.
79+
80+
Finalist teams who make it to the final evaluation round will receive honorable mentions in recognition of their contributions. These mentions will be announced during the competition session and highlighted in official competition materials.
81+
82+
Our goal is to celebrate creativity, rigor, and developmental insight—whether you win, place, or simply participate, your work helps advance the conversation. We look forward to seeing your innovative approaches and the impact they will have on our understanding of infant learning and development.
83+
84+
85+
86+
## Rules
87+
88+
The BabyBench Competition aims to bring together researchers and students in the fields of robotics, machine learning, developmental psychology, and beyond. We welcome participants from all backgrounds, institutions, and levels of experience. There are no excluding criteria for participation—collaboration across disciplines and perspectives is strongly encouraged.
89+
90+
While the competition is open and inclusive by design, we strongly encourage submissions that align with the spirit of ICDL. This means a focus on modeling, understanding, or replicating infant learning and behavior through mechanisms that are plausible, interpretable, or relevant to developmental processes. We are particularly interested in approaches that bridge the gap between artificial and biological learning systems, drawing insights from cognitive development, neuroscience, and behavioral science.
91+
92+
Submissions may employ any modeling framework or data modality, but should aim to contribute meaningfully to our understanding of how learning unfolds in early development—whether through simulation, robotics, theoretical modeling, or analysis of behavioral data. We encourage submissions that are reproducible, open-source, and accessible to the broader research community.
93+
94+
To keep the competition simple, we do not require submissions to include the full code used to learn the behaviors. Only the winning team will be expected to provide the code, which will be tested as a verification check before the announcement.
95+

wiki/faq.md renamed to faq.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,46 +5,47 @@ nav_order: 99
55
---
66

77
# FAQ
8-
{: .no_toc }
98

9+
{: .no_toc }
1010

1111
## Table of Contents
12+
1213
{: .no_toc .text-delta }
1314

1415
1. TOC
15-
{:toc}
16+
{:toc}
1617

1718
## What is BabyBench?
1819

1920
BabyBench is a multimodal benchmark of infant behaviors for developmental artificial intelligence. The BabyBench Competition hosted at IEEE ICDL invites participants to model infant-like learning in simulated environments using MIMo.
2021

2122
## What is MIMo?
2223

23-
MIMo, the multimodal infant model, is a simulation platform based on the MuJoCo physics engine. You can found out more about MIMo [here](mimo/). You can also download and use MIMo for your own experiments [here](https://github.com/trieschlab/MIMo).
24+
MIMo, the multimodal infant model, is a simulation platform based on the MuJoCo physics engine. You can found out more about MIMo [here](about/#mimo). You can also download and use MIMo for your own experiments [here](https://github.com/trieschlab/MIMo).
2425

2526
## Who can participate?
2627

2728
Anyone! Students, researchers, interdisciplinary teams—no matter your background, you’re welcome. If you're interested in developmental learning, this is for you.
2829

2930
## I don't have experience with reinforcement learning, can I still participate?
3031

31-
Absolutely, but training MIMo will require some basic knowledge of reinforcement learning. We recommend reading the [resources section](start/resources) or heading to the [discussion page on github](https://github.com/babybench/BabyBench2025) to connect with others.
32+
Absolutely, but training MIMo will require some basic knowledge of reinforcement learning. We recommend reading the [resources section](start/#resources) or heading to the [discussion page on github](https://github.com/babybench/BabyBench2025_Starter_Kit) to connect with others.
3233

33-
## Are there any baseline models or starter code?
34+
## Are there any baseline models or starter code?
3435

35-
Yes! We provide some basic examples with starter code to help you get up and running with training MIMo using reinforcement learning [here](start/examples).
36+
Yes! We provide some basic examples with starter code to help you get up and running with training MIMo using reinforcement learning [here](start/#examples).
3637

3738
## Are there restrictions on the algorithms or architectures I can use?
3839

39-
No, but we strongly encourage models that follow the spirit of the competition: unsupervised, self-supervised, or intrinsically motivated learning approaches that aim to recreate or explain developmental behaviors. Take into account that plausibility is one of the [evaluated criteria](competition/evaluation). Having said that, creativity and interpretability are valued!
40+
No, but we strongly encourage models that follow the spirit of the competition: unsupervised, self-supervised, or intrinsically motivated learning approaches that aim to recreate or explain developmental behaviors. Take into account that plausibility is one of the [evaluated criteria](competition/#evaluation). Having said that, creativity and interpretability are valued!
4041

4142
## What kind of submission is expected?
4243

43-
You’ll be asked to a 2-page abstract describing your approach, a log file automatically generated during training, and a video of the learned behavior. Details can be found [here](competition/submission).
44+
You’ll be asked to a 2-page abstract describing your approach, a log file automatically generated during training, and a video of the learned behavior. Details can be found [here](competition/#submission).
4445

4546
## How will the winner be chosen?
4647

47-
Each submission will be scored out of 10 points, using a mixture of qualitative and quantitative criteria explained [here](competition/evaluation).
48+
Each submission will be scored out of 10 points, using a mixture of qualitative and quantitative criteria explained [here](competition/#evaluation).
4849

4950
## What do I get if I win?
5051

@@ -54,8 +55,7 @@ The winning team of the BabyBench Competition will receive a **€150 prize** an
5455

5556
Yes, absolutely. Your BabyBench work can be a great foundation for a contribution to a conference or journal. We are working on a full version of the BabyBench platform in the fall of 2025. We’re happy to discuss follow-up opportunities!
5657

57-
5858
## More questions?
5959

60-
We’re happy to help! Contact us by [email](mailto:fcomlop@gmail.com) or leave a message in the [discussion page on github](https://github.com/babybench/BabyBench2025).
60+
We’re happy to help! Contact us by [email](mailto:fcomlop@gmail.com) or leave a message in the [discussion page on github](https://github.com/babybench/BabyBench2025_Starter_Kit).
6161

index.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,11 @@ The first **BabyBench Competition** will take place at the ***IEEE* ICDL 2025 Co
2525
<!--<iframe width="640" height="480" src="../../static/videos/handregard.mp4" frameborder="0" allowfullscreen></iframe>-->
2626

2727
---
28+
29+
```
30+
@misc{lopez2025babybench,
31+
title={BabyBench: A Multimodal Benchmark of Infant Behaviors for Developmental AI},
32+
author={Francisco M. López and Valentin Marcel and Xavier Hinaut and Jochen Triesch and Matej Hoffmann},
33+
year={2025},
34+
}
35+
```
Lines changed: 32 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,58 +1,59 @@
11
---
22
title: Installation
33
layout: default
4-
nav_order: 97
4+
nav_order: 2
55
---
66

77
# Installation
8+
89
{: .no_toc }
910

1011
## Table of Contents
12+
1113
{: .no_toc .text-delta }
1214

1315
1. TOC
14-
{:toc}
16+
{:toc}
1517

1618
## Option 1: Local installation
1719

1820
Pre-requisites: [Python](https://www.python.org/), [Git](https://git-scm.com/), and [Conda](https://www.anaconda.com/products/individual). All software has been tested on Ubuntu 18.04 and 24.04.
1921

2022
### Create a conda environment
21-
22-
```
23-
conda create --name babybench python=3.12
24-
conda activate babybench
25-
```
26-
27-
### Clone the BabyBench2025 repository
28-
29-
```
30-
git clone https://github.com/babybench/BabyBench2025.git
31-
cd BabyBench
32-
```
33-
23+
24+
```
25+
conda create --name babybench python=3.12
26+
conda activate babybench
27+
```
28+
29+
### Clone the BabyBench2025 repository
30+
31+
```
32+
git clone https://github.com/babybench/BabyBench2025_Starter_Kit.git
33+
cd BabyBench
34+
```
35+
3436
### Install requirements
35-
36-
```
37-
pip install -r requirements.txt
38-
```
39-
37+
38+
```
39+
pip install -r requirements.txt
40+
```
41+
4042
### Install MIMo
41-
42-
```
43-
pip install -e MIMo
44-
```
45-
43+
44+
```
45+
pip install -e MIMo
46+
```
4647

4748
All done! You are ready to start using BabyBench.
4849

4950
### Launch the installation test
50-
51-
```
52-
python test_installation.py
53-
```
5451

55-
This will run a test to check that the everything is correctly installed.
52+
```
53+
python test_installation.py
54+
```
55+
56+
This will run a test to check that the everything is correctly installed.
5657

5758
## Option 2: Singularity container
5859

@@ -76,4 +77,4 @@ This will run a test to check that the everything is correctly installed.
7677

7778
## Troubleshooting
7879

79-
If you encounter any issues, visit the ![troubleshooting page](https://babybench.github.io/babybench2025/wiki/troubleshooting)
80+
If you encounter any issues, visit the [troubleshooting page](https://babybench.github.io/babybench2025/wiki/troubleshooting)

start.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
title: Getting started
3+
layout: default
4+
nav_order: 3
5+
---
6+
7+
# Getting started
8+
9+
{: .no_toc }
10+
11+
## Table of Contents
12+
13+
{: .no_toc .text-delta }
14+
15+
1. TOC
16+
{:toc}
17+
18+
**---**
19+
20+
21+
22+
## Environments
23+
24+
25+
26+
## Configurations
27+
28+
29+
30+
## Examples
31+
32+
33+
34+
## Resources
35+

0 commit comments

Comments
 (0)