Skip to content

Commit 1da1414

Browse files
jaimergpocefpaf
andcommitted
add blog/2020-02-04-GSoC.md
Co-authored-by: ocefpaf <[email protected]>
1 parent 15fe8e9 commit 1da1414

File tree

1 file changed

+112
-0
lines changed

1 file changed

+112
-0
lines changed

blog/2020-02-04-GSoC.md

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
---
2+
authors:
3+
- ocefpaf
4+
tags: [autotick-bot, gsoc]
5+
---
6+
7+
# Google Summer of Code 2020 improved automatic maintenance of `conda-forge`
8+
9+
The `conda-forge` "autotick" bot is a crucial part of `conda-forge`'s
10+
infrastructure. It enables automatic maintenance of `conda-forge`
11+
packages by pushing version updates to the underlying software and
12+
enabling large migrations of packages from one dependency to another
13+
(e.g., Python 3.7 to Python 3.8). As `conda-forg` grows in size, with
14+
over 9,000 packages to date, automatic maintenance of the `conda-forge`
15+
ecosystem will become even more important.
16+
17+
<!--truncate-->
18+
19+
We here at `conda-forge` have a large number of potential Google Sumer
20+
of Code projects around maintenance and development of the autotick bot
21+
infrastructure. These projects are high-impact, affecting the entire
22+
`conda-forge` ecosystem. They also cover multiple systems including
23+
databases, `conda`'s CDN provider, continuous integration providers,
24+
and user interactions on GitHub.
25+
26+
Want to be a part of the team? Great! Take a look at the projects below
27+
and get in touch with us on GitHub! You can check the [GSoC
28+
label](<https://github.com/regro/cf-scripts/labels/GSOC>) for a
29+
detailed listing of the issues that need work.
30+
31+
1. Maintenance and Refactoring
32+
33+
We have a large backlog of maintenance and refactoring issues that
34+
are great for people with a range of experience from beginners to
35+
true code ninjas.
36+
37+
- Integration Testing for the Autotick Bot
38+
39+
> Run true integration testing on a copy of the graph to better
40+
> test code changes and improve our CI process.
41+
>
42+
> **Issue**: <https://github.com/regro/cf-scripts/issues/261>
43+
>
44+
> **Experience Level**: advanced
45+
46+
- Work on the "code hardening"
47+
[Milestone](<https://github.com/regro/cf-scripts/milestone/4>)
48+
49+
Address any of the issues in the milestone above related to code
50+
refactoring and cleanup.
51+
52+
**Issues**: <https://github.com/regro/cf-scripts/milestone/4>
53+
54+
**Experience Level**: beginner to advanced
55+
56+
2. Automated PRs for Globally Pinned Packages
57+
58+
`conda-forge` maintains a list of globally pinned packages. These
59+
are typically dependencies whose version needs to be the same across
60+
all of `conda-forge` (e.g., the compiler versions or packages like
61+
`HDF5`). While we have infrastructure to run a migration of the
62+
downstream packages from a given pinned package, we do not have
63+
automated infrastructure to propose the migration of the pin itself.
64+
The project here is to add this functionality to our infrastructure.
65+
66+
**Issue**: <https://github.com/regro/cf-scripts/issues/665>
67+
68+
**Experience Level**: advanced
69+
70+
3. Check the `conda` CDN for Updated Packages before Issuing PRs in a
71+
Migration
72+
73+
`conda` relies on a CDN provider to serve the index of available
74+
packages. There is a moderate ~30 minute delay between when a
75+
package is uploaded to `anaconda.org` and when it will appear in the
76+
`conda` index. We currently do not take this delay into account when
77+
issuing PRs in a migration.
78+
79+
**Issue**: <https://github.com/regro/cf-scripts/issues/595>
80+
81+
**Experience Level**: beginner
82+
83+
4. Finish Migrations with PR into the `conda-forge` Pinnings File
84+
85+
Right now, when the migration of say package `ABC` to version `X`
86+
from version `Y` is done, we do not automatically merge the change
87+
in the globally pinned value of `ABC` into our listing of global
88+
pinnings. We should be issuing a PR to the pinnings file once we
89+
have determined that a suitable fraction of the packages effected by
90+
a migration have been properly rebuilt.
91+
92+
**Issue**: <https://github.com/regro/cf-scripts/issues/595>
93+
94+
**Experience Level**: moderate
95+
96+
5. Fully Render `conda` Packages in order to Determine Migration
97+
Dependencies
98+
99+
Determining the dependencies of a given package is actually a
100+
computation expensive task due to the way `conda` recipes are
101+
structured and parametrized through the use of `Jinja2` and
102+
`conda-build-config.yaml` files. Currently, the autotick bot
103+
examines the static metadata in the `meta.yaml` file and not the
104+
fully rendered metadata. For this reason, we sometimes miss
105+
dependencies of a given package that need to be migrated first.
106+
Addressing this issue involves both calling the rendering process
107+
and also scaling that process to the entire set of `conda-forge`
108+
packages.
109+
110+
**Issue**: <https://github.com/regro/cf-scripts/issues/664>
111+
112+
**Experience Level**: moderate

0 commit comments

Comments
 (0)