|
| 1 | +## 19 December 2017 |
| 2 | + |
| 3 | + |
| 4 | + |
| 5 | +* [Volunteer for 1.10 release team!](https://github.com/kubernetes/sig-release/pull/49) |
| 6 | + |
| 7 | + |
| 8 | +## 7 November 2017 |
| 9 | + |
| 10 | +Agenda |
| 11 | + |
| 12 | + |
| 13 | + |
| 14 | +* Revisit release process automation discussion |
| 15 | + * Federation wants to release for 1.9 but is now in a separate repo |
| 16 | + * @david-mcmahon: I can’t make this time slot unfortunately, but please send me any docs or information on what exactly you're trying to achieve here. |
| 17 | + * My team is working on a few solutions to assist in releasing non-core Kubernetes components. Most don’t need the full orchestration that[ anago](https://github.com/kubernetes/release#kubernetes-release-process) provides for k/k. |
| 18 | + * See some related work[ here for cluster-registry](https://github.com/kubernetes/cluster-registry/compare/master...javier-b-perez:cloud-builder) building using Google Cloud Builder |
| 19 | + * Also the recent related[ update to push-build.sh](https://github.com/kubernetes/release/pull/444) to support federation pushes. |
| 20 | +* Can we apply[ milestone automation](https://github.com/kubernetes/community/blob/master/contributors/devel/release/issues.md) to[ PRs too](https://github.com/kubernetes/sig-release/issues/26)? [enisoc] |
| 21 | + |
| 22 | + |
| 23 | +## 24 October 2017 |
| 24 | + |
| 25 | +Agenda |
| 26 | + |
| 27 | + |
| 28 | + |
| 29 | +* [Code freeze enforcement](https://github.com/kubernetes/features/pull/493) |
| 30 | + |
| 31 | + |
| 32 | +## 10 October 2017 |
| 33 | + |
| 34 | +Agenda |
| 35 | + |
| 36 | + |
| 37 | + |
| 38 | +* Kops team would like to partner with sig-release to have kops release process automated. Draft outline:[ https://docs.google.com/a/cnmconsulting.net/document/ d/1i29ntaLjRVmQuUkUXfnkL3_jLby38At98kaE5VEJvg8/edit?usp=sharing](https://docs.google.com/a/cnmconsulting.net/document/d/1i29ntaLjRVmQuUkUXfnkL3_jLby38At98kaE5VEJvg8/edit?usp=sharing) |
| 39 | + * Want to partner with SIG Release |
| 40 | + * Focus on release mechanics |
| 41 | + * Where to put trusted keys? |
| 42 | + * Want to find common denominators around projects |
| 43 | + * General guidelines are important, and perhaps an end-state where artifacts get published |
| 44 | + * Define how we handle release artifacts and build backwards from there |
| 45 | + * Need to put something on the end of the test infra that does the push |
| 46 | + * Q: Who can make a call on where the bits in the registry live? \ |
| 47 | +A: |
| 48 | + * Notes |
| 49 | +* Reminder about[ release roles for 1.9](https://docs.google.com/spreadsheets/d/1NGEtufpSfYlJwsB_0D9gF80Y24JQEreut9gT_QIN6Kk/edit#gid=0) |
| 50 | + * solidify the roles |
| 51 | +* When/How do release roles get finalized? |
| 52 | + * Jaice will send announcement to begin timer on lazy consensus |
| 53 | +* Product Management reboot proposal[ draft](https://docs.google.com/document/d/113gnr3tKXv79J0IYHyg6VbQwLfb-sCYGphW4D9dLZ_E/edit#) |
| 54 | +* Updated milestone munger has been deployed |
| 55 | + * Next step is migrating to a prow plugin |
| 56 | + * Better maintainability |
| 57 | + * Enables slack notifications |
| 58 | + |
| 59 | + |
| 60 | +## 29 August 2017 |
| 61 | + |
| 62 | +Agenda - Recording |
| 63 | + |
| 64 | + |
| 65 | + |
| 66 | +* Attending |
| 67 | + * Jaice Singer DuMars |
| 68 | + * Radhika |
| 69 | + * Jennifer Rondeau |
| 70 | + * Mehdy Bohlool |
| 71 | + * Caleb Miles |
| 72 | + * Aaron Crickenberger |
| 73 | + |
| 74 | +Notes: |
| 75 | + |
| 76 | + |
| 77 | + |
| 78 | +* 1.8 Release update |
| 79 | + * [Issues](http://bit.ly/18issues) |
| 80 | + * [Release notes](http://bit.ly/18relnotes) |
| 81 | + * [Feature tracking](http://bit.ly/18features) |
| 82 | + * [Tests](http://k8s-testgrid.appspot.com/release-master-blocking#Summary) |
| 83 | + * Anything else we need to be tracking |
| 84 | + * Upgrade tests are failing - Eric and Mehdy will look at these |
| 85 | + * Serial tests are looking mostly ok |
| 86 | + * SIGs are not checking off their release note additions, so we need to closely monitor this |
| 87 | + * Issue automation for milestone labeling is merged, needs an announcement |
| 88 | + * Getting a Prow plugin to limit who can apply “approved-for-milestone” label, akin to LGTM controls |
| 89 | + * Use of[ OWNERS_ALIASES](https://github.com/kubernetes/kubernetes/blob/master/OWNERS_ALIASES) might be fraught with peril |
| 90 | + * Short term, we may not need auditability as a primary concern, and will defer to teams |
| 91 | + * Aaron: SIG Release creates a GitHub team for this purpose, specifically |
| 92 | + * Having a limited entry point into the team may increase security around the implementation |
| 93 | + * Notify: kubernetes-dev, kubernetes-sig-leads, kubernetes-sig-release, Community meeting |
| 94 | + * Caleb will help Maru create the team |
| 95 | + * What are the true release-blocking tests? How do we actually get a 100% passing version |
| 96 | + * A release-blocking job that only runs X days may not be useful |
| 97 | + * More frequent runs may be required, and ergo removed from required if it can’t be |
| 98 | + * Currently many failing GKE/GCE jobs |
| 99 | + * Need to establish accountability |
| 100 | + * Guidelines around “how long can something block the submit queue”? |
| 101 | + * Triage + plan to resolution / project-wide SLA = policy |
| 102 | + * Who actually owns these tests? |
| 103 | + * Jaice will draft _something_ here |
| 104 | + * |
| 105 | +* Announcements? |
| 106 | + * |
| 107 | + |
| 108 | + |
| 109 | +## 15 August 2017 |
| 110 | + |
| 111 | +Agenda - Recording |
| 112 | + |
| 113 | + |
| 114 | + |
| 115 | +* Attending |
| 116 | + * Aaron Crickenberger |
| 117 | + * Jaice Singer DuMars |
| 118 | + * Phillip Wittrock |
| 119 | + * Mehdy Bohlool |
| 120 | + * Steve Perry |
| 121 | + * Jennifer Rondeau |
| 122 | + * Radhika |
| 123 | + * Caleb Miles |
| 124 | + * Garrett Rodrigues |
| 125 | + * Brian Tardell |
| 126 | + * Ryan Kubiak |
| 127 | + * David McMahon |
| 128 | + |
| 129 | +Notes |
| 130 | + |
| 131 | + |
| 132 | + |
| 133 | +* 1.8 Release update |
| 134 | + * Feature freeze exceptions |
| 135 | + * [https://github.com/kubernetes/features/issues/382](https://github.com/kubernetes/features/issues/382) |
| 136 | + * [https://github.com/kubernetes/features/issues/384](https://github.com/kubernetes/features/issues/384) |
| 137 | + * [https://github.com/kubernetes/features/issues/383](https://github.com/kubernetes/features/issues/383) |
| 138 | + * [https://github.com/kubernetes/features/issues/386](https://github.com/kubernetes/features/issues/386) |
| 139 | + * [https://github.com/kubernetes/features/issues/387](https://github.com/kubernetes/features/issues/387) |
| 140 | + * Jaice would be deferent to features which are already in progress |
| 141 | + * Jaice to take ownership of the PRs with Ihor and see what their status is ~ _Done as of 8/16 JSD_ |
| 142 | + * **Question**: how does the feature freeze impact the release cadence? **<-** **Jaice **Idea is to have some idea what work is planned before coding starts. Are these features “funded” with development efforts. Ideally we would have a release planning session where SIGs present what work is scheduled, what cross SIG dependencies exist, and SIGs would be accountable to their commitments |
| 143 | + * [https://github.com/kubernetes/features/blob/master/release-1.8/release-1.8.md](https://github.com/kubernetes/features/blob/master/release-1.8/release-1.8.md) |
| 144 | + * Still no alphas cut? |
| 145 | + * Should be three already. Is this a risk? |
| 146 | + * Jaice: it is a risk, we have not have met the criteria for releasing with regards to CI |
| 147 | + * Caleb: I believe this is |
| 148 | + * Grod: next (last) alpha is scheduled for 23rd August. We should aim to have all tests passing by then. Suggest using 23 August as hard deadline for all tests passing |
| 149 | + * Thur 17 plan to cut an alpha - even if tests aren’t passing |
| 150 | + * Meant to validate release cutting infra is correct |
| 151 | + * Need to find a branch manager for weeks Adam is out |
| 152 | +* [Merged PRs in the 1.8 milestone](https://docs.google.com/spreadsheets/d/18V_8YvqemC96W17O_3WZ6H_kS7r77NAI9FqUHAM2MS0/edit?usp=sharing) with release-note=true |
| 153 | + * [GitHub query](https://github.com/kubernetes/kubernetes/pulls?utf8=%E2%9C%93&q=is%3Apr%20is%3Amerged%20label%3Arelease-note%20milestone%3Av1.8) |
| 154 | +* Dawn’s proposal lives! Jago hiring 2 people to work on the coding |
| 155 | + * Strive to link issues |
| 156 | + * How do we get more support for all the back end processes/bots/CI? |
| 157 | + * |
| 158 | +* [Docs PRs](https://github.com/kubernetes/features/pulls) in flight |
| 159 | + * Radhika sending an email to each group, then collect, edit and apply tech writer magic sprinkles |
| 160 | + * re: action required, need to untangle release note PRs |
| 161 | + * Better release note template might help |
| 162 | + * 2 classes of release notes: |
| 163 | + * Items in the k/features |
| 164 | + * items in PRs against k/k ~ not organized … |
| 165 | +* Planning for roll-out of milestone-maintaining munger |
| 166 | + * Proposal:[ https://github.com/kubernetes/community/blob/master/contributors/devel/release/issues.md](https://github.com/kubernetes/community/blob/master/contributors/devel/release/issues.md) |
| 167 | + * PR:[ https://github.com/kubernetes/test-infra/pull/4052](https://github.com/kubernetes/test-infra/pull/4052) |
| 168 | + * If you don’t label it sufficient for triage, it warns, then moves out of the milestone |
| 169 | + * priority, type, SIG |
| 170 | + * 3 day grace period + pings to GH UIDs (no email currently) |
| 171 | + * Need to send email to all the SIGs / present at community meeting |
| 172 | + * For critical/urgent they warn without kicking out |
| 173 | + * labels will allow us to track issues that get ejected |
| 174 | + * Initial grace period could be 1 weeks - Enough to give SIGs a chance to take action |
| 175 | + * This is step one, but the proposal has additional work items |
| 176 | + * Is there a notion of SIGs acknowledging ownership? |
| 177 | + * Maru needs a cool hat |
| 178 | + |
| 179 | + |
| 180 | +## 1 August 2017 |
| 181 | + |
| 182 | +Agenda - Recording |
| 183 | + |
| 184 | + |
| 185 | + |
| 186 | +* Attending |
| 187 | + * Jaice Singer DuMars |
| 188 | + * Erick Fejta |
| 189 | + * David McMahon |
| 190 | + * Aaron Crickenberger |
| 191 | + * Phillip Wittrock |
| 192 | + * Dan Gillespie |
| 193 | + * Joe Betz |
| 194 | + * Steve Perry |
| 195 | + * Maru Newby |
| 196 | + * Caleb Miles |
| 197 | + |
| 198 | +Notes: JSD |
| 199 | + |
| 200 | + |
| 201 | + |
| 202 | +* 1.8 Release update |
| 203 | + * Feature freeze status check |
| 204 | + * Any issues with no/wrong SIG |
| 205 | + * This is a “stability” release, so are there things that we see in the feature queue that should be deferred? |
| 206 | + * Does anyone know of notable missing features? |
| 207 | + * Release team roll call |
| 208 | + * [https://github.com/kubernetes/features/blob/master/release-1.8/release_team.md](https://github.com/kubernetes/features/blob/master/release-1.8/release_team.md) |
| 209 | + * Release risks |
| 210 | + * Upgrade testing |
| 211 | + * Issue inclusion process ~_ need a way to decide what a substantial PR is for the contrib ladder_, also ensure that PRs are properly documented for release notes |
| 212 | + * Is Dawn’s proposal being staffed? |
| 213 | + * Work to extract release notes from the issue has not been completed |
| 214 | + * This part happens in[ relnotes](https://github.com/kubernetes/release/blob/master/relnotes) and I can do this [@david-mcmahon] |
| 215 | + * Need to ensure there’s a clear value prop for the requirement |
| 216 | + * Phil’s proposal needs some dry run |
| 217 | +* kops test failures and escalation of[ this](https://github.com/kubernetes/kubernetes/issues/49981) |
| 218 | + * escalate to Justin SB / Kris Nova / Luis |
| 219 | + * kops is unique in that it’s fully owned |
| 220 | + * Test fails because of X and is owned by SIG Y, Z, B |
| 221 | + * Ownership gets dropped so it’s difficult to get traction |
| 222 | + * Release team has to find someone to fix it, which can be circular |
| 223 | + * When low-level or high-impact tests fail, hard to know where to send this |
| 224 | + * etcd testing specifically is tricky to understand |
| 225 | + * umbrella issue for putting [sig-foo] in e2e test names ~ on Aaron’s backlog + Phillip Wittrock |
| 226 | + * [https://github.com/kubernetes/kubernetes/issues/49161](https://github.com/kubernetes/kubernetes/issues/49161) |
| 227 | + * |
| 228 | + * Escalations for SIGs: mailing list / Slack / Individuals |
| 229 | + * Put ACTION REQUIRED in the subject or IMPORTANT |
| 230 | + * SIG rep for release if you have a SIG-owned issue or feature in the release ( if ! sig-lead ; then someone else) |
| 231 | + * |
| 232 | +* upgrade and serial tests - massive failures [pwittrock] |
| 233 | + * Upgrade - Mehdy Bohlool? |
| 234 | + * These have been highly problematic in past releases |
| 235 | + * Serial - ??? |
| 236 | + * Must pass before a release is cut currently, but not blocking |
| 237 | + * Could SIG merge be blocked by failing serial test? |
| 238 | + * SIG-specific tests are the only way to validate a SIG’s contribution, so if it’s not fixed, then merges for code needing its validation should be blocked |
| 239 | + * Break up serial jobs and apply them to a cluster that is not subject to prior changes |
| 240 | + * Write this up so we can get feedback ASAP [ Maru & Phillip ] |
| 241 | + * Execution frequency would be lowered |
| 242 | +* etcd3 test flakes [pwittrock] |
| 243 | +* [Hooking in deb/rpm publishing in anago](https://github.com/kubernetes/release/issues/365) [calebamiles] |
| 244 | + * A/B/RC/Final |
| 245 | + * Needs someone from Google because of the bucket (Mike Danese has done this up to now) |
| 246 | + * Permissions model may be interfering |
| 247 | + * Anago is OSS so someone could update it, but there may be some tripwire around Google accounts vs. not |
| 248 | + * Caleb will connect with mikedanese@ (@mikedanese) to get a walkthrough |
| 249 | + * Also reach out to djmm@ (@david-mcmahon) for any details on the included google layer that does google-specifics (push) |
| 250 | + |
| 251 | + |
| 252 | +## 18 July 2017 |
| 253 | + |
| 254 | +Agenda -[ Recording](https://youtu.be/I0KbWz8MTMk) |
| 255 | + |
| 256 | + |
| 257 | + |
| 258 | +* Attending |
| 259 | + * Jaice Singer DuMars |
| 260 | + * Phillip WIttrock |
| 261 | + * Caleb Miles |
| 262 | + * Dan Gillespie |
| 263 | + * Aaron Crickenberger |
| 264 | + * |
| 265 | +* Notes: JSD |
| 266 | +* 1.8 Release Team [DG] |
| 267 | + * Empty |
| 268 | + * Aaron willing to help but cannot commit to a role full-time |
| 269 | + * Assisted with issue triage and pushing PR’s forward last time |
| 270 | + * Trying to highlight escalation paths |
| 271 | + * Companies should be funding a role, especially in terms of commitment to the project |
| 272 | + * Going to advertise for more role involvement |
| 273 | + * At community meeting we will announce that we have a tentative lead/second & ask (final) if there are any other nominations |
| 274 | + * Also posting to SIG release list |
| 275 | + * Alpha release stability needs to be understood - no more releases without green tests |
| 276 | + * Phil is going to check on Google-specific roles |
| 277 | +* Establishing release criteria / standards / due dates [DG] |
| 278 | + * **Set[ release due dates](https://github.com/kubernetes/features/pull/305) for all of these items**: (merge[ https://github.com/kubernetes/features/pull/305](https://github.com/kubernetes/features/pull/305) ) |
| 279 | + * CI test signal |
| 280 | + * Which suites/jobs are required? |
| 281 | + * As of 1.7 anago polls testgrid’s config.yaml for release blocking jobs;[ issue for context](https://github.com/kubernetes/release/issues/340) |
| 282 | + * Anago polling testgrid’s config.yaml[ https://github.com/kubernetes/release/blob/master/lib/releaselib.sh#L115](https://github.com/kubernetes/release/blob/master/lib/releaselib.sh#L115) |
| 283 | + * The specific set of tesjobsts from config.yaml[ https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml#L2036-L2119](https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml#L2036-L2119) |
| 284 | + * Of note: upgrade jobs are not included in this list |
| 285 | + * Of velocity note: slow jobs are |
| 286 | + * What tolerance for flakes do we expect? |
| 287 | + * ie. 5 passes in a row on a commit we are releasing |
| 288 | + * In the past, we as humans have used “3 passes in a row” |
| 289 | + * Anago appears to look at 2 passes in a row if[ this comment in find_green_build](https://github.com/kubernetes/release/blob/2338077e3911ad2ede6473b2777ead38e9edf5b0/find_green_build#L32) is to be believed |
| 290 | + * When in the release cycle should this be stabilized? |
| 291 | + * Prior to cutting a build/release |
| 292 | + * Need to determine how long we support version-specific test cases, e.g. etcd2 upgrade path, beta upgrades ~ what SIG owns upgrade lifecycles? (SIG Cluster Lifecycle?) |
| 293 | + * API Policy |
| 294 | + * Component upgrades |
| 295 | + * Management of failing tests |
| 296 | + * Need to raise this in community and get test-owner reps in the release meetings |
| 297 | + * Upgrade test signal |
| 298 | + * Which upgrade conditions do we expect to validate? |
| 299 | + * Multi-master? |
| 300 | + * etcd version? |
| 301 | + * When in the release cycle should this be stabilized? |
| 302 | + * Release Notes |
| 303 | + * What format are they expected to be in? |
| 304 | + * When in the release should this be complete? |
| 305 | + * By the time feature freeze happens, we should have enough to create the PM-oriented “theme release notes” |
| 306 | + * Need to enact the ‘marketing’ and ‘release notes’ roles |
| 307 | + * A draft of the notes by code freeze will be required |
| 308 | + * Call out features with no docs |
| 309 | + * Call out features with no release notes |
| 310 | + * Cherry picks are hard, and porting info across PRs is difficult |
| 311 | + * Maybe this is not in the PR, and lives somewhere else |
| 312 | + * Documentation |
| 313 | + * When in the release should this be complete? |
| 314 | + * Bugs |
| 315 | +* **Action items summary from this meeting:** |
| 316 | + * ~~Raise awareness (again) about release roles~~ (@ community and @ SIG-Release Google Group) ~ **Jaice & Caleb** |
| 317 | + * **~~PWittrock to seek out Googlers for G-specific release roles~~** [ ~~branch manager~~ | patch release manager ] |
| 318 | + * ~~Give attention to Community PR 305 and ratify on Friday~~ ~ **SIG Release team** |
| 319 | + * ~~Schedule follow-up meeting for Friday at 2PM PT / 5PM ET / 9PM UTC~~ ~ **Caleb or Dan** |
| 320 | + * Determine if/how an Alpha is possible ~ **SIG Release team** |
| 321 | + * ~~Use Anago test list as the source of truth for release-blocking tests~~ ~ **Aaron** will provide this info |
| 322 | + |
| 323 | + |
| 324 | +## 6 June 2017 |
| 325 | + |
| 326 | +Agenda - Recording |
| 327 | + |
| 328 | + |
| 329 | + |
| 330 | +* Attending: |
| 331 | + * Jaice Singer DuMars |
| 332 | + * Anthony Yeh |
| 333 | + * Ihor Dvoretskyi |
| 334 | + * Maru Newby |
| 335 | + * Caleb Miles |
| 336 | + * Doug Claar |
| 337 | + * Javier Perez |
| 338 | + * David McMahon |
| 339 | + * Andrew Chen |
| 340 | + * Phil Wittrock |
| 341 | +* Notes: JSD |
| 342 | +* [ 0:01 ] What are we hoping to achieve from this meeting? |
| 343 | + * How do we make long-term improvements to the release process? |
| 344 | + * How do we empower teams to do more of the release tasking themselves? |
| 345 | + * Different than the work cutting releases |
| 346 | + * Iterating and improving on every release as a group |
| 347 | + * Release team forms/disbands around the release so not a durable group |
| 348 | + * Tools we need in order to improve the process, vs. release team uses current tool set |
| 349 | +* [ 0:09 ] Updates |
| 350 | + * SIG CLI was working on a page in the test grid, filtered for easy understanding |
| 351 | + * [1.6 Retro Action Items](https://docs.google.com/document/d/1JjQW14R9FCaKceZV2hE9Wx41LIUoZST8gp8FF5EqMmk/edit) |
| 352 | + * We need a measurable goal |
| 353 | + * Do a mock release weekly to verify the process and testing [ Phil leading this ] |
| 354 | + * What are the tools needed to make this happen? |
| 355 | + * Start with a PR for discussing how to enact this work |
| 356 | + * Release team retro [ Jaice ] |
| 357 | + * ID bugs and issues we already know about [ ID targets for continuous improvement ] |
| 358 | + * Schedule meeting |
0 commit comments