Skip to content

Conversation

parkan
Copy link
Collaborator

@parkan parkan commented Oct 16, 2025

(cleaned up version of #585 -- the history got so hairy that internal rebasing broke on that branch)

MOTIVATION

the workflows in this repo were historically based on PL templates/work patterns, with complex matrix builds, multi-container orchestration, and a huge number of action dependencies

this resulted in repeated breakage due to actions changing, versions not being pinned fully/going out of sync, new golang releases changing linting rules, etc

numerous efforts were made to get that CI workflow working reliably (#443, #436, #467, #581, etc) but ultimately this all feels like threading water at best

to make matters worse, testing the workflows locally is a giant headache because even act doesn't provide an isomorphic environment to what happens on the runners and is not feature complete; normal make test output diverges heavily between local dev and CI

PROPOSAL

  • use devcontainers for reproducible, isomorphic workflows
  • use go.mod as source of truth for go version as much as possible (already implemented)
  • run DB backends inside devcontainer, lock versions at build, avoid orchestration and IPC issues
  • run all actions that don't touch the releaser flow inside the devcontainer (much cheaper invocations, single container)

NOTE: we're using https://github.com/parkan/github-actions to run a rootless podman devcontainer because:

  • the official microsoft devcontainer ci actions are shockingly broken and don't match the local devcontainer cli/vscode behavior
  • I run a podman dev environment locally and see no reason to use legacy docker over rootless OCI, however my actions support docker if desired

NOTES/REMAINING

  • the releaser workflows still use the old paths; they are a bit brittle but seem to be working at the moment so I'm going to leave them alone
  • branch protection rules reference the removed workflows, we should repoint them to this one
  • probably needs a better name

- initial devcontainer configuration
- postgres and mysql database initialization scripts
- database startup scripts and permission fixes
- password and connectivity configuration
- workarounds for volume relabeling issues
- podman-specific configurations
- devcontainer re-engineering for podman compatibility
- missing scripts and permission fixes
- mysql user and permission fixes
- database role creation
- test database setup
- job selection logic fix
- suppress missing table warnings in tests
- disable dag generation during duplicate piece test
- resolve record changed since last read errors
- use v2 action
- reorder steps for fast fail
- remove stray files
@parkan parkan force-pushed the local/devcontainer-rebased branch from 934cfa8 to ed21663 Compare October 16, 2025 13:14
@parkan parkan force-pushed the local/devcontainer-rebased branch from ee6e85f to ccfaccd Compare October 16, 2025 14:43
@parkan parkan force-pushed the local/devcontainer-rebased branch from ccfaccd to cf692f7 Compare October 17, 2025 12:53
Copy link
Contributor

@lanzafame lanzafame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Minor comment on removing some of the old test util functions.

// GenerateUniqueName creates a unique name for testing by combining a prefix with a random suffix
// GenerateUniqueName creates a unique name for testing by combining a prefix with a UUID suffix
func GenerateUniqueName(prefix string) string {
return prefix + "-" + RandomLetterString(8) + "-" + RandomLetterString(4)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parkan Are we able to get rid of the RandomLetterString function while we are at it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants