feat: integrate automated PyPI oldest-release cleanup into wheels publishing workflow #364

Omar8345 · 2025-11-14T14:51:43Z

Description

This PR adds an automated cleanup step to the existing ci-wheels.yml workflow to ensure PyPI does not hit its version limit during publishing.

The cleanup script (Mediar Workflow) logs into PyPI's web UI and deletes the oldest release right before the terminator package is published. This prevents failed uploads when too many versions accumulate.

Key changes

Added TypeScript cleanup script under .github/scripts/pypi-cleanup
Added Node + Workflow setup steps inside ci-wheels.yml
Added cleanup execution step before the PyPI publish job

Added required GitHub Actions secrets

PYPI_UI_USERNAME — PyPI account username for UI login
PYPI_UI_PASSWORD — PyPI account password
PYPI_UI_COOKIES — Cookies from a 2FA authenticated session with Remember me on (must be updated every 30d)

example:

{
  "user_id__insecure": "d***********",
  "session_id": "d***********"
}

Video Demo

PyPI.Cleanup.TS.Workflow.Demo.mov

AI Review & Code Quality

I asked AI to critique my PR and incorporated feedback
I formatted my code properly
I tested my changes locally

Checklist

Code follows project style guidelines
Added video demo (recommended)
Updated documentation if needed

Additional Notes

/claim #357
Closes #357

cc @louis030195 for review 🙏

… release

Excellencedev

@Omar8345 This looks good but:

I think the workflow is supposed to use mediar-ai/workflow for the workflow instead of chromium etc
The workflow should have different steps in different files

Omar8345 · 2025-11-14T15:48:05Z

@Omar8345 This looks good but:

I think the workflow is supposed to use mediar-ai/workflow for the workflow instead of chromium etc

The workflow should have different steps in different files

Hiya @Excellencedev!

Thanks for the review!
I think there may be a misunderstanding — Louis’s request was for a TS Playwright workflow that deletes the oldest PyPI release via the UI.

The mediar-ai/workflow jobs (afaik) needs an active display and UI. While my solution can run on minimal CI runners without a UI/display stack, so that solution can't be working perfectly. Even if we tried, the cleanup step wouldn’t work there.

Excellencedev

Not a maintainer. but LGTM !

louis030195 · 2025-11-14T16:00:09Z

no i didnt ask for playwright ... missign the whole point, use terminator / workflow sdk

Excellencedev · 2025-11-14T16:08:33Z

no i didnt ask for playwright ... missign the whole point, use terminator / workflow sdk

@Omar8345 I was right

Omar8345 · 2025-11-14T16:49:11Z

no i didnt ask for playwright ... missign the whole point, use terminator / workflow sdk

@louis030195 but that can't run on GH actions, can it?

can you provide me with a more technical approach and what is exactly to be done, the issue doesn't have enough info

so I should create a new use case in the examples folder, like #360

louis030195 · 2025-11-14T23:58:57Z

gh action have UI

Excellencedev · 2025-11-15T01:27:53Z

@Omar8345 check this too to see if it is working in github actions https://github.com/nektos/act

Omar8345 · 2025-11-15T03:37:46Z

@Omar8345 check this too to see if it is working in github actions https://github.com/nektos/act

thanks!

@louis030195

so I should create a new use case in the examples folder, like #360

should i use like browser extension or do ? maybe can u give me general approach i can move on
should i do like simple_notepad_workflow?

Omar8345 · 2025-11-15T06:27:27Z

gm @louis030195 and @Excellencedev

While working on the PR, I faced this issue that it can't run on Linux:

❌ PyPI login failed: Linux implementation is not yet available
❌ Step failed: Login to PyPI (0ms)
   Error: Linux implementation is not yet available
❌ Step failed: Linux implementation is not yet available
{
  "status": "error",
  "lastStepId": "login-pypi",
  "lastStepIndex": 1,
  "error": "Linux implementation is not yet available"
}

execute: async ({ desktop, input, logger }: any) => {
    logger.info("🔐 Logging into PyPI...");

    try {
      await desktop.navigateBrowser("https://pypi.org/account/login/");
      await desktop.wait(2000);
      // ...

are we supposed to have a macos/windows image on the GitHub actions workflow instead?

Or write a script that simulates the login form and CSRF tokens manually, which is more brittle and will be a mess in maintenance.

Excellencedev · 2025-11-15T06:36:01Z

gm @louis030195 and @Excellencedev

While working on the PR, I faced this issue that it can't run on Linux:
❌ PyPI login failed: Linux implementation is not yet available
❌ Step failed: Login to PyPI (0ms)
   Error: Linux implementation is not yet available
❌ Step failed: Linux implementation is not yet available
{
  "status": "error",
  "lastStepId": "login-pypi",
  "lastStepIndex": 1,
  "error": "Linux implementation is not yet available"
}
execute: async ({ desktop, input, logger }: any) => {
    logger.info("🔐 Logging into PyPI...");

    try {
      await desktop.navigateBrowser("https://pypi.org/account/login/");
      await desktop.wait(2000);
      // ...
are we supposed to have a macos/windows image on the GitHub actions workflow instead?

Or write a script that simulates the login form and CSRF tokens manually, which is more brittle and will be a mess in maintenance.

@Omar8345 is it in ci or locally ?

Omar8345 · 2025-11-15T06:37:38Z

@Excellencedev i tried locally on my macOS M2 using the tool you shared, but it didn't work since there is no build for Linux nor even for my M2 (ARM). The error shared was on Linux (GH Actions)

Excellencedev · 2025-11-15T06:49:06Z

gm @louis030195 and @Excellencedev
While working on the PR, I faced this issue that it can't run on Linux:
❌ PyPI login failed: Linux implementation is not yet available
❌ Step failed: Login to PyPI (0ms)
   Error: Linux implementation is not yet available
❌ Step failed: Linux implementation is not yet available
{
  "status": "error",
  "lastStepId": "login-pypi",
  "lastStepIndex": 1,
  "error": "Linux implementation is not yet available"
}
execute: async ({ desktop, input, logger }: any) => {
    logger.info("🔐 Logging into PyPI...");

    try {
      await desktop.navigateBrowser("https://pypi.org/account/login/");
      await desktop.wait(2000);
      // ...
are we supposed to have a macos/windows image on the GitHub actions workflow instead?
Or write a script that simulates the login form and CSRF tokens manually, which is more brittle and will be a mess in maintenance.
@Omar8345 is it in ci or locally ?
@Excellencedev i tried locally on my macOS M2 using the tool you shared, but it didn't work since there is no build for Linux nor even for my M2 (ARM). The error shared was on Linux (GH Actions)

It's an issue with them. Right now, you have to

Switch to mediar ai/workflow and not playwright etc
Run it again to see if it works
Once done with those 2 you will request review from @louis030195 and he will approve workflow
Then you will know if it works in ci or not

Omar8345 · 2025-11-15T06:53:11Z

@Excellencedev

I did, and the issue I just shared is from my implementation using mediar workflow SDK
It is just that Terminator workflow doesn't have support for Linux yet, only Windows and macOS
let's see his reply, maybe we will have to approve the PR and wait for official support OR use windows for example in the workflow OR settle for Playwright solution
I can't test locally, on macOS directly (arch issue), and the tool you shared (no Linux support)

Omar8345 · 2025-11-15T13:22:35Z

Hey @louis030195 — quick update so we stay aligned:

I’ve switched the implementation to the Terminator Workflow SDK, but the workflow cannot run on Linux because the SDK currently has no Linux implementation.
This is the exact CI error:

❌ Linux implementation is not yet available

I also tried testing on my local macOS M2 machine, but the workflow binaries aren’t available for ARM either — so I cannot test locally. (maybe I missed something to install, if it works with ARM, please let me know to check the method once more, but we are thinking more of GitHub Actions compatibility right now)

I’ve put a lot of work into getting this running properly, and I want to avoid any delays with the bounty, so here are the practical options now:

Run the cleanup workflow on windows-latest
→ This should work with the Terminator Workflow SDK and avoids Linux limitations.
→ We can trigger it from the main Linux pipeline as a dependent job.
Wait for official Linux support in the Terminator SDK
→ This would delay the bounty, so not ideal.
Fallback to the Playwright implementation
→ It already works reliably on Linux CI, but I understand it's not the approach you originally asked for.

I’m committed to finishing this professionally — just let me know which direction you prefer, and I’ll update the PR right away. 🙏

divanshu-go · 2025-11-15T14:34:56Z

@Omar8345 only Windows runners like windows-latest support unrestricted UI automation, so that’s the right target. Windows is also the current priority, so we should switch the workflow to that.

Omar8345 · 2025-11-16T07:19:22Z

@louis030195 @Excellencedev @divanshu-go Hi all,

I discovered a new issue, due to GitHub CI logging in from unknown IPs, PyPI tends to send emails to open link to allow access even though 2FA is enabled and passed, should this be done manually or what?

Excellencedev · 2025-11-16T07:26:54Z

@louis030195 @Excellencedev @divanshu-go Hi all,

I discovered a new issue, due to GitHub CI logging in from unknown IPs, PyPI tends to send emails to open link to allow access even though 2FA is enabled and passed, should this be done manually or what?

@Omar8345 disable 2FA on your pypi account

Omar8345 · 2025-11-16T07:28:45Z

@louis030195 @Excellencedev @divanshu-go Hi all,
I discovered a new issue, due to GitHub CI logging in from unknown IPs, PyPI tends to send emails to open link to allow access even though 2FA is enabled and passed, should this be done manually or what?

@Omar8345 disable 2FA on your pypi account

really, man i only got one question, what will we have if we remove it, you can't publish pkgs without it, its mandatory, 2FA isn't the problem, that's fixed, the issue is, even if you supplied 2FA code, it sends email

Excellencedev · 2025-11-16T07:32:58Z

@Omar8345 test it yourself and see

Omar8345 · 2025-11-16T07:34:15Z

required

Excellencedev · 2025-11-16T07:46:54Z

@Omar8345 you hae already enabled 2FA before. It sais once it is enabled, it cannot be disabled

Omar8345 · 2025-11-16T07:49:41Z

@Omar8345 you hae already enabled 2FA before. It sais once it is enabled, it cannot be disabled

i beg u pls read

Excellencedev · 2025-11-16T07:56:34Z

Give me the link

Omar8345 · 2025-11-16T07:58:01Z

https://pypi.org/help/#:~:text=Two%2Dfactor%20authentication%20is%20required%20on%20your%20PyPI%20account.

Omar8345 · 2025-11-16T08:00:41Z

issue is that link for auth needs to somehow get to the CI vm

Excellencedev · 2025-11-16T08:07:44Z

@Omar8345 as you can see, 2FA is NOT enabled on my account

Omar8345 · 2025-11-16T08:08:57Z

@Omar8345 as you can see, 2FA is NOT enabled on my account

try publishing a throwaway package

Excellencedev · 2025-11-16T08:21:05Z

sorry @Omar8345
I now see why it won't work
reach out to @louis030195
i'm not the bounty or issue creator

Omar8345 · 2025-11-17T01:59:19Z

@louis030195

louis030195 · 2025-11-17T19:00:22Z

does it work, why there is 100000 comments

Omar8345 · 2025-11-18T03:22:14Z

mind reading for once? we got it to work, partially, the only problem we face right now even after doing 2FA successfully, it asks for email verification to continue, idk what do u think

Omar8345 · 2025-11-21T09:13:06Z

@louis030195

divanshu-go · 2025-11-21T15:58:32Z

@Omar8345 Could you please run the workflow locally (not through GitHub workflows) and provide a short recording demonstrating it functioning end-to-end? This will give us clear verification of the implementation and help move the review forward efficiently.

Omar8345 · 2025-11-22T05:22:23Z

@divanshu-go @louis030195

Just an update: I hit a hard blocker with PyPI’s email-verification step, which still triggers even after bypassing 2FA. That makes the workflow impossible to automate as-is, and I’m waiting for clear instructions on how you want this handled.

Two options from my side:

wait for the extension-installation workflow to be properly supported, or
I install the extension myself via scripted browser actions and rely on a persistent session (“remember me for 30 days”) so cookies stay valid and we avoid 2FA + email loops.

Either way, we do need reliable cookies — one approach or the other.

Alternatively, we can automate email handling using one of these approaches:

Gmail — we can set up a dedicated Gmail account or use the original account, and use the Gmail API with a refresh token. This allows the workflow to fetch verification emails programmatically, without any human intervention, on any VM or CI runner.
Mailgun — we can create a dedicated Mailgun inbox using their domain (even the free domain) and fetch inbound emails via their API (no OAuth required) to extract the verification link automatically. Setup is simpler than Gmail and works reliably in CI.

There’s also the option of a dedicated dummy PyPI account (with permission to delete releases) tied to whichever email solution we choose, so the workflow can fully automate verification and release deletion.

Pick one or specify another direction; otherwise I’m stuck here.

Omar8345 · 2025-11-30T13:31:19Z

@divanshu-go @louis030195

Just an update: I hit a hard blocker with PyPI’s email-verification step, which still triggers even after bypassing 2FA. That makes the workflow impossible to automate as-is, and I’m waiting for clear instructions on how you want this handled.

Two options from my side:

wait for the extension-installation workflow to be properly supported, or

I install the extension myself via scripted browser actions and rely on a persistent session (“remember me for 30 days”) so cookies stay valid and we avoid 2FA + email loops.

Either way, we do need reliable cookies — one approach or the other.

Alternatively, we can automate email handling using one of these approaches:

Gmail — we can set up a dedicated Gmail account or use the original account, and use the Gmail API with a refresh token. This allows the workflow to fetch verification emails programmatically, without any human intervention, on any VM or CI runner.

Mailgun — we can create a dedicated Mailgun inbox using their domain (even the free domain) and fetch inbound emails via their API (no OAuth required) to extract the verification link automatically. Setup is simpler than Gmail and works reliably in CI.

There’s also the option of a dedicated dummy PyPI account (with permission to delete releases) tied to whichever email solution we choose, so the workflow can fully automate verification and release deletion.

Pick one or specify another direction; otherwise I’m stuck here.

@m13v @louis030195 pls read and reply

in addition, please check screenpipe/screenpipe#1970 its been like 2 months

Omar8345 · 2025-12-05T06:20:40Z

@louis030195 @m13v

hm? fine.

TL;DR: The whole workflow is stuck on PyPI’s email-verification step. To unblock this, we need an automated way to read the verification emails. Options:

Gmail API (original or dedicated Gmail): use a refresh token so CI can fetch emails automatically on any VM.
Mailgun (simpler): create an inbox on their free domain and pull emails via API—no OAuth, works cleanly in CI.
We can attach this to either the main PyPI account or a dummy one with delete permissions.

Pick one so I can finally move forward.

Omar8345 · 2025-12-14T18:21:40Z

@louis030195 @m13v if you dont want the PR at least say so and tip me for my effort

Omar8345 · 2025-12-20T19:32:50Z

@m13v @louis030195 you closed issue as completed? and when I got a blockage when I was 90% of the way you ignore and don't reply? this is not fair at all? like you didn't apologize or give me a reply or neither compensate me not even partially?

this just doesn't feel fair, if you think everything's normal, i'm telling you it is the opposite 👍

Omar8345 added 3 commits November 14, 2025 18:42

feat(pypi-cleanup): add TypeScript project scaffold and config

81d76db

feat(pypi-cleanup): implement Playwright script to delete oldest PyPI…

f545cde

… release

ci(wheels): integrate PyPI cleanup step before publishing

7f53a91

algora-pbc bot added the 🙋 Bounty claim label Nov 14, 2025

algora-pbc bot mentioned this pull request Nov 14, 2025

create a TS workflow that delete the oldest version of PyPI before publishing to PyPI #357

Closed

Excellencedev suggested changes Nov 14, 2025

View reviewed changes

Excellencedev approved these changes Nov 14, 2025

View reviewed changes

Omar8345 added 7 commits November 15, 2025 09:48

feat(pypi-cleanup): use terminator sdk for workflow

6fb4992

Delete README.md

f4bd8ba

Update pypi-cleanup-example.yml

a3f4861

Update ci-wheels.yml

9111a48

install accessibility stack for workflow

a47d0c3

fix misplaced steps

10eca26

fix func calls

e88e254

Omar8345 added 2 commits November 15, 2025 19:35

switch workflow to windows latest

9dc15cf

replace desktop.wait with desktop.delay

4896a30

Omar8345 added 3 commits November 16, 2025 11:03

debug

3b9aca8

Update pypi-cleanup-example.yml

abc96fa

remove ctrl+a

880a2d1

Fix

16f02bd

Update 03-navigate-release.ts

f6ff7b5

m13v force-pushed the main branch from d4d30a7 to 6c374a4 Compare November 25, 2025 04:03

Merge branch 'mediar-ai:main' into feat/pypi-auto-cleanup

3abe8ab

feat: integrate automated PyPI oldest-release cleanup into wheels publishing workflow #364

Are you sure you want to change the base?

feat: integrate automated PyPI oldest-release cleanup into wheels publishing workflow #364

Uh oh!

Conversation

Omar8345 commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key changes

Added required GitHub Actions secrets

Video Demo

AI Review & Code Quality

Checklist

Additional Notes

Uh oh!

Excellencedev left a comment

Choose a reason for hiding this comment

Uh oh!

Omar8345 commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Excellencedev left a comment

Choose a reason for hiding this comment

Uh oh!

louis030195 commented Nov 14, 2025

Uh oh!

Excellencedev commented Nov 14, 2025

Uh oh!

Omar8345 commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

louis030195 commented Nov 14, 2025

Uh oh!

Excellencedev commented Nov 15, 2025

Uh oh!

Omar8345 commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Omar8345 commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Excellencedev commented Nov 15, 2025

Uh oh!

Omar8345 commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Excellencedev commented Nov 15, 2025

Uh oh!

Omar8345 commented Nov 15, 2025

Uh oh!

Omar8345 commented Nov 15, 2025

Uh oh!

divanshu-go commented Nov 15, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Excellencedev commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Excellencedev commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Excellencedev commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Excellencedev commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Excellencedev commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 16, 2025

Uh oh!

Excellencedev commented Nov 16, 2025

Uh oh!

Omar8345 commented Nov 14, 2025 •

edited

Loading

Omar8345 commented Nov 14, 2025 •

edited

Loading

Omar8345 commented Nov 14, 2025 •

edited

Loading

Omar8345 commented Nov 15, 2025 •

edited

Loading

Omar8345 commented Nov 15, 2025 •

edited

Loading

Omar8345 commented Nov 15, 2025 •

edited

Loading

Omar8345 commented Nov 18, 2025 •

edited

Loading

divanshu-go commented Nov 21, 2025 •

edited

Loading

Omar8345 commented Nov 30, 2025 •

edited

Loading