Contribution Pipeline and Development Transparency — Following Up on Announcement #500

[virtualian]

@danielmiessler (and @ksylvan),

I'm Ian Marr (@virtualian / linkedin.com/in/ianmarr).I've been using PAI since early in the v2.x cycle, have raised several issues, including #393 (voice server path mismatch) and #614, and have been following the project closely across Issues, Discussions, and releases.

This post is a direct follow-up to #500, where you outlined two priorities: a better install wizard and an improved contribution pipeline, and you formally welcomed Kayvan to professionalise the repo the way he did for Fabric. I'm not raising a complaint and simply want to follow up to solicit an update from you.

The current situation has real implications for PAI's community, and the path forward has already been described by both of you. It's been roughly five weeks, and I haven't seen any visible progress on the contribution pipeline. I perceive frustration, but maybe that's just me!

The problem, in your own words from #500:

"When I rolled out a new version in a week or so, guess what? A lot of those changes would get clobbered. That's extremely not cool."

That's still the structural reality. PRs are accumulating (44 open as of today), the merge path runs through a private staging repo that contributors can't see, and there's no reliable signal about whether a contribution is actively in review or sitting in a queue indefinitely.

What you promised in #500:

• A private staging branch with CI/CD, so PRs could eventually merge through the same pipeline you use rather than being manually re-implemented
• Automatic documentation updates on push, matching Fabric's model
• Kayvan explicitly committed in this thread to WSL testing before the next release

The ask — narrow and specific:

A designated contrib branch that accepts PRs directly, reviewed and merged by you or Kayvan as Collaborator, with you retaining sole authority over what is promoted to main for official releases. This is not a governance request. Architecture, design principles, and release cadence — all stay with you. This is purely about making the contribution pipeline visible and functional rather than opaque and manual.

Fabric already works this way. Kayvan built that. The model is proven.

Concrete questions:

1. Is the private staging repo active and in use?
2. Is there a timeline for when community PRs can merge through that pipeline rather than being manually re-implemented?
3. Is there anything blocking Kayvan's CI/CD work from being scoped and prioritised now?

The community building on PAI — and the quality of the project itself — would benefit from even a partial answer to those three questions.

I'm happy to help move this forward practically. I have a background in IT infrastructure, I'm passionate about AI and DAs, and have been running PAI hands-on since the v2.x cycle. If useful, and as inexperienced as I am with open-source projects, I have time to assist with whatever reduces the burden on you and Kayvan rather than adding to it. My goal: a more resilient project that can absorb community energy without creating overhead for the core team.

— Ian

[ksylvan]

Hi @virtualian

Ian, thanks for this nudge.

@danielmiessler and I will meet soon and discuss the best way forward. Thank you for offering to help, I may take you up on it.

Regarding your questions:

1. The staging repo - I'll be setting that up over the next few days.

2. Regarding a timeline for merging community PRs through an automated pipeline:
   I'll have to look at this with Daniel. I don't think we want an automated way to merge PRs without human oversight (there are serious security issues with that idea).

3. The only thing blocking @danielmiessler and me is our busy schedules. I just started a job as the Founding Platform and CLI Engineering Lead at Wizard.

Your concrete ask is reasonable and I'll look at how we want to set that up for PAI.

Feel free to message me privately or connect via LinkedIn if you'd like to continue the discussion.

[virtualian]

@ksylvan,

UPDATED

Thanks for the reply and answers to my questions. I've emailed you directly with my contact details.

One clarification on bullet 2:

... Is there a timeline for when community PRs can merge through that pipeline rather than being manually re-implemented? ...

My concern is that today community PRs have to be 'manually' re-applied to Daniel's/your private development repo. It is highly likely they will conflict with internal changes and you'll need to spend time verifying them.

I am NOT suggesting this can be automated, but giving the community a branch to develop and PR against, which you also use, will simplify and speed up rebasing and PR merging.

[ksylvan]

I don't understand this part of your reply:

x] Given the quality of VC's deliverables, and play into the Hasan dI heard that they are still being used, since my sister's wedding in 1985,
I agree that a is imperative.

[virtualian]

Neither did I! I've updated my post.

Sorry about that, it looked like Wispr Flow dictation went mad. I'll double-check next time. 🫣

[danielmiessler]

Yes, we haven't met yet to get this going, but we will soon.

[virtualian]

Nudge...

Have you met with @ksylvan on this yet?

[jlacour-git]

Dear all naturally I perceive the same issues and I might be able to offer an interim workaround if you're interested. With my assistant I created a tracking system for all the changes that I do, including changes to system files. In addition, I created a skill that continuously checks the upstream repo, is able to do updates to new Pi versions, check whether local fixes have now been included upstream or whether they need to reapply and make sure that after that update it is still a fully working setup including all my modifications.

Has worked pretty well now updating 4.0.0 or 4.0.3.

Happy to share if you want.

[jlacour-git]

@virtualian Here you go! Posted the full system in a separate discussion: #923

Includes the patch tracker template, the upstream scan tool, and the pre-upgrade checker. Let me know if you have questions or want to adapt anything.

[virtualian]

@jlacour-git Thanks! I have integrated this into my repo (@virtualian/pai).

On the first manual run, I get:

You must provide a `first` or `last` value to properly paginate the `comments` connection.
⚠ gh command failed: gh api graphql -f query='{ repository(owner:"danielmiessler", name:"Personal_AI_...
  Discussions: 0
  Our PRs: 0

Altho it does continue...

"I" made the following change:

Update(/Users/ianmarr/projects/pai/Tools/UpstreamScan.ts)                                                                      
  ⎿  Added 1 line, removed 1 line                                                                                                
                                                                                                                      
      211  function fetchDiscussions(): GitHubItem[] {                                                                           
      212 -  const query = `{ repository(owner:"danielmiessler", name:"Personal_AI_Infrastructure") { discussions(first:15     
          -, orderBy:{field:CREATED_AT, direction:DESC}) { nodes { number title createdAt author { login } category { name       
          - } comments { totalCount nodes { author { login } replies(first:10) { nodes { author { login } } } } } } } } }`     
          -;                                                                                                                   
      212 +  const query = `{ repository(owner:"danielmiessler", name:"Personal_AI_Infrastructure") { discussions(first:15     
          +, orderBy:{field:CREATED_AT, direction:DESC}) { nodes { number title createdAt author { login } category { name
          + } comments(first:20) { totalCount nodes { author { login } replies(first:10) { nodes { author { login } } } } 
          +} } } } }`;   

[virtualian]

@jlacour-git And another...

The problem:

gh api graphql -f query='${query}' — the query contains double quotes, but the whole thing is wrapped in backtick template literal passed to execSync, which runs it in a shell. The single quotes around ${query} work, but the shell may still have issues depending on the content.

The fix is to pass the query via stdin instead of as a shell argument:

⏺ Update(/Users/ianmarr/projects/pai/Tools/UpstreamScan.ts)                                                                      
  ⎿  Added 11 lines, removed 1 line                                                                                            
                                                                                                       
      213 -  const raw = ghCmd(`gh api graphql -f query='${query}'`);                                                          
      213 +  let raw: string | null;                                                                                           
      214 +  try {                                                                                                        
      215 +    raw = execSync(`gh api graphql -f query=@-`, {                                                             
      216 +      input: query,                                                                                            
      217 +      encoding: "utf-8",                                                                                       
      218 +      timeout: 30000,                                                                                          
      219 +    }).trim();                                                                                                 
      220 +  } catch {                                                                                                    
      221 +    console.error("⚠ gh graphql command failed for discussions");                                              
      222 +    raw = null;                                                                                                
      223 +  }           

[jlacour-git]

@virtualian Thanks for taking the time to report these! Both are real bugs:

1. Pagination: The comments connection needs first: N — GitHub's GraphQL API requires explicit pagination boundaries. Fixed locally, will update the gist.

2. Shell quoting: The double quotes inside the template literal break when execSync passes them through the shell. We'll switch to a file-based approach or proper escaping.

Your reports actually helped us uncover two more gaps in the tool:

3. No threaded reply fetching. The GraphQL query only pulls top-level comments, not nested replies. That's why we completely missed your bug reports here — they're threaded replies, invisible to the scan. Adding replies(first: N) to the query.

4. 15-discussion window too narrow. The scan only fetches the 15 most recent discussions by creation date. Older discussions with new activity (like this one!) get missed entirely. We're adding a "participated discussions" fetch so we never miss replies on threads we authored or commented on.

Updated gist coming once all four fixes are in. Thanks again for stress-testing it!

[virtualian]

New Gist available?

[Drizzt321]

I'm definitely interested in this. I'm developing a CalDAV skill (calendaring, tasks, contacts), and I want to contribute it back up, but right now the skills/etc are actually in release sub-dirs. I was expecting a singular directory that is the current main, with releases being tags and in the Release section, but otherwise a single spot to orient to push towards. It's likely there will be another point release or larger release before I'm done, so I'll have to port my work over into another directory on my fork, once you create that new release dir.

Is this the intention, that that's how this is going to be for contributions?

[virtualian]

@Drizzt321

PAI is only released as a complete version.

I believe that Daniel and Kayvan have one or more private repositories. While we can raise issues on the public repository, the pull requests (PRs) we submit cannot be merged directly by Daniel and Kayvan; they must manually reintegrate them into their own repositories. Although I am sure they are using their own DA/PAIs, this process is inherently error-prone and creates a disconnect from community contributions.

I am advocating for a more open development model where we can develop new features, fix bugs, and submit PRs for Daniel and Kayvan to review, accept, reject, and merge directly.

In the meantime, I do not see any value in submitting PRs. Therefore, I will only raise detailed issues for Daniel and Kayvan to address, and I trust they will resolve them in future releases.

However, I appreciate @jlacour-git's approach of sharing his code fixes and features via Gists.

[Drizzt321]

Unfortunately that leaves contributing entire chunks of code (such as an entirely new Skill) in a weird limbo. Since it's not a bug.

I don't want to mess with a somewhat working development/release process they have going, but yeah, as you point out in your OP, some way for the rest of us to submit fixes/features with an easier way for them to pull them in would definitely be very nice to have.

It's almost like we should do some automated diff patch sets, for them to pull in and apply. Old school.

[DolphusCY]

@Drizzt321 if you read the README there is a contributing section at the bottom and they mention skills and logging a PR. Not sure it that still the case

[virtualian]

@DolphusCY, you're right, but unfortunately, that's the process that's 'broken' and the disconnect between the public repo and their development environment.

[Drizzt321]

@DolphusCY it has a very, very small simplistic contributing section. However, the current PR/them manually copying to private repo is the process we're hoping can be changed, or mitigated somehow, to make it easier (and hopefully means quicker) for them to bring our PRs into the working private repo(s), for future releases.