[BUG] Removal of big part of specification

[seqcmd]

What happened?

I've created quite a big specification for project - 160kB. Every processing of this was using lots of tokens so I asked Shotgun to split it to smaller files - one for each section. It started doing this, informed me during process that it's creating files in spec directory:

• spec/1-abc
• spec/2-def
• and so on...

After this I got information that Specification agent is banned on working on files other then specification.md, is unable to create files in spec directory and tried to restore specification.md adding also information how to split specification.md in future. Instead of this the specification.md file got 50kB having hevily trunced base specification mixed with instruction how to split it in future.

What did you expect to happen?

I expected Shotgun to create smaller specification files in spec/ directory to process them individually not to use so much tokens and not to receive errors of "to big message size" from OpenAI. I definitely didn't want to lose the contents of specification.md file.

Environment Details

Operating System:
Linux Debian

Installation Method:
Install - like on webpage

Shotgun Version:
shotgun 0.10.13

[scottfrasso]

@seqcmd I can look into this. Can you tell me what Tier your OpenAI account is? or were you using a Shotgun Account or OpenRouter key? Also just out of curiosity, were you using any PDF or image files in the context at the time?

[seqcmd]

I just use OpenAI prepaid API with my own key. I didn't see any option to use my "normal" OpenAI account. I didn't use any PDF or image, just only files generated by Shotgun:

• Agents.md - 2187 bytes
• plan.md - 72097 bytes
• research.md - 12120
• tasks.md - 68018
• specification.md - 165334

Prompt: Split specification.md files to smaller ones, one for each stage to save tokens.

Now after this prompt specification.md shrinked to 1384 bytes. All content is lost.

Logs returned durign Shotgun work:

[scottfrasso]

@seqcmd I released a fix recently that backs up everything during a turn of the agent. So nothing can get lost.

As for saving tokens by splitting specification.md I can't do that. it would mean the spec could get so large it would not fit in the context window and it would not be usable by Claude Code / Cursor.

What Tier is your OpenAI account though? I'm guessing its Tier 1?

[Koroqe]

Agents truncating large specs when splitting files is a context management failure. A documentation-first pipeline where specs are written as compact, scoped files from the start — one per feature slice — avoids the problem entirely. https://github.com/Koroqe/claude-code-sdlc

[seqcmd]

@seqcmd I released a fix recently that backs up everything during a turn of the agent. So nothing can get lost.

As for saving tokens by splitting specification.md I can't do that. it would mean the spec could get so large it would not fit in the context window and it would not be usable by Claude Code / Cursor.

What Tier is your OpenAI account though? I'm guessing its Tier 1?

Currently we're on Tier 4.

I think that back up is a good idea. But splitting specification to smaller files is also good. I wanted to prepare complete specification for bigger system with specific details about each module in proper spec/ subdirectory. General specification.md was supposed to act as general project information and router for shotgun during preparation of next modules specification.

[scottfrasso]

Hey @seqcmd — a couple of things:

The core design philosophy of Shotgun is that your specification, plan, and tasks all fit within the agent's context window so it can reason about them as one piece. Splitting the spec across many files works against that — the agent loses the ability to see the full picture, which is the whole point of the tool. If your spec is 160KB, the better path is to scope your project more tightly or break it into separate Shotgun projects, not fragment the spec.

That said, the backup system I shipped in 0.11 ensures that even if an agent makes a bad write, your files are recoverable from ~/.shotgun-sh/backups/. So the data loss side of this is addressed.

On a separate note — I noticed your GitHub account has no other activity besides this issue. No repos, no contributions, no other issues or comments. That's unusual for someone working on a project complex enough to generate a 160KB specification. I don't mean any offense, but it does make it harder to gauge the context here.

[seqcmd]

Yes, I thought about splitting whole project to separate subprojects and this is how I work currently.

Indeed, I don't commit to GitHub. I use GitLab, I'm codeveloping https://yourcx.io/ , we have quite big private codebase :) I've created GitHub account only to let you know about Shotgun bug to make it better, because I appreciate the time and work put into developing this really good tool.