Workflows (#46)

Author: zah

.agents/tasks/2025/07/01-1348-workflows

@@ -0,0 +1,62 @@
+Let's implement and document a new feature:
+
+# Feature Description (Workflows)
+
+The task description entered by the user may include lines starting with /
+
+These are considered "workflow commands". They correspond to user-defined programs or shell scripts placed in the `.agents/workflows` directory of the repository under which `get-task` is executed. When `get-task` is executed, it looks for such workflow commands in the task description. For each such command, the matching programs is executed and its output is inserted in the place of the workflow command in the final text printed to `stdout` by `get-task` (a good analogy for this would be a macro in a programming language).
+
+The workflow command line may feature parameters. These should be parsed in the same way bash would parse them (i.e. `/some-workflow 10 "foo bar"` would mean two parameters, the second one being a string).
+
+Instead of a workflow program, in the `.agents/workflows` dir there might be a simple txt file named `<workflow-command>.txt`. In this case, the contents of this file are inserted as a verbatim copy in the place of the workflow command.
+
+In the output of each workflow command, there might be special lines such as "@agents-setup FOO=x BAR=10". Such lines indicate ENV variables that will be set in the environment by scripts such as `codex-setup`, `jules-setup`, etc, before the matching user-defined setup script from the `.agents` directory of the target repository are executed. The lines are stripped from the regular `get-task` output.
+
+The command `get-task --get-setup-env` lists all such ENV vars with their assignments from all @agents-setup directives that ended up in the task description (both inserted by the user directly and inserted by workflow commands). The `agents-workflow/***-setup` scripts use this command to set up the environment.
+
+## Validation of the task description
+
+After the user enters the task description (the EDITOR executed by `agent-task` quits with exit code 0), we would validate the entered task description. Any referenced workflows that don't have a matching definitions under `.agents/workflows` will be reported as errors. Conflicting variable assignments from `@agents-setup` directives will be reported as errors as well.
+
+If some of workflow commands exit with a non-zero exit code, their `stderr` is included in a diagnostic message like this:
+
+Failure in executing workflow commands:
+$ foo 10
+<stderr contents>
+$ bar
+<stderr contents>
+
+If the workflow command script exists, but it's not executable, try to automatically make it executable on platforms where this is possible. If this was not successful, report this with an appropriate diagnostic message.
+
+The error are reported by printing all diagnostic messages on the screen and asking the user to press any key to continue or Ctrl+C to abort. If the user continues, the editor is launched again, allowing the user to correct the issues.
+
+If the task description is directly supplied on the command line with `--prompt` or `--prompt-file`, `agent-task` reports the same diagnostic messages and exits with non-zero exit code.
+
+# Tasks
+
+* Implement the new features
+
+* Provide comprehensive user-facing documentation in the README.
+
+* Add tests for all of the described functionality. Make sure to test the happy path for shell scripts and ruby programs as workflow commands, as well as with txt files. Make sure to have tests for the correct production of error massages in all possible modes (interactive, --prompt, --prompt-file). Make sure to add test cases for all possible diagnostic message conditions.
+
+* Add a test case that setups a new temp repo that features setup scripts in its `.agents` directory. Simulate running scripts such as `agents-workflow/codex-setup`, `agents-workflow/jules-setup`, etc, when the CWD is set to the temp repository and verify that env vars provided with @agents-setup directives in the task description will be properly loaded and forwarded to the setup scripts from the temp repo. Please note that the `agents-workflow` repo will be moved as a result of calling the setup script, so invoke the setup command with a fresh local clone of the agents-workflow repo for each test or suppress the moving logic for most tests (keep at least one test that enables the logic though).
+
+--- FOLLOW UP TASK ---
+Please implement all test cases described near the end of .agents/tasks/2025/07/01-1348-workflows. Think of any additional test cases that would cover all of the described functionality.
+
+--- FOLLOW UP TASK ---
+Let's improve the test cases for the workflows feature:
+
+1) There is no need to have multiple test files. You can combine them into one, but don't delete any test.
+2) I'd like some of tests to involve more then one workflow commands.
+3) I'd like some of the task descriptions to be longer, where the workflow command appears as a first line, as a last line (with or without a trailing whitespace), in the middle of the text, with trailing whitespace on the line, etc.
+4) I'd like to see some workflow commands accepting parameters. Make sure some of the parameters are strings that contain spaces.
+
+--- FOLLOW UP TASK ---
+In the tests folder, use indented heredoc strings instead of single-line strings with "\n" characters in order to make the code easier to read.
+
+--- FOLLOW UP TASK ---
+Helpers like   are now defined in multiple places. Refactor the code to use a single definition (probably a regular global function). There are similar helpers for Linux and macOS. They should all be in a single place.
+--- FOLLOW UP TASK ---
+Helpers like `windows?`  are now defined in multiple places. Refactor the code to use a single definition (probably a regular global function). There are similar helpers for Linux and macOS. They should all be in a single place.

README.md

@@ -1,44 +1,68 @@
 ## Overview
 
-This repository provides an opinionated workflow designed to
-enhance the performace of teams working with coding agents,
-such as OpenAI Codex, Claude Code, Google Jules, GitHub Copilot,
-Goose, OpenHands and others.
+This repository provides a highly-opinionated workflow for working
+with cloud and local coding agents, such as Claude Code, Codex, GitHub
+Copilot, Jules, Gemini, Goose, OpenHands and others.
 
-The workflow standardizes how tasks are defined, and tracked,
-leveraging your VCS (e.g. git) as the primary driving interface.
+## Goals
 
-## Purpose
+The workflow adheres to the following principles, which are implemened
+both when workign with local agents and when working with remote agents:
 
-The primary goal of this workflow is to:
+1.  **The developer provides a coding task through a convenient command-line interface**
 
-1.  **Use git as the primary interface for driving Codex:**
+3.  **The agent works in a secure sandbox environment, without asking for confirmation when using tools**
 
-    Provides a convenient way to assign tasks to Codex right
-    from your command-line.
+4.  **The agent presents a complete patch/PR once it reaches a stage where all tests and linters are green**
 
-2.  **Maintain a transparent history:**
+5.  **It's easy to start multiple tasks in parallel from the current state of your working tree**
 
-    All task descriptions are committed to Git, creating an
-    auditable trail and a knowledge base demonstrating how
-    tasks are approached and solved.
+6.  **All tasks are recorded as commits/files in the history of the project**
 
-    This allows team members to learn from each other's practices
-    and makes `git blame` an effective tool for understanding the
-    intention behind all code. The workflow injects instructions
-    that teach the agents how to leverage this.
+Pushing to git becomes the primary interface for starting cloud agents.
+All other interactions with the web UIs of the agents are automated.
 
-3.  **Deal with the current limitations around internet connectivity:**
+Committing all task descriptions in git creates an auditable trail and
+a knowledge base demonstrating how tasks are approached and solved.
 
-    By pre-fetching internet resources mentioned in the task
-    descriptions, agents such as Codex are more successful at
-    dealing with problems that require information that is not
-    part of the codebase.
+This allows team members to learn from each other's practices and makes
+`git blame` an effective tool for understanding the intention behind all
+code. The workflow injects instructions that teach the agents how to
+leverage this.
 
-4.  **Simplify the agents workspace setup:**
+Local agents are started is devcontainers with rich support for different
+interaction patterns:
 
-    The `.agents/codex-setup` script is stored in your repository,
-    simplifying the maintainance of the workspace.
+- Start one Editor/IDE instance per task to observe the work of the agent
+  and review the final
+
+- Push to a designated branch automatically or create a PR.
+
+## Other Practical Benefits
+
+* Local agents can leverage ZFS and Btrfs snapshots to provide the best
+  possible agent-start up time. The agent takes advantage of incremental
+  compilation when building the project and its test suite.
+
+* The same start-up time and incremental compilations are possible when
+  you dispatch the coding tasks to a cluster of self-managed machines in
+  an office environment or a private cloud.
+
+* The workflow smooths out the differences between different agent tools
+  and cloud environments. Everything can be handled through shared config
+  and user interfaces.
+
+  The behavior of the cloud agents is modified through prompt engineering
+  and automation to implement new workflows such as automatically creating
+  PRs, automatically pushing to specific branches, etc.
+
+* The workflow provides a helpful framework for automatically downloading
+  relevant internet resources before coding tasks start for agents that
+  need to operate offline.
+
+* The workflow provides a framework for working in big monorepos that speeds
+  up agent start-up times (both locally and it the cloud) and helps with
+  managing the context of the agent in such repositories.
 
 ## Using the Workflow
 
@@ -84,6 +108,46 @@ The primary goal of this workflow is to:
     The `get-task` script will print the task description for the agent,
     along with instructions for accessing the downloaded internet resources
     and working with the git history.
+    It also supports a `--get-setup-env` option which prints only the
+    environment variable assignments gathered from `@agents-setup` lines.
+
+### Workflow Commands
+
+Task descriptions may include lines beginning with `/` (e.g. `/front-end-task`).
+
+When `get-task` is executed, these lines are replaced with the output of a
+matching programs or text files in the `.agents/workflows` folder of your
+repository.
+
+In other words, in the example above, `get-task` will look either for an
+executable stored in `.agents/workflows/front-end-task` or for a text file
+located at `.agents/workflows/front-end-task.txt` (the contents of this file
+will take the place of the workflow command in the task description, like a
+macro in a programming language).
+
+Executables are typically used when the workflow command has parameters.
+
+Lines starting with `@agents-setup` in either the task file or the workflow
+output are stripped from the final message and interpreted as environment
+variable assignments for the `*-setup` scripts described below.
+
+```shell
+@agent-setup DEV_SHELL=csharp TESTED_COMPONENTS+=backend,db
+```
+
+A directive may either assign a value (`VAR=value`) or append entries to a
+comma‑separated set using the `VAR+=val1,val2` syntax. When multiple directives
+affect the same variable, the following rules apply:
+
+1. Conflicting direct assignments (different values for the same variable)
+   result in an error.
+2. A direct assignment can be combined with one or more appends. The final value
+   contains the assigned value plus all appended entries, regardless of their
+   order.
+3. One or more append operations without a direct assignment simply combine
+   their entries.
+
+Duplicate directives or values are ignored.
 
 ## Supported Agent Systems
 

codex-setup

@@ -48,6 +48,13 @@ EOF
 
 bash "$AGENTS_WORKFLOW_DIR/common-pre-setup"
 
+SETUP_ENV="$("$AGENTS_WORKFLOW_DIR/bin/get-task" --get-setup-env 2>/dev/null)"
+if [ -n "$SETUP_ENV" ]; then
+  while IFS= read -r line; do
+    export "$line"
+  done <<< "$SETUP_ENV"
+fi
+
 if [ -f .agents/codex-setup ]; then
   .agents/codex-setup
 fi

copilot-setup

@@ -4,6 +4,13 @@ AGENTS_WORKFLOW_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 
 bash "$AGENTS_WORKFLOW_DIR/common-pre-setup"
 
+SETUP_ENV="$("$AGENTS_WORKFLOW_DIR/bin/get-task" --get-setup-env 2>/dev/null)"
+if [ -n "$SETUP_ENV" ]; then
+  while IFS= read -r line; do
+    export "$line"
+  done <<< "$SETUP_ENV"
+fi
+
 if [ -f .agents/copilot-setup ]; then
   .agents/copilot-setup
 fi

goose-setup

@@ -4,6 +4,13 @@ AGENTS_WORKFLOW_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 
 bash "$AGENTS_WORKFLOW_DIR/common-pre-setup"
 
+SETUP_ENV="$("$AGENTS_WORKFLOW_DIR/bin/get-task" --get-setup-env 2>/dev/null)"
+if [ -n "$SETUP_ENV" ]; then
+  while IFS= read -r line; do
+    export "$line"
+  done <<< "$SETUP_ENV"
+fi
+
 if [ -f .agents/goose-setup ]; then
   .agents/goose-setup
 fi

jules-setup

@@ -4,6 +4,13 @@ AGENTS_WORKFLOW_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 
 bash "$AGENTS_WORKFLOW_DIR/common-pre-setup"
 
+SETUP_ENV="$("$AGENTS_WORKFLOW_DIR/bin/get-task" --get-setup-env 2>/dev/null)"
+if [ -n "$SETUP_ENV" ]; then
+  while IFS= read -r line; do
+    export "$line"
+  done <<< "$SETUP_ENV"
+fi
+
 if [ -f .agents/jules-setup ]; then
   .agents/jules-setup
 fi

lib/agent_task/cli.rb

@@ -266,6 +266,9 @@ def run_get_task(args = [])
         opts.on('--autopush', 'Tells the agent to automatically push its changes') do
           options[:autopush] = true
         end
+        opts.on('--get-setup-env', 'Print ENV vars from @agents-setup directives') do
+          options[:get_setup_env] = true
+        end
       end.parse!(args)
 
       repos = discover_repos
@@ -275,17 +278,28 @@ def run_get_task(args = [])
       end
 
       if repos.length == 1 && repos[0][0].nil?
-        puts repos[0][1].agent_prompt_with_autopush_setup(autopush: options[:autopush])
+        at = repos[0][1]
+        if options[:get_setup_env]
+          _, env = at.agent_prompt_with_env
+          env.each { |k, v| puts "#{k}=#{v}" }
+        else
+          puts at.agent_prompt_with_autopush_setup(autopush: options[:autopush])
+        end
         return
       end
 
       dir_messages = []
-      repos.each do |dir, at|
+      repos.each do |dir, agent_tasks|
         next if dir.nil?
 
         begin
-          msg = at.agent_prompt_with_autopush_setup(autopush: options[:autopush])
-          dir_messages << [dir, msg] if msg && !msg.empty?
+          if options[:get_setup_env]
+            _, env = agent_tasks.agent_prompt_with_env
+            dir_messages << [dir, env.map { |k, v| "#{k}=#{v}" }.join("\n")]
+          else
+            msg = agent_tasks.agent_prompt_with_autopush_setup(autopush: options[:autopush])
+            dir_messages << [dir, msg] if msg && !msg.empty?
+          end
         rescue StandardError
           next
         end
@@ -295,10 +309,8 @@ def run_get_task(args = [])
         puts "Error: Could not find repository root from #{Dir.pwd}"
         exit 1
       elsif dir_messages.length == 1
-        # Single repo case: display without directory hints
         puts dir_messages[0][1]
       else
-        # Multiple repos case: display with directory hints
         output = dir_messages.map { |dir, msg| "In directory `#{dir}`:\n#{msg}" }.join("\n\n")
         puts output
       end