Improve docs on authoring prompts with more examples

Author: colinmcneil

docs/content/tools/docs/authoring-prompts.md

@@ -7,11 +7,31 @@ weight: 2
 
 A prompt is markdown content with a preamble describing tools available to the agent when executing this prompt.
 
-We use `h1` headers to delineate sections that should be exposed as prompts. When authoring this markdown, you can include non-prompt sections 
-using headers that don't start with the word "prompt". 
+## Prompt Headers
 
+We use `h1` headers to delineate sections that should be exposed as prompts. `h1` headers that start with `prompt` will be sent to the model.
+
+We use `h2` headers to indicate metadata about the prompt. 
+
+Both h1 and h2 headers are reserved keywords and should not be used in the markdown.
+
+```markdown
+# prompt I am a prompt header
+## metadata I am metadata header
+### header I am a regular header
+```
+
+You can also use h1 headers that don't start with the word "prompt" to organize your markdown.
+
+```markdown
+# I am not a prompt header
+```
+
+## Front-matter
+Prompt files support a yaml front-matter section that can be used to specify tools, models, and other metadata.
 Here's a simple prompt that will use Docker official curl container to fetch gists from Github.
 
+## Example
 
 ```markdown
 ---
@@ -24,9 +44,9 @@ tools:
 Run the curl command, in silent mode, to fetch gists for user slimslenderslacks from GitHub.
 ```
 
-## Tools
+## Adding Tools
 
-Some tools, like curl, are available by default.  However, the container for a tool can also be defined inline.
+Some tools, like curl, are available by default. However, you can add your own tools by adding a `tools` section to the front-matter.
 
 
 ```markdown
@@ -54,65 +74,62 @@ Use ffmpeg to convert the file UsingPuppeteer.mp4 into an animated gif file at 1
 The output file should be named UsingPuppeteer.gif.
 ```
 
-The `name`, `description`, and `container` fields are mandatory, and you'll typically also have a `parameters` field to descript the json schema
-of the parameters that the agent will extract from the conversation.
+The `name`, `description`, and `container` fields are mandatory, and you'll typically also have a `parameters` field to descript the json schema of the parameters that the agent will extract from the conversation.
 
 * `name` should uniquely identify the tool.
 * `description` is important. Good descriptions help the agent understand the tool and how to use it.
 * `container` sticks close to the format of a [compose service definition](https://docs.docker.com/reference/compose-file/services/) but does
   fully implement it. Many of the most common top-level arguments are supported (`image`, `command`, `volumes`)
 
-You can interpolate into parameter properties into the container definition using strings with double curly braces. These substitutions also
-support filters, such as `into`, which spreads an array parameter.  There are some common patterns for moving parameters into the container
-runtime that we support for making it easier to use standard images.
+You can interpolate into parameter properties into the container definition using strings with double curly braces. These substitutions also support filters, such as `into`, which spreads an array parameter.  There are some common patterns for moving parameters into the container runtime that we support for making it easier to use standard images.
+
+## Changing the Default Model
 
-## Models
+If you don't specify a model in the preamble, the default model is currently set to be `gpt-4` on OpenAI. You can refer to other models and their endpoints by specifying the `model` and `url` fields.
 
-If you don't specify a model in the preamble, the default model is currently set to be gpt-4 on openai. You can refer to other models and 
-other endpoints using `model` and `url`.
+**We currentlysupport all models compatible with the OpenAI API or Anthropic API.**
 
 ```markdown
 ---
 tools:
   - name: curl
-model: claude-3-5-sonnet-20241022
+model: llama3.2
+url: http://localhost:11434/v1
 ---
 
 # prompt
 
 Run the curl command, in silent mode, to fetch gists for user slimslenderslacks from GitHub.
 ```
 
-If you specify an anthropic model, such as `model: claude-3-5-sonnet-20241022`, you do not need to specify the url. The correct anthropic
-endpoint will be used.
-
-{{< callout type="info" >}}
-When using anthropic, do not change tool definitions to use `json_schema`. Use the openai standard of `parameters` - this will be automatically converted before making the api call.
-{{< /callout >}}
-
-### OpenAI-compatible endpoints (Ollama)
-
-You can specify other openai-compatible endpoints by including a `url`.
+### Pre-configured endpoints
+If you specify a model that we know about, such as `model: claude-3-5-sonnet-20241022`, you do not need to specify the url. The correct anthropic endpoint will be used. Currently, we support pre-configured endpoints for Anthropic and OpenAI.
 
 ```markdown
 ---
 tools:
   - name: curl
-url:  http://localhost/v1/chat/completions
-stream: false
-model: llama3.1
+model: claude-3-5-sonnet-20241022
 ---
 
 # prompt
 
 Run the curl command, in silent mode, to fetch gists for user slimslenderslacks from GitHub.
 ```
 
+{{< callout type="info" >}}
+When using Anthropic models, do not change tool definitions to use `json_schema`. Use the OpenAI standard of `parameters` - this will be automatically converted before making the api call.
+{{< /callout >}}
+
+### Streaming
+
+The parameter `stream` can be used to control whether the tool call is streamed to the model. This is useful for models that do not support streaming. The default is `true`.
+
 {{< callout type="info" >}}
 Set streaming to false if you're using Ollama for tool calling. Ollama does not currently stream tool calls.
 {{< /callout >}}
 
-## Prompt Templates
+## Using Templates and Parameters in Prompts
 
 It is common for prompts to contain parameters that are either extracted from a user interaction 
 or, in the case of RAG, are populated by some sort of retrieval process. Markdown prompts can also
@@ -167,7 +184,7 @@ prompt-format: "django"
 ---
 ```
 
-### Binding values during testing
+### Testing Templates
 
 When running in VSCode, you can set values of the parameters in the markdown preamble. This is
 a great way to quickly test your prompt during development.
@@ -178,45 +195,3 @@ parameter-values:
   user: slimslenderslacks
 ---
 ```
-
-### Extractors
-
-Extractors are container functions that can be used to extract values when the prompt has been deployed
-to a server. These extractors are also used to populate default values for a prompt when it is used from
-an MCP client.
-
-Extractor definitions also follow the pattern of compose services. They are just docker images but with 
-the additional requirement that they should write `application/json` to stdout. This json will be used to
-populate the context for binding parameters in the prompt template.
-
-```markdown
----
-extractors:
-  - name: linguist
-    image: vonwig/go-linguist:latest
-    command:
-      - -json
----
-```
-
-We can create lists if the extractor json output has array types.  For example,
-if we run the linguist tool to extract language from a project, our prompt can list
-them using the following template.  You need to be familar with the json format output
-by linguist (eg that it creates lists of maps with a `language` key).
-
-```markdown
----
-extractors:
-  - name: linguist 
----
-
-# prompt
-
-{{#linguist}}
-
-This project contains {{language}} code.
-
-{{/linguist}}
-
-```
-