added a tester profile, this will show the test prompts

Author: dtehan-td

docs/developer_guide/DEVELOPER_GUIDE.md

@@ -125,8 +125,13 @@ Two guides have been created to show how to add tools and prompts:
 ## Tools testing
 Every module will have at least one prompt that would be responsible for testing all of the tools in the module.  
 
+Use the tester profile to see the test prompts.
+
+`uv run teradata-mcp-server --profile tester`
+
 The following test prompt template should be modified and placed in the module_objects.ylm file (e.g. base_objects.yml), the name of the testig prompt should be test_\<module name\>Tools. (e.g. test_baseTools):
 
+
 ```
 test_baseTools:
   type: prompt

profiles.yml

@@ -1,4 +1,12 @@
 all:
+  tool:
+    - .*
+  prompt:
+    - ^(?!test_).*
+  resource:
+    - .*
+
+tester:
   tool:
     - .*
   prompt:
@@ -8,20 +16,20 @@ all:
 
 dba:
   tool:
-    - dba_*
-    - base_*
-    - sec_*
+    - ^dba_*
+    - ^base_*
+    - ^sec_*
   prompt:
-    - dba_*
+    - ^dba_*
 
 dataScientist:
   tool:
-    - base_*
-    - rag_*
-    - fs_*
-    - qlty_*
-    - sec_userDbPermissions
-    - dba_userSqlList
+    - ^base_*
+    - ^rag_*
+    - ^fs_*
+    - ^qlty_*
+    - ^sec_userDbPermissions
+    - ^dba_userSqlList
 
 eda:
   tool:

src/teradata_mcp_server/tools/base/base_objects.yml

@@ -22,37 +22,47 @@ test_baseTools:
    Perform the phases in order, and do not skip any phase.
    
    ## Phase 0 - Get a list of databases
-   - Get a list of databases from the user. Use the base_databaseList function to get the list.  The tool should return a list of databases in the Teradata system.
+   - Get a list of databases from the user. Use the base_databaseList function to get the list.  The tool should return a list of databases in the Teradata system, if no data is returned then fail this test.
    ## Phase 1 - Get the list of tables in each database
-   - For the DBC database, get the list of tables. Use the base_tableList function to get the list. The tool should return a list of tables in the DBC database.
+   - For the DBC database, get the list of tables. Use the base_tableList function to get the list. The tool should return a list of tables in the DBC database, if no data is returned then fail this test.
    ## Phase 2 - Create a test table in your default database
    - Create a customer table in your default database called test_customer, the table should have an Cust_id column. Use the base_writeQuery function to create the table. A test_customer table should be created.
    - Add 10 rows to the test_customer table. Use the base_writeQuery function to add the rows.
    ## Phase 3 - Test the query tool
-   - test the query tool by running a query on the test_customer table. Use the base_readQuery function to run the query. The query should return the 10 rows you added in the previous phase.
-   - test the query tool by running a query on the test_customer table with a filter. Use the base_readQuery function to run the query. The query should return only the rows that match the filter. (example of a filter is Cust_id > 5)
+   - test the query tool by running a query on the test_customer table. Use the base_readQuery function to run the query. The query should return the 10 rows you added in the previous phase, if no data is returned then fail this test.
+   - test the query tool by running a query on the test_customer table with a filter. Use the base_readQuery function to run the query. The query should return only the rows that match the filter, if no data is returned then fail this test. (example of a filter is Cust_id > 5)
    ## Phase 4 - Test the table DDL tool
-   - Get the DDL of the test_customer table. Use the base_tableDDL function to get the DDL. The tool should return the DDL of the test_customer table.
+   - Get the DDL of the test_customer table. Use the base_tableDDL function to get the DDL. The tool should return the DDL of the test_customer table, if no data is returned then fail this test.
    ## Phase 5 - Test the column description tool
-   - Get the column description of the test_customer table. Use the base_columnDescription function to get the columns description. The tool should return the column description of the test_customer table.
+   - Get the column description of the test_customer table. Use the base_columnDescription function to get the columns description. The tool should return the column description of the test_customer table, if no data is returned then fail this test.
    ## Phase 6 - Test the table preview tool
-   - Get the preview of the test_customer table. Use the base_tablePreview function to get the preview. The tool should return the first 5 rows of the test_customer table and the column information.
+   - Get the preview of the test_customer table. Use the base_tablePreview function to get the preview. The tool should return the first 5 rows of the test_customer table and the column information, if no data is returned then fail this test.
    ## Phase 7 - Test the table affinity tool
-   - Get the table affinity of the test_customer table. Use the base_tableAffinity function to get the affinity. The tool should return the table affinity of the test_customer table.
+   - Get the table affinity of the test_customer table. Use the base_tableAffinity function to get the affinity. The tool should return the table affinity of the test_customer table, if no data is returned then fail this test.
    ## Phase 8 - Test the table usage tool
-   - Get the table usage of the test_customer table. Use the base_tableUsage function to get the usage. The tool should return the table usage of the test_customer table.
+   - Get the table usage of the test_customer table. Use the base_tableUsage function to get the usage. The tool should return the table usage of the test_customer table, if no data is returned then fail this test.
    ## Phase 9 - Clean up
    - Drop the test_customer table. Use the base_writeQuery function to drop the table. The tool should return a success message.
    ## Communication guidelines:
          - Be concise but informative in your explanations
          - Clearly indicate which phase the process is currently in
          - summarize the success or any failure of the phase before moving to the next phase
-   ## Final output guidelines:
-         - return in markdown results for all phases
-         - Example:
-         ***Phase 0:*** list databases:  Successful
-         ***Phase 1:*** list tables in DBC database:  Successful
-         ***Phase 2:*** Create test_customer table:  Failed
+    ## Final output guidelines:
+          - return in markdown results for all phases as a table including the phase number, description, success or failure status, and a description of the outcome.
+          - if a tool does not return any results, then fail that test and indicate it in the outcome.
+    ## Example output:
+          - | Phase | Description | Status | Outcome |
+          - |-------|-------------|--------|---------|
+          - | 0     | Get a list of databases | Success | List of databases returned |
+          - | 1     | Get the list of tables in DBC database | Success | List of tables returned |
+          - | 2     | Create a test table in default database | Success | test_customer table created |
+          - | 3     | Test the query tool on test_customer table | Success | 10 rows returned |
+          - | 4     | Get the DDL of the test_customer table | Success | DDL returned |
+          - | 5     | Get the column description of the test_customer table | Success | Column description returned |
+          - | 6     | Get the preview of the test_customer table | Success | First 5 rows and column info returned |
+          - | 7     | Get the table affinity of the test_customer table | Success | Table affinity returned |
+          - | 8     | Get the table usage of the test_customer table | Success | Table usage returned |
+          - | 9     | Drop the test_customer table | Success | Table dropped successfully |
 
          
 base_query:

src/teradata_mcp_server/tools/dba/dba_objects.yml

@@ -135,40 +135,40 @@ test_dbaTools:
     - Add 10 rows to the test_customer table using the base_writeQuery tool.
 
     ## Phase 1 - Get a list of all the SQL executed by a user
-    - Get a list of all the SQL executed by a user (e.g. demo_user) in the last 7 days using the dba_userSqlList tool. A list of all SQL statements should be returned
+    - Get a list of all the SQL executed by a user (e.g. demo_user) in the last 7 days using the dba_userSqlList tool. A list of all SQL statements should be returned, if no data is returned then fail this test
     
     ## Phase 2 - Get a list of all the SQL executed against a table
-    - Get a list of all the SQL executed against a table (e.g. test_customer) in the last 7 days using the dba_tableSqlList tool. A list of all SQL statements should be returned
+    - Get a list of all the SQL executed against a table (e.g. test_customer) in the last 7 days using the dba_tableSqlList tool. A list of all SQL statements should be returned, if no data is returned then fail this test
 
     ## Phase 3 - get a list of the current perm table space for a tables
-    - Get the current perm table space for a test_customer table using the dba_tableSpace tool. The current perm table space should be returned.
+    - Get the current perm table space for a test_customer table using the dba_tableSpace tool. The current perm table space should be returned, if no data is returned then fail this test.
 
     ## Phase 4 - get a list of the current perm database space for a database
-    - Get the current perm database space for a database (e.g. demo_user) using the dba_databaseSpace tool. The current perm database space should be returned.
+    - Get the current perm database space for a database (e.g. demo_user) using the dba_databaseSpace tool. The current perm database space should be returned, if no data is returned then fail this test.
 
     ## Phase 5 - get a list of the current database version
-    - Get the current database version using the dba_databaseVersion tool. The current database version should be returned. 
+    - Get the current database version using the dba_databaseVersion tool. The current database version should be returned, if no data is returned then fail this test. 
 
     ## Phase 6 - get a list of the system usage summary metrics
-    - Get the Teradata system usage summary metrics by weekday and hour for each workload type and query complexity bucket using the dba_resusageSummary tool. The system usage summary metrics should be returned  
+    - Get the Teradata system usage summary metrics by weekday and hour for each workload type and query complexity bucket using the dba_resusageSummary tool. The system usage summary metrics should be returned, if no data is returned then fail this test  
 
     ## Phase 7 - get a list of the system usage for a user
-    - Get the system usage for a user (e.g. demo_user) using the dba_resusageSummary tool. The system resusage for the user (e.g. demo_user) should be returned.
+    - Get the system usage for a user (e.g. demo_user) using the dba_resusageSummary tool. The system resusage for the user (e.g. demo_user) should be returned, if no data is returned then fail this test.
 
     ## Phase 8 - get a list of the system flow control metrics
-    - Get the Teradata system flow control metrics by day and hour using the dba_flowControl tool. The system flow control metrics should be returned.
+    - Get the Teradata system flow control metrics by day and hour using the dba_flowControl tool. The system flow control metrics should be returned, if no data is returned then fail this test.
 
     ## Phase 9 - get a list of the user feature usage metrics
-    - Get the user feature usage metrics using the dba_featureUsage tool. The user feature usage metrics should be returned.
+    - Get the user feature usage metrics using the dba_featureUsage tool. The user feature usage metrics should be returned, if no data is returned then fail this test.
 
     ## Phase 10 - get a list of the user delay metrics
-    - Get the Teradata user delay metrics using the dba_userDelay tool. The user delay metrics should be returned.
+    - Get the Teradata user delay metrics using the dba_userDelay tool. The user delay metrics should be returned, if no data is returned then fail this test.
 
     ## Phase 11 - get a list of the table usage impact by a user
-    - Get the usage of a table (e.g. test_customer) by a user (e.g. demo_user) using the dba_tableUsageImpact tool. The usage of the table by the user should be returned.
+    - Get the usage of a table (e.g. test_customer) by a user (e.g. demo_user) using the dba_tableUsageImpact tool. The usage of the table by the user should be returned, if no data is returned then fail this test.
 
     ## Phase 12 - get a list of the session information for a user
-    - Get the session information for a user (e.g. demo_user) using the dba_sessionInfo tool. The session information for the user should be returned.
+    - Get the session information for a user (e.g. demo_user) using the dba_sessionInfo tool. The session information for the user should be returned, if no data is returned then fail this test.
 
     ## Phase 13 - Clean up
     - Drop the test_customer table. Use the base_writeQuery function to drop the table. The tool should return a success message.
@@ -177,11 +177,24 @@ test_dbaTools:
           - Clearly indicate which phase the process is currently in
           - summarize the success or any failure of the phase before moving to the next phase
     ## Final output guidelines:
-          - return in markdown results for all phases
-          - Example:
-          ***Phase 0:*** list databases:  Successful
-          ***Phase 1:*** list tables in DBC database:  Successful
-          ***Phase 2:*** Create test_customer table:  Failed
+          - return in markdown results for all phases as a table including the phase number, description, success or failure status, and a description of the outcome.
+          - if a tool does not return any results, then fail that test and indicate it in the outcome.
+    ## Example output:
+
+          | Phase | Description | Status | Outcome |
+          |-------|-------------|--------|---------|
+          | 1     | Get a list of all the SQL executed against a table (e.g. test_customer) in the last 7 days | Successful |  |
+          | 2     | Get the current perm table space for a test_customer table | Successful |  |
+          | 3     | Get the current perm database space for a database (e.g. demo_user) | Successful |  |
+          | 4     | Get the current database version | Successful |  |
+          | 5     | Get the Teradata system usage summary metrics | Successful |  |
+          | 6     | Get the system usage for a user (e.g. demo_user) | Successful |  |
+          | 7     | Get the Teradata system flow control metrics | Successful |  |
+          | 8     | Get the user feature usage metrics | Successful |  |
+          | 9     | Get the Teradata user delay metrics | Successful |  |
+          | 10    | Get the usage of a table (e.g. test_customer) by a user (e.g. demo_user) | Successful |  |
+          | 11    | Get the session information for a user (e.g. demo_user) | Successful |  |
+          | 12    | Drop the test_customer table | Successful |  |
 
 
 dba_tableArchive:

src/teradata_mcp_server/tools/fs/fs_objects.yml

@@ -14,8 +14,9 @@ test_fsTools:
             - Clearly indicate which phase the process is currently in
             - summarize the success or any failure of the phase before moving to the next phase
         ## Final output guidelines:
-            - return in markdown results for all phases
-            - Example:
-            ***Phase 0:*** list databases:  Successful
-            ***Phase 1:*** list tables in DBC database:  Successful
-            ***Phase 2:*** Create test_customer table:  Failed
+            - return in markdown results for all phases as a table including the phase number, description, success or failure status, and a description of the outcome.
+            - if a tool does not return any results, then fail that test and indicate it in the outcome.
+        ## Example output:
+          - | Phase | Description | Status | Outcome |
+          - |-------|-------------|--------|---------|
+          - | 0     | Set up test environment | Success | Test environment set up successfully |