Skip to content

Commit 00b699d

Browse files
vburckhardtin-1911
andauthored
feat: initial full version of the stack (#3)
Co-authored-by: Igor Naumov <[email protected]>
1 parent 070010a commit 00b699d

File tree

5 files changed

+492
-5
lines changed

5 files changed

+492
-5
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.def.json

README.md

Lines changed: 70 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,70 @@
1-
# stack-ibm-template
1+
# Retrievel Augmented Generation (RAG) stack
2+
3+
To run the full stack. These steps will be updated as development progresses on the stack and underlaying DAs.
4+
5+
## 1. Deploy the stack in a new project from catalog
6+
7+
Catalog url: https://cloud.ibm.com/catalog/7df1e4ca-d54c-4fd0-82ce-3d13247308cd/architecture/Retrieval_Augmented_Generation_Pattern-5fdd0045-30fc-4013-a8bc-6db9d5447a52?bss_account=9f9af00a96104f49b6509aa715f9d6a5
8+
9+
Click the "Add to project" button, and select create in new project.
10+
11+
## 2. Prereqs in target account
12+
13+
- Create an API key in the target account. Keep note of it. Give it admin privilege for now. Exact permissions will be narrowed down in future version.
14+
- Create a resource group in the target account. This steps is temporary in current alpha version - resource groups will be created in the base account DA in future versions.
15+
16+
17+
## 3. Set the input configuration for the stack
18+
19+
- Clone this repository locally - and checkout the dev branch.
20+
- Create a file with name ".def.json" with the following content. Important: ensure region is either us-south or eu-de as watsonx can only be deployed in those 2 locations for now.
21+
22+
```json
23+
{
24+
"inputs": {
25+
"prefix": "<prefix for resources name>",
26+
"ibmcloud_api_key": "<API Key of the target account with sufficient permissions>",
27+
"resource_group_name": "<target resource group - must be existing in account>",
28+
"region": "<region where resources are deployed>",
29+
"sample_app_git_url": "https://github.com/IBM/gen-ai-rag-watsonx-sample-application"
30+
}
31+
}
32+
```
33+
34+
Example:
35+
```json
36+
{
37+
"inputs": {
38+
"prefix": "0410",
39+
"ibmcloud_api_key": "<your api key>",
40+
"resource_group_name": "0411-stack-service-rg",
41+
"region": "eu-de",
42+
"sample_app_git_url": "https://github.com/IBM/gen-ai-rag-watsonx-sample-application"
43+
}
44+
}
45+
```
46+
47+
48+
## 4. Run ./deploy-many.sh
49+
50+
- Ensure you are login into the account containing the Cloud project with the stack using ibmcloud login --sso
51+
- Execute ./deploy-many.sh with project name, stack name and optional configuration name pattern. The selected non-stack configruations will be processed by their name in alphabetical order. Using configuration name pattern (regex can be used - make sure to enclose it in quotes) you can chose which configurations are deployed
52+
53+
Example 1 - update stack inputs for stack configuration `RAG` and process all non-stack configurations in the project:
54+
```bash
55+
./deploy-many.sh my-test-project RAG
56+
```
57+
58+
Example 2 - update stack inputs and process some configurations in the project:
59+
```bash
60+
./deploy-many.sh my-test-project RAG 'RAG-1|RAG-4|RAG-5'
61+
```
62+
63+
Example 2 - simulate updating stack inputs and validating some configurations in the project in dry-run mode (no changes or actual validation or deployments is done):
64+
```bash
65+
DRY_RUN=true ./deploy-many.sh my-test-project RAG 'RAG-1|RAG-4|RAG-5'
66+
```
67+
68+
Tips: If deployment fail in one of the DA, you may need to remove the configuration name of the deployment that already passes from the pattern before re-running the script.
69+
70+
Tips: to accelerate iteration you may deploy only a subset of the configurations: the bare minimum are key management, security manager, watson saas, alm and rag configuration da. Base account, observability and SCC are not on the critical path to get the app running.

deploy-many.sh

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
#!/bin/bash
2+
3+
set -x
4+
5+
# PROJECT_ID="546993cb-c1ba-49cd-a975-187a2b924c21"
6+
# CONFIG_IDS=("b79d03d5-edfe-4cd4-8787-7d70738c6529" "853c369d-3dbd-41e4-8e79-006d2c3408b2" "736fd03d-190a-4bcd-a1a5-18334d868f20" "3d76faf5-663e-4a7a-bb1d-3c98a9fa6628" "70667b97-6bb5-425b-8e6a-47c931740604")
7+
# STACK_CONFIG_ID="53ef9a25-3aad-4167-b548-41b19c57cad4"
8+
9+
function parse_params() {
10+
PROJECT_NAME=$1
11+
STACK_NAME=$2
12+
CONFIG_PATTERN=$3
13+
[[ -z "$PROJECT_NAME" && -z "$PROJECT_ID" || -z "$STACK_NAME" && -z "$STACK_CONFIG_ID" ]] && \
14+
die "Usage: $(basename "${BASH_SOURCE[0]}") project_name stack_name [config_name_pattern]"
15+
16+
if [[ -z $DRY_RUN ]]; then
17+
CLI_CMD=ibmcloud
18+
else
19+
CLI_CMD=echo
20+
fi
21+
}
22+
23+
function get_config_ids() {
24+
25+
if [[ -z "$PROJECT_ID" ]]; then
26+
PROJECT_ID=$( ibmcloud project list --all-pages --output json | jq -r --arg project_name "$PROJECT_NAME" '.projects[]? | select(.definition.name == $project_name) | .id' )
27+
fi
28+
[[ -z "$PROJECT_ID" ]] && die "ERROR!!! Project $PROJECT_NAME is not found"
29+
30+
if [[ -z "$STACK_CONFIG_ID" ]]; then
31+
STACK_CONFIG_ID=$(ibmcloud project configs --project-id $PROJECT_ID --output json | jq -r --arg conf "$STACK_NAME" '.configs[]? | select(.definition.name==$conf) | .id ')
32+
fi
33+
[[ -z "$STACK_CONFIG_ID" ]] && die "ERROR!!! Stack Configuration $STACK_NAME is not found in project $PROJECT_NAME"
34+
35+
36+
if [[ -z "$CONFIG_IDS" ]]; then
37+
CONFIG_IDS=($(ibmcloud project configs --project-id $PROJECT_ID --output json | jq -r --arg pattern "$CONFIG_PATTERN" '[.configs[]? | select((.definition.name | test($pattern)) and (.deployment_model != "stack"))] | sort_by(.definition.name)[] | .id'))
38+
fi
39+
[[ -z "$CONFIG_IDS" ]] && die "ERROR!!! No configurations found matching '$CONFIG_PATTERN' in project $PROJECT_NAME"
40+
}
41+
42+
function set_stack_inputs() {
43+
$CLI_CMD project config-update --project-id $PROJECT_ID --id $STACK_CONFIG_ID --definition @.def.json
44+
}
45+
46+
function validate_config() {
47+
echo "=========> Starting validation for $(ibmcloud project config --project-id $PROJECT_ID --id $CONFIG_ID --output json| jq -r '.definition.name')"
48+
$CLI_CMD project config-validate --project-id $PROJECT_ID --id $CONFIG_ID --output json > /tmp/validation.json
49+
}
50+
51+
function wait_for_validation() {
52+
# Loop until the state is set to validated
53+
while true; do
54+
55+
# Get the current state of the configuration
56+
STATE=$(ibmcloud project config --project-id $PROJECT_ID --id $CONFIG_ID --output json | jq -r '.state')
57+
if [[ ! -z $DRY_RUN ]]; then
58+
STATE=validated
59+
fi
60+
61+
if [[ "$STATE" == "validated" ]]; then
62+
break
63+
fi
64+
65+
if [[ "$STATE" != "validating" ]]; then
66+
echo "Error: Unexpected state $STATE"
67+
exit 1
68+
fi
69+
70+
sleep 10
71+
done
72+
}
73+
74+
function approve_config() {
75+
$CLI_CMD project config-approve --project-id $PROJECT_ID --id $CONFIG_ID --comment "I approve through CLI"
76+
}
77+
78+
function deploy_config() {
79+
$CLI_CMD project config-deploy --project-id $PROJECT_ID --id $CONFIG_ID
80+
}
81+
82+
function wait_for_deployment() {
83+
while true; do
84+
# Retrieve the configuration
85+
RESPONSE=$(ibmcloud project config --project-id $PROJECT_ID --id $CONFIG_ID --output json)
86+
87+
# Check the state of the configuration under approved_version
88+
STATE=$(echo "$RESPONSE" | jq -r ".approved_version.state")
89+
if [[ ! -z $DRY_RUN ]]; then
90+
STATE=deployed
91+
fi
92+
93+
94+
# If the state is "deployed" or "deploying_failed", exit the loop
95+
if [[ "$STATE" == "deployed" || "$STATE" == "deploying_failed" ]]; then
96+
break
97+
fi
98+
99+
# If the state is not "deploying", print an error message and exit
100+
if [[ "$STATE" != "deploying" ]]; then
101+
echo "Error: Unexpected state $STATE"
102+
exit 1
103+
fi
104+
105+
# Sleep for a few seconds before checking the state again
106+
sleep 10
107+
done
108+
}
109+
110+
function die()
111+
{
112+
local message=$1
113+
local exit_code=${2-1}
114+
echo >&2 -e "$message"
115+
exit $exit_code
116+
}
117+
118+
parse_params "$@"
119+
get_config_ids
120+
set_stack_inputs
121+
122+
# 6. Loop through the configuration IDs and execute the functions
123+
for CONFIG_ID in "${CONFIG_IDS[@]}"
124+
do
125+
validate_config
126+
wait_for_validation
127+
approve_config
128+
deploy_config
129+
wait_for_deployment
130+
done

ibm_catalog.json

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
{
44
"label": "Retrieval Augmented Generation Pattern",
55
"name": "Retrieval_Augmented_Generation_Pattern",
6-
"version": "0.0.2",
6+
"version": "0.1.0-dev",
77
"product_kind": "solution",
88
"tags": [
99
"watson",
@@ -59,10 +59,10 @@
5959
"diagrams": [
6060
{
6161
"diagram": {
62-
"url": "https://raw.githubusercontent.com/terraform-ibm-modules/terraform-ibm-rag-sample-da/init-da/reference-architecture/stack.svg",
62+
"url": "https://raw.githubusercontent.com/terraform-ibm-modules/terraform-ibm-rag-sample-da/main/reference-architecture/stack.svg",
6363
"caption": "Solution components",
6464
"type": "image/svg+xml",
65-
"thumbnail_url": "https://raw.githubusercontent.com/terraform-ibm-modules/terraform-ibm-rag-sample-da/init-da/reference-architecture/stack.svg"
65+
"thumbnail_url": "https://raw.githubusercontent.com/terraform-ibm-modules/terraform-ibm-rag-sample-da/main/reference-architecture/stack.svg"
6666
},
6767
"description": "Solution components"
6868
}
@@ -74,6 +74,25 @@
7474
"type": "string",
7575
"default_value": "stack-demo",
7676
"required": false
77+
},
78+
{
79+
"name": "ibmcloud_api_key",
80+
"required": true,
81+
"type": "password",
82+
"hidden": false
83+
},
84+
{
85+
"name": "resource_group_name",
86+
"required": false,
87+
"type": "string",
88+
"hidden": false
89+
},
90+
{
91+
"name": "region",
92+
"required": false,
93+
"type": "string",
94+
"hidden": false,
95+
"default": "us-south"
7796
}
7897
],
7998
"install_type": "fullstack"

0 commit comments

Comments
 (0)