-
Notifications
You must be signed in to change notification settings - Fork 55
Importing sites numIds #1186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Importing sites numIds #1186
Changes from 68 commits
f10d837
e5cfd06
bb90ef8
954a485
8ccdc7f
0f113e5
7d70341
b8aa290
61dff24
c7fdbf5
1817747
397f2ca
cd45896
545962f
58c24dd
2ebe717
66cdee0
5120a8d
036c44e
d438a13
3c3accc
f111bfa
9763401
7651530
52e8f8a
455e667
38d873f
f6769c1
4730cea
a07b2a9
98386b9
2f58d60
965eb10
e6cbb48
a259336
54ec570
fa54d67
340c59f
56b19fc
f06413f
651b939
c6e3bdb
69c4c11
ab05976
d6ec35c
6c1e51c
9e9b4b1
9e7e3ab
ba49f74
8270d43
37caa4f
53df7ae
7ffdfe1
2473296
b43e99e
9b077ee
bb3539f
99ef254
e910900
4b180e3
98e7b7a
378ee26
8e53e83
5babcf4
e4ff0eb
2bb785b
643072e
d2f5fc8
a52c0b2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,156 @@ | ||
#!/bin/bash -e | ||
|
||
# This script automates the process of updating num_ids for a list of site models | ||
|
||
# --- USER ACTION REQUIRED --- | ||
# The script will now prompt for the project specification. | ||
|
||
# Please populate this array with the list of your site models. | ||
# For example: SITE_MODELS=("US-SVL-BRGUP1" "US-SVL-OTHER" "etc") | ||
SITE_MODELS=() | ||
|
||
UDMI_ROOT=$(dirname $0)/.. | ||
cd $UDMI_ROOT | ||
|
||
# ============================================================================== | ||
|
||
# Get the gcloud email address. | ||
GCLOUD_EMAIL=$(gcloud config get-value account 2>/dev/null) | ||
|
||
if [ -z "$GCLOUD_EMAIL" ]; then | ||
echo "Error: Could not get gcloud email address." | ||
echo "Please make sure you are authenticated with gcloud ('gcloud auth login')." | ||
exit 1 | ||
fi | ||
|
||
echo "Using gcloud email: $GCLOUD_EMAIL" | ||
|
||
# Prompt user for the project specification. | ||
read -p "Enter the project_spec (e.g., //gbos/bos-platform-prod or //gbos/bos-platform-dev/namespace): " project_spec | ||
jainrocks marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
# Validate the project_spec format and extract the project ID. | ||
# The project ID is expected to be the component after "//gbos/". | ||
if [[ ! "$project_spec" =~ ^//gbos/ ]]; then | ||
echo "Error: project_spec must start with //gbos/" | ||
exit 1 | ||
fi | ||
|
||
# Extracts the part like 'bos-platform-prod' or 'bos-platform-dev' | ||
project_id=$(echo "$project_spec" | cut -d'/' -f4) | ||
|
||
if [ -z "$project_id" ]; then | ||
echo "Error: Could not extract project ID from project_spec '$project_spec'." | ||
echo "Ensure it has at least one component after //gbos/ (e.g., //gbos/project-id)." | ||
exit 1 | ||
fi | ||
|
||
echo "Using Project ID: $project_id" | ||
|
||
# Check if the SITE_MODELS array is empty. | ||
if [ ${#SITE_MODELS[@]} -eq 0 ]; then | ||
echo "Please edit this script and populate the SITE_MODELS array with your site models." | ||
exit 1 | ||
fi | ||
|
||
# Loop through each site model provided in the array. | ||
for site_model in "${SITE_MODELS[@]}"; do | ||
echo "-----------------------------------------------------" | ||
echo "Processing site model: $site_model" | ||
echo "-----------------------------------------------------" | ||
|
||
SITE_PATH="$UDMI_ROOT/sites/$site_model" | ||
|
||
# Prepare a clean directory for the site model. | ||
if [ -d "$SITE_PATH/.git" ]; then | ||
echo "Repository exists. Cleaning and pulling latest changes." | ||
( | ||
cd "$SITE_PATH" || exit | ||
git reset --hard origin/main | ||
git clean -fdx | ||
git pull origin main | ||
) | ||
else | ||
echo "Repository not found. Cloning a new one." | ||
rm -rf "$SITE_PATH" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if it's not found then there's no reason to remove it! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right that the rm is redundant if the directory doesn't exist at all. But, I've kept it in as a safeguard for the case where the path might exist but isn't a git repo (e.g., from a previous failed run). This ensures git clone always has a clean target. |
||
GIT_CLONE_URL="ssh://${GCLOUD_EMAIL}@source.developers.google.com:2022/p/${project_id}/r/${site_model}" | ||
git clone "$GIT_CLONE_URL" "$SITE_PATH" | ||
fi | ||
|
||
# Create a new branch for this site model's changes | ||
BRANCH_NAME="feat/import-numids-${site_model}-$(date +%s)" | ||
( | ||
cd "$SITE_PATH" || exit | ||
|
||
git checkout -b "$BRANCH_NAME" | ||
) | ||
|
||
DEVICES_PATH="$SITE_PATH/udmi/devices" | ||
|
||
if [ ! -d "$DEVICES_PATH" ]; then | ||
echo " WARNING: Devices directory not found at '$DEVICES_PATH'. Skipping device-level operations." | ||
continue | ||
fi | ||
|
||
# Step A: Normalize metadata.json files in the devices directory. | ||
echo "Normalizing metadata for $site_model..." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we have a separate script for normalizing metadata? Normalizing might introduce a lot of changes and the diff might become difficult to review. Normalizing as a separate step will increase the confidence in review when adding the actual num ids There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The normalization stuff is already captured in a separate commit, so at last the churn will be compartmentalized. I'm not sure what the overall intended flow here is b/c I'm not sure what actual "review" we'd do on a 100 site models (I'm actually not even sure what they are). Do we have a clearly identified "recipient" for this (is it Vik?), and maybe get them to sign off on what the review flow needs to be? From a technical perspective I think having them split as separate commits is fine, because then they could be reviewed separately. OTOH, another reason to split out the script would be that there might be other times when we want to normalize a site model. I just hate the thought of replicating all the common code (extracting site models)... and wouldn't want this "simple thing" to balloon into a bigger mess! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, Vik is the recipient for the same, will check with him and get sign off on review flow. |
||
( | ||
cd "$DEVICES_PATH" || exit | ||
for device in */; do | ||
device_name=${device%/} | ||
if [ -f "$device_name/metadata.json" ]; then | ||
echo " Normalizing $device_name/metadata.json" | ||
jq -S . "$device_name/metadata.json" | sponge "$device_name/metadata.json" | ||
fi | ||
done | ||
) | ||
|
||
# --- Commit --- | ||
echo "Committing normalized files for $site_model..." | ||
( | ||
cd "$SITE_PATH" || exit | ||
git add . | ||
git commit -m "Normalize metadata for $site_model" | ||
) | ||
|
||
# Step B: Run the registrar utility to update from IoT Core. | ||
cd $UDMI_ROOT | ||
jainrocks marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
echo "Running registrar for $site_model..." | ||
"$UDMI_ROOT/bin/registrar" "$SITE_PATH/udmi" "$project_spec" -u | ||
|
||
# Step C: Update the num_id in metadata.json from the registrar's output. | ||
echo "Updating num_id for $site_model..." | ||
( | ||
cd "$DEVICES_PATH" || exit | ||
for device in */; do | ||
device_name=${device%/} | ||
echo " Updating num_id for $device_name" | ||
if [ -f "$device_name/out/metadata_norm.json" ]; then | ||
id=$(jq .cloud.num_id "$device_name/out/metadata_norm.json") | ||
jq -S ".cloud.num_id = $id" "$device_name/metadata.json" | sponge "$device_name/metadata.json" | ||
else | ||
echo " WARNING: metadata_norm.json not found for $device_name. Skipping num_id update." | ||
fi | ||
done | ||
) | ||
|
||
# Step D: Commit and Push the num_id changes to git for Review. | ||
echo "Committing and pushing num_id changes for $site_model..." | ||
( | ||
cd "$SITE_PATH" || exit | ||
git add . | ||
git commit -m "Import num_ids for site model $site_model" | ||
) | ||
|
||
# Push the new feature branch to the remote repository | ||
echo "Pushing changes to new branch: $BRANCH_NAME" | ||
( | ||
cd "$SITE_PATH" || exit | ||
git push origin "$BRANCH_NAME" | ||
) | ||
|
||
|
||
echo "Done processing $site_model." | ||
done | ||
|
||
echo "=====================================================" | ||
echo "All site models processed." | ||
echo "=====================================================" |
Uh oh!
There was an error while loading. Please reload this page.