Skip to content

Commit c0b22a8

Browse files
committed
merge: PRs #61, #39, #37, #38 resolved and merged
4 parents b93c188 + 3083770 + 845b9b3 + aa0b224 commit c0b22a8

File tree

7 files changed

+637
-113
lines changed

7 files changed

+637
-113
lines changed

package-lock.json

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
"enhance:language": "node scripts/standardize-languages.js",
1414
"enhance:metadata": "node scripts/scrape-metadata.js",
1515
"enhance:slugs": "node scripts/generate-slugs.js",
16+
"enhance:merge": "node scripts/merge-enhanced-metadata.js",
1617
"validate:data": "node .github/scripts/validate-data.mjs",
1718
"validate:build": "node .github/scripts/validate-build.mjs",
1819
"validate:links": "node .github/scripts/check-links.mjs"
@@ -22,6 +23,7 @@
2223
"@astrojs/react": "^4.4.0",
2324
"astro": "^5.13.8",
2425
"framer-motion": "^12.23.24",
26+
"fuse.js": "^7.1.0",
2527
"papaparse": "^5.5.3",
2628
"react": "^19.2.0",
2729
"react-dom": "^19.2.0",
@@ -37,4 +39,4 @@
3739
"pa11y-ci": "^4.0.1",
3840
"puppeteer": "^24.36.1"
3941
}
40-
}
42+
}

scripts/README.md

Lines changed: 26 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,10 @@ These scripts automate metadata extraction and enhancement for the 56 lessons in
2929

3030
```bash
3131
npm run enhance:phase1
32+
npm run enhance:merge
3233
```
3334

34-
This runs all 3 Phase 1 scripts:
35+
This runs all 3 Phase 1 scripts and merges the output:
3536
1. Language standardization (fast, no web requests)
3637
2. Time estimation (makes web requests, ~5 min)
3738
3. Metadata scraping (makes web requests, ~10 min)
@@ -52,6 +53,9 @@ npm run enhance:time
5253

5354
# Scrape author, license, dates (slow, fetches pages)
5455
npm run enhance:metadata
56+
57+
# Merge all records into a single master CSV
58+
npm run enhance:merge
5559
```
5660

5761
## Output
@@ -67,10 +71,9 @@ scripts/output/
6771
```
6872

6973
**Import to Google Sheets**:
70-
1. Open each CSV file
71-
2. Review automated values
72-
3. Spot-check for accuracy
73-
4. Copy-paste columns into Google Sheets
74+
75+
1. **(Recommended)** Use the **Inventory Tools** Google Apps Script. See [UPDATING_GOOGLE_SHEETS.md](UPDATING_GOOGLE_SHEETS.md) for setup and usage.
76+
2. Alternatively, manually copy-paste columns from the individual CSVs or the merged CSV.
7477

7578
## Script Details
7679

@@ -150,6 +153,21 @@ scripts/output/
150153
- Footer text parsing for licenses
151154
- JSON-LD schema.org extraction
152155

156+
---
157+
158+
### 5. `merge-enhanced-metadata.js`
159+
160+
**What it does**: Consolidates all phase-specific CSV outputs into a single master CSV.
161+
162+
**Goal**: Simplifies the bulk update process for Google Sheets by providing one file with all enhanced fields.
163+
164+
**Execution**:
165+
```bash
166+
npm run enhance:merge
167+
```
168+
169+
**Output**: `scripts/output/MERGED-enhanced-metadata-[YYYY-MM-DD].csv`
170+
153171
## Configuration
154172

155173
### Adjust Reading Speed
@@ -231,10 +249,9 @@ Review 10-15 random lessons for accuracy:
231249

232250
### 4. Import to Google Sheets
233251

234-
Copy the columns you want to update:
235-
- Open CSV in Excel/Numbers/Sheets
236-
- Select relevant columns (`timeRequired`, `author`, `license`, etc.)
237-
- Copy-paste into Google Sheets
252+
The recommended way to update the inventory is using the Google Apps Script tool. See [UPDATING_GOOGLE_SHEETS.md](UPDATING_GOOGLE_SHEETS.md) for full instructions.
253+
254+
Alternatively, you can copy the columns you want to update from the merged CSV and paste them into the Google Sheet.
238255

239256
## Troubleshooting
240257

scripts/UPDATING_GOOGLE_SHEETS.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Updating Google Sheets Inventory with Enhanced Metadata
2+
3+
This guide explains how to update the canonical Google Sheets inventory with enhanced metadata (slugs, time estimates, languages, etc.) generated by the automation scripts.
4+
5+
## Prerequisite: Generating the Merged CSV
6+
7+
Before updating the Google Sheet, you must generate the consolidated metadata file:
8+
9+
1. Ensure you have run all the individual enhancement scripts (or they have been run in CI):
10+
```bash
11+
npm run enhance:metadata
12+
npm run enhance:slugs
13+
npm run enhance:time
14+
npm run enhance:language
15+
```
16+
2. Run the merge script to create the final update file:
17+
```bash
18+
node scripts/merge-enhanced-metadata.js
19+
```
20+
3. Locate the generated file in `scripts/output/MERGED-enhanced-metadata-[YYYY-MM-DD].csv`.
21+
22+
---
23+
24+
## Method 1: Google Apps Script Tool (Recommended)
25+
26+
This is the safest method as it preserves all formatting (colors, conditional formatting, fonts) and only updates the specific metadata columns.
27+
28+
### Initial Setup (One-time only)
29+
1. Open the [Google Sheets Inventory](https://docs.google.com/spreadsheets/d/1Xl27XUuV_n5zG9_8Xq_87wTzN-P_A2W_Zz4k5_W-Q4/edit).
30+
2. Go to **Extensions > Apps Script**.
31+
3. In the script editor, paste the contents of `scripts/google-apps-script-updater.gs`.
32+
4. Click the **Save** icon (diskette) and name the project "Inventory Updater".
33+
5. Close the tab and refresh your Google Sheet.
34+
35+
### Running the Update
36+
1. Open the **Inventory Tools** menu that now appears in the top menu bar.
37+
2. Select **Update Enhanced Columns**.
38+
3. Open your generated CSV file (`MERGED-enhanced-metadata-...csv`) in a text editor (e.g., Notepad or VS Code).
39+
4. Select all text (Ctrl+A / Cmd+A) and copy it.
40+
5. Paste it into the dialog box in Google Sheets and click **Update Sheet**.
41+
6. The script will match lessons by their **name** and update the metadata columns.
42+
43+
---
44+
45+
## Method 2: Manual VLOOKUP (Excel/PowerShell)
46+
47+
If you prefer not to use scripts, you can use VLOOKUP in Google Sheets or Excel.
48+
49+
1. **Import the CSV** into a new tab (named `Updates`) in the Google Sheet.
50+
2. In the main tab, use a VLOOKUP formula for the columns you want to update:
51+
```excel
52+
=VLOOKUP(A2, Updates!A:Z, column_index, FALSE)
53+
```
54+
3. **Copy and Paste Values Only** once you are satisfied with the results to remove the formulas.
55+
56+
---
57+
58+
## Troubleshooting
59+
60+
### "Name not found" errors
61+
The script matches rows by the **name** column. If a lesson name has changed in either the script output or the Google Sheet, it won't be matched. Verify that the names match exactly (including spacing and capitalization).
62+
63+
### Column Mapping
64+
The following columns are updated by the script:
65+
- `author`
66+
- `license`
67+
- `contributor`
68+
- `dateCreated`
69+
- `dateModified`
70+
- `datePublished`
71+
- `inLanguage`
72+
- `timeRequired`
73+
- `identifier`
74+
- `slug`
75+
- `about`
76+
- `abstract`
77+
- `accessibilitySummary`
78+
79+
If you need to update other columns, you will need to modify the `colMap` in `google-apps-script-updater.gs`.
Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
/**
2+
* Google Apps Script for updating inventory columns based on Lesson Name.
3+
*
4+
* INSTRUCTIONS:
5+
* 1. Open your Google Sheet.
6+
* 2. Go to Extensions > Apps Script.
7+
* 3. Delete any code in the editor and paste this code.
8+
* 4. Click the Save icon and name it "Inventory Updater".
9+
* 5. Refresh your Google Sheet. You will see a new menu: "Inventory Tools".
10+
*/
11+
12+
function onOpen() {
13+
const ui = SpreadsheetApp.getUi();
14+
ui.createMenu('Inventory Tools')
15+
.addItem('Update Enhanced Columns', 'showUpdateDialog')
16+
.addToUi();
17+
}
18+
19+
/**
20+
* Shows a dialog to paste CSV data.
21+
*/
22+
function showUpdateDialog() {
23+
const html = HtmlService.createHtmlOutput(
24+
'<p>Paste the content of your MERGED-enhanced-metadata CSV file below:</p>' +
25+
'<textarea id="csvData" style="width: 100%; height: 300px; font-family: monospace;"></textarea>' +
26+
'<br><br>' +
27+
'<button onclick="runUpdate()" style="padding: 10px 20px; background: #1a73e8; color: white; border: none; border-radius: 4px; cursor: pointer;">Update Sheet</button>' +
28+
'<script>' +
29+
' function runUpdate() {' +
30+
' const data = document.getElementById("csvData").value;' +
31+
' google.script.run.withSuccessHandler(closeDialog).processCsv(data);' +
32+
' }' +
33+
' function closeDialog() {' +
34+
' google.script.host.close();' +
35+
' }' +
36+
'</script>'
37+
).setWidth(600).setHeight(450);
38+
39+
SpreadsheetApp.getUi().showModalDialog(html, 'Update Inventory Data');
40+
}
41+
42+
/**
43+
* Processes the CSV data and updates the sheet.
44+
*/
45+
function processCsv(csvString) {
46+
const sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
47+
const data = Utilities.parseCsv(csvString);
48+
49+
if (data.length < 2) {
50+
SpreadsheetApp.getUi().alert('Invalid CSV data.');
51+
return;
52+
}
53+
54+
const headers = data[0];
55+
const csvRows = data.slice(1);
56+
57+
// Get sheet data
58+
const sheetData = sheet.getDataRange().getValues();
59+
const sheetHeaders = sheetData[0];
60+
const sheetRows = sheetData.slice(1);
61+
62+
// Find column indices in CSV
63+
const colMap = {
64+
name: headers.indexOf('name'),
65+
author: headers.indexOf('author'),
66+
license: headers.indexOf('license'),
67+
contributor: headers.indexOf('contributor'),
68+
dateCreated: headers.indexOf('dateCreated'),
69+
dateModified: headers.indexOf('dateModified'),
70+
datePublished: headers.indexOf('datePublished'),
71+
inLanguage: headers.indexOf('inLanguage'),
72+
timeRequired: headers.indexOf('timeRequired'),
73+
identifier: headers.indexOf('identifier'),
74+
slug: headers.indexOf('slug'),
75+
'@id': headers.indexOf('@id'),
76+
Topic: headers.indexOf('Topic'),
77+
about: headers.indexOf('about'),
78+
abstract: headers.indexOf('abstract'),
79+
accessibilitySummary: headers.indexOf('accessibilitySummary')
80+
};
81+
82+
// Find column indices in Sheet
83+
const sheetColMap = {};
84+
Object.keys(colMap).forEach(key => {
85+
sheetColMap[key] = sheetHeaders.indexOf(key);
86+
});
87+
88+
if (sheetColMap.name === -1) {
89+
SpreadsheetApp.getUi().alert('Error: "name" column not found in Google Sheet.');
90+
return;
91+
}
92+
93+
let updateCount = 0;
94+
let skipCount = 0;
95+
96+
// Create a map for CSV data for fast lookup
97+
const csvDataMap = {};
98+
csvRows.forEach(row => {
99+
const name = row[colMap.name];
100+
if (name) csvDataMap[name] = row;
101+
});
102+
103+
// Iterate through sheet rows and update
104+
for (let i = 0; i < sheetRows.length; i++) {
105+
const sheetName = sheetRows[i][sheetColMap.name];
106+
const csvRow = csvDataMap[sheetName];
107+
108+
if (csvRow) {
109+
const rowNum = i + 2; // 1-indexed + header
110+
111+
Object.keys(colMap).forEach(key => {
112+
if (key === 'name') return; // Don't update name
113+
114+
const sheetCol = sheetColMap[key];
115+
const csvCol = colMap[key];
116+
117+
if (sheetCol !== -1 && csvCol !== -1) {
118+
const newValue = csvRow[csvCol];
119+
const oldValue = sheetRows[i][sheetCol];
120+
121+
if (newValue && newValue !== String(oldValue)) {
122+
sheet.getRange(rowNum, sheetCol + 1).setValue(newValue);
123+
updateCount++;
124+
}
125+
}
126+
});
127+
} else {
128+
skipCount++;
129+
}
130+
}
131+
132+
SpreadsheetApp.getUi().alert('Update Complete!\nUpdated cells: ' + updateCount + '\nLessons not found in CSV: ' + skipCount);
133+
}

0 commit comments

Comments
 (0)