Skip to content

Commit a441ce2

Browse files
themactepCopilot
andcommitted
scripts: improve manage-package-overrides.sh clone workflow
- Clone with --recurse-submodules so submodules are initialised - Use --recurse-submodules on git pull during updates too - Add fork-first workflow for GitHub repos: fork via gh, clone the fork as origin, add original as upstream remote - Add -f/--fork flag to auto-accept the fork prompt - Create a local branch at the pinned version commit instead of leaving the repo in detached HEAD state - Redirect all print_* output inside fork_github_repo to stderr so the fork URL captured by command substitution stays clean - Abort override registration when git clone fails docs: add local-overrides.md newbie guide covering the full workflow from listing packages through cloning, building, and opening a PR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent d10a41a commit a441ce2

File tree

2 files changed

+214
-6
lines changed

2 files changed

+214
-6
lines changed

docs/local-overrides.md

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Working with Local Package Overrides
2+
3+
Buildroot's **override** mechanism lets you replace a package's downloaded source
4+
with a local directory on your machine. This is the standard way to develop or
5+
patch a package without touching the Buildroot internals.
6+
7+
The `scripts/manage-package-overrides.sh` helper automates the whole workflow.
8+
9+
---
10+
11+
## How it works
12+
13+
When a package has an override, Buildroot reads its source from the directory you
14+
specify instead of downloading the upstream tarball or cloning the upstream repo.
15+
The mapping lives in `local.mk`:
16+
17+
```
18+
EXFAT_NOFUSE_OVERRIDE_SRCDIR = $(BR2_EXTERNAL)/overrides/exfat-nofuse
19+
```
20+
21+
During a build, Buildroot uses `rsync` to copy that directory into the build tree,
22+
so you can edit files in `overrides/<package>/`, run `make <package>-rebuild`, and
23+
see your changes without a full rebuild.
24+
25+
---
26+
27+
## Quick-start: set up an override
28+
29+
### 1. Find the package name
30+
31+
```sh
32+
./scripts/manage-package-overrides.sh -l # list all packages
33+
./scripts/manage-package-overrides.sh -l thingino-* # filter by pattern
34+
```
35+
36+
### 2. Clone and register an override (interactive)
37+
38+
```sh
39+
./scripts/manage-package-overrides.sh <package-name>
40+
```
41+
42+
The script reads the package's `.mk` file, shows you the upstream URL and pinned
43+
version, then asks:
44+
45+
- **Fork this GitHub repo before cloning?** – say `y` if this is a foreign
46+
(third-party) repo you may want to send changes back to. The script forks it
47+
under your GitHub account, clones the fork, and adds the original as `upstream`.
48+
- **Download/clone this package?** – say `y` to proceed.
49+
50+
The clone lands in `overrides/<package>/` and the entry is written to `local.mk`.
51+
52+
#### Auto mode (no prompts)
53+
54+
```sh
55+
./scripts/manage-package-overrides.sh -a <pattern> # clone without asking
56+
./scripts/manage-package-overrides.sh -a -f <pattern> # also fork automatically
57+
```
58+
59+
---
60+
61+
## Making and building changes
62+
63+
```sh
64+
# edit source files
65+
vim overrides/exfat-nofuse/exfat.c
66+
67+
# rebuild just that package
68+
make exfat-nofuse-rebuild
69+
70+
# rebuild and re-generate the firmware image
71+
make exfat-nofuse-rebuild all
72+
```
73+
74+
---
75+
76+
## Contributing changes back
77+
78+
When you cloned via a fork (`-f`), the repo is pre-configured for the full
79+
fork → branch → PR workflow:
80+
81+
| Remote | Points to |
82+
|--------|-----------|
83+
| `origin` | Your fork on GitHub |
84+
| `upstream` | The original repo |
85+
86+
A local branch named `local-<short-hash>` was created at the pinned commit, so
87+
you are never in a detached HEAD state.
88+
89+
```sh
90+
cd overrides/exfat-nofuse
91+
92+
# commit your work
93+
git add -p
94+
git commit -m "fix: description of the change"
95+
96+
# push to your fork
97+
git push origin local-01c30ad...
98+
99+
# open a pull request against the original repo
100+
gh pr create --repo dorimanx/exfat-nofuse --base master \
101+
--title "fix: description" --body "Details..."
102+
```
103+
104+
---
105+
106+
## Managing existing overrides
107+
108+
| Task | Command |
109+
|------|---------|
110+
| List all overrides and their paths | `./scripts/manage-package-overrides.sh -l` |
111+
| Temporarily disable (comment out) | `./scripts/manage-package-overrides.sh -d <package>` |
112+
| Re-enable a disabled override | `./scripts/manage-package-overrides.sh -e <package>` |
113+
| Pull latest changes in an override | `./scripts/manage-package-overrides.sh -u <package>` |
114+
| Update all overrides at once | `./scripts/manage-package-overrides.sh -u --all` |
115+
| Remove an override entry | `./scripts/manage-package-overrides.sh -r <package>` |
116+
| Remove all override entries | `./scripts/manage-package-overrides.sh --clean` |
117+
118+
Disabling an override comments out the `local.mk` line so Buildroot falls back to
119+
the upstream source, without losing the local clone.
120+
121+
---
122+
123+
## Directory layout
124+
125+
```
126+
firmware/
127+
├── local.mk ← override mappings (git-ignored, machine-local)
128+
├── overrides/ ← git-ignored; holds your local clones
129+
│ └── exfat-nofuse/ ← working clone (origin = fork, upstream = original)
130+
└── scripts/
131+
└── manage-package-overrides.sh
132+
```
133+
134+
Both `local.mk` and `overrides/` are git-ignored, so your local development setup
135+
never pollutes the firmware repository.

scripts/manage-package-overrides.sh

Lines changed: 79 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ BASE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
77
PACKAGE_DIR="$BASE_DIR/package"
88
OVERRIDES_DIR="$BASE_DIR/overrides"
99
LOCAL_MK="$BASE_DIR/local.mk"
10+
FORK_MODE="no"
1011

1112
# Colors for output
1213
RED='\033[0;31m'
@@ -40,6 +41,7 @@ Traverse packages in package/ directory and manage local source overrides.
4041
OPTIONS:
4142
-h, --help Show this help message
4243
-a, --auto Automatically download all matching packages without prompting
44+
-f, --fork Fork GitHub repos before cloning (enables contribution workflow)
4345
-l, --list List packages and their override status only
4446
-r, --remove PACKAGE Remove override for specified package
4547
-e, --enable PACKAGE Enable (uncomment) override for specified package
@@ -317,7 +319,7 @@ update_override() {
317319

318320
# Pull latest changes
319321
print_info "Pulling latest changes on $current_branch..."
320-
if git pull --ff-only origin "$current_branch" 2>&1; then
322+
if git pull --recurse-submodules --ff-only origin "$current_branch" 2>&1; then
321323
print_success "Updated $pkg_name successfully"
322324
else
323325
print_error "Failed to update $pkg_name (conflicts or non-fast-forward)"
@@ -329,6 +331,33 @@ update_override() {
329331
return 0
330332
}
331333

334+
# Check if a URL points to a GitHub repository
335+
is_github_url() {
336+
[[ "$1" =~ github\.com ]]
337+
}
338+
339+
# Fork a GitHub repo and echo the fork's clone URL
340+
fork_github_repo() {
341+
local repo_url="$1"
342+
local repo_path
343+
repo_path=$(echo "$repo_url" | sed -E 's|https://github\.com/||; s|git@github\.com:||; s|\.git$||')
344+
345+
print_info "Forking $repo_path on GitHub..." >&2
346+
if ! gh repo fork "$repo_path" --clone=false >&2 2>&1; then
347+
print_error "Failed to fork repository" >&2
348+
return 1
349+
fi
350+
351+
local gh_user
352+
gh_user=$(gh api user --jq .login 2>/dev/null)
353+
if [ -z "$gh_user" ]; then
354+
print_error "Could not determine GitHub username" >&2
355+
return 1
356+
fi
357+
358+
echo "https://github.com/$gh_user/$(basename "$repo_path")"
359+
}
360+
332361
# Clone or download package source
333362
download_package() {
334363
local pkg_name="$1"
@@ -355,18 +384,58 @@ download_package() {
355384
fi
356385

357386
if [ "$method" = "git" ]; then
358-
print_info "Cloning $site to $dest_dir"
387+
local clone_url="$site"
388+
local upstream_url=""
389+
390+
# Offer to fork GitHub repos for an easier contribution workflow
391+
if is_github_url "$site"; then
392+
local do_fork="$FORK_MODE"
393+
if [ "$do_fork" != "yes" ]; then
394+
read -p "Fork this GitHub repo before cloning? [y/N]: " -n 1 -r
395+
echo
396+
[[ $REPLY =~ ^[Yy]$ ]] && do_fork="yes"
397+
fi
398+
if [ "$do_fork" = "yes" ]; then
399+
local fork_url
400+
if fork_url=$(fork_github_repo "$site"); then
401+
upstream_url="$site"
402+
clone_url="$fork_url"
403+
print_success "Will clone fork: $clone_url"
404+
else
405+
print_warning "Forking failed, cloning original..."
406+
fi
407+
fi
408+
fi
409+
410+
print_info "Cloning $clone_url to $dest_dir"
359411

360412
if [ -n "$branch" ]; then
361-
git clone --branch "$branch" "$site" "$dest_dir"
413+
if ! git clone --recurse-submodules --branch "$branch" "$clone_url" "$dest_dir"; then
414+
print_error "Clone failed"
415+
return 1
416+
fi
362417
else
363-
git clone "$site" "$dest_dir"
418+
if ! git clone --recurse-submodules "$clone_url" "$dest_dir"; then
419+
print_error "Clone failed"
420+
return 1
421+
fi
422+
fi
423+
424+
# Add upstream remote if we cloned a fork
425+
if [ -n "$upstream_url" ]; then
426+
git -C "$dest_dir" remote add upstream "$upstream_url"
427+
print_info "Added upstream remote: $upstream_url"
364428
fi
365429

366-
# Checkout specific version if not HEAD
430+
# Checkout specific version if not HEAD; create a local branch to avoid detached HEAD
367431
if [ -n "$version" ] && [ "$version" != "HEAD" ]; then
368432
cd "$dest_dir"
369-
git checkout "$version" 2>/dev/null || print_warning "Could not checkout version: $version"
433+
local local_branch="local-$(echo "$version" | tr '/' '-' | cut -c1-30)"
434+
if git checkout -b "$local_branch" "$version" 2>/dev/null; then
435+
print_info "Created local branch '$local_branch' at $version"
436+
else
437+
print_warning "Could not create branch at version: $version"
438+
fi
370439
cd - >/dev/null
371440
fi
372441

@@ -630,6 +699,10 @@ main() {
630699
auto_mode="yes"
631700
shift
632701
;;
702+
-f|--fork)
703+
FORK_MODE="yes"
704+
shift
705+
;;
633706
-l|--list)
634707
mode="list"
635708
shift

0 commit comments

Comments
 (0)