|
| 1 | +# Upgrading to GitProxy v2 |
| 2 | + |
| 3 | +This guide attempts to cover everything needed for a seamless upgrade from GitProxy v1 (`1.19.2`) to v2. |
| 4 | + |
| 5 | +Most errors will be related to invalid database records added in v1 - mainly in the `user` and `repo` databases. As of writing, database migration files are not provided. |
| 6 | + |
| 7 | +## Breaking changes |
| 8 | + |
| 9 | +Two important breaking changes were made: |
| 10 | + |
| 11 | +### Associate commits by email |
| 12 | + |
| 13 | +Commits are no longer associated by Git's `user.name`. Now, they're associated by email (to match the handling of commits by GitHub, GitLab and other SCM providers), which allows Git Proxy to handle multiple SCM providers. [#973](github.com/finos/git-proxy/pull/973) |
| 14 | + |
| 15 | +In practice, pushes that were working in v1 (made with an improperly configured git client) may be blocked in v2 due to the change in requirements. The user's GitProxy email must match the commit's email (Git's `user.email`). This is often already required by a firm's contribution policy or to pass a CLA (Contributor License Agreement) check on a project. |
| 16 | + |
| 17 | +### Support for GitLab and other Git hosts |
| 18 | + |
| 19 | +Added support for Git SCM hosts other than GitHub. Eliminated assumptions about GitHub as the Git repository host. [#1043](https://github.com/finos/git-proxy/pull/1043) |
| 20 | + |
| 21 | +Repositories are no longer identified by name, but by internal ID instead. This means that multiple forks of the same repo are now supported, as well as repos for other Git host (GitLab, etc.). |
| 22 | + |
| 23 | +From v2 onwards, Git Proxy git URLs include the domain of the git host (e.g. https://git-proxydomain.net:8443/org/project.git has changed to https://git-proxydomain.net:8443/github.com/org/project.git). Backwards compatibility was implemented to ensure that these older URLs don't break. However, users should be advised to update the URL used in their remote in case this is removed in a subsequent major release. |
| 24 | + |
| 25 | +## Troubleshooting typical errors |
| 26 | + |
| 27 | +Most of these errors can be easily **fixed by simply accessing the UI** to delete the offending repository, add it again, and restore all the allowed users. Manually editing the database entries is not recommended, but also works. |
| 28 | + |
| 29 | +If you encounter any errors not on this guide, feel free to [open a discussion](https://github.com/finos/git-proxy/discussions). |
| 30 | + |
| 31 | +### Errors when pushing to a repo that was working in v1: |
| 32 | + |
| 33 | +#### fatal: <repo-url>/info/refs not valid: is this git repository? |
| 34 | + |
| 35 | +`git push` returns: |
| 36 | + |
| 37 | +``` |
| 38 | +fatal: <repo-url>/info/refs not valid: is this git repository? |
| 39 | +``` |
| 40 | + |
| 41 | +This error happens when pushing to GitProxy with a mismatched URL. |
| 42 | + |
| 43 | +In v1, Git URLs without the trailing `.git` were considered valid: |
| 44 | + |
| 45 | +``` |
| 46 | +"url": "https://github.com/my-org/my-repo" |
| 47 | +``` |
| 48 | + |
| 49 | +In v2, URLs are automatically formatted when adding a repo. **Repos added in v1 must be edited or re-added to fix this error**: |
| 50 | + |
| 51 | +``` |
| 52 | +"url": "https://github.com/my-org/my-repo.git" |
| 53 | +``` |
| 54 | + |
| 55 | +#### Your push has been blocked (<email> is not allowed to push on repo <repo-url>) |
| 56 | + |
| 57 | +`git push` returns: |
| 58 | + |
| 59 | +``` |
| 60 | +Your push has been blocked (<email> is not allowed to push on repo <repo-url>) |
| 61 | +``` |
| 62 | + |
| 63 | +This error occurs when pushing to GitProxy without being in the `canPush` list. This error can also occur when no GitProxy users match the given email. |
| 64 | + |
| 65 | +In v1, authorised users were matched based on `gitAccount` (which was actually the Git `user.name` and mistakenly being used as the GitHub username in the UI): |
| 66 | + |
| 67 | +``` |
| 68 | +"users":{"canPush":["John Doe"],"canAuthorise":["John Doe","admin"]} |
| 69 | +``` |
| 70 | + |
| 71 | +In v2, authorised users are identified by the email address associated with their GitProxy username. The email associated with the push (Git config `user.email`) must match their GitProxy email: |
| 72 | + |
| 73 | +Repo data: |
| 74 | + |
| 75 | +``` |
| 76 | +{"users":{"canPush":["johndoe123"],"canAuthorise":["johndoe123","admin"]"}, ...}` |
| 77 | +``` |
| 78 | + |
| 79 | +User data: |
| 80 | + |
| 81 | +``` |
| 82 | +{"username":"johndoe123","gitAccount":"<does-not-matter>","email":"<email>", ...} |
| 83 | +``` |
| 84 | + |
| 85 | +Changing the email address associated with commits can be accomplished via a number of routes, including 'rewriting history' [using rebase](https://stackoverflow.com/questions/750172/how-do-i-change-the-author-and-committer-name-email-for-multiple-commits) (dangerous but preserves the commits) or creating new commits with the correct metadata (safer but involves creating a new history/branch). |
| 86 | + |
| 87 | +## Other notable changes |
| 88 | + |
| 89 | +### Features |
| 90 | + |
| 91 | +- Replaced `getMissingData` action with `checkEmptyBranch` to handle empty branch processing in [#1134](https://github.com/finos/git-proxy/pull/1134) |
| 92 | + - `getMissingData` was setting the `Commit` object's `committer` to the `author_name` which is not always true. Furthermore, the edge case that `getMissingData` was trying to solve was already covered by the `checkHiddenCommits` action |
| 93 | + - `checkEmptyBranch` simply checks whether the branch has had any new commits (if not, the push will be rejected) |
| 94 | +- Added a settings page for configuring the JWT token to authenticate UI requests to API when `apiAuthentication` is enabled in [#1096](https://github.com/finos/git-proxy/pull/1096) |
| 95 | + - Previously, requests from the UI were bypassing the JWT check if the user was logged in, and failing otherwise when `apiAuthentication` was set |
| 96 | + - For more details on setting JWT, check the [architecture documentation](./Architecture.md#setting-up-jwt-authentication): |
| 97 | +- Added the ability to create new users via the GitProxy CLI in [#981](https://github.com/finos/git-proxy/pull/981) |
| 98 | +- Added `/healthcheck` endpoint for AWS Load Balancer support [#1197](https://github.com/finos/git-proxy/pull/1197) |
| 99 | +- Improved login page flexibility, error handling and visibility of available auth methods in [#1227](https://github.com/finos/git-proxy/pull/1227) |
| 100 | +- Added config schema for `commitConfig`, `attestationConfig` and `domains` in [#1243](https://github.com/finos/git-proxy/pull/1243) |
| 101 | + - See the [schema reference](https://git-proxy.finos.org/docs/configuration/reference) for a detailed description of each |
| 102 | + - Also removes the defunct `api.github` config element |
| 103 | +- Added confirmation dialog to `RepoDetails` page to prevent accidental repository deletions in [#1267](https://github.com/finos/git-proxy/pull/1267) |
| 104 | +- Added support for using AWS Credential Provider to authenticate MongoDB connections in [#1319](https://github.com/finos/git-proxy/pull/1319) |
| 105 | +- Optimized push speed by performing shallow clones by default in [#1189](https://github.com/finos/git-proxy/pull/1189) |
| 106 | + - Increased push speeds for larger repos [by around 30~50%](https://github.com/finos/git-proxy/issues/985) |
| 107 | +- Improved configuration validation and typing in [#1140](https://github.com/finos/git-proxy/pull/1140) |
| 108 | + |
| 109 | +### Bugfixes |
| 110 | + |
| 111 | +- Fixed issue where requests for unknown repos were being forwarded to GitHub instead of being blocked as expected in [#1163](https://github.com/finos/git-proxy/issues/1163) |
| 112 | + - Improved error handling on chain execution to ensure errors always block pushes |
| 113 | + - Ensured `checkRepoInAuthList` is run for all requests |
| 114 | +- Fixed MongoDB client implementation issues (not awaiting promises, searching repos against the wrong field) in [#1167](https://github.com/finos/git-proxy/pull/1167) |
| 115 | +- Fixed issues with Git client not rendering error messages on rejected pushes in [#1178](https://github.com/finos/git-proxy/pull/1178) |
| 116 | + - Reverted previous changes to status codes on rejected pushes since the Git client only renders errors on `200 OK` |
| 117 | +- Fixed Push table committer and author links, replaced links to profile with `mailto:` in [#1179](https://github.com/finos/git-proxy/pull/1179) |
| 118 | +- Fixed display errors when adding a new repo in [#1120](https://github.com/finos/git-proxy/pull/1120) |
| 119 | + - Caused by an issue with server side errors being silently ignored |
| 120 | +- Fixed `--force` pushes failing due to the `getDiff` action blocking legitimate empty diffs in [#1182](https://github.com/finos/git-proxy/pull/1182) |
| 121 | +- Fixed incorrect error message on cloning unauthorized repos in [#1204](https://github.com/finos/git-proxy/pull/1204) |
| 122 | + - Caused by improper Git protocol error handling for `GET /info/refs` requests, resulting in Git client receiving malformed `upload-pack` data |
| 123 | +- Fixed duplicated chain execution when pushing a PR that has been approved in [#1209](https://github.com/finos/git-proxy/pull/1209) |
| 124 | + - Caused by an issue with raw body extraction on `POST git-pack` requests |
| 125 | +- Reimplemented push parsing to fix various issues related to packfile decoding in [#1187](https://github.com/finos/git-proxy/pull/1187) |
| 126 | + - Fixed `Z_DATA_ERROR` when pushing |
| 127 | + - Fixed Git object header parsing and packfile metadata reading |
| 128 | + - Reimplemented decompression to better replicate how Git handles it (replaced inflating/deflating the object) |
| 129 | +- Fixed logout failure in production caused by UI defaulting to `http://localhost:3000` when `VITE_API_URI` is unset in [#1201](https://github.com/finos/git-proxy/pull/1201) |
| 130 | + - Refactors API URL usages to rely on a single source of truth, sets default values |
| 131 | +- Fixed a potential denial-of-service vulnerability when pushing to an unknown repository in [#1095](https://github.com/finos/git-proxy/pull/1095) |
| 132 | + - Caused by a bug in the MongoDB implementation `isUserPushAllowed` which assumed that the repository exists. If the repository wasn't found, the backend crashed when attempting to access its properties |
| 133 | +- Fixed `MongoServerError` when updating user due to attempting to override the pre-existent `_id` in [#1230](https://github.com/finos/git-proxy/pull/1230) |
| 134 | +- Fixed error with `commitConfig.diff.block.literals` entry being matched as regular expressions instead in [#1251](https://github.com/finos/git-proxy/pull/1251) |
| 135 | +- Fixed infinite loop in `UserList` component causing excessive API requests and preventing proper rendering in [#1255](https://github.com/finos/git-proxy/pull/1255) |
| 136 | +- Fixed broken user links in `PushDetails` and `RepoDetails` components in [#1268](https://github.com/finos/git-proxy/pull/1268) |
| 137 | + - Created `UserLink` component to centralise user navigation |
| 138 | +- Fixed pagination component to show correct page count when no data is available in [#1274](https://github.com/finos/git-proxy/pull/1274) |
| 139 | +- Fixed proxy startup failure due to default repo mismatch in [#1284](https://github.com/finos/git-proxy/pull/1284) |
| 140 | + - Caused by matching repos by name instead of URL on calling `proxyPreparations` |
| 141 | +- Fixed error when making subsequent pushes to a new branch in [#1291](https://github.com/finos/git-proxy/pull/1291) |
| 142 | + - `Error: fatal: Invalid revision range` was being thrown on valid pushes to new branches |
| 143 | + - Caused by setting `singleBranch: true` when pulling the remote repo for optimization purposes |
| 144 | + - Removal of this option does not affect pull/push times considerably. Rudimentary benchmarks show that despite removing the option, push speeds [are still considerably faster](https://github.com/finos/git-proxy/pull/1305#issuecomment-3611774012) than without the `depth: 1` optimization |
| 145 | +- Fixed misleading backend status codes and improved UI error handling in [#1293](https://github.com/finos/git-proxy/pull/1293) |
| 146 | + - Also removed redundant `/api/auth/me` endpoint |
| 147 | +- Fixed race condition preventing MongoDB connection when loading configuration in [#1316](https://github.com/finos/git-proxy/pull/1316) |
| 148 | + - Deferred retrieval of database config allowing the user configuration to be loaded before attempting to use it |
| 149 | +- Replaced `jwk-to-pem` dependency with native `crypto` to remove vulnerable dependency (`elliptic`) in [#1283](https://github.com/finos/git-proxy/pull/1283) |
0 commit comments