diff --git a/docs/usage/bot-comparison.md b/docs/usage/bot-comparison.md index bb4a4acacfa..b372e116cd4 100644 --- a/docs/usage/bot-comparison.md +++ b/docs/usage/bot-comparison.md @@ -12,7 +12,7 @@ If you see anything wrong on this page, please let us know by creating a [Discus | Dependency Dashboard | Yes | No | | Grouped updates | Yes, use community-provided groups, or create your own | Yes, create [`groups`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups) manually or [handled automatically by dependabot](https://docs.github.com/en/code-security/dependabot/dependabot-security-updates/about-dependabot-security-updates#about-grouped-security-updates) | | Upgrades common monorepo packages at once | Yes | Yes | -| Officially supported platforms | Azure, Bitbucket, Forgejo, Gitea, GitHub, GitLab, see [full list](./index.md#supported-platforms) | GitHub and Azure DevOps | +| Officially supported platforms | Azure, Bitbucket, Forgejo, Gitea, GitHub, GitLab, SCM-Manager, see [full list](./index.md#supported-platforms) | GitHub and Azure DevOps | | Supported languages | [List for Renovate](./modules/manager/index.md) | [List for Dependabot](https://docs.github.com/en/code-security/dependabot/ecosystems-supported-by-dependabot/supported-ecosystems-and-repositories#supported-ecosystems-and-repositories) | | Show changelogs | Yes | Yes | | Compatibility score badges | Four badges showing: Age, Adoption, Passing, Confidence | One badge with overall compatibility score | diff --git a/docs/usage/configuration-options.md b/docs/usage/configuration-options.md index dc02a313af5..cabb410b6e6 100644 --- a/docs/usage/configuration-options.md +++ b/docs/usage/configuration-options.md @@ -4712,7 +4712,7 @@ Please see the above link for valid timezone names. ## toolSettings -When Renovate updates a dependency and needs to invoke processes leveraging Java, for example Gradle for [the `gradle-wrapper` manager](./modules/manager/gradle-wrapper/index.md), the repository's Gradle Wrapper will be invoked, if present. +When Renovate updates a dependency and needs to invoke processes leveraging Java, for example Gradle for [the `gradle`](./modules/manager/gradle/index.md) or [the `gradle-wrapper`](./modules/manager/gradle-wrapper/index.md) managers, the repository's Gradle Wrapper will be invoked, if present. The JVM heap size for the Java invocations is 512m by default. This can be overridden using the following options. @@ -4726,14 +4726,14 @@ This option can be used on the repository level and in the [Renovate configurati !!! note - The JVM memory settings are considered for the `gradle-wrapper` manager. + The JVM memory settings are considered for the `gradle` and `gradle-wrapper` manager. ### jvmMaxMemory Maximum heap size in MB for Java VMs. Defaults to `512` for both the repository level and self-hosted configuration. -To allow repositories to use _more_ than 512m of heap during the Gradle Wrapper update, configure the `jvmMaxMemory` option in the [`toolSettings.jvmMaxMemory`](./self-hosted-configuration.md). +To allow repositories to use _more_ than 512m of heap during any invocations of the Gradle Wrapper, configure the `jvmMaxMemory` option in the [`toolSettings.jvmMaxMemory`](./self-hosted-configuration.md). ### jvmMemory diff --git a/docs/usage/faq.md b/docs/usage/faq.md index 6b59b5d4174..9bb64d87e4c 100644 --- a/docs/usage/faq.md +++ b/docs/usage/faq.md @@ -31,10 +31,10 @@ If you're self hosting Renovate, use the latest release if possible. ## Renovate core features not supported on all platforms -| Feature | Platforms which lack feature | See Renovate issue(s) | -| --------------------- | ----------------------------------------------- | ------------------------------------------------------------ | -| Dependency Dashboard | Azure, Bitbucket, Bitbucket Server, Gerrit | [#9592](https://github.com/renovatebot/renovate/issues/9592) | -| The Mend Renovate App | Azure, Bitbucket Server, Forgejo, Gitea, GitLab | | +| Feature | Platforms which lack feature | See Renovate issue(s) | +| --------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| Dependency Dashboard | Azure, Bitbucket, Bitbucket Server, Gerrit, SCM-Manager | [#9592](https://github.com/renovatebot/renovate/issues/9592) | +| The Mend Renovate App | Azure, Bitbucket Server, Forgejo, Gitea, GitLab, SCM-Manager | | ## Major platform features not supported by Renovate diff --git a/docs/usage/getting-started/running.md b/docs/usage/getting-started/running.md index 2789a19c6a9..5253d8c8cb3 100644 --- a/docs/usage/getting-started/running.md +++ b/docs/usage/getting-started/running.md @@ -217,6 +217,7 @@ Read the platform-specific docs to learn how to setup authentication on your pla - [Gitea](../modules/platform/gitea/index.md) - [github.com and GitHub Enterprise Server](../modules/platform/github/index.md) - [GitLab](../modules/platform/gitlab/index.md) +- [SCM-Manager](../modules/platform/scm-manager/index.md) ### GitHub.com token for changelogs (and tools) diff --git a/docs/usage/self-hosted-configuration.md b/docs/usage/self-hosted-configuration.md index cbfe03fc04f..ef081c0617f 100644 --- a/docs/usage/self-hosted-configuration.md +++ b/docs/usage/self-hosted-configuration.md @@ -1130,6 +1130,10 @@ It also may mean that ignored directories like `node_modules` can be preserved a ## platform +## prCacheSyncMaxPages + +Maximum number of pages to fetch when syncing the pull request cache. + ## prCommitsPerRunLimit Parameter to reduce CI load. diff --git a/lib/config/global.ts b/lib/config/global.ts index 6affca9596d..a8810c47fc8 100644 --- a/lib/config/global.ts +++ b/lib/config/global.ts @@ -45,6 +45,7 @@ export class GlobalConfig { 'onboardingNoDeps', 'onboardingPrTitle', 'platform', + 'prCacheSyncMaxPages', 'presetCachePersistence', 'repositoryCacheForceLocal', 's3Endpoint', diff --git a/lib/config/options/index.ts b/lib/config/options/index.ts index 8e214d33b59..25e71e8d075 100644 --- a/lib/config/options/index.ts +++ b/lib/config/options/index.ts @@ -514,7 +514,14 @@ const options: Readonly[] = [ 'If set to `true` then Renovate creates draft PRs, instead of normal status PRs.', type: 'boolean', default: false, - supportedPlatforms: ['azure', 'forgejo', 'gitea', 'github', 'gitlab'], + supportedPlatforms: [ + 'azure', + 'forgejo', + 'gitea', + 'github', + 'gitlab', + 'scm-manager', + ], }, { name: 'dryRun', @@ -1050,7 +1057,12 @@ const options: Readonly[] = [ description: 'Username for authentication.', stage: 'repository', type: 'string', - supportedPlatforms: ['azure', 'bitbucket', 'bitbucket-server'], + supportedPlatforms: [ + 'azure', + 'bitbucket', + 'bitbucket-server', + 'scm-manager', + ], globalOnly: true, }, { @@ -3190,7 +3202,13 @@ const options: Readonly[] = [ description: 'Overrides the default resolution for Git remote, e.g. to switch GitLab from HTTPS to SSH-based.', type: 'string', - supportedPlatforms: ['bitbucket-server', 'forgejo', 'gitea', 'gitlab'], + supportedPlatforms: [ + 'bitbucket-server', + 'forgejo', + 'gitea', + 'gitlab', + 'scm-manager', + ], allowedValues: ['default', 'ssh', 'endpoint'], default: 'default', stage: 'repository', @@ -3311,6 +3329,17 @@ const options: Readonly[] = [ default: 90, globalOnly: true, }, + { + name: 'prCacheSyncMaxPages', + description: + 'Maximum number of pages to fetch when syncing the pull request cache.', + type: 'integer', + default: 100, + globalOnly: true, + supportedPlatforms: ['github'], + experimental: true, + experimentalIssues: [41485], + }, { name: 'dockerMaxPages', description: diff --git a/lib/config/presets/index.spec.ts b/lib/config/presets/index.spec.ts index ebaf0670aa7..aa7bd14b0aa 100644 --- a/lib/config/presets/index.spec.ts +++ b/lib/config/presets/index.spec.ts @@ -617,6 +617,13 @@ describe('config/presets/index', () => { }); describe('getPreset', () => { + it('does not use cache for internal presets', async () => { + const memCacheGetSpy = vi.spyOn(memCache, 'get'); + expect(await presets.getPreset(':dependencyDashboard', {})).toBeDefined(); + expect(memCacheGetSpy).not.toHaveBeenCalled(); + expect(packageCache.get).not.toHaveBeenCalled(); + }); + it('handles removed presets with a migration', async () => { const res = await presets.getPreset(':base', {}); expect(res).toEqual({ diff --git a/lib/config/presets/index.ts b/lib/config/presets/index.ts index 1f00c13cf14..fdb3643e897 100644 --- a/lib/config/presets/index.ts +++ b/lib/config/presets/index.ts @@ -46,7 +46,6 @@ const presetSources: Record = { github, gitlab, http, - internal, local, npm, }; @@ -120,33 +119,49 @@ export async function getPreset( } const { presetSource, repo, presetPath, presetName, tag, params, rawParams } = parsePreset(preset); - const cacheKey = `preset:${preset}`; - const presetCachePersistence = GlobalConfig.get( - 'presetCachePersistence', - false, - ); let presetConfig: Preset | null | undefined; - if (presetCachePersistence) { - presetConfig = await packageCache.get(presetCacheNamespace, cacheKey); - } else { - presetConfig = memCache.get(cacheKey); - } - - if (isNullOrUndefined(presetConfig)) { - presetConfig = await presetSources[presetSource].getPreset({ + if (presetSource === 'internal') { + presetConfig = internal.getPreset({ repo, presetPath, presetName, tag, }); + } else { + const cacheKey = `preset:${preset}`; + const presetCachePersistence = GlobalConfig.get( + 'presetCachePersistence', + false, + ); + if (presetCachePersistence) { - await packageCache.set(presetCacheNamespace, cacheKey, presetConfig, 15); + presetConfig = await packageCache.get(presetCacheNamespace, cacheKey); } else { - memCache.set(cacheKey, presetConfig); + presetConfig = memCache.get(cacheKey); + } + + if (isNullOrUndefined(presetConfig)) { + presetConfig = await presetSources[presetSource].getPreset({ + repo, + presetPath, + presetName, + tag, + }); + if (presetCachePersistence) { + await packageCache.set( + presetCacheNamespace, + cacheKey, + presetConfig, + 15, + ); + } else { + memCache.set(cacheKey, presetConfig); + } } } + if (!presetConfig) { throw new Error(PRESET_DEP_NOT_FOUND); } diff --git a/lib/config/presets/local/index.ts b/lib/config/presets/local/index.ts index 59827bca511..631441d6099 100644 --- a/lib/config/presets/local/index.ts +++ b/lib/config/presets/local/index.ts @@ -29,6 +29,7 @@ const resolvers = { github, gitlab, local: null, + 'scm-manager': null, } satisfies Record; export function getPreset({ diff --git a/lib/config/types.ts b/lib/config/types.ts index 450150bc56f..d9e262e5085 100644 --- a/lib/config/types.ts +++ b/lib/config/types.ts @@ -245,6 +245,7 @@ export interface RepoGlobalConfig extends GlobalInheritableConfig { localDir?: string; migratePresets?: Record; platform?: PlatformId; + prCacheSyncMaxPages?: number; presetCachePersistence?: boolean; httpCacheTtlDays?: number; autodiscoverRepoSort?: RepoSortMethod; diff --git a/lib/constants/platforms.ts b/lib/constants/platforms.ts index 93e709c5ffc..8b16552f143 100644 --- a/lib/constants/platforms.ts +++ b/lib/constants/platforms.ts @@ -9,6 +9,7 @@ export const PLATFORM_HOST_TYPES = [ 'github', 'gitlab', 'local', + 'scm-manager', ] as const; export type PlatformId = (typeof PLATFORM_HOST_TYPES)[number]; diff --git a/lib/logger/index.spec.ts b/lib/logger/index.spec.ts index f5058751026..921bb216c3a 100644 --- a/lib/logger/index.spec.ts +++ b/lib/logger/index.spec.ts @@ -370,7 +370,12 @@ describe('logger/index', () => { expect(logged.msg).toBe('foo'); expect(logged.foo.foo).toBe('[Circular]'); expect(logged.foo.bar).toEqual(['[Circular]']); - expect(logged.bar).toBe('[Circular]'); + expect(logged.bar).toEqual([ + { + bar: '[Circular]', + foo: '[Circular]', + }, + ]); }); it('sanitizes secrets', () => { diff --git a/lib/logger/utils.ts b/lib/logger/utils.ts index 96108324546..963649adbcc 100644 --- a/lib/logger/utils.ts +++ b/lib/logger/utils.ts @@ -20,6 +20,7 @@ import { ZodError } from 'zod/v3'; import { ExecError } from '../util/exec/exec-error.ts'; import { regEx } from '../util/regex.ts'; import { redactedFields, sanitize } from '../util/sanitize.ts'; +import { quickStringify } from '../util/stringify.ts'; import type { BunyanRecord, BunyanStream } from './types.ts'; const excludeProps = ['pid', 'time', 'v', 'hostname']; @@ -284,10 +285,7 @@ export function withSanitizer(streamConfig: bunyan.Stream): bunyan.Stream { const result = streamConfig.type === 'raw' ? raw - : JSON.stringify(raw, bunyan.safeCycles()).replace( - regEx(/\n?$/), - '\n', - ); + : quickStringify(raw)?.replace(regEx(/\n?$/), '\n'); stream.write(result, enc, cb); }; diff --git a/lib/modules/manager/cake/index.spec.ts b/lib/modules/manager/cake/index.spec.ts index ff3cd71bcd6..7f7202278c3 100644 --- a/lib/modules/manager/cake/index.spec.ts +++ b/lib/modules/manager/cake/index.spec.ts @@ -1,3 +1,4 @@ +import { codeBlock } from 'common-tags'; import { Fixtures } from '~test/fixtures.ts'; import { extractPackageFile } from './index.ts'; @@ -19,4 +20,79 @@ describe('modules/manager/cake/index', () => { ], }); }); + + it('extracts dotnet tools from single sdk style build file', () => { + const content = codeBlock` + #:sdk Cake.Sdk + + // Install single tool + InstallTool("dotnet:https://api.nuget.org/v3/index.json?package=SingleTool.Install.First&version=1.0.0"); + InstallTool("dotnet:?package=SingleTool.Install.Second&version=1.2.0"); + + // Install multiple tools at once + InstallTools( + "dotnet:https://api.nuget.org/v3/index.json?package=MultipleTools.Install.First&version=2.0.0", + "dotnet:?package=MultipleTools.Install.Second&version=2.1.1" + ); + + var target = Argument("target", "Default"); + + Task("Default") + .Does(() => + { + Information("Hello from Cake.Sdk!"); + }); + + var installTools = "dotnet:?Should.Not.Match&version=1.0.0"; + + RunTarget(target); + `; + expect(extractPackageFile(content)).toMatchObject({ + deps: [ + { + depName: 'SingleTool.Install.First', + currentValue: '1.0.0', + datasource: 'nuget', + registryUrls: ['https://api.nuget.org/v3/index.json'], + }, + { + depName: 'SingleTool.Install.Second', + currentValue: '1.2.0', + datasource: 'nuget', + }, + { + depName: 'MultipleTools.Install.First', + currentValue: '2.0.0', + datasource: 'nuget', + registryUrls: ['https://api.nuget.org/v3/index.json'], + }, + { + depName: 'MultipleTools.Install.Second', + currentValue: '2.1.1', + datasource: 'nuget', + }, + ], + }); + }); + + it('skips invalid entries in InstallTools', () => { + const content = codeBlock` + #:sdk Cake.Sdk + + // One invalid and one valid tool entry + InstallTools( + "dotnet:bad uri", + "dotnet:?package=Good.Tool&version=1.2.3" + ); + `; + expect(extractPackageFile(content)).toMatchObject({ + deps: [ + { + depName: 'Good.Tool', + currentValue: '1.2.3', + datasource: 'nuget', + }, + ], + }); + }); }); diff --git a/lib/modules/manager/cake/index.ts b/lib/modules/manager/cake/index.ts index c9098e5e588..84f06ffd28e 100644 --- a/lib/modules/manager/cake/index.ts +++ b/lib/modules/manager/cake/index.ts @@ -25,6 +25,10 @@ const lexer = moo.states({ match: /^#(?:addin|tool|module|load|l)\s+"(?:nuget|dotnet):[^"]+"\s*$/, // TODO #12870 value: (s: string) => s.trim().slice(1, -1), }, + dependencyFromInstallTools: { + match: /(?:InstallTools?\s*\()[^)]+(?:\s*\)\s*;)/, + lineBreaks: true, + }, unknown: moo.fallback, }, }); @@ -74,6 +78,15 @@ export function extractPackageFile(content: string): PackageFileContent { if (dep) { deps.push(dep); } + } else if (type === 'dependencyFromInstallTools') { + const matches = value.matchAll(regEx(/"dotnet:[^"]+"/g)); + for (const match of matches) { + const withoutQuote = match.toString().slice(1, -1); + const dep = parseDependencyLine(withoutQuote); + if (dep) { + deps.push(dep); + } + } } token = lexer.next(); } diff --git a/lib/modules/manager/cake/readme.md b/lib/modules/manager/cake/readme.md index 37d6820850a..00d610afaed 100644 --- a/lib/modules/manager/cake/readme.md +++ b/lib/modules/manager/cake/readme.md @@ -1 +1,3 @@ Extracts dependencies from `*.cake` files. + +It can also extract `dotnet:` tool packages when used in .NET single file builds, when they are used with the `InstallTool` or `InstallTools` method. Keep in mind that those files are usually not `*.cake` files but C# code files. Those are not included in the `managerFilePatterns` per default - make sure to include them manually in your configuration if you want to use this feature.`#:package`or`#:sdk` directives are not handled by the cake manager as this is a dotnet feature. Those are separately handled by the nuget manager. diff --git a/lib/modules/manager/gomod/artifacts.spec.ts b/lib/modules/manager/gomod/artifacts.spec.ts index e4148e20c2f..34b99242ae3 100644 --- a/lib/modules/manager/gomod/artifacts.spec.ts +++ b/lib/modules/manager/gomod/artifacts.spec.ts @@ -8,7 +8,7 @@ import type { RepoGlobalConfig } from '../../../config/types.ts'; import { TEMPORARY_ERROR } from '../../../constants/error-messages.ts'; import * as docker from '../../../util/exec/docker/index.ts'; import type { StatusResult } from '../../../util/git/types.ts'; -import * as _hostRules from '../../../util/host-rules.ts'; +import * as hostRules from '../../../util/host-rules.ts'; import * as _datasource from '../../datasource/index.ts'; import type { UpdateArtifactsConfig } from '../types.ts'; import * as _artifactsExtra from './artifacts-extra.ts'; @@ -17,7 +17,6 @@ import * as gomod from './index.ts'; type FS = typeof import('../../../util/fs/index.ts'); vi.mock('../../../util/exec/env.ts'); -vi.mock('../../../util/host-rules.ts', () => mockDeep()); vi.mock('../../../util/http/index.ts'); vi.mock('../../../util/fs/index.ts', async () => { // restore @@ -32,7 +31,6 @@ vi.mock('./artifacts-extra.ts', () => mockDeep()); process.env.CONTAINERBASE = 'true'; const datasource = vi.mocked(_datasource); -const hostRules = vi.mocked(_hostRules); const artifactsExtra = vi.mocked(_artifactsExtra); const gomod1 = codeBlock` @@ -84,7 +82,7 @@ describe('modules/manager/gomod/artifacts', () => { env.getChildProcessEnv.mockReturnValue({ ...envMock.basic, ...goEnv }); GlobalConfig.set(adminConfig); docker.resetPrefetchedImages(); - hostRules.getAll.mockReturnValue([]); + hostRules.clear(); }); afterEach(() => { @@ -934,17 +932,16 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with credentials', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({ + hostRules.add({ token: 'some-token', + hostType: 'github', + matchHost: 'api.github.com', }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-token', - hostType: 'github', - matchHost: 'api.github.com', - }, - { token: 'some-other-token', matchHost: 'https://gitea.com' }, - ]); + hostRules.add({ + token: 'some-other-token', + matchHost: 'https://gitea.com', + }); + fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1040,21 +1037,16 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with 2 credentials', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({ + hostRules.add({ token: 'some-token', + hostType: 'github', + matchHost: 'api.github.com', + }); + hostRules.add({ + token: 'some-enterprise-token', + matchHost: 'github.enterprise.com', + hostType: 'github', }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-token', - hostType: 'github', - matchHost: 'api.github.com', - }, - { - token: 'some-enterprise-token', - matchHost: 'github.enterprise.com', - hostType: 'github', - }, - ]); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1117,13 +1109,11 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with single credential', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-enterprise-token', - matchHost: 'gitlab.enterprise.com', - hostType: 'gitlab', - }, - ]); + hostRules.add({ + token: 'some-enterprise-token', + matchHost: 'gitlab.enterprise.com', + hostType: 'gitlab', + }); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1178,18 +1168,16 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with multiple credentials for different paths', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-enterprise-token-repo1', - matchHost: 'https://gitlab.enterprise.com/repo1', - hostType: 'gitlab', - }, - { - token: 'some-enterprise-token-repo2', - matchHost: 'https://gitlab.enterprise.com/repo2', - hostType: 'gitlab', - }, - ]); + hostRules.add({ + token: 'some-enterprise-token-repo1', + matchHost: 'https://gitlab.enterprise.com/repo1', + hostType: 'gitlab', + }); + hostRules.add({ + token: 'some-enterprise-token-repo2', + matchHost: 'https://gitlab.enterprise.com/repo2', + hostType: 'gitlab', + }); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1253,18 +1241,16 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode and ignores non http credentials', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-token', - matchHost: 'ssh://github.enterprise.com', - hostType: 'github', - }, - { - token: 'some-gitlab-token', - matchHost: 'gitlab.enterprise.com', - hostType: 'gitlab', - }, - ]); + hostRules.add({ + token: 'some-token', + matchHost: 'ssh://github.enterprise.com', + hostType: 'github', + }); + hostRules.add({ + token: 'some-gitlab-token', + matchHost: 'gitlab.enterprise.com', + hostType: 'gitlab', + }); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1319,26 +1305,21 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with many credentials', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({ + hostRules.add({ token: 'some-token', + matchHost: 'api.github.com', + hostType: 'github', + }); + hostRules.add({ + token: 'some-enterprise-token', + matchHost: 'github.enterprise.com', + hostType: 'github', + }); + hostRules.add({ + token: 'some-gitlab-token', + matchHost: 'gitlab.enterprise.com', + hostType: 'gitlab', }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-token', - matchHost: 'api.github.com', - hostType: 'github', - }, - { - token: 'some-enterprise-token', - matchHost: 'github.enterprise.com', - hostType: 'github', - }, - { - token: 'some-gitlab-token', - matchHost: 'gitlab.enterprise.com', - hostType: 'gitlab', - }, - ]); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1410,16 +1391,15 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode and ignores non git credentials', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({ + hostRules.add({ token: 'some-token', + matchHost: 'github.com', + }); + hostRules.add({ + token: 'some-enterprise-token', + matchHost: 'github.enterprise.com', + hostType: 'npm', }); - hostRules.getAll.mockReturnValueOnce([ - { - token: 'some-enterprise-token', - matchHost: 'github.enterprise.com', - hostType: 'npm', - }, - ]); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); // TODO: #22198 can be null fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename @@ -1474,7 +1454,6 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with goModTidy', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({}); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1539,7 +1518,6 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with gomodTidy1.17', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({}); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1604,7 +1582,6 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with gomodTidyE and gomodTidy1.17', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({}); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); @@ -1669,7 +1646,6 @@ describe('modules/manager/gomod/artifacts', () => { it('supports docker mode with gomodTidyE', async () => { fs.findLocalSiblingOrParent.mockResolvedValueOnce('vendor'); GlobalConfig.set({ ...adminConfig, binarySource: 'docker' }); - hostRules.find.mockReturnValueOnce({}); fs.readLocalFile.mockResolvedValueOnce('Current go.sum'); fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename const execSnapshots = mockExecAll(); diff --git a/lib/modules/manager/gomod/extract.spec.ts b/lib/modules/manager/gomod/extract.spec.ts index 20f3e63269f..59304b70327 100644 --- a/lib/modules/manager/gomod/extract.spec.ts +++ b/lib/modules/manager/gomod/extract.spec.ts @@ -124,6 +124,55 @@ describe('modules/manager/gomod/extract', () => { }); }); + it('extracts replace directives from non-public module path', () => { + const goMod = codeBlock` + module github.com/JamieTanna-Mend-testing/tka-9783-golang-pro-main + go 1.25.5 + require pro-lib v0.0.0-00010101000000-000000000000 + replace pro-lib => github.com/ns-rpro-dev-tests/golang-pro-lib/libs/src/ns v0.0.0-20260219031232-e6910bd8fb97 + `; + const res = extractPackageFile(goMod); + expect(res).toEqual({ + deps: [ + { + managerData: { + lineNumber: 1, + }, + depName: 'go', + depType: 'golang', + currentValue: '1.25.5', + datasource: 'golang-version', + versioning: 'go-mod-directive', + }, + { + managerData: { + lineNumber: 2, + }, + depName: 'pro-lib', + depType: 'require', + currentValue: 'v0.0.0-00010101000000-000000000000', + currentDigest: '000000000000', + datasource: 'go', + digestOneAndOnly: true, + versioning: 'loose', + skipReason: 'invalid-version', + }, + { + managerData: { + lineNumber: 3, + }, + depName: 'github.com/ns-rpro-dev-tests/golang-pro-lib/libs/src/ns', + depType: 'replace', + currentValue: 'v0.0.0-20260219031232-e6910bd8fb97', + currentDigest: 'e6910bd8fb97', + datasource: 'go', + digestOneAndOnly: true, + versioning: 'loose', + }, + ], + }); + }); + // https://go.dev/doc/modules/gomod-ref#exclude it('ignores exclude directives from multi-line and single line', () => { const goMod = codeBlock` @@ -348,4 +397,99 @@ describe('modules/manager/gomod/extract', () => { const res = extractPackageFile(goMod); expect(res).toBeNull(); }); + + it('marks placeholder pseudo versions with skipReason invalid-version', () => { + const goMod = codeBlock` + module github.com/renovate-tests/gomod + go 1.19 + require ( + github.com/foo/bar v1.2.3 + github.com/baz/qux v0.0.0-00010101000000-000000000000 + github.com/example/local v0.0.0-00010101000000-000000000000 // indirect + github.com/non/placeholder v1.2.4-0.20230101120000-abcdef123456 + monorepo v0.0.0-00010101000000-000000000000 + ) + `; + const res = extractPackageFile(goMod); + expect(res).toEqual({ + deps: [ + { + managerData: { + lineNumber: 1, + }, + depName: 'go', + depType: 'golang', + currentValue: '1.19', + datasource: 'golang-version', + versioning: 'go-mod-directive', + }, + { + managerData: { + lineNumber: 3, + multiLine: true, + }, + depName: 'github.com/foo/bar', + depType: 'require', + currentValue: 'v1.2.3', + datasource: 'go', + }, + { + managerData: { + lineNumber: 4, + multiLine: true, + }, + depName: 'github.com/baz/qux', + depType: 'require', + currentValue: 'v0.0.0-00010101000000-000000000000', + datasource: 'go', + skipReason: 'invalid-version', + currentDigest: '000000000000', + digestOneAndOnly: true, + versioning: 'loose', + }, + { + managerData: { + lineNumber: 5, + multiLine: true, + }, + depName: 'github.com/example/local', + depType: 'indirect', + currentValue: 'v0.0.0-00010101000000-000000000000', + datasource: 'go', + skipReason: 'invalid-version', + enabled: false, + currentDigest: '000000000000', + digestOneAndOnly: true, + versioning: 'loose', + }, + { + managerData: { + lineNumber: 6, + multiLine: true, + }, + depName: 'github.com/non/placeholder', + depType: 'require', + currentValue: 'v1.2.4-0.20230101120000-abcdef123456', + datasource: 'go', + currentDigest: 'abcdef123456', + digestOneAndOnly: true, + versioning: 'loose', + }, + { + managerData: { + lineNumber: 7, + multiLine: true, + }, + depName: 'monorepo', + depType: 'require', + currentValue: 'v0.0.0-00010101000000-000000000000', + datasource: 'go', + currentDigest: '000000000000', + digestOneAndOnly: true, + versioning: 'loose', + skipReason: 'invalid-version', + }, + ], + }); + }); }); diff --git a/lib/modules/manager/gomod/line-parser.spec.ts b/lib/modules/manager/gomod/line-parser.spec.ts index d2e9cbd0878..2d97c4e03cb 100644 --- a/lib/modules/manager/gomod/line-parser.spec.ts +++ b/lib/modules/manager/gomod/line-parser.spec.ts @@ -80,6 +80,21 @@ describe('modules/manager/gomod/line-parser', () => { }); }); + it('should parse require definition with placeholder pseudo-version', () => { + const line = 'require foo/foo v0.0.0-00010101000000-000000000000'; + const res = parseLine(line); + expect(res).toStrictEqual({ + currentDigest: '000000000000', + currentValue: 'v0.0.0-00010101000000-000000000000', + datasource: 'go', + depName: 'foo/foo', + depType: 'require', + digestOneAndOnly: true, + skipReason: 'invalid-version', + versioning: 'loose', + }); + }); + it('should parse require multi-line', () => { const line = ' foo/foo v1.2'; const res = parseLine(line); @@ -250,6 +265,22 @@ describe('modules/manager/gomod/line-parser', () => { }); }); + it('should parse replace definition with placeholder pseudo-version', () => { + const line = + 'replace foo/foo => bar/bar v0.0.0-00010101000000-000000000000'; + const res = parseLine(line); + expect(res).toStrictEqual({ + currentDigest: '000000000000', + currentValue: 'v0.0.0-00010101000000-000000000000', + datasource: 'go', + depName: 'bar/bar', + depType: 'replace', + digestOneAndOnly: true, + skipReason: 'invalid-version', + versioning: 'loose', + }); + }); + it('should parse replace indirect definition', () => { const line = 'replace foo/foo => bar/bar v1.2 // indirect'; const res = parseLine(line); diff --git a/lib/modules/manager/gomod/line-parser.ts b/lib/modules/manager/gomod/line-parser.ts index 0f9ac3b17fc..2c518257dd5 100644 --- a/lib/modules/manager/gomod/line-parser.ts +++ b/lib/modules/manager/gomod/line-parser.ts @@ -14,7 +14,7 @@ const requireRegex = regEx( ); const replaceRegex = regEx( - /^(?replace)?\s+(?[^\s]+\/[^\s]+)\s*=>\s*(?[^\s]+)(?:\s+(?[^\s]+))?(?:\s*\/\/\s*(?[^\s]+)\s*)?$/, + /^(?replace)?\s+(?[^\s]+\/?[^\s]+)\s*=>\s*(?[^\s]+)(?:\s+(?[^\s]+))?(?:\s*\/\/\s*(?[^\s]+)\s*)?$/, ); export const excludeBlockStartRegex = regEx(/^(?exclude)\s+\(\s*$/); @@ -29,11 +29,17 @@ const toolchainVersionRegex = regEx(/^\s*toolchain\s+go(?[^\s]+)\s*$/); const pseudoVersionRegex = regEx(GoDatasource.pversionRegexp); +const placeholderPseudoVersion = 'v0.0.0-00010101000000-000000000000'; + function extractDigest(input: string): string | undefined { const match = pseudoVersionRegex.exec(input); return match?.groups?.digest; } +function isPlaceholderPseudoVersion(version: string): boolean { + return version === placeholderPseudoVersion; +} + export function parseLine(input: string): PackageDependency | null { const goVersionMatches = goVersionRegex.exec(input)?.groups; if (goVersionMatches) { @@ -91,6 +97,9 @@ export function parseLine(input: string): PackageDependency | null { dep.currentDigest = digest; dep.digestOneAndOnly = true; dep.versioning = 'loose'; + if (isPlaceholderPseudoVersion(currentValue)) { + dep.skipReason = 'invalid-version'; + } } } else { dep.skipReason = 'invalid-version'; @@ -132,6 +141,9 @@ export function parseLine(input: string): PackageDependency | null { dep.currentDigest = digest; dep.digestOneAndOnly = true; dep.versioning = 'loose'; + if (isPlaceholderPseudoVersion(currentValue)) { + dep.skipReason = 'invalid-version'; + } } } else if (currentValue) { dep.skipReason = 'invalid-version'; diff --git a/lib/modules/manager/gradle-wrapper/artifacts.spec.ts b/lib/modules/manager/gradle-wrapper/artifacts.spec.ts index e56c402b725..9156b8ebb5f 100644 --- a/lib/modules/manager/gradle-wrapper/artifacts.spec.ts +++ b/lib/modules/manager/gradle-wrapper/artifacts.spec.ts @@ -13,7 +13,7 @@ import type { StatusResult } from '../../../util/git/types.ts'; import { getPkgReleases } from '../../datasource/index.ts'; import { updateArtifacts as gradleUpdateArtifacts } from '../gradle/index.ts'; import type { UpdateArtifactsConfig, UpdateArtifactsResult } from '../types.ts'; -import { gradleJvmArg, updateBuildFile, updateLockFiles } from './artifacts.ts'; +import { updateBuildFile, updateLockFiles } from './artifacts.ts'; import { updateArtifacts } from './index.ts'; vi.mock('../../../util/fs/index.ts'); @@ -73,13 +73,6 @@ describe('modules/manager/gradle-wrapper/artifacts', () => { }); }); - describe('gradleJvmArg()', () => { - it('takes the values given to it, and returns the JVM arguments', () => { - const result = gradleJvmArg({ jvmMemory: 256, jvmMaxMemory: 768 }); - expect(result).toBe(' -Dorg.gradle.jvmargs="-Xms256m -Xmx768m"'); - }); - }); - describe('updateArtifacts()', () => { it('Custom Gradle Wrapper heap settings are populated', async () => { const execSnapshots = mockExecAll(); diff --git a/lib/modules/manager/gradle-wrapper/artifacts.ts b/lib/modules/manager/gradle-wrapper/artifacts.ts index 3d4df01a7fc..f21993ca2b7 100644 --- a/lib/modules/manager/gradle-wrapper/artifacts.ts +++ b/lib/modules/manager/gradle-wrapper/artifacts.ts @@ -2,10 +2,13 @@ import { lang, query as q } from '@renovatebot/good-enough-parser'; import { isTruthy } from '@sindresorhus/is'; import { quote } from 'shlex'; import upath from 'upath'; -import type { ToolSettingsOptions } from '../../../config/types.ts'; import { TEMPORARY_ERROR } from '../../../constants/error-messages.ts'; import { logger } from '../../../logger/index.ts'; -import { exec, getToolSettingsOptions } from '../../../util/exec/index.ts'; +import { + exec, + getToolSettingsOptions, + gradleJvmArg, +} from '../../../util/exec/index.ts'; import type { ExecOptions } from '../../../util/exec/types.ts'; import { localPathExists, @@ -132,10 +135,6 @@ export async function updateLockFiles( }); } -export function gradleJvmArg(config: ToolSettingsOptions): string { - return ` -Dorg.gradle.jvmargs="-Xms${config.jvmMemory}m -Xmx${config.jvmMaxMemory}m"`; -} - export async function updateArtifacts({ packageFileName, newPackageFileContent, diff --git a/lib/modules/manager/gradle/artifacts.spec.ts b/lib/modules/manager/gradle/artifacts.spec.ts index 4b82727d26d..1d8244798b1 100644 --- a/lib/modules/manager/gradle/artifacts.spec.ts +++ b/lib/modules/manager/gradle/artifacts.spec.ts @@ -199,6 +199,51 @@ describe('modules/manager/gradle/artifacts', () => { expect(execSnapshots).toBeEmptyArray(); }); + it('uses custom JVM heap settings when toolSettings are configured', async () => { + const execSnapshots = mockExecAll(); + GlobalConfig.set({ + ...adminConfig, + toolSettings: { jvmMaxMemory: 600 }, + }); + + const res = await updateArtifacts({ + packageFileName: 'build.gradle', + updatedDeps: [ + { depName: 'org.junit.jupiter:junit-jupiter-api' }, + { depName: 'org.junit.jupiter:junit-jupiter-engine' }, + ], + newPackageFileContent: '', + config: {}, + }); + + expect(res).toEqual([ + { + file: { + type: 'addition', + path: 'gradle.lockfile', + contents: 'New gradle.lockfile', + }, + }, + ]); + expect(execSnapshots).toMatchObject([ + { + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms600m -Xmx600m" --console=plain --dependency-verification lenient -q properties', + options: { + cwd: '/tmp/github/some/repo', + }, + }, + { + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms600m -Xmx600m" --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', + options: { + cwd: '/tmp/github/some/repo', + stdin: 'pipe', + stdout: 'ignore', + stderr: 'pipe', + }, + }, + ]); + }); + it('updates lock file', async () => { const execSnapshots = mockExecAll(); @@ -223,13 +268,13 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -268,13 +313,13 @@ describe('modules/manager/gradle/artifacts', () => { // In win32, gradle.bat will be used and /dev/null redirection isn't used yet expect(execSnapshots).toMatchObject([ { - cmd: 'gradlew.bat --console=plain --dependency-verification lenient -q properties', + cmd: 'gradlew.bat -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, }, { - cmd: 'gradlew.bat --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', + cmd: 'gradlew.bat -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -313,13 +358,13 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies --update-locks org.springframework.boot:org.springframework.boot.gradle.plugin', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --update-locks org.springframework.boot:org.springframework.boot.gradle.plugin', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -366,13 +411,13 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies --write-locks', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --write-locks', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -418,7 +463,7 @@ describe('modules/manager/gradle/artifacts', () => { ' bash -l -c "' + 'install-tool java 16.0.1' + ' && ' + - './gradlew --console=plain --dependency-verification lenient -q properties' + + './gradlew -Dorg.gradle.jvmargs=\\"-Xms512m -Xmx512m\\" --console=plain --dependency-verification lenient -q properties' + '"', options: { cwd: '/tmp/github/some/repo' }, }, @@ -435,7 +480,7 @@ describe('modules/manager/gradle/artifacts', () => { ' bash -l -c "' + 'install-tool java 16.0.1' + ' && ' + - './gradlew --console=plain --dependency-verification lenient -q :dependencies --write-locks' + + './gradlew -Dorg.gradle.jvmargs=\\"-Xms512m -Xmx512m\\" --console=plain --dependency-verification lenient -q :dependencies --write-locks' + '"', options: { cwd: '/tmp/github/some/repo', @@ -470,12 +515,12 @@ describe('modules/manager/gradle/artifacts', () => { expect(execSnapshots).toMatchObject([ { cmd: 'install-tool java 16.0.1' }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo' }, }, { cmd: 'install-tool java 16.0.1' }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies --write-locks', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --write-locks', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -513,13 +558,13 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies :sub1:dependencies :sub2:dependencies --write-locks', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies :sub1:dependencies :sub2:dependencies --write-locks', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -565,7 +610,7 @@ describe('modules/manager/gradle/artifacts', () => { expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, @@ -618,12 +663,12 @@ describe('modules/manager/gradle/artifacts', () => { expect(execSnapshots).toMatchObject([ { cmd: 'install-tool java 11.0.1' }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo' }, }, { cmd: 'install-tool java 11.0.1' }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies --write-locks', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --write-locks', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -672,7 +717,7 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -761,7 +806,7 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -820,13 +865,13 @@ describe('modules/manager/gradle/artifacts', () => { ]); expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q properties', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q properties', options: { cwd: '/tmp/github/some/repo', }, }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q :dependencies --update-locks org.junit.jupiter:junit-jupiter-api,org.junit.jupiter:junit-jupiter-engine', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -835,7 +880,7 @@ describe('modules/manager/gradle/artifacts', () => { }, }, { - cmd: './gradlew --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -880,7 +925,7 @@ describe('modules/manager/gradle/artifacts', () => { expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q --write-verification-metadata sha256 dependencies', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', @@ -924,7 +969,7 @@ describe('modules/manager/gradle/artifacts', () => { expect(execSnapshots).toMatchObject([ { - cmd: './gradlew --console=plain --dependency-verification lenient -q --write-verification-metadata sha256,pgp dependencies', + cmd: './gradlew -Dorg.gradle.jvmargs="-Xms512m -Xmx512m" --console=plain --dependency-verification lenient -q --write-verification-metadata sha256,pgp dependencies', options: { cwd: '/tmp/github/some/repo', stdin: 'pipe', diff --git a/lib/modules/manager/gradle/artifacts.ts b/lib/modules/manager/gradle/artifacts.ts index 23026ae7a7f..345b4c26faa 100644 --- a/lib/modules/manager/gradle/artifacts.ts +++ b/lib/modules/manager/gradle/artifacts.ts @@ -4,7 +4,11 @@ import upath from 'upath'; import { GlobalConfig } from '../../../config/global.ts'; import { TEMPORARY_ERROR } from '../../../constants/error-messages.ts'; import { logger } from '../../../logger/index.ts'; -import { exec } from '../../../util/exec/index.ts'; +import { + exec, + getToolSettingsOptions, + gradleJvmArg, +} from '../../../util/exec/index.ts'; import type { ExecOptions } from '../../../util/exec/types.ts'; import { findUpLocal, @@ -208,7 +212,7 @@ export async function updateArtifacts({ const oldLockFileContentMap = await getFiles(lockFiles); await prepareGradleCommand(gradlewFile); - const baseCmd = `${gradlewName} --console=plain --dependency-verification lenient -q`; + const baseCmd = `${gradlewName}${gradleJvmArg(getToolSettingsOptions(config.toolSettings))} --console=plain --dependency-verification lenient -q`; const execOptions: ExecOptions = { cwdFile: gradlewFile, docker: {}, diff --git a/lib/modules/platform/api.ts b/lib/modules/platform/api.ts index 7dedecd4b1d..0d86a2f7605 100644 --- a/lib/modules/platform/api.ts +++ b/lib/modules/platform/api.ts @@ -9,6 +9,7 @@ import * as gitea from './gitea/index.ts'; import * as github from './github/index.ts'; import * as gitlab from './gitlab/index.ts'; import * as local from './local/index.ts'; +import * as scmm from './scm-manager/index.ts'; import type { Platform } from './types.ts'; const api = new Map(); @@ -24,3 +25,4 @@ api.set(gitea.id, gitea); api.set(github.id, github); api.set(gitlab.id, gitlab); api.set(local.id, local); +api.set(scmm.id, scmm); diff --git a/lib/modules/platform/github/index.spec.ts b/lib/modules/platform/github/index.spec.ts index 0820d65bbfe..0d592d8b29e 100644 --- a/lib/modules/platform/github/index.spec.ts +++ b/lib/modules/platform/github/index.spec.ts @@ -1771,6 +1771,51 @@ describe('modules/platform/github/index', () => { ); }); + it('stops at custom max sync pages', async () => { + GlobalConfig.set({ prCacheSyncMaxPages: 2 }); + const scope = httpMock.scope(githubApiHost); + + // Run 1: initial fetch — cache gets lastModified = t1 + initRepoMock(scope, 'some/repo'); + scope.get(pagePath(1)).reply(200, [renovatePr]); + await github.initRepo({ + repository: 'some/repo', + renovateUsername: 'renovate-bot', + }); + await github.getPrList(); + + // Run 2: sync — 2 pages of non-Renovate PRs all newer than lastModified + initRepoMock(scope, 'some/repo'); + for (let i = 1; i <= 2; i++) { + scope.get(pagePath(i, 20)).reply( + 200, + [ + { + ...pr1, + number: 100 + i, + updated_at: t4, + user: { login: 'other' }, + }, + ], + { link: pageLink(i + 1) }, + ); + } + // Page 3 NOT mocked — must stop at 2 + + await github.initRepo({ + repository: 'some/repo', + renovateUsername: 'renovate-bot', + }); + const res = await github.getPrList(); + + expect(res).toHaveLength(1); + expect(res).toMatchObject([{ number: 1, title: 'Renovate PR' }]); + expect(logger.logger.warn).toHaveBeenCalledWith( + { repo: 'some/repo', pages: 2 }, + 'PR cache: hit max sync pages, stopping', + ); + }); + it('reconciles mixed pages with both Renovate and non-Renovate PRs', async () => { const scope = httpMock.scope(githubApiHost); diff --git a/lib/modules/platform/github/pr.ts b/lib/modules/platform/github/pr.ts index 215660a75f5..e432266004d 100644 --- a/lib/modules/platform/github/pr.ts +++ b/lib/modules/platform/github/pr.ts @@ -1,5 +1,6 @@ import { isEmptyArray, isNonEmptyArray } from '@sindresorhus/is'; import { DateTime } from 'luxon'; +import { GlobalConfig } from '../../../config/global.ts'; import { instrument } from '../../../instrumentation/index.ts'; import { logger } from '../../../logger/index.ts'; import { ExternalHostError } from '../../../types/errors/external-host-error.ts'; @@ -14,8 +15,6 @@ import { ApiCache } from './api-cache.ts'; import { coerceRestPr } from './common.ts'; import type { ApiPageCache, GhPr, GhRestPr } from './types.ts'; -const MAX_SYNC_PAGES = 100; - function getPrApiCache(): ApiCache { const repoCache = getCache(); if (!repoCache?.platform?.github?.pullRequestsCache) { @@ -81,8 +80,9 @@ export async function getPrCache( } const cutoffTime = lastModifiedRaw ? DateTime.fromISO(lastModifiedRaw) : null; - const startTime = Date.now(); try { + const maxSyncPages = GlobalConfig.get('prCacheSyncMaxPages', 100); + const startTime = Date.now(); let requestsTotal = 0; let apiQuotaAffected = false; let needNextPageFetch = true; @@ -163,7 +163,7 @@ export async function getPrCache( !isInitial && needNextPageFetch && needNextPageSync && - pageIdx >= MAX_SYNC_PAGES + pageIdx >= maxSyncPages ) { logger.warn( { repo, pages: pageIdx }, diff --git a/lib/modules/platform/scm-manager/index.spec.ts b/lib/modules/platform/scm-manager/index.spec.ts new file mode 100644 index 00000000000..83f4583e03a --- /dev/null +++ b/lib/modules/platform/scm-manager/index.spec.ts @@ -0,0 +1,667 @@ +import { git } from '~test/util.ts'; +import * as httpMock from '../../../../test/http-mock.ts'; +import { GlobalConfig } from '../../../config/global.ts'; +import * as hostRules from '../../../util/host-rules.ts'; +import type { Pr } from '../types.ts'; +import * as util from '../util.ts'; +import * as scmPlatform from './index.ts'; +import { mapPrFromScmToRenovate } from './mapper.ts'; +import type { PullRequest, Repo, User } from './schema.ts'; +import type { PrFilterByState } from './types.ts'; + +vi.mock('../util'); +vi.mock('../../../util/git'); + +const endpoint = 'https://localhost:8080'; +const baseUrl = `${endpoint}/scm/api/v2`; +const token = 'TEST_TOKEN'; + +const user: User = { + mail: 'test@user.de', + displayName: 'Test User', + name: 'testUser1337', +}; + +const repo: Repo = { + contact: 'test@test.com', + creationDate: '2023-08-02T10:48:24.762Z', + description: 'Default Repo', + lastModified: '2023-08-10T10:48:24.762Z', + namespace: 'default', + name: 'repo', + type: 'git', + archived: false, + exporting: false, + healthCheckRunning: false, + _links: { + protocol: [ + { name: 'http', href: 'https://localhost:8080/scm/default/repo' }, + ], + defaultBranch: { + href: 'https://localhost:8080/scm/api/v2/config/git/default/repo/default-branch', + }, + }, +}; + +const pullRequest: PullRequest = { + id: '1', + author: { displayName: 'Thomas Zerr', id: 'tzerr' }, + source: 'feature/test', + target: 'develop', + title: 'The PullRequest', + description: 'Another PullRequest', + creationDate: '2023-08-02T10:48:24.762Z', + status: 'OPEN', + labels: [], + tasks: { todo: 2, done: 4 }, + _links: {}, + _embedded: { + defaultConfig: { + mergeStrategy: 'SQUASH', + deleteBranchOnMerge: true, + }, + }, +}; + +const renovatePr = mapPrFromScmToRenovate(pullRequest); + +describe('modules/platform/scm-manager/index', () => { + beforeEach(() => { + GlobalConfig.reset(); + vi.resetAllMocks(); + hostRules.add({ token, username: user.name }); + scmPlatform.invalidatePrCache(); + }); + + describe(scmPlatform.initPlatform, () => { + it('should throw error, when endpoint is not configured', async () => { + await expect(scmPlatform.initPlatform({ token })).rejects.toThrow( + 'SCM-Manager endpoint not configured', + ); + }); + + it('should throw error, when token is not configured', async () => { + await expect(scmPlatform.initPlatform({ endpoint })).rejects.toThrow( + 'SCM-Manager API token not configured', + ); + }); + + it('should throw error, when token is invalid', async () => { + httpMock.scope(baseUrl).get(`/me`).reply(401); + + await expect( + scmPlatform.initPlatform({ endpoint, token: 'invalid' }), + ).rejects.toThrow('Init: Authentication failure'); + }); + + it('should init platform', async () => { + httpMock.scope(baseUrl).get('/me').reply(200, user); + expect(await scmPlatform.initPlatform({ endpoint, token })).toEqual({ + endpoint: baseUrl, + gitAuthor: 'Test User ', + }); + }); + }); + + describe(scmPlatform.initRepo, () => { + it('should init repo', async () => { + const repository = `${repo.namespace}/${repo.name}`; + const expectedFingerprint = 'expectedFingerprint'; + const expectedDefaultBranch = 'expectedDefaultBranch'; + GlobalConfig.set({ ignorePrAuthor: true }); + + httpMock + .scope(baseUrl) + .get(`/repositories/${repository}`) + .reply(200, repo); + httpMock + .scope(baseUrl) + .get(`/config/git/${repository}/default-branch`) + .reply(200, { defaultBranch: expectedDefaultBranch }); + + vi.mocked(util.repoFingerprint).mockReturnValueOnce(expectedFingerprint); + + expect( + await scmPlatform.initRepo({ + repository: `${repo.namespace}/${repo.name}`, + }), + ).toEqual({ + defaultBranch: expectedDefaultBranch, + isFork: false, + repoFingerprint: expectedFingerprint, + }); + + expect(git.initRepo).toHaveBeenCalledExactlyOnceWith({ + url: `https://${user.name}:${token}@localhost:8080/scm/default/repo`, + repository, + defaultBranch: expectedDefaultBranch, + ignorePrAuthor: true, + }); + }); + }); + + describe(scmPlatform.getRepos, () => { + it('should return all available repos', async () => { + httpMock + .scope(baseUrl) + .get(`/repositories?pageSize=1000000`) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + repositories: [ + repo, + { ...repo, namespace: 'other', name: 'repository' }, + { ...repo, namespace: 'other', name: 'mercurial', type: 'hg' }, + { ...repo, namespace: 'other', name: 'subversion', type: 'svn' }, + ], + }, + }); + + expect(await scmPlatform.getRepos()).toEqual([ + 'default/repo', + 'other/repository', + ]); + }); + }); + + describe(scmPlatform.getPrList, () => { + it('should return empty array, because no PR could be found', async () => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [], + }, + }); + + expect(await scmPlatform.getPrList()).toBeEmptyArray(); + }); + + it('should return empty array, because API request failed', async () => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(400); + + expect(await scmPlatform.getPrList()).toBeEmptyArray(); + }); + + it('should return all PRs of a repo', async () => { + const expectedResult: Pr[] = [ + { + sourceBranch: pullRequest.source, + createdAt: pullRequest.creationDate, + labels: pullRequest.labels, + number: parseInt(pullRequest.id), + state: pullRequest.status, + targetBranch: pullRequest.target, + title: pullRequest.title, + hasAssignees: false, + isDraft: false, + reviewers: [], + }, + ]; + + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [pullRequest], + }, + }); + + //Fetching from client + expect(await scmPlatform.getPrList()).toIncludeAllMembers(expectedResult); + //Fetching from cache + expect(await scmPlatform.getPrList()).toIncludeAllMembers(expectedResult); + }); + }); + + describe(scmPlatform.findPr, () => { + it('search in Pull Request without explicitly setting the state as argument', async () => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [pullRequest], + }, + }); + + expect( + await scmPlatform.findPr({ + branchName: pullRequest.source, + prTitle: pullRequest.title, + }), + ).toEqual(renovatePr); + }); + + it.each` + availablePullRequest | branchName | prTitle | state | result + ${[]} | ${pullRequest.source} | ${pullRequest.title} | ${'all'} | ${null} + ${[pullRequest]} | ${'invalid branchName'} | ${pullRequest.title} | ${'all'} | ${null} + ${[pullRequest]} | ${pullRequest.source} | ${'invalid title'} | ${'all'} | ${null} + ${[pullRequest]} | ${pullRequest.source} | ${null} | ${'all'} | ${renovatePr} + ${[pullRequest]} | ${pullRequest.source} | ${undefined} | ${'all'} | ${renovatePr} + ${[pullRequest]} | ${pullRequest.source} | ${pullRequest.title} | ${'all'} | ${renovatePr} + ${[pullRequest]} | ${pullRequest.source} | ${pullRequest.title} | ${'open'} | ${renovatePr} + ${[pullRequest]} | ${pullRequest.source} | ${pullRequest.title} | ${'!open'} | ${null} + ${[pullRequest]} | ${pullRequest.source} | ${pullRequest.title} | ${'closed'} | ${null} + `( + 'search within available pull requests for branch name "$branchName", pr title "$prTitle" and state "$state" with result $result', + async ({ + availablePullRequest, + branchName, + prTitle, + state, + result, + }: { + availablePullRequest: PullRequest[]; + branchName: string; + prTitle: string | undefined | null; + state: string; + result: Pr | null; + }) => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: availablePullRequest, + }, + }); + + expect( + await scmPlatform.findPr({ + branchName, + prTitle, + state: state as PrFilterByState, + }), + ).toEqual(result); + }, + ); + }); + + describe(scmPlatform.getBranchPr, () => { + it.each` + availablePullRequest | branchName | result + ${[]} | ${pullRequest.source} | ${null} + ${[pullRequest]} | ${'invalid branchName'} | ${null} + ${[pullRequest]} | ${pullRequest.source} | ${renovatePr} + `( + 'search within available pull requests for branch name "$branchName" with result $result', + async ({ + availablePullRequest, + branchName, + result, + }: { + availablePullRequest: PullRequest[]; + branchName: string; + result: Pr | null; + }) => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: availablePullRequest, + }, + }); + + expect(await scmPlatform.getBranchPr(branchName)).toEqual(result); + }, + ); + }); + + describe(scmPlatform.getPr, () => { + it('should return null, because PR was not found', async () => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [], + }, + }); + + httpMock + .scope(baseUrl) + .get(`/pull-requests/${repo.namespace}/${repo.name}/${pullRequest.id}`) + .reply(404); + + expect(await scmPlatform.getPr(1)).toBeNull(); + }); + + it('should return PR from cache', async () => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [pullRequest], + }, + }); + + expect(await scmPlatform.getPr(parseInt(pullRequest.id))).toEqual( + renovatePr, + ); + }); + + it('should return fetched pr', async () => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [], + }, + }); + + httpMock + .scope(baseUrl) + .get(`/pull-requests/${repo.namespace}/${repo.name}/${pullRequest.id}`) + .reply(200, pullRequest); + + expect(await scmPlatform.getPr(parseInt(pullRequest.id))).toEqual( + renovatePr, + ); + }); + }); + + describe(scmPlatform.createPr, () => { + it.each` + draftPr | expectedState | expectedIsDraft + ${undefined} | ${'OPEN'} | ${false} + ${false} | ${'OPEN'} | ${false} + ${true} | ${'DRAFT'} | ${true} + `( + 'should create PR with $draftPR and state $expectedState', + async ({ + draftPr, + expectedState, + expectedIsDraft, + }: { + draftPr: boolean | undefined; + expectedState: string; + expectedIsDraft: boolean; + }) => { + httpMock + .scope(baseUrl) + .post(`/pull-requests/${repo.namespace}/${repo.name}`) + .reply(201, undefined, { + location: `${baseUrl}/pull-requests/${repo.namespace}/${repo.name}/1337`, + }); + + httpMock + .scope(baseUrl) + .get(`/pull-requests/${repo.namespace}/${repo.name}/1337`) + .reply(200, { + id: '1337', + source: 'feature/test', + target: 'develop', + title: 'PR Title', + description: 'PR Body', + creationDate: '2023-01-01T13:37:00.000Z', + status: draftPr ? 'DRAFT' : 'OPEN', + labels: [], + tasks: { todo: 0, done: 0 }, + _links: {}, + _embedded: { + defaultConfig: { + mergeStrategy: 'FAST_FORWARD_IF_POSSIBLE', + deleteBranchOnMerge: false, + }, + }, + }); + + expect( + await scmPlatform.createPr({ + sourceBranch: 'feature/test', + targetBranch: 'develop', + prTitle: 'PR Title', + prBody: 'PR Body', + draftPR: draftPr, + }), + ).toEqual({ + sourceBranch: 'feature/test', + targetBranch: 'develop', + title: 'PR Title', + createdAt: '2023-01-01T13:37:00.000Z', + hasAssignees: false, + isDraft: expectedIsDraft, + labels: [], + number: 1337, + reviewers: [], + state: expectedState, + }); + }, + ); + }); + + describe(scmPlatform.updatePr, () => { + it.each` + state | body + ${'open'} | ${'prBody'} + ${'closed'} | ${'prBody'} + ${undefined} | ${'prBody'} + ${'open'} | ${undefined} + `( + 'should update PR with state $state and bdoy $body', + async ({ + state, + body, + }: { + state: string | undefined; + body: string | undefined; + }) => { + httpMock + .scope(baseUrl) + .get( + `/pull-requests/${repo.namespace}/${repo.name}/${pullRequest.id}`, + ) + .reply(200, pullRequest); + + httpMock + .scope(baseUrl) + .put(`/pull-requests/${repo.namespace}/${repo.name}/1`) + .reply(204); + + await expect( + scmPlatform.updatePr({ + number: 1, + prTitle: 'PR Title', + prBody: body, + state: state as 'open' | 'closed' | undefined, + targetBranch: 'Target/Branch', + }), + ).resolves.not.toThrow(); + }, + ); + }); + + describe(scmPlatform.mergePr, () => { + it('should Not implemented and return false', async () => { + const result = await scmPlatform.mergePr({ id: 1 }); + expect(result).toBeFalse(); + }); + }); + + describe(scmPlatform.getBranchStatus, () => { + it('should Not implemented and return red', async () => { + const result = await scmPlatform.getBranchStatus('test/branch', false); + expect(result).toBe('red'); + }); + }); + + describe(scmPlatform.setBranchStatus, () => { + it('should Not implemented', async () => { + await expect( + scmPlatform.setBranchStatus({ + branchName: 'test/branch', + context: 'context', + description: 'description', + state: 'red', + }), + ).resolves.not.toThrow(); + }); + }); + + describe(scmPlatform.getBranchStatusCheck, () => { + it('should Not implemented and return null', async () => { + const result = await scmPlatform.getBranchStatusCheck( + 'test/branch', + null, + ); + expect(result).toBeNull(); + }); + }); + + describe(scmPlatform.addReviewers, () => { + it('should Not implemented', async () => { + await expect( + scmPlatform.addReviewers(1, ['reviewer']), + ).resolves.not.toThrow(); + }); + }); + + describe(scmPlatform.addAssignees, () => { + it('should Not implemented', async () => { + await expect( + scmPlatform.addAssignees(1, ['assignee']), + ).resolves.not.toThrow(); + }); + }); + + describe(scmPlatform.deleteLabel, () => { + it('should Not implemented', async () => { + await expect(scmPlatform.deleteLabel(1, 'label')).resolves.not.toThrow(); + }); + }); + + describe(scmPlatform.getIssueList, () => { + it('should Not implemented and return empty list', async () => { + const result = await scmPlatform.getIssueList(); + expect(result).toEqual([]); + }); + }); + + describe(scmPlatform.findIssue, () => { + it('should Not implemented and return null', async () => { + const result = await scmPlatform.findIssue('issue'); + expect(result).toBeNull(); + }); + }); + + describe(scmPlatform.ensureIssue, () => { + it('should Not implemented and return null', async () => { + const result = await scmPlatform.ensureIssue({ + title: 'issue', + body: 'body', + }); + expect(result).toBeNull(); + }); + }); + + describe(scmPlatform.ensureIssueClosing, () => { + it('should Not implemented', async () => { + await expect( + scmPlatform.ensureIssueClosing('issue'), + ).resolves.not.toThrow(); + }); + }); + + describe(scmPlatform.ensureCommentRemoval, () => { + it('should Not implemented', async () => { + await expect( + scmPlatform.ensureCommentRemoval({ + type: 'by-content', + number: 1, + content: 'content', + }), + ).resolves.not.toThrow(); + }); + }); + + describe(scmPlatform.ensureComment, () => { + it('should Not implemented', async () => { + expect( + await scmPlatform.ensureComment({ + number: 1, + topic: 'comment', + content: 'content', + }), + ).toBeFalse(); + }); + }); + + describe(scmPlatform.massageMarkdown, () => { + it('should adjust smart link for Pull Requests', () => { + const result = scmPlatform.massageMarkdown('[PR](../pull/1)'); + expect(result).toBe('[PR](pulls/1)'); + }); + }); + + describe(scmPlatform.getRepoForceRebase, () => { + it('should Not implemented and return false', async () => { + const result = await scmPlatform.getRepoForceRebase(); + expect(result).toBeFalse(); + }); + }); + + describe(scmPlatform.getRawFile, () => { + it('should Not implemented and return null', async () => { + const result = await scmPlatform.getRawFile('file'); + expect(result).toBeNull(); + }); + }); + + describe(scmPlatform.getJsonFile, () => { + it('should Not implemented and return undefined', async () => { + const result = await scmPlatform.getJsonFile('package.json'); + expect(result).toBeNull(); + }); + }); + + describe(scmPlatform.maxBodyLength, () => { + it('should return the max body length allowed for an SCM-Manager request body', () => { + expect(scmPlatform.maxBodyLength()).toBe(200000); + }); + }); +}); diff --git a/lib/modules/platform/scm-manager/index.ts b/lib/modules/platform/scm-manager/index.ts new file mode 100644 index 00000000000..7ff143d1758 --- /dev/null +++ b/lib/modules/platform/scm-manager/index.ts @@ -0,0 +1,340 @@ +import { GlobalConfig } from '../../../config/global.ts'; +import { logger } from '../../../logger/index.ts'; +import type { BranchStatus } from '../../../types/index.ts'; +import * as git from '../../../util/git/index.ts'; +import { getBaseUrl, setBaseUrl } from '../../../util/http/scm-manager.ts'; +import { sanitize } from '../../../util/sanitize.ts'; +import type { + BranchStatusConfig, + CreatePRConfig, + EnsureCommentConfig, + EnsureCommentRemovalConfigByContent, + EnsureCommentRemovalConfigByTopic, + EnsureIssueConfig, + FindPRConfig, + Issue, + MergePRConfig, + PlatformParams, + PlatformResult, + Pr, + RepoParams, + RepoResult, + UpdatePrConfig, +} from '../types.ts'; +import { repoFingerprint } from '../util.ts'; +import { smartTruncate } from '../utils/pr-body.ts'; +import { mapPrFromScmToRenovate } from './mapper.ts'; +import { + createScmPr, + getAllRepoPrs, + getAllRepos, + getCurrentUser, + getDefaultBranch, + getRepo, + getRepoPr, + updateScmPr, +} from './scm-manager-helper.ts'; +import { getRepoUrl, mapPrState, matchPrState, smartLinks } from './utils.ts'; + +interface SCMMRepoConfig { + repository: string; + prList: Pr[] | null; + defaultBranch: string; + ignorePrAuthor: boolean; +} + +export const id = 'scm-manager'; +export const experimental = true; + +let config: SCMMRepoConfig = {} as any; + +export async function initPlatform({ + endpoint, + token, +}: PlatformParams): Promise { + if (!endpoint) { + throw new Error('Init: SCM-Manager endpoint not configured'); + } + + if (!token) { + throw new Error('Init: SCM-Manager API token not configured'); + } + + const baseUrl = `${endpoint}/scm/api/v2`; + setBaseUrl(baseUrl); + + try { + const me = await getCurrentUser(token); + const gitAuthor = `${me.displayName} <${me.mail}>`; + const result = { endpoint: baseUrl, gitAuthor }; + + logger.debug({ result }, 'Platform result'); + + return result; + } catch (err) { + logger.debug( + { err }, + 'Init: Error authenticating with SCM-Manager. Check your token', + ); + throw new Error('Init: Authentication failure'); + } +} + +export async function initRepo({ + repository, + gitUrl, +}: RepoParams): Promise { + const repo = await getRepo(repository); + const defaultBranch = await getDefaultBranch(repo); + const url = getRepoUrl(repo, gitUrl, getBaseUrl()); + + config = {} as any; + config.repository = repository; + config.defaultBranch = defaultBranch; + config.ignorePrAuthor = GlobalConfig.get('ignorePrAuthor', false); + + await git.initRepo({ + ...config, + url, + }); + + // Reset cached resources + invalidatePrCache(); + + const result = { + defaultBranch: config.defaultBranch, + isFork: false, + repoFingerprint: repoFingerprint(config.repository, getBaseUrl()), + }; + + logger.debug({ result }, `Repo initialized`); + + return result; +} + +export async function getRepos(): Promise { + const repos = (await getAllRepos()).filter((repo) => repo.type === 'git'); + return repos.map((repo) => `${repo.namespace}/${repo.name}`); +} + +export async function getBranchPr(branchName: string): Promise { + return await findPr({ branchName, state: 'open' }); +} + +export async function findPr({ + branchName, + prTitle, + state = 'all', +}: FindPRConfig): Promise { + const inProgressPrs = await getPrList(); + const result = inProgressPrs.find( + (pr) => + branchName === pr.sourceBranch && + (!prTitle || prTitle === pr.title) && + matchPrState(pr, state), + ); + + if (result) { + logger.debug({ result }, `Found PR`); + return result; + } + + logger.debug( + `Could not find PR with source branch ${branchName} and title ${ + prTitle ?? '' + } and state ${state}`, + ); + + return null; +} + +export async function getPr(number: number): Promise { + const inProgressPrs = await getPrList(); + const cachedPr = inProgressPrs.find((pr) => pr.number === number); + + if (cachedPr) { + logger.debug('Returning from cached PRs'); + return cachedPr; + } + + try { + const result = await getRepoPr(config.repository, number); + logger.debug('Returning PR from API'); + return mapPrFromScmToRenovate(result); + } catch (error) { + logger.error({ error }, `Can not find a PR with id ${number}`); + return null; + } +} + +export async function getPrList(): Promise { + if (config.prList === null) { + try { + config.prList = ( + await getAllRepoPrs(config.repository, config.ignorePrAuthor) + ).map((pr) => mapPrFromScmToRenovate(pr)); + } catch (error) { + logger.error(error); + } + } + + return config.prList ?? []; +} + +export async function createPr({ + sourceBranch, + targetBranch, + prTitle, + prBody, + draftPR, +}: CreatePRConfig): Promise { + const createdPr = await createScmPr(config.repository, { + source: sourceBranch, + target: targetBranch, + title: prTitle, + description: sanitize(prBody), + status: draftPR ? 'DRAFT' : 'OPEN', + }); + + logger.debug( + `PR created with title '${createdPr.title}' from source '${createdPr.source}' to target '${createdPr.target}'`, + ); + + return mapPrFromScmToRenovate(createdPr); +} + +export async function updatePr({ + number, + prTitle, + prBody, + state, + targetBranch, +}: UpdatePrConfig): Promise { + await updateScmPr(config.repository, number, { + title: prTitle, + description: sanitize(prBody) ?? undefined, + target: targetBranch, + status: mapPrState(state), + }); + + logger.debug(`Updated PR #${number} with title ${prTitle}`); +} + +export function mergePr(_config: MergePRConfig): Promise { + logger.debug('Not implemented mergePr'); + return Promise.resolve(false); +} + +export function getBranchStatus( + _branchName: string, + _internalChecksAsSuccess: boolean, +): Promise { + logger.debug('Not implemented getBranchStatus'); + return Promise.resolve('red'); +} + +export function setBranchStatus( + _branchStatusConfig: BranchStatusConfig, +): Promise { + logger.debug('Not implemented setBranchStatus'); + return Promise.resolve(); +} + +export function getBranchStatusCheck( + _branchName: string, + _context: string | null | undefined, +): Promise { + logger.debug('Not implemented setBranchStatus'); + return Promise.resolve(null); +} + +export function addReviewers( + _number: number, + _reviewers: string[], +): Promise { + logger.debug('Not implemented addReviewers'); + return Promise.resolve(); +} + +export function addAssignees( + _number: number, + _assignees: string[], +): Promise { + logger.debug('Not implemented addAssignees'); + return Promise.resolve(); +} + +export function deleteLabel(_number: number, _label: string): Promise { + logger.debug('Not implemented deleteLabel'); + return Promise.resolve(); +} + +export function getIssueList(): Promise { + logger.debug('Not implemented getIssueList'); + return Promise.resolve([]); +} + +export function findIssue(_title: string): Promise { + logger.debug('Not implemented findIssue'); + return Promise.resolve(null); +} + +export function ensureIssue( + _config: EnsureIssueConfig, +): Promise<'updated' | 'created' | null> { + logger.debug('Not implemented ensureIssue'); + return Promise.resolve(null); +} + +export function ensureIssueClosing(_title: string): Promise { + logger.debug('Not implemented ensureIssueClosing'); + return Promise.resolve(); +} + +export function ensureComment(_config: EnsureCommentConfig): Promise { + logger.debug('Not implemented ensureComment'); + return Promise.resolve(false); +} + +export function ensureCommentRemoval( + _ensureCommentRemoval: + | EnsureCommentRemovalConfigByTopic + | EnsureCommentRemovalConfigByContent, +): Promise { + logger.debug('Not implemented ensureCommentRemoval'); + return Promise.resolve(); +} + +export function massageMarkdown(prBody: string): string { + return smartTruncate(smartLinks(prBody), maxBodyLength()); +} + +export function getRepoForceRebase(): Promise { + return Promise.resolve(false); +} + +export function getRawFile( + _fileName: string, + _repoName?: string, + _branchOrTag?: string, +): Promise { + logger.debug('Not implemented getRawFile'); + return Promise.resolve(null); +} + +export function getJsonFile( + _fileName: string, + _repoName?: string, + _branchOrTag?: string, +): Promise { + logger.debug('Not implemented getJsonFile'); + return Promise.resolve(null); +} + +export function maxBodyLength(): number { + return 200000; +} + +export function invalidatePrCache(): void { + config.prList = null; +} diff --git a/lib/modules/platform/scm-manager/mapper.spec.ts b/lib/modules/platform/scm-manager/mapper.spec.ts new file mode 100644 index 00000000000..b1142fb48b1 --- /dev/null +++ b/lib/modules/platform/scm-manager/mapper.spec.ts @@ -0,0 +1,44 @@ +import { mapPrFromScmToRenovate } from './mapper.ts'; +import type { PullRequest } from './schema.ts'; + +describe('modules/platform/scm-manager/mapper', () => { + it('should correctly map the scm-manager type of a PR to the Renovate PR type', () => { + const scmPr: PullRequest = { + source: 'feat/new', + target: 'develop', + creationDate: '2024-12-24T18:21Z', + closeDate: '2024-12-25T18:21Z', + reviewer: [ + { id: 'id', displayName: 'user', mail: 'user@user.de', approved: true }, + ], + labels: ['label'], + id: '1', + status: 'OPEN', + title: 'Merge please', + description: 'Description', + tasks: { todo: 0, done: 0 }, + _links: {}, + _embedded: { + defaultConfig: { + mergeStrategy: 'SQUASH', + deleteBranchOnMerge: true, + }, + }, + }; + + const result = mapPrFromScmToRenovate(scmPr); + expect(result).toEqual({ + sourceBranch: 'feat/new', + targetBranch: 'develop', + createdAt: '2024-12-24T18:21Z', + closedAt: '2024-12-25T18:21Z', + hasAssignees: true, + labels: ['label'], + number: 1, + reviewers: ['user'], + state: 'OPEN', + title: 'Merge please', + isDraft: false, + }); + }); +}); diff --git a/lib/modules/platform/scm-manager/mapper.ts b/lib/modules/platform/scm-manager/mapper.ts new file mode 100644 index 00000000000..eb21c39596c --- /dev/null +++ b/lib/modules/platform/scm-manager/mapper.ts @@ -0,0 +1,18 @@ +import type { Pr } from '../types.ts'; +import type { PullRequest } from './schema.ts'; + +export function mapPrFromScmToRenovate(pr: PullRequest): Pr { + return { + sourceBranch: pr.source, + targetBranch: pr.target, + createdAt: pr.creationDate, + closedAt: pr.closeDate ?? undefined, + hasAssignees: !!pr.reviewer?.length, + labels: pr.labels, + number: parseInt(pr.id), + reviewers: pr.reviewer?.map((review) => review.displayName) ?? [], + state: pr.status, + title: pr.title, + isDraft: pr.status === 'DRAFT', + }; +} diff --git a/lib/modules/platform/scm-manager/readme.md b/lib/modules/platform/scm-manager/readme.md new file mode 100644 index 00000000000..6a991c9f521 --- /dev/null +++ b/lib/modules/platform/scm-manager/readme.md @@ -0,0 +1,40 @@ +# SCM-Manager + +Renovate supports the [SCM-Manager](https://scm-manager.org) platform. +This platform is considered experimental. + +## Authentication + +1. Create an API Key for your technical Renovate user in SCM-Manager +1. The technical user _must_ have a valid name and email address +1. Put the API key in the `RENOVATE_TOKEN` environment variable, so that Renovate can use it + +## Set correct platform + +You must set the [`platform`](../../../self-hosted-configuration.md#platform) config option to `scm-manager` in your admin config file. + +## Set permissions + +The technical user must have the permissions to: + +1. read the repository +1. pull/checkout the repository +1. push/commit the repository +1. create pull requests for the repository + +Those permissions can be granted on a repository level within the permission settings of each repository. + +## Install Review Plugin + +To let Renovate access the Pull Request API, you must install the Review Plugin with at least version 3.11.0. +Find the list of available plugins by going to Administration -> Plugins -> Available. + +## Supported versions of SCM-Manager + +Renovate supports SCM-Manager major version `3.x`. +The minimum version for the `3.x` range is `3.10.0`. + +## Automerge + +Currently, the Renovate automerge feature is not supported by the SCM-Manager platform. +Every pull request requires merging them manually for now. diff --git a/lib/modules/platform/scm-manager/schema.ts b/lib/modules/platform/scm-manager/schema.ts new file mode 100644 index 00000000000..bf98fc3cbc6 --- /dev/null +++ b/lib/modules/platform/scm-manager/schema.ts @@ -0,0 +1,123 @@ +import { z } from 'zod/v3'; +import { EmailAddress } from '../../../util/schema-utils/index.ts'; + +export const User = z.object({ + mail: EmailAddress, + displayName: z.string(), + name: z.string(), +}); +export type User = z.infer; + +export const DefaultBranchSchema = z.object({ + defaultBranch: z.string(), +}); + +export const LinkSchema = z.object({ + href: z.string(), + name: z.string().optional().nullable(), + templated: z.boolean().optional().nullable(), +}); +export type Link = z.infer; + +export const LinksSchema = z.record( + z.string(), + z.union([LinkSchema, z.array(LinkSchema)]), +); +export type Links = z.infer; + +export const PrStateSchema = z.enum(['DRAFT', 'OPEN', 'REJECTED', 'MERGED']); +export type PrState = z.infer; + +export const PrMergeMethodSchema = z.enum([ + 'MERGE_COMMIT', + 'REBASE', + 'FAST_FORWARD_IF_POSSIBLE', + 'FAST_FORWARD_ONLY', + 'SQUASH', +]); +export type PrMergeMethod = z.infer; + +export const PullRequestSchema = z.object({ + id: z.string(), + author: z + .object({ + mail: z.string().optional().nullable(), + displayName: z.string(), + id: z.string(), + }) + .optional() + .nullable(), + reviser: z + .object({ + id: z.string().optional().nullable(), + displayName: z.string().optional().nullable(), + }) + .optional() + .nullable(), + closeDate: z.string().optional().nullable(), + source: z.string(), + target: z.string(), + title: z.string(), + description: z.string().optional().nullable(), + creationDate: z.string(), + lastModified: z.string().optional().nullable(), + status: PrStateSchema, + reviewer: z + .array( + z.object({ + id: z.string(), + displayName: z.string(), + mail: z.string().optional().nullable(), + approved: z.boolean(), + }), + ) + .optional() + .nullable(), + labels: z.string().array(), + tasks: z.object({ + todo: z.number(), + done: z.number(), + }), + _links: LinksSchema, + _embedded: z.object({ + defaultConfig: z.object({ + mergeStrategy: PrMergeMethodSchema, + deleteBranchOnMerge: z.boolean(), + }), + }), +}); +export type PullRequest = z.infer; + +const RepoTypeSchema = z.enum(['git', 'svn', 'hg']); + +export const RepoSchema = z.object({ + contact: z.string().optional().nullable(), + creationDate: z.string().optional().nullable(), + description: z.string().optional().nullable(), + lastModified: z.string().optional().nullable(), + namespace: z.string(), + name: z.string(), + type: RepoTypeSchema, + archived: z.boolean(), + exporting: z.boolean(), + healthCheckRunning: z.boolean(), + _links: LinksSchema, +}); +export type Repo = z.infer; + +const PagedSchema = z.object({ + page: z.number(), + pageTotal: z.number(), +}); + +export const PagedPullRequestSchema = PagedSchema.extend({ + _embedded: z.object({ + pullRequests: z.array(PullRequestSchema), + }), +}); + +export const PagedRepoSchema = PagedSchema.extend({ + _embedded: z.object({ + repositories: z.array(RepoSchema), + }), +}); diff --git a/lib/modules/platform/scm-manager/scm-manager-helper.spec.ts b/lib/modules/platform/scm-manager/scm-manager-helper.spec.ts new file mode 100644 index 00000000000..45f6bea20f3 --- /dev/null +++ b/lib/modules/platform/scm-manager/scm-manager-helper.spec.ts @@ -0,0 +1,402 @@ +import * as httpMock from '~test/http-mock.ts'; +import { setBaseUrl } from '../../../util/http/scm-manager.ts'; +import type { PullRequest, Repo, User } from './schema.ts'; +import { + createScmPr, + getAllRepoPrs, + getAllRepos, + getCurrentUser, + getDefaultBranch, + getRepo, + getRepoPr, + updateScmPr, +} from './scm-manager-helper.ts'; +import type { + PullRequestCreateParams, + PullRequestUpdateParams, +} from './types.ts'; + +describe('modules/platform/scm-manager/scm-manager-helper', () => { + const endpoint = 'http://localhost:8080/scm/api/v2'; + const token = 'apiToken'; + setBaseUrl(endpoint); + + const repo: Repo = { + contact: 'test@test.com', + creationDate: '2023-08-02T10:48:24.762Z', + description: 'Default Repo', + lastModified: '2023-08-10T10:48:24.762Z', + namespace: 'default', + name: 'repo', + type: 'git', + archived: false, + exporting: false, + healthCheckRunning: false, + _links: { + protocol: [ + { name: 'http', href: 'http://localhost:8080/scm/default/repo' }, + ], + defaultBranch: { + href: `${endpoint}/config/git/default/repo/default-branch`, + }, + }, + }; + + const pullRequest: PullRequest = { + id: '1337', + author: { displayName: 'Thomas Zerr', id: 'tzerr' }, + source: 'feature/test', + target: 'develop', + title: 'The PullRequest', + description: 'Another PullRequest', + creationDate: '2023-08-02T10:48:24.762Z', + status: 'OPEN', + labels: [], + tasks: { todo: 2, done: 4 }, + _links: {}, + _embedded: { + defaultConfig: { + mergeStrategy: 'SQUASH', + deleteBranchOnMerge: true, + }, + }, + }; + + describe(getCurrentUser, () => { + it('should return the current user', async () => { + const expectedUser: User = { + mail: 'test@test.de', + displayName: 'Test User', + name: 'test', + }; + + httpMock.scope(endpoint).get('/me').reply(200, expectedUser); + + expect(await getCurrentUser(token)).toEqual(expectedUser); + }); + + it.each` + expectedResponse + ${401} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock.scope(endpoint).get('/me').reply(expectedResponse); + await expect(getCurrentUser(token)).rejects.toThrow(); + }, + ); + }); + + describe(getRepo, () => { + it('should return the repo', async () => { + httpMock + .scope(endpoint) + .get(`/repositories/${repo.namespace}/${repo.name}`) + .reply(200, repo); + + expect(await getRepo(`${repo.namespace}/${repo.name}`)).toEqual(repo); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${404} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock + .scope(endpoint) + .get(`/repositories/${repo.namespace}/${repo.name}`) + .reply(expectedResponse); + + await expect( + getRepo(`${repo.namespace}/${repo.name}`), + ).rejects.toThrow(); + }, + ); + }); + + describe(getAllRepos, () => { + it('should return all repos', async () => { + httpMock + .scope(endpoint) + .get('/repositories?pageSize=1000000') + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { repositories: [repo] }, + }); + + expect(await getAllRepos()).toEqual([repo]); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock + .scope(endpoint) + .get('/repositories?pageSize=1000000') + .reply(expectedResponse); + + await expect(getAllRepos()).rejects.toThrow(); + }, + ); + }); + + describe(getDefaultBranch, () => { + it('should return the default branch', async () => { + httpMock + .scope(endpoint) + .get('/config/git/default/repo/default-branch') + .reply(200, { + defaultBranch: 'develop', + }); + + expect(await getDefaultBranch(repo)).toBe('develop'); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${404} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock + .scope(endpoint) + .get('/config/git/default/repo/default-branch') + .reply(expectedResponse); + + await expect(getDefaultBranch(repo)).rejects.toThrow(); + }, + ); + }); + + describe(getAllRepoPrs, () => { + it('should return all repo PRs', async () => { + httpMock + .scope(endpoint) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [pullRequest], + }, + }); + + expect( + await getAllRepoPrs(`${repo.namespace}/${repo.name}`, true), + ).toEqual([pullRequest]); + }); + + it('should return all of my PRs', async () => { + httpMock + .scope(endpoint) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=MINE&pageSize=1000000`, + ) + .reply(200, { + page: 0, + pageTotal: 1, + _embedded: { + pullRequests: [pullRequest], + }, + }); + + expect( + await getAllRepoPrs(`${repo.namespace}/${repo.name}`, false), + ).toEqual([pullRequest]); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${404} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock + .scope(endpoint) + .get( + `/pull-requests/${repo.namespace}/${repo.name}?status=ALL&pageSize=1000000`, + ) + .reply(expectedResponse); + + await expect( + getAllRepoPrs(`${repo.namespace}/${repo.name}`, true), + ).rejects.toThrow(); + }, + ); + }); + + describe(getRepoPr, () => { + it('should return the repo PR', async () => { + httpMock + .scope(endpoint) + .get(`/pull-requests/${repo.namespace}/${repo.name}/${pullRequest.id}`) + .reply(200, pullRequest); + + expect(await getRepoPr(`${repo.namespace}/${repo.name}`, 1337)).toEqual( + pullRequest, + ); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${404} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock + .scope(endpoint) + .get( + `/pull-requests/${repo.namespace}/${repo.name}/${pullRequest.id}`, + ) + .reply(expectedResponse); + + await expect( + getRepoPr(`${repo.namespace}/${repo.name}`, 1337), + ).rejects.toThrow(); + }, + ); + }); + + describe(createScmPr, () => { + it('should create PR for a repo', async () => { + const expectedCreateParams: PullRequestCreateParams = { + source: 'feature/test', + target: 'develop', + title: 'Test Title', + description: 'PR description', + status: 'OPEN', + }; + + const expectedPrId = 1337; + + httpMock + .scope(endpoint) + .post(`/pull-requests/${repo.namespace}/${repo.name}`) + .reply(201, undefined, { + location: `${endpoint}/pull-requests/${repo.namespace}/${repo.name}/${expectedPrId}`, + }); + + httpMock + .scope(endpoint) + .get(`/pull-requests/${repo.namespace}/${repo.name}/${expectedPrId}`) + .reply(200, pullRequest); + + expect( + await createScmPr( + `${repo.namespace}/${repo.name}`, + expectedCreateParams, + ), + ).toEqual(pullRequest); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${404} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + httpMock + .scope(endpoint) + .post(`/pull-requests/${repo.namespace}/${repo.name}`) + .reply(expectedResponse); + + await expect( + createScmPr(`${repo.namespace}/${repo.name}`, { + source: 'feature/test', + target: 'develop', + title: 'Test Title', + description: 'PR description', + status: 'OPEN', + }), + ).rejects.toThrow(); + }, + ); + }); + + describe(updateScmPr, () => { + it('should update PR for a repo', async () => { + const expectedUpdateParams: PullRequestUpdateParams = { + title: 'Test Title', + description: 'PR description', + status: 'OPEN', + target: 'new/target', + }; + + const expectedPrId = 1337; + + httpMock + .scope(endpoint) + .get(`/pull-requests/${repo.namespace}/${repo.name}/${expectedPrId}`) + .reply(200, pullRequest); + + httpMock + .scope(endpoint) + .put(`/pull-requests/${repo.namespace}/${repo.name}/${expectedPrId}`) + .reply(204); + + await expect( + updateScmPr( + `${repo.namespace}/${repo.name}`, + expectedPrId, + expectedUpdateParams, + ), + ).resolves.not.toThrow(); + }); + + it.each` + expectedResponse + ${401} + ${403} + ${404} + ${500} + `( + 'should throw expected response $expectedResponse', + async ({ expectedResponse }: { expectedResponse: number }) => { + const expectedPrId = 1337; + + httpMock + .scope(endpoint) + .get(`/pull-requests/${repo.namespace}/${repo.name}/${expectedPrId}`) + .reply(200, pullRequest); + + httpMock + .scope(endpoint) + .put(`/pull-requests/${repo.namespace}/${repo.name}/${expectedPrId}`) + .reply(expectedResponse); + + await expect( + updateScmPr(`${repo.namespace}/${repo.name}`, expectedPrId, { + title: 'Test Title', + description: 'PR description', + status: 'OPEN', + }), + ).rejects.toThrow(); + }, + ); + }); +}); diff --git a/lib/modules/platform/scm-manager/scm-manager-helper.ts b/lib/modules/platform/scm-manager/scm-manager-helper.ts new file mode 100644 index 00000000000..0cf8d86805e --- /dev/null +++ b/lib/modules/platform/scm-manager/scm-manager-helper.ts @@ -0,0 +1,170 @@ +import { ScmManagerHttp } from '../../../util/http/scm-manager.ts'; +import type { Link, PullRequest, Repo } from './schema.ts'; +import { + DefaultBranchSchema, + PagedPullRequestSchema, + PagedRepoSchema, + PullRequestSchema, + RepoSchema, + User, +} from './schema.ts'; +import type { + PullRequestCreateParams, + PullRequestUpdateParams, +} from './types.ts'; + +export const scmManagerHttp = new ScmManagerHttp(); + +const URLS = { + ME: 'me', + ALL_REPOS: 'repositories?pageSize=1000000', + REPO: (repoPath: string) => `repositories/${repoPath}`, + PULLREQUESTS: (repoPath: string) => `pull-requests/${repoPath}`, + ALL_PULLREQUESTS_WITH_PAGINATION: (repoPath: string) => + `pull-requests/${repoPath}?status=ALL&pageSize=1000000`, + MY_PULLREQUESTS_WITH_PAGINATION: (repoPath: string) => + `pull-requests/${repoPath}?status=MINE&pageSize=1000000`, + PULLREQUEST_BY_ID: (repoPath: string, id: number) => + `pull-requests/${repoPath}/${id}`, +}; + +const CONTENT_TYPES = { + ME: 'application/vnd.scmm-me+json;v=2', + REPOSITORY: 'application/vnd.scmm-repository+json;v=2', + REPOSITORIES: 'application/vnd.scmm-repositoryCollection+json;v=2', + GIT_CONFIG: 'application/vnd.scmm-gitDefaultBranch+json;v=2', + PULLREQUEST: 'application/vnd.scmm-pullRequest+json;v=2', + PULLREQUESTS: 'application/vnd.scmm-pullRequestCollection+json;v=2', +}; + +export async function getCurrentUser(token: string): Promise { + const response = await scmManagerHttp.getJson( + URLS.ME, + { + headers: { accept: CONTENT_TYPES.ME }, + token, + }, + User, + ); + return response.body; +} + +export async function getRepo(repoPath: string): Promise { + const response = await scmManagerHttp.getJson( + URLS.REPO(repoPath), + { + headers: { accept: CONTENT_TYPES.REPOSITORY }, + }, + RepoSchema, + ); + return response.body; +} + +export async function getAllRepos(): Promise { + const response = await scmManagerHttp.getJson( + URLS.ALL_REPOS, + { + headers: { accept: CONTENT_TYPES.REPOSITORIES }, + }, + PagedRepoSchema, + ); + + return response.body._embedded.repositories; +} + +export async function getDefaultBranch(repo: Repo): Promise { + const defaultBranchUrl = repo._links.defaultBranch as Link; + const response = await scmManagerHttp.getJson( + defaultBranchUrl.href, + { + headers: { accept: CONTENT_TYPES.GIT_CONFIG }, + }, + DefaultBranchSchema, + ); + + return response.body.defaultBranch; +} + +export async function getAllRepoPrs( + repoPath: string, + ignorePrAuthor: boolean, +): Promise { + const response = await scmManagerHttp.getJson( + ignorePrAuthor + ? URLS.ALL_PULLREQUESTS_WITH_PAGINATION(repoPath) + : URLS.MY_PULLREQUESTS_WITH_PAGINATION(repoPath), + { + headers: { accept: CONTENT_TYPES.PULLREQUESTS }, + }, + PagedPullRequestSchema, + ); + return response.body._embedded.pullRequests; +} + +export async function getRepoPr( + repoPath: string, + id: number, +): Promise { + const response = await scmManagerHttp.getJson( + URLS.PULLREQUEST_BY_ID(repoPath, id), + { + headers: { accept: CONTENT_TYPES.PULLREQUEST }, + }, + PullRequestSchema, + ); + + return response.body; +} + +export async function createScmPr( + repoPath: string, + params: PullRequestCreateParams, +): Promise { + const createPrResponse = await scmManagerHttp.postJson( + URLS.PULLREQUESTS(repoPath), + { + body: params, + headers: { + 'Content-Type': CONTENT_TYPES.PULLREQUEST, + accept: CONTENT_TYPES.PULLREQUEST, + }, + }, + ); + + const getCreatedPrResponse = await scmManagerHttp.getJson( + createPrResponse.headers.location!, + { + headers: { accept: CONTENT_TYPES.PULLREQUEST }, + }, + PullRequestSchema, + ); + + return getCreatedPrResponse.body; +} + +export async function updateScmPr( + repoPath: string, + id: number, + params: PullRequestUpdateParams, +): Promise { + const currentPr = await getRepoPr(repoPath, id); + await scmManagerHttp.putJson(URLS.PULLREQUEST_BY_ID(repoPath, id), { + body: mergePullRequestWithUpdate(currentPr, params), + headers: { + 'Content-Type': CONTENT_TYPES.PULLREQUEST, + }, + }); +} + +function mergePullRequestWithUpdate( + pr: PullRequest, + updateParams: PullRequestUpdateParams, +): PullRequest { + return { + ...pr, + title: updateParams.title, + description: updateParams.description ?? pr.description, + status: updateParams.status ?? pr.status, + target: updateParams.target ?? pr.target, + }; +} diff --git a/lib/modules/platform/scm-manager/types.ts b/lib/modules/platform/scm-manager/types.ts new file mode 100644 index 00000000000..9a508461e1d --- /dev/null +++ b/lib/modules/platform/scm-manager/types.ts @@ -0,0 +1,15 @@ +import type { PrState } from './schema.ts'; + +export interface PullRequestCreateParams extends PullRequestUpdateParams { + source: string; + target: string; +} + +export interface PullRequestUpdateParams { + title: string; + description?: string; + status?: PrState; + target?: string; +} + +export type PrFilterByState = 'open' | 'closed' | '!open' | 'all'; diff --git a/lib/modules/platform/scm-manager/utils.spec.ts b/lib/modules/platform/scm-manager/utils.spec.ts new file mode 100644 index 00000000000..5bb7771c68b --- /dev/null +++ b/lib/modules/platform/scm-manager/utils.spec.ts @@ -0,0 +1,283 @@ +import type { MergeStrategy } from '../../../config/types.ts'; +import * as hostRules from '../../../util/host-rules.ts'; +import type { GitUrlOption, Pr } from '../types.ts'; +import { invalidatePrCache } from './index.ts'; +import type { Repo } from './schema.ts'; +import type { PrFilterByState } from './types.ts'; +import { + getMergeMethod, + getRepoUrl, + matchPrState, + smartLinks, +} from './utils.ts'; + +describe('modules/platform/scm-manager/utils', () => { + describe(getMergeMethod, () => { + it.each` + strategy | method + ${undefined} | ${null} + ${'auto'} | ${null} + ${'fast-forward'} | ${'FAST_FORWARD_ONLY'} + ${'merge-commit'} | ${'MERGE_COMMIT'} + ${'rebase'} | ${'REBASE'} + ${'squash'} | ${'SQUASH'} + `( + 'map merge strategy $strategy on PR merge method $method', + ({ + strategy, + method, + }: { + strategy: string | undefined; + method: string | null; + }) => { + expect(getMergeMethod(strategy as MergeStrategy)).toEqual(method); + }, + ); + }); + + describe(smartLinks, () => { + it.each` + body | result + ${''} | ${''} + ${'](../pull/'} | ${'](pulls/'} + `( + 'adjust $body to smart link $result', + ({ body, result }: { body: string; result: string }) => { + expect(smartLinks(body)).toEqual(result); + }, + ); + }); + + describe(matchPrState, () => { + const defaultPr: Pr = { + sourceBranch: 'feature/test', + createdAt: '2023-08-02T10:48:24.762Z', + number: 1, + state: '', + title: 'Feature Test PR', + isDraft: false, + }; + + it.each` + pr | state | expectedResult + ${{ ...defaultPr, state: 'OPEN' }} | ${'all'} | ${true} + ${{ ...defaultPr, state: 'DRAFT' }} | ${'all'} | ${true} + ${{ ...defaultPr, state: 'MERGED' }} | ${'all'} | ${true} + ${{ ...defaultPr, state: 'REJECTED' }} | ${'all'} | ${true} + ${{ ...defaultPr, state: 'OPEN' }} | ${'open'} | ${true} + ${{ ...defaultPr, state: 'DRAFT' }} | ${'open'} | ${true} + ${{ ...defaultPr, state: 'MERGED' }} | ${'open'} | ${false} + ${{ ...defaultPr, state: 'REJECTED' }} | ${'open'} | ${false} + ${{ ...defaultPr, state: 'OPEN' }} | ${'!open'} | ${false} + ${{ ...defaultPr, state: 'DRAFT' }} | ${'!open'} | ${false} + ${{ ...defaultPr, state: 'MERGED' }} | ${'!open'} | ${true} + ${{ ...defaultPr, state: 'REJECTED' }} | ${'!open'} | ${true} + ${{ ...defaultPr, state: 'OPEN' }} | ${'closed'} | ${false} + ${{ ...defaultPr, state: 'DRAFT' }} | ${'closed'} | ${false} + ${{ ...defaultPr, state: 'MERGED' }} | ${'closed'} | ${true} + ${{ ...defaultPr, state: 'REJECTED' }} | ${'closed'} | ${true} + `( + 'match scm pr state $pr.state to renovate pr state $state', + ({ + pr, + state, + expectedResult, + }: { + pr: Pr; + state: string; + expectedResult: boolean; + }) => { + expect(matchPrState(pr, state as PrFilterByState)).toEqual( + expectedResult, + ); + }, + ); + }); + + describe(getRepoUrl, () => { + const repo: Repo = { + contact: 'test@test.com', + creationDate: '2023-08-02T10:48:24.762Z', + description: 'Default Repo', + lastModified: '2023-08-10T10:48:24.762Z', + namespace: 'default', + name: 'repo', + type: 'git', + archived: false, + exporting: false, + healthCheckRunning: false, + _links: {}, + }; + + const endpoint = 'http://localhost:8081/scm/api/v2'; + const gitHttpEndpoint = 'http://localhost:8081/scm/repo/default/repo'; + const gitSshEndpoint = 'ssh://localhost:2222/scm/repo/default/repo'; + + beforeEach(() => { + hostRules.add({ token: 'token', username: 'tzerr' }); + invalidatePrCache(); + }); + + it.each` + gitUrl + ${'ssh'} + ${'default'} + ${'endpoint'} + ${undefined} + `( + 'should throw error for option $gitUrl, because protocol links are missing', + ({ gitUrl }: { gitUrl: string | undefined }) => { + expect(() => + getRepoUrl(repo, gitUrl as GitUrlOption, endpoint), + ).toThrow('Missing protocol links.'); + }, + ); + + it('should throw error because of missing SSH link', () => { + expect(() => + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'http', href: gitHttpEndpoint }] }, + }, + 'ssh', + endpoint, + ), + ).toThrow('MISSING_SSH_LINK'); + }); + + it('should throw error because protocol links are not an array', () => { + expect(() => + getRepoUrl( + { + ...repo, + _links: { protocol: { name: 'http', href: gitHttpEndpoint } }, + }, + 'ssh', + endpoint, + ), + ).toThrow('Expected protocol links to be an array of links.'); + }); + + it('should use the provided ssh link', () => { + expect( + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'ssh', href: gitSshEndpoint }] }, + }, + 'ssh', + endpoint, + ), + ).toEqual(gitSshEndpoint); + }); + + it.each` + gitUrl + ${'default'} + ${'endpoint'} + ${undefined} + `( + 'should throw error because of missing HTTP link for option $gitUrl', + ({ gitUrl }: { gitUrl: string | undefined }) => { + expect(() => + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'ssh', href: gitSshEndpoint }] }, + }, + gitUrl as GitUrlOption | undefined, + endpoint, + ), + ).toThrow('MISSING_HTTP_LINK'); + }, + ); + + it.each` + gitUrl + ${'default'} + ${'endpoint'} + ${undefined} + `( + 'should throw error because of malformed HTTP link with option $gitUrl', + ({ gitUrl }: { gitUrl: string | undefined }) => { + expect(() => + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'http', href: 'invalid url' }] }, + }, + gitUrl as GitUrlOption | undefined, + endpoint, + ), + ).toThrow('MALFORMED_HTTP_LINK'); + }, + ); + + it.each` + gitUrl + ${'default'} + ${'endpoint'} + ${undefined} + `( + 'should use empty string, because username was not provided with option $gitUrl', + ({ gitUrl }: { gitUrl: string | undefined }) => { + hostRules.clear(); + expect( + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'http', href: gitHttpEndpoint }] }, + }, + gitUrl as GitUrlOption | undefined, + endpoint, + ), + ).toBe('http://localhost:8081/scm/repo/default/repo'); + }, + ); + + it.each` + gitUrl + ${'default'} + ${'endpoint'} + ${undefined} + `( + 'should use empty string, because token was not provided. With option $gitUrl', + ({ gitUrl }: { gitUrl: string | undefined }) => { + hostRules.clear(); + hostRules.add({ username: 'tzerr' }); + expect( + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'http', href: gitHttpEndpoint }] }, + }, + gitUrl as GitUrlOption | undefined, + endpoint, + ), + ).toBe('http://tzerr@localhost:8081/scm/repo/default/repo'); + }, + ); + + it.each` + gitUrl + ${'default'} + ${'endpoint'} + ${undefined} + `( + 'should provide the HTTP link with username, for option $gitUrl', + ({ gitUrl }: { gitUrl: string | undefined }) => { + expect( + getRepoUrl( + { + ...repo, + _links: { protocol: [{ name: 'http', href: gitHttpEndpoint }] }, + }, + gitUrl as GitUrlOption | undefined, + endpoint, + ), + ).toBe('http://tzerr:token@localhost:8081/scm/repo/default/repo'); + }, + ); + }); +}); diff --git a/lib/modules/platform/scm-manager/utils.ts b/lib/modules/platform/scm-manager/utils.ts new file mode 100644 index 00000000000..fa9606ce7b7 --- /dev/null +++ b/lib/modules/platform/scm-manager/utils.ts @@ -0,0 +1,113 @@ +import type { MergeStrategy } from '../../../config/types.ts'; +import { logger } from '../../../logger/index.ts'; +import * as hostRules from '../../../util/host-rules.ts'; +import { regEx } from '../../../util/regex.ts'; +import { parseUrl } from '../../../util/url.ts'; +import type { GitUrlOption, Pr } from '../types.ts'; +import type { PrMergeMethod, Repo } from './schema.ts'; +import type { PrFilterByState } from './types.ts'; + +export function mapPrState( + state: 'open' | 'closed' | undefined, +): 'OPEN' | 'REJECTED' | undefined { + switch (state) { + case 'open': + return 'OPEN'; + case 'closed': + return 'REJECTED'; + default: + return undefined; + } +} + +export function matchPrState(pr: Pr, state: PrFilterByState): boolean { + if (state === 'all') { + return true; + } + + if (state === 'open' && (pr.state === 'OPEN' || pr.state === 'DRAFT')) { + return true; + } + + if (state === '!open' && (pr.state === 'MERGED' || pr.state === 'REJECTED')) { + return true; + } + + if ( + state === 'closed' && + (pr.state === 'MERGED' || pr.state === 'REJECTED') + ) { + return true; + } + + return false; +} + +export function smartLinks(body: string): string { + return body.replace(regEx(/]\(\.\.\/pull\//g), '](pulls/'); +} + +export function getRepoUrl( + repo: Repo, + gitUrl: GitUrlOption | undefined, + endpoint: string, +): string { + const protocolLinks = repo._links.protocol; + + if (!protocolLinks) { + throw new Error('Missing protocol links.'); + } + + if (!Array.isArray(protocolLinks)) { + throw new Error('Expected protocol links to be an array of links.'); + } + + if (gitUrl === 'ssh') { + const sshUrl = protocolLinks.find((l) => l.name === 'ssh')?.href; + if (!sshUrl) { + throw new Error('MISSING_SSH_LINKS'); + } + + logger.debug(`Using SSH URL: ${sshUrl}`); + return sshUrl; + } + + const httpUrl = protocolLinks.find((l) => l.name === 'http')?.href; + if (!httpUrl) { + throw new Error('MISSING_HTTP_LINK'); + } + + logger.debug(`Using HTTP URL: ${httpUrl}`); + + const repoUrl = parseUrl(httpUrl); + if (!repoUrl) { + throw new Error('MALFORMED_HTTP_LINK'); + } + + const hostOptions = hostRules.find({ + hostType: 'scm-manager', + url: endpoint, + }); + + repoUrl.username = hostOptions.username ?? ''; + repoUrl.password = hostOptions.token ?? ''; + + return repoUrl.toString(); +} + +export function getMergeMethod( + strategy: MergeStrategy | undefined, +): PrMergeMethod | null { + switch (strategy) { + case 'fast-forward': + return 'FAST_FORWARD_ONLY'; + case 'merge-commit': + return 'MERGE_COMMIT'; + case 'rebase': + return 'REBASE'; + case 'squash': + return 'SQUASH'; + default: + return null; + } +} diff --git a/lib/modules/platform/scm.ts b/lib/modules/platform/scm.ts index 379870f62c7..d1e00faa042 100644 --- a/lib/modules/platform/scm.ts +++ b/lib/modules/platform/scm.ts @@ -18,6 +18,7 @@ platformScmImpls.set('gitea', DefaultGitScm); platformScmImpls.set('github', GithubScm); platformScmImpls.set('gitlab', DefaultGitScm); platformScmImpls.set('local', LocalFs); +platformScmImpls.set('scm-manager', DefaultGitScm); let _scm: PlatformScm | undefined; diff --git a/lib/util/exec/index.spec.ts b/lib/util/exec/index.spec.ts index 550432385ab..04bc4728564 100644 --- a/lib/util/exec/index.spec.ts +++ b/lib/util/exec/index.spec.ts @@ -8,7 +8,7 @@ import type { UpdateArtifactsConfig } from '../../modules/manager/types.ts'; import { setCustomEnv } from '../env.ts'; import * as dockerModule from './docker/index.ts'; import { getHermitEnvs } from './hermit.ts'; -import { exec, getToolSettingsOptions } from './index.ts'; +import { exec, getToolSettingsOptions, gradleJvmArg } from './index.ts'; import type { CommandWithOptions, ExecOptions, @@ -1415,4 +1415,11 @@ describe('util/exec/index', () => { }); }); }); + + describe('gradleJvmArg()', () => { + it('takes the values given to it, and returns the JVM arguments', () => { + const result = gradleJvmArg({ jvmMemory: 256, jvmMaxMemory: 768 }); + expect(result).toBe(' -Dorg.gradle.jvmargs="-Xms256m -Xmx768m"'); + }); + }); }); diff --git a/lib/util/exec/index.ts b/lib/util/exec/index.ts index e67d87913b8..825c7cf4271 100644 --- a/lib/util/exec/index.ts +++ b/lib/util/exec/index.ts @@ -273,3 +273,7 @@ export function getToolSettingsOptions( return options; } + +export function gradleJvmArg(config: ToolSettingsOptions): string { + return ` -Dorg.gradle.jvmargs="-Xms${config.jvmMemory}m -Xmx${config.jvmMaxMemory}m"`; +} diff --git a/lib/util/http/scm-manager.spec.ts b/lib/util/http/scm-manager.spec.ts new file mode 100644 index 00000000000..4c3647b9879 --- /dev/null +++ b/lib/util/http/scm-manager.spec.ts @@ -0,0 +1,30 @@ +import * as httpMock from '../../../test/http-mock.ts'; +import { ScmManagerHttp, setBaseUrl } from './scm-manager.ts'; + +describe('util/http/scm-manager', () => { + const baseUrl = 'http://localhost:8080/scm/api/v2'; + let scmManagerHttp: ScmManagerHttp; + + beforeEach(() => { + scmManagerHttp = new ScmManagerHttp(); + setBaseUrl(baseUrl); + }); + + it('supports custom accept header', async () => { + const expectedAcceptHeader = 'application/vnd.scmm-me+json;v=2'; + httpMock + .scope(baseUrl, { + reqheaders: { + accept: expectedAcceptHeader, + }, + }) + .get('/example') + .reply(200); + + const response = await scmManagerHttp.getJsonUnchecked('example', { + headers: { accept: expectedAcceptHeader }, + }); + + expect(response.statusCode).toEqual(200); + }); +}); diff --git a/lib/util/http/scm-manager.ts b/lib/util/http/scm-manager.ts new file mode 100644 index 00000000000..b27ac0030cd --- /dev/null +++ b/lib/util/http/scm-manager.ts @@ -0,0 +1,18 @@ +import { ensureTrailingSlash } from '../url.ts'; +import { HttpBase } from './http.ts'; + +let baseUrl: string; +export const setBaseUrl = (newBaseUrl: string): void => { + baseUrl = ensureTrailingSlash(newBaseUrl); +}; +export const getBaseUrl = (): string => baseUrl; + +export class ScmManagerHttp extends HttpBase { + constructor() { + super('scm-manager'); + } + + protected override get baseUrl(): string | undefined { + return baseUrl; + } +} diff --git a/lib/workers/repository/extract/index.ts b/lib/workers/repository/extract/index.ts index 901c20c662d..61a67d2880a 100644 --- a/lib/workers/repository/extract/index.ts +++ b/lib/workers/repository/extract/index.ts @@ -73,8 +73,16 @@ export async function extractAllDependencies( // De-duplicate results using supersedesManagers processSupersedesManagers(extractResults); + // Sort alphabetically for stable log output. See #40091 + const sortedExtractDurations = Object.keys(extractDurations) + .sort() + .reduce>((acc, key) => { + acc[key] = extractDurations[key]; + return acc; + }, {}); + logger.debug( - { managers: extractDurations }, + { managers: sortedExtractDurations }, 'manager extract durations (ms)', ); let fileCount = 0; diff --git a/lib/workers/repository/process/vulnerabilities.ts b/lib/workers/repository/process/vulnerabilities.ts index d7f618e0928..ad29fd1b5eb 100644 --- a/lib/workers/repository/process/vulnerabilities.ts +++ b/lib/workers/repository/process/vulnerabilities.ts @@ -8,7 +8,7 @@ import { isTruthy, } from '@sindresorhus/is'; import type { CvssVector } from 'ae-cvss-calculator'; -import * as _aeCvss from 'ae-cvss-calculator'; +import * as aeCvss from 'ae-cvss-calculator'; import { z } from 'zod/v3'; import { getManagerConfig, mergeChildConfig } from '../../../config/index.ts'; import type { PackageRule, RenovateConfig } from '../../../config/types.ts'; @@ -31,9 +31,6 @@ import type { Vulnerability, } from './types.ts'; -const { fromVector } = (_aeCvss as unknown as { default: typeof _aeCvss }) - .default; - export class Vulnerabilities { private static osvOffline: Promise | undefined; @@ -533,7 +530,7 @@ export class Vulnerabilities { }); try { - const parsedCvssScore: CvssVector | null = fromVector(vector); + const parsedCvssScore: CvssVector | null = aeCvss.fromVector(vector); const res = CvssJsonSchema.parse(parsedCvssScore?.createJsonSchema()); return [res.baseScore.toFixed(1), res.baseSeverity]; diff --git a/package.json b/package.json index e5db79e7865..630de282ce4 100644 --- a/package.json +++ b/package.json @@ -220,7 +220,7 @@ "fs-extra": "11.3.3", "git-url-parse": "16.1.0", "github-url-from-git": "1.5.0", - "glob": "13.0.5", + "glob": "13.0.6", "global-agent": "3.0.0", "google-auth-library": "10.5.0", "got": "14.6.6", @@ -237,7 +237,7 @@ "luxon": "3.7.2", "markdown-it": "14.1.1", "markdown-table": "3.0.4", - "minimatch": "10.2.1", + "minimatch": "10.2.2", "moo": "0.5.2", "ms": "2.1.3", "neotraverse": "0.6.18", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9a66880306c..b7f72f23a6b 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -216,8 +216,8 @@ importers: specifier: 1.5.0 version: 1.5.0 glob: - specifier: 13.0.5 - version: 13.0.5 + specifier: 13.0.6 + version: 13.0.6 global-agent: specifier: 3.0.0 version: 3.0.0 @@ -267,8 +267,8 @@ importers: specifier: 3.0.4 version: 3.0.4 minimatch: - specifier: 10.2.1 - version: 10.2.1 + specifier: 10.2.2 + version: 10.2.2 moo: specifier: 0.5.2 version: 0.5.2 @@ -4084,9 +4084,9 @@ packages: deprecated: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me hasBin: true - glob@13.0.5: - resolution: {integrity: sha512-BzXxZg24Ibra1pbQ/zE7Kys4Ua1ks7Bn6pKLkVPZ9FZe4JQS6/Q7ef3LG1H+k7lUf5l4T3PLSyYyYJVYUvfgTw==} - engines: {node: 20 || >=22} + glob@13.0.6: + resolution: {integrity: sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==} + engines: {node: 18 || 20 || >=22} glob@7.2.3: resolution: {integrity: sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q==} @@ -5166,9 +5166,9 @@ packages: resolution: {integrity: sha512-ethXTt3SGGR+95gudmqJ1eNhRO7eGEGIgYA9vnPatK4/etz2MEVDno5GMCibdMTuBMyElzIlgxMna3K94XDIDQ==} engines: {node: 20 || >=22} - minimatch@10.2.1: - resolution: {integrity: sha512-MClCe8IL5nRRmawL6ib/eT4oLyeKMGCghibcDWK+J0hh0Q8kqSdia6BvbRMVk6mPa6WqUa5uR2oxt6C5jd533A==} - engines: {node: 20 || >=22} + minimatch@10.2.2: + resolution: {integrity: sha512-+G4CpNBxa5MprY+04MbgOw1v7So6n5JY166pFi9KfYwT78fxScCeSNQSNzp6dpPSW2rONOps6Ocam1wFhCgoVw==} + engines: {node: 18 || 20 || >=22} minimatch@3.1.2: resolution: {integrity: sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw==} @@ -5208,8 +5208,8 @@ packages: resolution: {integrity: sha512-DxiNidxSEK+tHG6zOIklvNOwm3hvCrbUrdtzY74U6HKTJxvIDfOUL5W5P2Ghd3DTkhhKPYGqeNUIh5qcM4YBfw==} engines: {node: '>=8'} - minipass@7.1.2: - resolution: {integrity: sha512-qOOzS1cBTWYF4BH8fVePDBOO9iptMnGUEZwNc/cMWnTV2nVLZ7VoNWEPHkYczZA0pdoA7dl6e7FL659nX9S2aw==} + minipass@7.1.3: + resolution: {integrity: sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==} engines: {node: '>=16 || 14 >=14.17'} minizlib@3.1.0: @@ -5683,9 +5683,9 @@ packages: resolution: {integrity: sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA==} engines: {node: '>=16 || 14 >=14.18'} - path-scurry@2.0.1: - resolution: {integrity: sha512-oWyT4gICAu+kaA7QWk/jvCHWarMKNs6pXOGWKDTr7cw4IGcUbW+PeTfbaQiLGheFRpjo6O9J0PmyMfQPjH71oA==} - engines: {node: 20 || >=22} + path-scurry@2.0.2: + resolution: {integrity: sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==} + engines: {node: 18 || 20 || >=22} path-to-regexp@8.3.0: resolution: {integrity: sha512-7jdwVIRtsP8MYpdXSwOS0YdD0Du+qOoF/AEPIt88PcCFrZCzx41oxku1jD88hZBwbNUIEfpqvuhjFaMAqMTWnA==} @@ -8316,7 +8316,7 @@ snapshots: '@isaacs/fs-minipass@4.0.1': dependencies: - minipass: 7.1.2 + minipass: 7.1.3 '@istanbuljs/load-nyc-config@1.1.0': dependencies: @@ -9480,7 +9480,7 @@ snapshots: '@types/cacache@20.0.1': dependencies: '@types/node': 24.10.13 - minipass: 7.1.2 + minipass: 7.1.3 '@types/cacheable-request@6.0.3': dependencies: @@ -10228,9 +10228,9 @@ snapshots: dependencies: '@npmcli/fs': 5.0.0 fs-minipass: 3.0.3 - glob: 13.0.5 + glob: 13.0.6 lru-cache: 11.2.5 - minipass: 7.1.2 + minipass: 7.1.3 minipass-collect: 2.0.1 minipass-flush: 1.0.5 minipass-pipeline: 1.2.4 @@ -10899,7 +10899,7 @@ snapshots: eslint: 9.39.2 eslint-import-context: 0.1.9(unrs-resolver@1.11.1) is-glob: 4.0.3 - minimatch: 10.2.1 + minimatch: 10.2.2 semver: 7.7.4 stable-hash-x: 0.2.0 unrs-resolver: 1.11.1 @@ -11229,7 +11229,7 @@ snapshots: fs-minipass@3.0.3: dependencies: - minipass: 7.1.2 + minipass: 7.1.3 fs.realpath@1.0.0: {} @@ -11369,15 +11369,15 @@ snapshots: foreground-child: 3.3.1 jackspeak: 3.4.3 minimatch: 9.0.5 - minipass: 7.1.2 + minipass: 7.1.3 package-json-from-dist: 1.0.1 path-scurry: 1.11.1 - glob@13.0.5: + glob@13.0.6: dependencies: - minimatch: 10.2.1 - minipass: 7.1.2 - path-scurry: 2.0.1 + minimatch: 10.2.2 + minipass: 7.1.3 + path-scurry: 2.0.2 glob@7.2.3: dependencies: @@ -12192,7 +12192,7 @@ snapshots: '@npmcli/agent': 4.0.0 cacache: 20.0.3 http-cache-semantics: 4.2.0 - minipass: 7.1.2 + minipass: 7.1.3 minipass-fetch: 5.0.1 minipass-flush: 1.0.5 minipass-pipeline: 1.2.4 @@ -12659,7 +12659,7 @@ snapshots: dependencies: brace-expansion: 2.0.2 - minimatch@10.2.1: + minimatch@10.2.2: dependencies: brace-expansion: 5.0.2 @@ -12681,11 +12681,11 @@ snapshots: minipass-collect@2.0.1: dependencies: - minipass: 7.1.2 + minipass: 7.1.3 minipass-fetch@5.0.1: dependencies: - minipass: 7.1.2 + minipass: 7.1.3 minipass-sized: 2.0.0 minizlib: 3.1.0 optionalDependencies: @@ -12702,18 +12702,18 @@ snapshots: minipass-sized@2.0.0: dependencies: - minipass: 7.1.2 + minipass: 7.1.3 optional: true minipass@3.3.6: dependencies: yallist: 4.0.0 - minipass@7.1.2: {} + minipass@7.1.3: {} minizlib@3.1.0: dependencies: - minipass: 7.1.2 + minipass: 7.1.3 mkdirp-classic@0.5.3: optional: true @@ -13162,12 +13162,12 @@ snapshots: path-scurry@1.11.1: dependencies: lru-cache: 10.4.3 - minipass: 7.1.2 + minipass: 7.1.3 - path-scurry@2.0.1: + path-scurry@2.0.2: dependencies: lru-cache: 11.2.5 - minipass: 7.1.2 + minipass: 7.1.3 path-to-regexp@8.3.0: {} @@ -13516,7 +13516,7 @@ snapshots: rimraf@6.1.3: dependencies: - glob: 13.0.5 + glob: 13.0.6 package-json-from-dist: 1.0.1 roarr@2.15.4: @@ -13836,7 +13836,7 @@ snapshots: ssri@13.0.0: dependencies: - minipass: 7.1.2 + minipass: 7.1.3 stable-hash-x@0.2.0: {} @@ -13991,7 +13991,7 @@ snapshots: dependencies: '@isaacs/fs-minipass': 4.0.1 chownr: 3.0.0 - minipass: 7.1.2 + minipass: 7.1.3 minizlib: 3.1.0 yallist: 5.0.0 diff --git a/readme.md b/readme.md index 48a2e040ee8..fd2bd93b9ef 100644 --- a/readme.md +++ b/readme.md @@ -30,7 +30,7 @@ Supports over [90 different package managers](https://docs.renovatebot.com/modul ### Platforms -Renovate updates code repositories on the following platforms: GitHub, GitLab, Bitbucket, Azure DevOps, AWS Code Commit, Gitea, Forgejo, Gerrit (experimental) +Renovate updates code repositories on the following platforms: GitHub, GitLab, Bitbucket, Azure DevOps, AWS Code Commit (experimental), Gitea, Forgejo, Gerrit (experimental), SCM-Manager (experimental) ## Ways to run Renovate