diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index fd24ae666..fd5219d69 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -1,4 +1,4 @@ -FROM registry.gitlab.com/finestructure/spi-base:1.1.1 +FROM registry.gitlab.com/finestructure/spi-base:1.2.0 # Install SPM build dependencies RUN apt-get update && apt-get install -y curl git make unzip \ diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c996440a4..4595d956f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -29,7 +29,7 @@ jobs: name: Test runs-on: ubuntu-latest container: - image: registry.gitlab.com/finestructure/spi-base:1.1.1 + image: registry.gitlab.com/finestructure/spi-base:1.2.0 options: --privileged services: postgres: @@ -64,7 +64,7 @@ jobs: name: Release build runs-on: ubuntu-latest container: - image: registry.gitlab.com/finestructure/spi-base:1.1.1 + image: registry.gitlab.com/finestructure/spi-base:1.2.0 options: --privileged steps: - name: GH Runner bug workaround @@ -72,8 +72,16 @@ jobs: - name: Checkout code uses: actions/checkout@v4 with: { 'fetch-depth': 0 } - - name: Build release - run: swift build -c release --static-swift-stdlib -Xlinker -ljemalloc + - name: Test release + run: | + # Since we're only pinging the version api endpoint, no database is needed and + # we can just copy the development template to ensure the db env variables are set. + cp .env.development.template .env.development + swift build -c release --static-swift-stdlib -Xlinker -ljemalloc + $(swift build --show-bin-path -c release)/Run serve --port 8080 --hostname 0.0.0.0 & + sleep 10 + echo Probing api/version... + bash -c '[ "$(curl -sL -w "%{http_code}" -o /dev/null http://127.0.0.1:8080/api/version)" == "200" ] || (echo Connection failed ; exit 1)' # test-macos: # name: Test macOS diff --git a/.github/workflows/query-performance.yml b/.github/workflows/query-performance.yml index 3125a3e7f..70b29c357 100644 --- a/.github/workflows/query-performance.yml +++ b/.github/workflows/query-performance.yml @@ -28,7 +28,7 @@ jobs: runs-on: ubuntu-latest continue-on-error: true container: - image: registry.gitlab.com/finestructure/spi-base:1.1.1 + image: registry.gitlab.com/finestructure/spi-base:1.2.0 steps: - name: Checkout code uses: actions/checkout@v4 diff --git a/Dockerfile b/Dockerfile index 839d3939c..c90ff28dc 100644 --- a/Dockerfile +++ b/Dockerfile @@ -18,7 +18,7 @@ # ================================ # Build image # ================================ -FROM registry.gitlab.com/finestructure/spi-base:1.1.1 as build +FROM registry.gitlab.com/finestructure/spi-base:1.2.0 as build # Set up a build area WORKDIR /build @@ -61,7 +61,7 @@ RUN [ -d /build/Resources ] && { mv /build/Resources ./Resources && chmod -R a-w # ================================ # Run image # ================================ -FROM registry.gitlab.com/finestructure/spi-base:1.1.1 +FROM registry.gitlab.com/finestructure/spi-base:1.2.0 # NB sas 2022-09-23: We're not using a dedicated `vapor` user to run the executable, because it # makes managing the data in the checkouts volume difficult. See diff --git a/LOCAL_DEVELOPMENT_SETUP.md b/LOCAL_DEVELOPMENT_SETUP.md index 56fdbcc69..be3a562f2 100644 --- a/LOCAL_DEVELOPMENT_SETUP.md +++ b/LOCAL_DEVELOPMENT_SETUP.md @@ -236,7 +236,7 @@ The trickiest part of this is to ensure the test or app container can connect to So, in order to run the tests in a Linux container run: ``` -docker run --rm -v "$PWD":/host -w /host --add-host=host.docker.internal:host-gateway registry.gitlab.com/finestructure/spi-base:1.1.1 swift test +docker run --rm -v "$PWD":/host -w /host --add-host=host.docker.internal:host-gateway registry.gitlab.com/finestructure/spi-base:1.2.0 swift test ``` Make sure you use the most recent `spi-base` image. You can find the latest image name in the `test-docker` target, which also provides a convenient way to run all all tests in a docker container. diff --git a/Makefile b/Makefile index af61190a3..692dfdea9 100644 --- a/Makefile +++ b/Makefile @@ -68,7 +68,7 @@ test-docker: @# run tests inside a docker container docker run --rm -v "$(PWD)":/host -w /host \ --add-host=host.docker.internal:host-gateway \ - registry.gitlab.com/finestructure/spi-base:1.1.1 \ + registry.gitlab.com/finestructure/spi-base:1.2.0 \ make test test-e2e: db-reset reconcile ingest analyze diff --git a/Package.resolved b/Package.resolved index 35cf56e96..4500acf01 100644 --- a/Package.resolved +++ b/Package.resolved @@ -1,13 +1,13 @@ { - "originHash" : "0b2db4c57b28a53a06ca50890187dbcb7d0638c3f544b81849d4404b8ac6f698", + "originHash" : "164d58b993750ebc70c5988d01cb84a4e748196f26cab9a6869dcce591a1640e", "pins" : [ { "identity" : "async-http-client", "kind" : "remoteSourceControl", "location" : "https://github.com/swift-server/async-http-client.git", "state" : { - "revision" : "2119f0d9cc1b334e25447fe43d3693c0e60e6234", - "version" : "1.24.0" + "revision" : "333f51104b75d1a5b94cb3b99e4c58a3b442c9f7", + "version" : "1.25.2" } }, { @@ -33,8 +33,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/SwiftPackageIndex/CanonicalPackageURL.git", "state" : { - "revision" : "0652109907f980a4020e158f67d154424e3caa30", - "version" : "0.0.8" + "revision" : "f7dc7d49061fef2d347dabc532fb587fad17cc32", + "version" : "1.0.0" } }, { @@ -60,8 +60,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/SwiftPackageIndex/DependencyResolution.git", "state" : { - "revision" : "9e0373749a7577384c058a507716abc59edcbfb6", - "version" : "1.2.0" + "revision" : "700dba4ca633b541e4693cb81049aa9cb4a00352", + "version" : "1.3.0" } }, { @@ -78,8 +78,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/vapor/fluent-kit.git", "state" : { - "revision" : "614d3ec27cdef50cfb9fc3cfd382b6a4d9578cff", - "version" : "1.49.0" + "revision" : "0f1b8bf3e0d4355e6c77a2c54a59db1c3063c7fd", + "version" : "1.52.0" } }, { @@ -87,8 +87,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/vapor/fluent-postgres-driver.git", "state" : { - "revision" : "fd57101e426d3edf66a32ba63a7d0b8ced4d7499", - "version" : "2.10.0" + "revision" : "095bc5a17ab3363167f4becb270b6f8eb790481c", + "version" : "2.10.1" } }, { @@ -127,6 +127,15 @@ "version" : "4.7.1" } }, + { + "identity" : "packagecollections", + "kind" : "remoteSourceControl", + "location" : "https://github.com/SwiftPackageIndex/PackageCollections.git", + "state" : { + "branch" : "main", + "revision" : "158a19b90e3a8daa051c1e9378a74356b409a54b" + } + }, { "identity" : "plot", "kind" : "remoteSourceControl", @@ -150,8 +159,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/vapor/postgres-nio.git", "state" : { - "revision" : "fd0e415a705c490499f983639b04f491a2ed9d99", - "version" : "1.23.0" + "revision" : "5d817be55cae8b00003b7458944954558302d006", + "version" : "1.25.0" } }, { @@ -258,8 +267,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/vapor/sql-kit.git", "state" : { - "revision" : "e0b35ff07601465dd9f3af19a1c23083acaae3bd", - "version" : "3.32.0" + "revision" : "baf0d8684a43f16cd11ebcc67300c8ab5cb5d078", + "version" : "3.33.0" } }, { @@ -271,24 +280,6 @@ "version" : "1.2.1" } }, - { - "identity" : "swift-argument-parser", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-argument-parser.git", - "state" : { - "revision" : "8f4d2753f0e4778c76d5f05ad16c74f707390531", - "version" : "1.2.3" - } - }, - { - "identity" : "swift-asn1", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-asn1.git", - "state" : { - "revision" : "ae33e5941bb88d88538d0a6b19ca0b01e6c76dcf", - "version" : "1.3.1" - } - }, { "identity" : "swift-async-algorithms", "kind" : "remoteSourceControl", @@ -316,15 +307,6 @@ "version" : "1.6.1" } }, - { - "identity" : "swift-certificates", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-certificates.git", - "state" : { - "revision" : "83640c8097acaec17c9835a083e89678cb0f2b66", - "version" : "1.3.0" - } - }, { "identity" : "swift-clocks", "kind" : "remoteSourceControl", @@ -375,35 +357,35 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/pointfreeco/swift-dependencies", "state" : { - "revision" : "4e6b6a814675daf2c1973514314283448f95f941", - "version" : "1.9.0" + "revision" : "fee6aa29908a75437506ddcbe7434c460605b7e6", + "version" : "1.9.1" } }, { - "identity" : "swift-driver", + "identity" : "swift-distributed-tracing", "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-driver.git", + "location" : "https://github.com/apple/swift-distributed-tracing.git", "state" : { - "branch" : "release/5.10", - "revision" : "b461fd4fc51be8e1f2a3f4a2184b664b8846b46f" + "revision" : "a64a0abc2530f767af15dd88dda7f64d5f1ff9de", + "version" : "1.2.0" } }, { - "identity" : "swift-http-types", + "identity" : "swift-http-structured-headers", "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-http-types", + "location" : "https://github.com/apple/swift-http-structured-headers.git", "state" : { - "revision" : "ef18d829e8b92d731ad27bb81583edd2094d1ce3", - "version" : "1.3.1" + "revision" : "8e769facea6b7d46ea60e5e93635a384fd573480", + "version" : "1.2.1" } }, { - "identity" : "swift-llbuild", + "identity" : "swift-http-types", "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-llbuild.git", + "location" : "https://github.com/apple/swift-http-types.git", "state" : { - "branch" : "release/5.10", - "revision" : "fd7c2e0d9279edd023ced6b0a590f8407f5472f9" + "revision" : "a0a57e949a8903563aba4615869310c0ebf14c03", + "version" : "1.4.0" } }, { @@ -429,8 +411,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio.git", "state" : { - "revision" : "f6230d37089fbb52612caf338dc6671e027817b0", - "version" : "2.76.0" + "revision" : "c51907a839e63ebf0ba2076bba73dd96436bd1b9", + "version" : "2.81.0" } }, { @@ -438,8 +420,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio-extras.git", "state" : { - "revision" : "2e9746cfc57554f70b650b021b6ae4738abef3e6", - "version" : "1.24.1" + "revision" : "00f3f72d2f9942d0e2dc96057ab50a37ced150d4", + "version" : "1.25.0" } }, { @@ -478,15 +460,6 @@ "version" : "1.0.3" } }, - { - "identity" : "swift-package-manager", - "kind" : "remoteSourceControl", - "location" : "https://github.com/swiftlang/swift-package-manager.git", - "state" : { - "branch" : "release/5.10", - "revision" : "b5f8ad931b7a40b81f64765fa08c2751164759b4" - } - }, { "identity" : "swift-parsing", "kind" : "remoteSourceControl", @@ -505,6 +478,15 @@ "version" : "1.0.2" } }, + { + "identity" : "swift-service-context", + "kind" : "remoteSourceControl", + "location" : "https://github.com/apple/swift-service-context.git", + "state" : { + "revision" : "8946c930cae601452149e45d31d8ddfac973c3c7", + "version" : "1.2.0" + } + }, { "identity" : "swift-service-lifecycle", "kind" : "remoteSourceControl", @@ -526,7 +508,7 @@ { "identity" : "swift-syntax", "kind" : "remoteSourceControl", - "location" : "https://github.com/swiftlang/swift-syntax", + "location" : "https://github.com/swiftlang/swift-syntax.git", "state" : { "revision" : "0687f71944021d616d34d922343dcef086855920", "version" : "600.0.1" @@ -537,8 +519,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-system.git", "state" : { - "revision" : "d2ba781702a1d8285419c15ee62fd734a9437ff5", - "version" : "1.3.2" + "revision" : "a34201439c74b53f0fd71ef11741af7e7caf01e1", + "version" : "1.4.2" } }, { @@ -546,8 +528,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-tools-support-core.git", "state" : { - "branch" : "release/5.10", - "revision" : "90bdc2a157ebacc5d3de0c83e085d05d22ca5fa0" + "revision" : "b464fcd8d884e599e3202d9bd1eee29a9e504069", + "version" : "0.7.2" } }, { @@ -564,8 +546,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/vapor/vapor.git", "state" : { - "revision" : "fb619ab6485a88787ef9c78ba70e7415f8ebf981", - "version" : "4.106.4" + "revision" : "87b0edd2633c35de543cb7573efe5fbf456181bc", + "version" : "4.114.1" } }, { diff --git a/Package.swift b/Package.swift index 0f675cba2..c1867bbeb 100644 --- a/Package.swift +++ b/Package.swift @@ -29,13 +29,13 @@ let package = Package( dependencies: [ .package(url: "https://github.com/JohnSundell/Ink.git", from: "0.5.1"), .package(url: "https://github.com/swift-server/swift-prometheus.git", from: "1.0.0"), - .package(url: "https://github.com/SwiftPackageIndex/Plot.git", branch: "main"), - .package(url: "https://github.com/SwiftPackageIndex/CanonicalPackageURL.git", from: "0.0.8"), + .package(url: "https://github.com/SwiftPackageIndex/CanonicalPackageURL.git", from: "1.0.0"), .package(url: "https://github.com/SwiftPackageIndex/DependencyResolution.git", from: "1.1.2"), + .package(url: "https://github.com/SwiftPackageIndex/PackageCollections.git", branch: "main"), + .package(url: "https://github.com/SwiftPackageIndex/Plot.git", branch: "main"), .package(url: "https://github.com/SwiftPackageIndex/SPIManifest.git", from: "1.2.0"), .package(url: "https://github.com/SwiftPackageIndex/SemanticVersion.git", from: "0.3.0"), .package(url: "https://github.com/SwiftPackageIndex/ShellOut.git", from: "3.1.4"), - .package(url: "https://github.com/swiftlang/swift-package-manager.git", branch: "release/5.10"), .package(url: "https://github.com/pointfreeco/swift-custom-dump.git", from: "1.0.0"), .package(url: "https://github.com/pointfreeco/swift-dependencies", from: "1.8.0"), .package(url: "https://github.com/pointfreeco/swift-parsing.git", from: "0.12.0"), @@ -68,13 +68,13 @@ let package = Package( .product(name: "DependencyResolution", package: "DependencyResolution"), .product(name: "Fluent", package: "fluent"), .product(name: "FluentPostgresDriver", package: "fluent-postgres-driver"), + .product(name: "PackageCollections", package: "PackageCollections"), .product(name: "Parsing", package: "swift-parsing"), .product(name: "Redis", package: "redis"), .product(name: "ShellOut", package: "ShellOut"), - .product(name: "SwiftPMDataModel-auto", package: "swift-package-manager"), - .product(name: "SwiftPMPackageCollections", package: "swift-package-manager"), .product(name: "Vapor", package: "vapor"), - .product(name: "SotoCognitoAuthentication", package: "soto-cognito-authentication") + .product(name: "SotoCognitoAuthentication", package: "soto-cognito-authentication"), + .product(name: "JWTKit", package: "jwt-kit") ], swiftSettings: swiftSettings, linkerSettings: [.unsafeFlags(["-Xlinker", "-interposable"], diff --git a/Sources/App/Controllers/API/API+PackageCollectionController.swift b/Sources/App/Controllers/API/API+PackageCollectionController.swift index a52a92147..d5478dad8 100644 --- a/Sources/App/Controllers/API/API+PackageCollectionController.swift +++ b/Sources/App/Controllers/API/API+PackageCollectionController.swift @@ -14,7 +14,6 @@ import Fluent import PackageCollectionsModel -import PackageCollectionsSigning import Vapor @@ -23,14 +22,14 @@ extension API { enum PackageCollectionController { @Sendable - static func generate(req: Request) async throws -> SignedCollection { + static func generate(req: Request) async throws -> PackageCollection { AppMetrics.apiPackageCollectionGetTotal?.inc() let dto = try req.content.decode(PostPackageCollectionDTO.self) switch dto.selection { case let .author(author): - return try await SignedCollection.generate( + return try await PackageCollection.generate( db: req.db, filterBy: .author(author), authorName: dto.authorName ?? "Swift Package Index", @@ -44,7 +43,7 @@ extension API { guard packageURLs.count <= 20 else { throw Abort(.badRequest) } - return try await SignedCollection.generate( + return try await PackageCollection.generate( db: req.db, filterBy: .urls(packageURLs), authorName: dto.authorName ?? "Swift Package Index", @@ -61,7 +60,7 @@ extension API { } -extension PackageCollectionSigning.Model.SignedCollection: Vapor.Content {} +extension PackageCollectionModel.V1.Collection: Vapor.Content {} extension API { diff --git a/Sources/App/Controllers/API/API+PackageController+GetRoute+Model.swift b/Sources/App/Controllers/API/API+PackageController+GetRoute+Model.swift index e0f5d4c53..abce3f6c7 100644 --- a/Sources/App/Controllers/API/API+PackageController+GetRoute+Model.swift +++ b/Sources/App/Controllers/API/API+PackageController+GetRoute+Model.swift @@ -319,7 +319,7 @@ extension API.PackageController.GetRoute.Model { var latest: Version? } - struct BuildInfo: Codable, Equatable { + struct BuildInfo: Codable, Equatable, Sendable { var stable: NamedBuildResults? var beta: NamedBuildResults? var latest: NamedBuildResults? @@ -335,7 +335,7 @@ extension API.PackageController.GetRoute.Model { } } - struct NamedBuildResults: Codable, Equatable { + struct NamedBuildResults: Codable, Equatable, Sendable { var referenceName: String var results: T } diff --git a/Sources/App/Controllers/PackageCollectionController.swift b/Sources/App/Controllers/PackageCollectionController.swift index dabbcf4ee..17a15f8ce 100644 --- a/Sources/App/Controllers/PackageCollectionController.swift +++ b/Sources/App/Controllers/PackageCollectionController.swift @@ -17,7 +17,7 @@ import Vapor enum PackageCollectionController { @Sendable - static func generate(req: Request) async throws -> SignedCollection { + static func generate(req: Request) async throws -> PackageCollection { AppMetrics.packageCollectionGetTotal?.inc() guard let collectionType = getCollectionType(req: req) else { @@ -27,13 +27,13 @@ enum PackageCollectionController { do { switch collectionType { case let .author(owner): - return try await SignedCollection.generate( + return try await PackageCollection.generate( db: req.db, filterBy: .author(owner), authorName: "\(owner) via the Swift Package Index" ) case let .keyword(keyword): - return try await SignedCollection.generate( + return try await PackageCollection.generate( db: req.db, filterBy: .keyword(keyword), authorName: "Swift Package Index", @@ -45,7 +45,7 @@ enum PackageCollectionController { case let .custom(key): let collection = try await CustomCollection.find(on: req.db, key: key) .unwrap(or: Abort(.notFound)) - return try await SignedCollection.generate( + return try await PackageCollection.generate( db: req.db, filterBy: .customCollection(key), authorName: "Swift Package Index", diff --git a/Sources/App/Core/Dependencies/EnvironmentClient.swift b/Sources/App/Core/Dependencies/EnvironmentClient.swift index 46d00bcfe..8e7bcf625 100644 --- a/Sources/App/Core/Dependencies/EnvironmentClient.swift +++ b/Sources/App/Core/Dependencies/EnvironmentClient.swift @@ -100,13 +100,7 @@ extension EnvironmentClient: DependencyKey { .flatMap(Double.init) ?? 1.0 }, - collectionSigningCertificateChain: { - [ - "package_collections.cer", - "AppleWWDRCAG3.cer", - "AppleIncRootCertificate.cer", - ].map { SignedCollection.certsDir.appendingPathComponent($0) } - }, + collectionSigningCertificateChain: { [] }, collectionSigningPrivateKey: { Environment.get("COLLECTION_SIGNING_PRIVATE_KEY").map { Data($0.utf8) } }, diff --git a/Sources/App/Core/PackageCollection+signing.swift b/Sources/App/Core/PackageCollection+signing.swift deleted file mode 100644 index d52aadfaf..000000000 --- a/Sources/App/Core/PackageCollection+signing.swift +++ /dev/null @@ -1,84 +0,0 @@ -// Copyright Dave Verwer, Sven A. Schmidt, and other contributors. -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -import Basics -import Dependencies -import Fluent -import Vapor -@preconcurrency import PackageCollectionsModel -@preconcurrency import PackageCollectionsSigning - - -typealias SignedCollection = PackageCollectionSigning.Model.SignedCollection - - -extension SignedCollection { - - static func generate(db: Database, - filterBy filter: PackageCollection.Filter, - authorName: String? = nil, - collectionName: String? = nil, - keywords: [String]? = nil, - overview: String? = nil, - revision: Int? = nil, - limit maxResults: Int? = nil) async throws -> SignedCollection { - let collection = try await PackageCollection.generate(db: db, - filterBy: filter, - authorName: authorName, - collectionName: collectionName, - keywords: keywords, - overview: overview, - revision: revision, - limit: maxResults) - return try await sign(collection: collection) - } - - static func sign(collection: PackageCollection) async throws -> SignedCollection { - @Dependency(\.environment) var environment - - guard let privateKey = environment.collectionSigningPrivateKey() else { - throw AppError.envVariableNotSet("COLLECTION_SIGNING_PRIVATE_KEY") - } - - return try await signer.sign(collection: collection, - certChainPaths: environment.collectionSigningCertificateChain(), - privateKeyPEM: privateKey) - } - - static func validate(signedCollection: SignedCollection) async throws -> Bool { - try await signer.validate(signedCollection: signedCollection) - return true - } - - static var certsDir: URL { - @Dependency(\.fileManager) var fileManager - return URL(fileURLWithPath: fileManager.workingDirectory()) - .appendingPathComponent("Resources") - .appendingPathComponent("Certs") - } - - static let signer = PackageCollectionSigning( - trustedRootCertsDir: certsDir, - additionalTrustedRootCerts: nil, - observabilityScope: .ignored - ) - -} - - -private extension ObservabilityScope { - static var ignored: ObservabilityScope { - ObservabilitySystem { _, _ in }.topScope - } -} diff --git a/Sources/App/Views/PackageController/PackageShow.swift b/Sources/App/Views/PackageController/PackageShow.swift index f8eebed25..5520178a2 100644 --- a/Sources/App/Views/PackageController/PackageShow.swift +++ b/Sources/App/Views/PackageController/PackageShow.swift @@ -36,7 +36,7 @@ enum PackageShow { } } - struct BuildStatusRow: Codable, Equatable { + struct BuildStatusRow: Codable, Equatable, Sendable { var references: [Reference] var results: T diff --git a/Tests/AppTests/AllTests.swift b/Tests/AppTests/AllTests.swift index a7f4c67d0..ca659459e 100644 --- a/Tests/AppTests/AllTests.swift +++ b/Tests/AppTests/AllTests.swift @@ -15,12 +15,14 @@ @testable import App import Dependencies +import SnapshotTesting import Testing @Suite( .dependency(\.date.now, .t0), - .dependency(\.metricsSystem, .mock) + .dependency(\.metricsSystem, .mock), + .snapshots(record: .failed) ) struct AllTests { } diff --git a/Tests/AppTests/ApiTests.swift b/Tests/AppTests/ApiTests.swift index 906232bc0..71f789075 100644 --- a/Tests/AppTests/ApiTests.swift +++ b/Tests/AppTests/ApiTests.swift @@ -15,7 +15,6 @@ @testable import App import Dependencies -import PackageCollectionsSigning import SnapshotTesting import Testing import XCTVapor @@ -845,76 +844,76 @@ extension AllTests.ApiTests { } } - @Test(.disabled(if: !isRunningInCI() && EnvironmentClient.liveValue.collectionSigningPrivateKey() == nil, - "Skip test for local user due to unset COLLECTION_SIGNING_PRIVATE_KEY env variable")) - func package_collections_owner() async throws { - let event = App.ActorIsolated(nil) - try await withDependencies { - $0.date.now = .t0 - $0.environment.apiSigningKey = { "secret" } - $0.environment.collectionSigningCertificateChain = EnvironmentClient.liveValue.collectionSigningCertificateChain - $0.environment.collectionSigningPrivateKey = EnvironmentClient.liveValue.collectionSigningPrivateKey - $0.httpClient.postPlausibleEvent = { @Sendable kind, path, _ in - await event.setValue(.init(kind: kind, path: path)) - } - } operation: { - try await withApp { app in - // setup - let p1 = Package(id: .id1, url: "1") - try await p1.save(on: app.db) - try await Repository(package: p1, - defaultBranch: "main", - name: "name 1", - owner: "foo", - summary: "foo bar package").save(on: app.db) - let v = try Version(package: p1, - latest: .release, - packageName: "Foo", - reference: .tag(1, 2, 3), - toolsVersion: "5.0") - try await v.save(on: app.db) - try await Product(version: v, type: .library(.automatic), name: "lib") - .save(on: app.db) - - do { // MUT - let body: ByteBuffer = .init(string: """ - { - "revision": 3, - "authorName": "author", - "keywords": [ - "a", - "b" - ], - "selection": { - "author": { - "_0": "foo" - } - }, - "collectionName": "my collection", - "overview": "my overview" - } - """) - - try await app.test(.POST, "api/package-collections", - headers: .bearerApplicationJSON(try .apiToken(secretKey: "secret", tier: .tier3)), - body: body, - afterResponse: { res async throws in - // validation - #expect(res.status == .ok) - let container = try res.content.decode(SignedCollection.self) - #expect(!container.signature.signature.isEmpty) - // more details are tested in PackageCollectionTests - #expect(container.collection.name == "my collection") - }) - } - - // ensure API event has been reported - await event.withValue { - #expect($0 == .some(.init(kind: .pageview, path: .packageCollections))) - } - } - } - } +// @Test(.disabled(if: !isRunningInCI() && EnvironmentClient.liveValue.collectionSigningPrivateKey() == nil, +// "Skip test for local user due to unset COLLECTION_SIGNING_PRIVATE_KEY env variable")) +// func package_collections_owner() async throws { +// let event = App.ActorIsolated(nil) +// try await withDependencies { +// $0.date.now = .t0 +// $0.environment.apiSigningKey = { "secret" } +// $0.environment.collectionSigningCertificateChain = EnvironmentClient.liveValue.collectionSigningCertificateChain +// $0.environment.collectionSigningPrivateKey = EnvironmentClient.liveValue.collectionSigningPrivateKey +// $0.httpClient.postPlausibleEvent = { @Sendable kind, path, _ in +// await event.setValue(.init(kind: kind, path: path)) +// } +// } operation: { +// try await withApp { app in +// // setup +// let p1 = Package(id: .id1, url: "1") +// try await p1.save(on: app.db) +// try await Repository(package: p1, +// defaultBranch: "main", +// name: "name 1", +// owner: "foo", +// summary: "foo bar package").save(on: app.db) +// let v = try Version(package: p1, +// latest: .release, +// packageName: "Foo", +// reference: .tag(1, 2, 3), +// toolsVersion: "5.0") +// try await v.save(on: app.db) +// try await Product(version: v, type: .library(.automatic), name: "lib") +// .save(on: app.db) +// +// do { // MUT +// let body: ByteBuffer = .init(string: """ +// { +// "revision": 3, +// "authorName": "author", +// "keywords": [ +// "a", +// "b" +// ], +// "selection": { +// "author": { +// "_0": "foo" +// } +// }, +// "collectionName": "my collection", +// "overview": "my overview" +// } +// """) +// +// try await app.test(.POST, "api/package-collections", +// headers: .bearerApplicationJSON(try .apiToken(secretKey: "secret", tier: .tier3)), +// body: body, +// afterResponse: { res async throws in +// // validation +// #expect(res.status == .ok) +// let container = try res.content.decode(SignedCollection.self) +// #expect(!container.signature.signature.isEmpty) +// // more details are tested in PackageCollectionTests +// #expect(container.collection.name == "my collection") +// }) +// } +// +// // ensure API event has been reported +// await event.withValue { +// #expect($0 == .some(.init(kind: .pageview, path: .packageCollections))) +// } +// } +// } +// } @Test(.disabled(if: !isRunningInCI() && EnvironmentClient.liveValue.collectionSigningPrivateKey() == nil, "Skip test for local user due to unset COLLECTION_SIGNING_PRIVATE_KEY env variable")) diff --git a/Tests/AppTests/PackageCollectionTests.swift b/Tests/AppTests/PackageCollectionTests.swift index 03ef768d6..3d2f66036 100644 --- a/Tests/AppTests/PackageCollectionTests.swift +++ b/Tests/AppTests/PackageCollectionTests.swift @@ -14,9 +14,7 @@ @testable import App -import Basics import Dependencies -import PackageCollectionsSigning import SnapshotTesting import Testing import Vapor @@ -831,71 +829,71 @@ extension AllTests.PackageCollectionTests { } } - @Test( - .disabled( - if: !isRunningInCI() && EnvironmentClient.liveValue.collectionSigningPrivateKey() == nil, - "Skip test for local user due to unset COLLECTION_SIGNING_PRIVATE_KEY env variable" - ) - ) - func sign_collection() async throws { - try await withDependencies { - $0.environment.collectionSigningCertificateChain = EnvironmentClient.liveValue.collectionSigningCertificateChain - $0.environment.collectionSigningPrivateKey = EnvironmentClient.liveValue.collectionSigningPrivateKey - } operation: { - // setup - let collection: PackageCollection = .mock - - // MUT - let signedCollection = try await SignedCollection.sign(collection: collection) - - // validate signed collection content - #expect(!signedCollection.signature.signature.isEmpty) - assertSnapshot(of: signedCollection, as: .json(encoder)) - - // validate signature - let validated = try await SignedCollection.validate(signedCollection: signedCollection) - #expect(validated) - } - } - - @Test( - .disabled("Skipping until issue is resolved"), - .bug("https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/1583#issuecomment-1066592057") - ) - func sign_collection_revoked_key() async throws { - // setup - let collection: PackageCollection = .mock - // get cert and key and make sure the inputs are valid (apart from being revoked) - // so we don't fail for that reason - let revokedUrl = fixtureUrl(for: "revoked.cer") - #expect(Foundation.FileManager.default.fileExists(atPath: revokedUrl.path)) - let revokedKey = try fixtureData(for: "revoked.pem") - - await withDependencies { - $0.environment.collectionSigningCertificateChain = { - [ - revokedUrl, - SignedCollection.certsDir.appendingPathComponent("AppleWWDRCAG3.cer"), - SignedCollection.certsDir.appendingPathComponent("AppleIncRootCertificate.cer") - ] - } - $0.environment.collectionSigningPrivateKey = { revokedKey } - } operation: { - do { - // MUT - let signedCollection = try await SignedCollection.sign(collection: collection) - // NB: signing _can_ succeed in case of reachability issues to verify the cert - // in this case we need to check the signature - // https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/1583#issuecomment-1048408400 - let validated = try await SignedCollection.validate(signedCollection: signedCollection) - #expect(!validated) - } catch PackageCollectionSigningError.invalidCertChain { - // ok - } catch { - Issue.record("unexpected signing error: \(error)") - } - } - } +// @Test( +// .disabled( +// if: !isRunningInCI() && EnvironmentClient.liveValue.collectionSigningPrivateKey() == nil, +// "Skip test for local user due to unset COLLECTION_SIGNING_PRIVATE_KEY env variable" +// ) +// ) +// func sign_collection() async throws { +// try await withDependencies { +// $0.environment.collectionSigningCertificateChain = EnvironmentClient.liveValue.collectionSigningCertificateChain +// $0.environment.collectionSigningPrivateKey = EnvironmentClient.liveValue.collectionSigningPrivateKey +// } operation: { +// // setup +// let collection: PackageCollection = .mock +// +// // MUT +// let signedCollection = try await SignedCollection.sign(collection: collection) +// +// // validate signed collection content +// #expect(!signedCollection.signature.signature.isEmpty) +// assertSnapshot(of: signedCollection, as: .json(encoder)) +// +// // validate signature +// let validated = try await SignedCollection.validate(signedCollection: signedCollection) +// #expect(validated) +// } +// } + +// @Test( +// .disabled("Skipping until issue is resolved"), +// .bug("https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/1583#issuecomment-1066592057") +// ) +// func sign_collection_revoked_key() async throws { +// // setup +// let collection: PackageCollection = .mock +// // get cert and key and make sure the inputs are valid (apart from being revoked) +// // so we don't fail for that reason +// let revokedUrl = fixtureUrl(for: "revoked.cer") +// #expect(Foundation.FileManager.default.fileExists(atPath: revokedUrl.path)) +// let revokedKey = try fixtureData(for: "revoked.pem") +// +// await withDependencies { +// $0.environment.collectionSigningCertificateChain = { +// [ +// revokedUrl, +// SignedCollection.certsDir.appendingPathComponent("AppleWWDRCAG3.cer"), +// SignedCollection.certsDir.appendingPathComponent("AppleIncRootCertificate.cer") +// ] +// } +// $0.environment.collectionSigningPrivateKey = { revokedKey } +// } operation: { +// do { +// // MUT +// let signedCollection = try await SignedCollection.sign(collection: collection) +// // NB: signing _can_ succeed in case of reachability issues to verify the cert +// // in this case we need to check the signature +// // https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/issues/1583#issuecomment-1048408400 +// let validated = try await SignedCollection.validate(signedCollection: signedCollection) +// #expect(!validated) +// } catch PackageCollectionSigningError.invalidCertChain { +// // ok +// } catch { +// Issue.record("unexpected signing error: \(error)") +// } +// } +// } } diff --git a/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.linux.json b/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.linux.json index c5f316a6c..d2e2c2a86 100644 --- a/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.linux.json +++ b/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.linux.json @@ -21,7 +21,12 @@ "upperBound" : "510.0.0" } ] - } + }, + "traits" : [ + { + "name" : "default" + } + ] } ] } @@ -204,5 +209,8 @@ ], "toolsVersion" : { "_version" : "5.9.0" - } + }, + "traits" : [ + + ] } \ No newline at end of file diff --git a/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.macos.json b/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.macos.json index 783d7af20..9ac50bfc0 100644 --- a/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.macos.json +++ b/Tests/AppTests/__Snapshots__/AnalyzerTests/dumpPackage_format.macos.json @@ -21,7 +21,12 @@ "upperBound" : "510.0.0" } ] - } + }, + "traits" : [ + { + "name" : "default" + } + ] } ] } @@ -210,5 +215,8 @@ ], "toolsVersion" : { "_version" : "5.9.0" - } + }, + "traits" : [ + + ] } \ No newline at end of file