Support recursive types (#330)

Support recursive types

### Motivation

Fixes #70.

### Modifications

To start, read the new docc [article](https://github.com/apple/swift-openapi-generator/blob/ceb51fa1a4f1f858590b22da75162c4bf999719b/Sources/swift-openapi-generator/Documentation.docc/Development/Supporting-recursive-types.md) about how reference types are implemented, then most of the PR should make sense.

As suggested by @simonjbeaumont, boxing of recursive types happens on the Swift representation, as opposed to my original approach, which tried to do this early in the translation layer. This massively simplified the problem and definitely seems like the better way to do it.

Highlights:
- In `validateDoc`, removed the dereferencing, which we previously used to catch cycles early and emit descriptive errors.
- Introduced an efficient stack type caller `ReferenceStack` that makes checking if an item is present in the stack fast, on top of being a stack (represented as an array).
- Helper methods like `isSchemaSupported` and `isKeyValuePair` gained an inout parameter of the stack, to allow it to break infinite recursion.
- The actual algorithm for walking the graph, detecting cycles, and deciding which types to box is implemented in `RecursionDetector`, which is a generic algorithm on top of nodes with edges.
- Then `DeclarationRecursionDetector` provides concrete types that glue it with our structured Swift representation's `Declaration`.
- The algorithm runs in `translateSchemas` where we're iterating over the items in `#/components/schemas`, as those are the only ones that can produce a cycle (as schemas in other parts of the document can refer to items in `#/components/schemas`, but not the other way around: items in `#/components/schemas` cannot refer to schemas outside of it.)

### Result

OpenAPI documents with recursive schemas are now supported.

### Test Plan

- Added unit tests for the recursion detector.
- Adapted other tests, of `isSchemaSupported` and `isKeyValuePair`.
- Added examples to `petstore.yaml`, as this one introduces quite a lot of new code that we want to make sure compiles without warnings.
- Also added examples to snippet tests, to allow us to expand those later with edge cases we haven't thought about yet.



Reviewed by: dnadoba

Builds:
     ✔︎ pull request validation (5.10) - Build finished. 
     ✔︎ pull request validation (5.8) - Build finished. 
     ✔︎ pull request validation (5.9) - Build finished. 
     ✔︎ pull request validation (compatibility test) - Build finished. 
     ✔︎ pull request validation (docc test) - Build finished. 
     ✔︎ pull request validation (integration test) - Build finished. 
     ✔︎ pull request validation (nightly) - Build finished. 
     ✔︎ pull request validation (soundness) - Build finished. 

#330

Author: czechboy0

Package.swift

@@ -89,7 +89,7 @@ let package = Package(
         // Tests-only: Runtime library linked by generated code, and also
         // helps keep the runtime library new enough to work with the generated
         // code.
-        .package(url: "https://github.com/apple/swift-openapi-runtime", .upToNextMinor(from: "0.3.2")),
+        .package(url: "https://github.com/apple/swift-openapi-runtime", .upToNextMinor(from: "0.3.3")),
 
         // Build and preview docs
         .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),

Sources/_OpenAPIGeneratorCore/Parser/validateDoc.swift

@@ -44,29 +44,5 @@ func validateDoc(_ doc: ParsedOpenAPIRepresentation, config: Config) throws -> [
             ]
         )
     }
-
-    // Validate that the document is dereferenceable, which
-    // catches reference cycles, which we don't yet support.
-    _ = try doc.locallyDereferenced()
-
-    // Also explicitly dereference the parts of components
-    // that the generator uses. `locallyDereferenced()` above
-    // only dereferences paths/operations, but not components.
-    let components = doc.components
-    try components.schemas.forEach { schema in
-        _ = try schema.value.dereferenced(in: components)
-    }
-    try components.parameters.forEach { schema in
-        _ = try schema.value.dereferenced(in: components)
-    }
-    try components.headers.forEach { schema in
-        _ = try schema.value.dereferenced(in: components)
-    }
-    try components.requestBodies.forEach { schema in
-        _ = try schema.value.dereferenced(in: components)
-    }
-    try components.responses.forEach { schema in
-        _ = try schema.value.dereferenced(in: components)
-    }
     return diagnostics
 }

Sources/_OpenAPIGeneratorCore/Translator/CommonTranslations/translateAllAnyOneOf.swift

@@ -85,8 +85,10 @@ extension FileTranslator {
                     associatedDeclarations: associatedDeclarations,
                     asSwiftSafeName: swiftSafeName
                 )
+                var referenceStack = ReferenceStack.empty
                 let isKeyValuePairSchema = try TypeMatcher.isKeyValuePair(
                     schema,
+                    referenceStack: &referenceStack,
                     components: components
                 )
                 return (blueprint, isKeyValuePairSchema)
@@ -196,8 +198,10 @@ extension FileTranslator {
                     } else {
                         associatedDeclarations = []
                     }
+                    var referenceStack = ReferenceStack.empty
                     let isKeyValuePair = try TypeMatcher.isKeyValuePair(
                         schema,
+                        referenceStack: &referenceStack,
                         components: components
                     )
                     return (caseName, nil, isKeyValuePair, comment, childType, associatedDeclarations)

Sources/_OpenAPIGeneratorCore/Translator/CommonTypes/Constants.swift

@@ -312,6 +312,12 @@ enum Constants {
 
             /// The name of the namespace.
             static let namespace: String = "Schemas"
+
+            /// The full namespace components.
+            static let components: [String] = [
+                Constants.Components.namespace,
+                Constants.Components.Schemas.namespace,
+            ]
         }
 
         /// Constants related to the Parameters namespace.

Sources/_OpenAPIGeneratorCore/Translator/Recursion/DeclarationRecursionDetector.swift

@@ -0,0 +1,204 @@
+//===----------------------------------------------------------------------===//
+//
+// This source file is part of the SwiftOpenAPIGenerator open source project
+//
+// Copyright (c) 2023 Apple Inc. and the SwiftOpenAPIGenerator project authors
+// Licensed under Apache License v2.0
+//
+// See LICENSE.txt for license information
+// See CONTRIBUTORS.txt for the list of SwiftOpenAPIGenerator project authors
+//
+// SPDX-License-Identifier: Apache-2.0
+//
+//===----------------------------------------------------------------------===//
+
+/// A set of specialized types for using the recursion detector for
+/// declarations.
+struct DeclarationRecursionDetector {
+
+    /// A node for a pair of a Swift type name and a corresponding declaration.
+    struct Node: TypeNode, Equatable {
+
+        /// The type of the name is a string.
+        typealias NameType = String
+
+        /// The name of the node.
+        var name: NameType
+
+        /// Whether the type can be boxed.
+        var isBoxable: Bool
+
+        /// The names of nodes pointed to by this node.
+        var edges: [NameType]
+
+        /// The declaration represented by this node.
+        var decl: Declaration
+
+        /// Creates a new node.
+        /// - Parameters:
+        ///   - name: The name of the node.
+        ///   - isBoxable: Whether the type can be boxed.
+        ///   - edges: The names of nodes pointed to by this node.
+        ///   - decl: The declaration represented by this node.
+        private init(name: NameType, isBoxable: Bool, edges: [NameType], decl: Declaration) {
+            self.name = name
+            self.isBoxable = isBoxable
+            self.edges = edges
+            self.decl = decl
+        }
+
+        /// Creates a new node from the provided declaration.
+        ///
+        /// Returns nil when the declaration is missing a name.
+        /// - Parameter decl: A declaration.
+        init?(_ decl: Declaration) {
+            guard let name = decl.name else {
+                return nil
+            }
+            let edges = decl.schemaComponentNamesOfUnbreakableReferences
+            self.init(
+                name: name,
+                isBoxable: decl.isBoxable,
+                edges: edges,
+                decl: decl
+            )
+        }
+    }
+
+    /// A container for declarations.
+    struct Container: TypeNodeContainer {
+
+        /// The type of the node.
+        typealias Node = DeclarationRecursionDetector.Node
+
+        /// An error thrown by the container.
+        enum ContainerError: Swift.Error {
+
+            /// The node for the provided name was not found.
+            case nodeNotFound(Node.NameType)
+        }
+
+        /// The lookup map from the name to the node.
+        var lookupMap: [String: Node]
+
+        func lookup(_ name: String) throws -> DeclarationRecursionDetector.Node {
+            guard let node = lookupMap[name] else {
+                throw ContainerError.nodeNotFound(name)
+            }
+            return node
+        }
+    }
+}
+
+extension Declaration {
+
+    /// A name of the declaration, if it has one.
+    var name: String? {
+        switch self {
+        case .struct(let desc):
+            return desc.name
+        case .enum(let desc):
+            return desc.name
+        case .typealias(let desc):
+            return desc.name
+        case .commentable(_, let decl), .deprecated(_, let decl):
+            return decl.name
+        case .variable, .extension, .protocol, .function, .enumCase:
+            return nil
+        }
+    }
+
+    /// A Boolean value representing whether this declaration can be boxed.
+    var isBoxable: Bool {
+        switch self {
+        case .struct, .enum:
+            return true
+        case .commentable(_, let decl), .deprecated(_, let decl):
+            return decl.isBoxable
+        case .typealias, .variable, .extension, .protocol, .function, .enumCase:
+            return false
+        }
+    }
+
+    /// An array of names that can be found in `#/components/schemas` in
+    /// the OpenAPI document that represent references that can cause
+    /// a reference cycle.
+    var schemaComponentNamesOfUnbreakableReferences: [String] {
+        switch self {
+        case .struct(let desc):
+            return desc
+                .members
+                .compactMap { (member) -> [String]? in
+                    switch member.strippingTopComment {
+                    case .variable,  // A reference to a reusable type.
+                        .struct, .enum:  // An inline type.
+                        return member.schemaComponentNamesOfUnbreakableReferences
+                    default:
+                        return nil
+                    }
+                }
+                .flatMap { $0 }
+        case .enum(let desc):
+            return desc
+                .members
+                .compactMap { (member) -> [String]? in
+                    guard case .enumCase = member.strippingTopComment else {
+                        return nil
+                    }
+                    return member
+                        .schemaComponentNamesOfUnbreakableReferences
+                }
+                .flatMap { $0 }
+        case .commentable(_, let decl), .deprecated(_, let decl):
+            return decl
+                .schemaComponentNamesOfUnbreakableReferences
+        case .typealias(let desc):
+            return desc
+                .existingType
+                .referencedSchemaComponentName
+                .map { [$0] } ?? []
+        case .variable(let desc):
+            return desc.type?.referencedSchemaComponentName.map { [$0] } ?? []
+        case .enumCase(let desc):
+            switch desc.kind {
+            case .nameWithAssociatedValues(let values):
+                return values.compactMap { $0.type.referencedSchemaComponentName }
+            default:
+                return []
+            }
+        case .extension, .protocol, .function:
+            return []
+        }
+    }
+}
+
+fileprivate extension Array where Element == String {
+
+    /// The name in the `Components.Schemas.` namespace.
+    var nameIfTopLevelSchemaComponent: String? {
+        let components = self
+        guard
+            components.count == 3,
+            components.starts(with: Constants.Components.Schemas.components)
+        else {
+            return nil
+        }
+        return components[2]
+    }
+}
+
+extension ExistingTypeDescription {
+
+    /// The name in the `Components.Schemas.` namespace, if the type can appear
+    /// there. Nil otherwise.
+    var referencedSchemaComponentName: String? {
+        switch self {
+        case .member(let components):
+            return components.nameIfTopLevelSchemaComponent
+        case .array(let desc), .dictionaryValue(let desc), .any(let desc), .optional(let desc):
+            return desc.referencedSchemaComponentName
+        case .generic:
+            return nil
+        }
+    }
+}