Allow custom scalars in schema (aws#97)

If a given input schema contained a custom scalar, the utility would generate an invalid schema and any field which has a custom scalar type will not be able to be used as input for a query or mutation.

Changes include:
-fixed `schemaModelValidator.js` to no longer interpret custom scalars as edges and also allow fields of custom scalar types to be included as part of input types
-fixed `schemaModelValidator.js` to add the `Options` parameter to generated queries that retrieve an array of objects instead of the queries that retrieve single objects
-added documentation for how to use custom scalars with Apollo Server and scalar restrictions with App Sync

Author: andreachild

CHANGELOG.md

@@ -20,28 +20,31 @@ This release contains new support for Apollo Server integration.
 
 ### Bug Fixes
 
-* Don't cast integers to floats in Neptune schema (#62)
-* Fix query from AppSync with an empty filter object (#61)
-* Retain numeric parameter value type when creating open cypher query (#63)
-* Fixed bug with ID argument type conversion and added Apollo arguments to help menu (#74)
-* Upgraded axios and babel versions to fix security warnings (#90)
-* Fixed failing integration test by excluding `node_modules` from Apollo zip (#94)
-* Fixed enum types in schema to be included in input types (#95)
-* Fixed bug where id fields without @id directives are not accounted for (#96)
+* Don't cast integers to floats in Neptune schema ([#62](https://github.com/aws/amazon-neptune-for-graphql/pull/62))
+* Fix query from AppSync with an empty filter object ([#61](https://github.com/aws/amazon-neptune-for-graphql/pull/61))
+* Retain numeric parameter value type when creating open cypher query ([#63](https://github.com/aws/amazon-neptune-for-graphql/pull/63))
+* Fixed bug with ID argument type conversion and added Apollo arguments to help menu ([#74](https://github.com/aws/amazon-neptune-for-graphql/pull/74))
+* Upgraded axios and babel versions to fix security warnings ([#90](https://github.com/aws/amazon-neptune-for-graphql/pull/90))
+* Fixed failing integration test by excluding `node_modules` from Apollo zip ([#94](https://github.com/aws/amazon-neptune-for-graphql/pull/94))
+* Fixed enum types in schema to be included in input types ([#95](https://github.com/aws/amazon-neptune-for-graphql/pull/95))
+* Fixed bug where id fields without @id directives are not accounted for ([#96](https://github.com/aws/amazon-neptune-for-graphql/pull/96))
+* Fixed custom scalar types in schema to be included in input types ([#97](https://github.com/aws/amazon-neptune-for-graphql/pull/97))
+* Fixed queries generated from an input schema which retrieve an array to have an option parameter with limit ([#97](https://github.com/aws/amazon-neptune-for-graphql/pull/97))
+
 
 ### Features
 
-* Support output of zip package of Apollo Server artifacts (#70, #72, #73, #75, #76)
+* Support output of zip package of Apollo Server artifacts (([#70](https://github.com/aws/amazon-neptune-for-graphql/pull/70)), ([#72](https://github.com/aws/amazon-neptune-for-graphql/pull/72)), ([#73](https://github.com/aws/amazon-neptune-for-graphql/pull/73)), ([#75](https://github.com/aws/amazon-neptune-for-graphql/pull/75)), ([#76](https://github.com/aws/amazon-neptune-for-graphql/pull/76)))
 
 ### Improvements
 
-* Increased graphdb.js test coverage using sample data (#53)
-* Saved the neptune schema to file early so that it can be used for troubleshooting (#56)
-* Alias edges with same label as a node (#57)
-* Cap concurrent requests to get Neptune schema (#58)
-* Honour @id directive on type fields (#60)
-* Changed lambda template to use ECMAScripts modules (#68)
-* Add template file missing from packaging (#71)
-* Separated graphQL schema from resolver template (#79)
-* Added unit tests for resolver and moved resolver integration tests to be unit tests (#83)
-* Set limit on the expensive query which is retrieving distinct to and from labels for edges (#89)
+* Increased graphdb.js test coverage using sample data ([#53](https://github.com/aws/amazon-neptune-for-graphql/pull/53))
+* Saved the neptune schema to file early so that it can be used for troubleshooting ([#56](https://github.com/aws/amazon-neptune-for-graphql/pull/56))
+* Alias edges with same label as a node ([#57](https://github.com/aws/amazon-neptune-for-graphql/pull/57))
+* Cap concurrent requests to get Neptune schema ([#58](https://github.com/aws/amazon-neptune-for-graphql/pull/58))
+* Honour @id directive on type fields ([#60](https://github.com/aws/amazon-neptune-for-graphql/pull/60))
+* Changed lambda template to use ECMAScripts modules ([#68](https://github.com/aws/amazon-neptune-for-graphql/pull/68))
+* Add template file missing from packaging ([#71](https://github.com/aws/amazon-neptune-for-graphql/pull/71))
+* Separated graphQL schema from resolver template ([#79](https://github.com/aws/amazon-neptune-for-graphql/pull/79))
+* Added unit tests for resolver and moved resolver integration tests to be unit tests ([#83](https://github.com/aws/amazon-neptune-for-graphql/pull/83))
+* Set limit on the expensive query which is retrieving distinct to and from labels for edges ([#89](https://github.com/aws/amazon-neptune-for-graphql/pull/89))

README.md

@@ -302,12 +302,17 @@ The example commands above will generate the file `apollo-server-<identifier>-<t
 > [!NOTE] 
 > Node's default AWS credentials provider is used for authentication with Neptune. See [AWS SDK credential providers](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-credential-providers/#fromnodeproviderchain) for more information.
 
+### Using Custom Scalars with Apollo Server
+When using custom scalars in your schema (specified via `--input-schema-file`), you should add corresponding scalar resolvers to the index.mjs file in the generated Apollo Server artifact. Although queries will execute without these resolvers, the absence of the custom scalar resolvers will bypass important data validation for fields using custom scalar types. See [Apollo Custom Scalars](https://www.apollographql.com/docs/apollo-server/schema/custom-scalars) for more details on how to attach custom scalar resolvers with Apollo Server.
+
 # Known limitations
 - @graphQuery using Gremlin works only if the query returns a scalar value, one elementMap(), or list as elementMap().fold(), this feature is under development.
 - Neptune RDF database and SPARQL language is not supported.
 - Querying Neptune via SDK is not yet supported for Apollo Server, only HTTPS is supported.
 - Mutations are not yet supported for Apollo Server
-<br>
+- Schemas specified by `--input-schema-file` with `--create-update-aws-pipeline` may not contain custom scalars. See [AWS App Sync Scalar types in GraphQL](https://docs.aws.amazon.com/appsync/latest/devguide/scalars.html) for more information.
+- Schemas specified by `--input-schema-file` with `--create-update-apollo-server` or `--create-update-apollo-server-subgraph` which contain custom scalars require manual steps to add custom scalar resolvers for additional query validation.
+  <br>
 
 # Roadmap
 - Gremlin resolver.

doc/todoExample.md

@@ -73,10 +73,10 @@ input Options {
 }
 
 type Query {
-  getNodeTodo(filter: TodoInput, options: Options): Todo
-  getNodeTodos(filter: TodoInput): [Todo]
-  getNodeComment(filter: CommentInput, options: Options): Comment
-  getNodeComments(filter: CommentInput): [Comment]
+  getNodeTodo(filter: TodoInput): Todo
+  getNodeTodos(filter: TodoInput, options: Options): [Todo]
+  getNodeComment(filter: CommentInput): Comment
+  getNodeComments(filter: CommentInput, options: Options): [Comment]
 }
 
 type Mutation {

src/schemaModelValidator.js

@@ -22,6 +22,7 @@ const typesToAdd = [];
 const queriesToAdd = [];
 const mutationsToAdd = [];
 const enumTypes = [];
+const customScalarTypes = [];
 
 function lowercaseFirstCharacter(inputString) {
     if (inputString.length === 0) {     
@@ -145,8 +146,8 @@ function addNode(def) {
     typesToAdd.push(`input ${name}Input {\n${print(getInputFields(def))}\n}`);    
 
     // Create query
-    queriesToAdd.push(`getNode${name}(filter: ${name}Input, options: Options): ${name}\n`);
-    queriesToAdd.push(`getNode${name}s(filter: ${name}Input): [${name}]\n`);
+    queriesToAdd.push(`getNode${name}(filter: ${name}Input): ${name}\n`);
+    queriesToAdd.push(`getNode${name}s(filter: ${name}Input, options: Options): [${name}]\n`);
 
     // Create mutation
     mutationsToAdd.push(`createNode${name}(input: ${name}Input!): ${name}\n`);
@@ -256,7 +257,7 @@ function nullable(type) {
 
 
 function isScalarOrEnum(type) {
-    const scalarOrEnumTypes = ['String', 'Int', 'Float', 'Boolean', 'ID', ...enumTypes];
+    const scalarOrEnumTypes = ['String', 'Int', 'Float', 'Boolean', 'ID', ...enumTypes, ...customScalarTypes];
     return type.kind === 'NamedType' && scalarOrEnumTypes.includes(type.name.value);
 }
 
@@ -266,11 +267,14 @@ function inferGraphDatabaseDirectives(schemaModel) {
     var currentType = '';
     let referencedType = '';
     let edgeName = '';
-
     schemaModel.definitions
         .filter(definition => definition.kind === 'EnumTypeDefinition')
         .forEach(definition => enumTypes.push(definition.name.value));
 
+    schemaModel.definitions
+        .filter(definition => definition.kind === 'ScalarTypeDefinition')
+        .forEach(definition => customScalarTypes.push(definition.name.value));
+
     schemaModel.definitions.forEach(def => {
         if (def.kind == 'ObjectTypeDefinition') {
             if (!(def.name.value == 'Query' || def.name.value == 'Mutation')) {
@@ -294,6 +298,7 @@ function inferGraphDatabaseDirectives(schemaModel) {
                 // add relationships
                 def.fields.forEach(field => {                    
                     if (field.type.type !== undefined) {
+                        // FIXME handle NonNullType wrapper
                         if (field.type.type.kind === 'NamedType' && field.type.type.name.value !== 'ID')
                         {
                             try {
@@ -316,7 +321,8 @@ function inferGraphDatabaseDirectives(schemaModel) {
                                field.type.name.value !== 'Int' && 
                                field.type.name.value !== 'Float' && 
                                field.type.name.value !== 'Boolean' &&
-                               !enumTypes.includes(field.type.name.value)) {
+                               !enumTypes.includes(field.type.name.value) &&
+                               !customScalarTypes.includes(field.type.name.value)) {
 
                         referencedType = field.type.name.value;
                         edgeName = referencedType + 'Edge';

src/test/schemaModelValidator.test.js

@@ -1,15 +1,15 @@
 import { readFileSync } from 'node:fs';
 import { loggerInit } from '../logger.js';
 import { validatedSchemaModel } from '../schemaModelValidator.js';
-import { schemaParser } from '../schemaParser.js';
+import { schemaParser, schemaStringify } from '../schemaParser.js';
 
 describe('validatedSchemaModel', () => {
     let model;
 
     beforeAll(() => {
         loggerInit('./output', false, 'silent');
 
-        const schema = readFileSync('./src/test/directive-id.graphql');
+        const schema = readFileSync('./src/test/user-group.graphql');
         model = validatedSchemaModel(schemaParser(schema));
     });
 
@@ -28,9 +28,9 @@ describe('validatedSchemaModel', () => {
         const groupType = objTypeDefs.find(def => def.name.value === 'Group');
         const moderatorType = objTypeDefs.find(def => def.name.value === 'Moderator');
 
-        expect(userType.fields).toHaveLength(4);
+        expect(userType.fields).toHaveLength(5);
         expect(groupType.fields).toHaveLength(2);
-        expect(moderatorType.fields).toHaveLength(2);
+        expect(moderatorType.fields).toHaveLength(4);
 
         const userIdFields = getIdFields(userType);
         const groupIdFields = getIdFields(groupType);
@@ -74,6 +74,12 @@ describe('validatedSchemaModel', () => {
         const userRoleField = userInput.fields.find(field => field.name.value === 'role');
         expect(userRoleField.type.name.value).toEqual('Role');
     });
+    
+    test('should output expected validated schema', () => {
+        const actual = schemaStringify(model, true);
+        const expected = readFileSync('./src/test/user-group-validated.graphql', 'utf8')
+        expect(actual).toBe(expected); 
+    });
 
     function getIdFields(objTypeDef) {
         return objTypeDef.fields.filter(

src/test/user-group-validated.graphql

@@ -0,0 +1,82 @@
+type User {
+  userId: ID! @id
+  firstName: String
+  lastName: String
+  role: Role
+  email: EmailAddress
+}
+
+type Group {
+  _id: ID! @id
+  name: String
+}
+
+type Moderator {
+  moderatorId: ID! @id
+  name: String
+  moderates: Group @relationship(type: "GroupEdge", direction: OUT)
+  groupEdge: GroupEdge
+}
+
+enum Role {
+  ADMIN
+  USER
+  GUEST
+}
+
+"https://the-guild.dev/graphql/scalars/docs/scalars/email-address"
+scalar EmailAddress
+
+input UserInput {
+  userId: ID! @id
+  firstName: String
+  lastName: String
+  role: Role
+  email: EmailAddress
+}
+
+input GroupInput {
+  _id: ID! @id
+  name: String
+}
+
+input ModeratorInput {
+  moderatorId: ID! @id
+  name: String
+}
+
+type GroupEdge {
+  _id: ID! @id
+}
+
+input Options {
+  limit: Int
+}
+
+type Query {
+  getNodeUser(filter: UserInput): User
+  getNodeUsers(filter: UserInput, options: Options): [User]
+  getNodeGroup(filter: GroupInput): Group
+  getNodeGroups(filter: GroupInput, options: Options): [Group]
+  getNodeModerator(filter: ModeratorInput): Moderator
+  getNodeModerators(filter: ModeratorInput, options: Options): [Moderator]
+}
+
+type Mutation {
+  createNodeUser(input: UserInput!): User
+  updateNodeUser(input: UserInput!): User
+  deleteNodeUser(userId: ID!): Boolean
+  createNodeGroup(input: GroupInput!): Group
+  updateNodeGroup(input: GroupInput!): Group
+  deleteNodeGroup(_id: ID!): Boolean
+  createNodeModerator(input: ModeratorInput!): Moderator
+  updateNodeModerator(input: ModeratorInput!): Moderator
+  deleteNodeModerator(moderatorId: ID!): Boolean
+  connectNodeModeratorToNodeGroupEdgeGroupEdge(from_id: ID!, to_id: ID!): GroupEdge
+  deleteEdgeGroupEdgeFromModeratorToGroup(from_id: ID!, to_id: ID!): Boolean
+}
+
+schema {
+  query: Query
+  mutation: Mutation
+}

src/test/directive-id.graphql src/test/user-group.graphqlsrc/test/directive-id.graphql renamed to src/test/user-group.graphql

@@ -3,6 +3,7 @@ type User {
     firstName: String
     lastName: String
     role: Role
+    email: EmailAddress
 }
 
 type Group {
@@ -12,10 +13,14 @@ type Group {
 type Moderator {
     moderatorId: ID!
     name: String
+    moderates: Group
 }
 
 enum Role {
     ADMIN
     USER
     GUEST
 }
+
+"https://the-guild.dev/graphql/scalars/docs/scalars/email-address"
+scalar EmailAddress