Meta Evaluation of Semantic Nullability Proposals

[janettec]

This document assesses each open source proposal for semantic nullability in terms of ease of adoption at Meta. We will reference proposal numbers outlined in the open source document.

Obviously ease of adoption is not the only consideration. This is meant to serve as a case study of how adoption of the different proposals would play out at a company of Meta’s size/with a similar GraphQL stack.

Meta’s GraphQL Stack

• Our schema is generated from resolver implementations (“implementation first”)
  • A field’s nullability in the schema is determined from the return type of its resolver function (it is not manually annotated)
• There are potentially many consumers of the GraphQL schema (including obscure consumers we won’t be aware of until they break)
• The most important schema consumer is our model generation built on top of the Relay compiler.
  • We generate models for use on clients where the nullability of field accessor return types are based off GraphQL schema nullability
    • Currently only strict non-null fields are typed as null in client models
  • Changing a field in a model from non-null to nullable will break app builds
  • When these models are consumed in null-safe languages, going from nullable to non-null will also break app builds (null unwrapping using ? in Swift on a type that is not nullable will result in a compilation error)
  • The Relay compiler ignores unknown directives (new directives are a no-op until we decide to consume them).
  • The input to Relay is the SDL. We do not use introspection to consume the schema.
• Many use cases of GraphQL use smart clients like Relay that are able to null bubble on the client

What makes new SDL syntax easy to adopt

• Not changing behavior
  • Client model generation (a field on a model doesn’t change nullability when the syntax changes)
    • This is counter to Criteria L. Model generation needs to be able to continue distinguishing between semantic non-null and strict non-null
  • Error propagation/null-bubbling doesn’t change (we don’t null-out any more data than we were before the syntax change)
    • This is counter to Criteria P. Instead of “The solution should result in users marking all semantically non-null fields as such,” we need “The solution should allow for an incremental path towards marking all semantically non-null fields as such.”
  • Hard breaks (schema parse failure, app build failure) are preferred over silent behavior changes
• Not changing interpretation of existing syntax
  • Changing interpretation of existing syntax in executable documents is basically a nonstarter (Criteria B, Criteria O)
    • Executable documents live forever and knowing to interpret documents from older clients differently is a mess
  • Changing interpretation of existing syntax (i.e. ! ) in the SDL is more likely to lead to unintentional changes in behavior

Solutions 1 & 6

(Grouped together as the adoption story is the same for these two solutions)

Adopting this new syntax in the SDL is easy (and in fact we are already using Solution 6 internally).

Because these solutions propose purely new syntax, these changes are no-ops without changes to the logic that consumes the schema.

Solution 2

TL;DR Schema-parsing might break, but the fix (per schema consumer) is easy

Rejected - click for details

• This solution would require coordination between our server team and schema consumers.
  • Consumers’ schema parsing logic should (ideally) fail given both the schema-level directive @strictNullability and ? are new to the SDL.
  • Assuming silent failure during schema parsing would result in ignoring ? and continuing to interpret those fields as nullable, there will be no changes to generated models (as no annotation will still be interpreted as nullable until the consumer logic is updated)
  • To resolve each consumer needs to:
    • Know that now there are schema-level directives (like @strictNullability) that change how syntax is interpreted.
    • Change the interpretation of syntax accordingly
    • Note: These changes are not difficult to implement, so difficulty depends on how many schema consumers there are.

Solution 3

TL;DR Schema parsing might break; app builds might break; could have more unintentional null bubbling, but the fix (per schema consumer) is easy

Rejected - click for details

• Generated models will have a lot more non-null fields (semanticNonNull types previously generated with nullable accessors will now be generated with non-null accessors)
  • In some null-safe languages, this will result in builds breaking (? to unwrap nulls cannot be used on non-null types in Swift)
  • In non-null safe languages like ObjC and Java, we will start null-bubbling more on the client (this is really bad)
• Assuming silent failure would result in previously ! types (now !!) being interpreted as having no annotation (and thus being nullable):
  • In null-safe languages like Swift, this will result in builds breaking
  • In non-null safe languages like ObjC and Java, this will also result in builds breaking
  • Remember, hard breaks are better than silent unintended changes in behavior!
• Risks above can be mitigated by
  • Making sure important consumers (i.e. model generation) have logic ready to interpret the @strictNullability document-level directive
  • Only applying the @strictNullability document-level directive to new fields (initially)
• To resolve schema parsing, build failure, or unintended new behavior, each consumer needs to:
  • Know that now there are document-level directives (like @strictNullability) that change how syntax is interpreted.
  • Change the interpretation of syntax accordingly
  • Note: These changes are not difficult to implement, so difficulty depends on how many schema consumers there are.

Solution 4 (assuming input syntax is unchanged)

TL;DR schema-parsing might break, but the fix (per schema consumer) is easy

Rejected - click for details

• Similar to solution 2 above, except the use of document directive allows for a slower rollout of the new syntax
• Assuming silent failure would result in ignoring ? and continuing to interpret those fields as nullable, there will be no unintended change to generated models
• To resolve each consumer needs to:
  • Know that now there are document-level directives (like @strictNullability) that change how syntax is interpreted.
  • Change the interpretation of syntax accordingly
  • Note: These changes are not difficult to implement, so difficulty depends on how many schema consumers there are.

Solution 5

TL;DR Things might break badly (a lot more null-bubbling), AND the fix is hard

Rejected - click for details

• As of now, there is no directive gating proposed for this solution and no new syntax introduced
  • This means we cannot rely on schema parse failures to identify schema consumers that need to be updated
• Generated models will have a lot more non-null fields (semanticNonNull types previously generated with nullable accessors will now be generated with non-null accessors). This change would happen suddenly because our schema is generated based off of resolver implementations (we are not manually annotating types with !)
  • In some null-safe languages, this will result in builds breaking (? to unwrap nulls cannot be used on non-null types in Swift)
  • In non-null safe languages like ObjC and Java, we will start null-bubbling more on the client (this is really bad)
• Fix is hard because schema consumer loses ability to distinguish between semantic and strict non-null
  • Model generation cannot continue typing semantic non-null fields as nullable
• Internally, we would probably keep the @semanticNonNull annotation from proposal 6 so schema consumers are able to continue distinguishing between strict and semantic non-null
  • This allows model generation to continue to distinguish semantic non-null fields, and continue generating them as nullable to preserve existing behavior (@semanticNonNull = still generate this field as nullable in the model)
  • If we want to eventually converge with open source and remove @semanticNonNull, we need to carefully audit when and where semantic non-null fields throw errors and add proper logic that catches those errors on the client (this would likely be a years-long effort)

Solution 7

TL;DR Things might break badly (a lot more null-bubbling), but the fix (per schema consumer) is easy

• As of now, there is no directive gating proposed for this solution and no new syntax introduced
  • This means we cannot rely on schema parse failures to identify schema consumers that need to be updated
• Generated models will have a lot more non-null fields (semanticNonNull types previously generated with nullable accessors will now be generated with non-null accessors). This change would happen suddenly because our schema is generated based off of resolver implementations (we are not manually annotating types with !)
  • In some null-safe languages, this will result in builds breaking (? to unwrap nulls cannot be used on non-null types in Swift)
  • In non-null safe languages like ObjC and Java, we will start null-bubbling more on the client (this is really bad)
• Fix is easy because schema consumer keeps the ability to distinguish between semantic and strict non-null
  • Model generation can continue typing semantic non-null fields as nullable by looking for @propogateError

[martinbonnin]

Thank you so much for writing this! Question about solution 5. You write:

In non-null safe languages like ObjC and Java, we will start null-bubbling more on the client (this is really bad)

Can you ellaborate? Assuming we have this:

type Query {
  user: User
}
type User {
  name: String
  email: String
}

Which is turned into the non-nullable version:

type Query {
  user: User!
}
type User {
  name: String!
  email: String!
}

The Java Model probably looks something like this:

public class User {
  String name;
  String email;
}

My understanding was that Relay (or the Java client you're using) would be able to detect errors and process them to the closest error boundary so that User.name and User.email are never null in practice. It could probably be a net zero change for the client developper? But looks like it is not the case?

[mjmahone]

In our Java client, we do null bubbling locally, i.e. if a non-nullable field ends up being null (many reasons this can happen), then we bubble that to the nearest nullable parent.

I'm not sure if this is the same problem @janettec is describing, but one problem is if you go from:

type Query {
  user: User
}
type User {
  name: String
  email: String
}

where the Java product is treating user as non-null because in practice it is and there are never, and then you change the schema to:

type Query {
  user: User
}
type User {
  name: String!
  email: String!
}

Now you'll get an NPE on user when name is unexpectedly null, whereas before you'd be fine checking name for null. You would potentially solve this by making user non-nullable.

In a language with actual null safety, you'd have a compile-time issue already and be forced to check user for null, even if it never is (because the schema says it might be at some point!)

[martinbonnin]

Thanks for the walkthrough! It does sound like a headache indeed.

My 5. solution was be to use an error-handling client that can read any error and branch it to the nearest UI error boundary, and then make sure the calling code always checks actually nullable positions (really hard to do with Java). So it would probably take years, which is why we rejected #5, it all adds up, Thanks!