Skip to content

feat: Fast schema file format #254

@rstata

Description

@rstata

This issue describes the work submitted in this commit.

Under the hood, Viaduct is slowly moving away from GraphQL Java's graphql.schema.GraphQLSchema as a representation of a GraphQL schema and towards using its own ViaductSchema representation instead. However, to date ViaductSchema itself has been a thin wrapper around GraphQL Java's parser and validator, which can be slow and memory intensive on very large schemas (like the "central schema" at Airbnb). In addition, because Viaduct still depends on GraphQLSchema in many places, this PR contains a function that uses GraphQLSchema.Builder to convert a ViaductSchema into a GraphQLSchema, which cuts construction time in half.

This PR introduces a new implementation of ViaductSchema that is constructed by reading a binary file format specially designed to support fast, lightweight construction of even very large schemas. Here are some numbers:

LargeSchema4 Benchmarks (~6MB schema)

Benchmark Avg Time Memory
SDL Text → GraphQLSchema 2676 ms 3.5 GB/op
SDL Text → GJSchemaRaw 481 ms 626 MB/op
Binary → GraphQLSchema 1447 ms 2.2 GB/op
Binary → BSchema 59 ms 166 MB/op

where GraphQLSchema is GraphQL Java's schema, GJSchemaRaw is a ViaductSchema instance backed by TypeDefinitionRegistry rather than a GraphQLSchema (and thus is faster to construct), and BSchema is a ViaductSchema instance read from the binary file format

LargeSchema5 Benchmarks (~11MB schema)

Benchmark Avg Time Memory
SDL Text → GraphQLSchema 11983 ms 14.6 GB/op
SDL Text → GJSchemaRaw 904 ms 1.1 GB/op
Binary → GraphQLSchema 6671 ms 8.5 GB/op
Binary → ViaductSchema 185 ms 290 MB/op

In addition to the implementation itself, the PR includes extensive testing and documentation for the new implementation, as well as JMH benchmarks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions