Batching Requests

When an API is distributed across many different services, we need to take special care to make sure that we don't run into performance issues when queries cross over domain boundaries.

To illustrate this, consider two relati vely small schemas in different services and a query that looks up information in both:

Service One:

type User {
    firstName: String!
}

type Query {
    allUsers: [User!]!
}

Service Two:

type User {
    lastName: String!
}

The query:

{
    allUsers {
        lastName
    }    
}

Even for a small query like this, we can run into problems if the number of users returned by Query.allUsers is more than around 50. That's because the gateway will perform a separate request for each user to look up their last name. That's where batching comes in.

For more information on batching techniques, you can read this post by the Apollo team that goes into more depth. While their recommendations are from the point of view of a frontend library, the techniques work equally well for server-side requests.

At the moment, the gateway only supports a single form of batching which we call "Multiple Operation" batching. This is the technique described under "Transport Level Batching" in the post that's linked above. It does put additional constraints on the service but the community has started to converge on this approach and more often than not, this additional requirement is statisfied for the developer without extra work.

To enable batching, the gateway allows us to pass in a QueryerFactory that will tell the planner to use a Queryer that will batch its requests:

import (
    "time"

    "github.com/nautilus/gateway"
    "github.com/nautilus/graphql"
)

// create queryer that can batch requests whenever we query a service
factory := gateway.QueryerFactory(func(ctx *gateway.PlanningContext, url string) Queryer {
    return graphql.NewMultiOpQueryer(url, 10*time.Millisecond, 1000)
})

// instantiate the gateway with our factory
gw, err := gateway.New(..., gateway.WithQueryerFactory(factory))

Keep in mind that this Queryer gets injected into each step in the plan. If we create a new instance of a Queryer in the factory, then only those requests from the one plan step could get batched together. For most situations, this is a totally fine compromise.