Download as PDF, PPTX
![Protocol Buffer Serialization
A message is just a stream of bytes
[field_number<<3 + wire_type] [length if necessary][data]...
$ hexdump /tmp/request.bin
0000000 0a 05 68 65 6c 6c 6f
0a is “0000 1010”, so
field_number = 1 and wire_type = 2](https://image.slidesharecdn.com/implementingopenapiandgraphqlserviceswithgrpc-191015163736/75/Implementing-OpenAPI-and-GraphQL-services-with-gRPC-28-2048.jpg)
![service Display {
rpc CreateKiosk(Kiosk) returns (Kiosk) {
option (google.api.http) = {post:"/v1/kiosks"};
}
rpc GetKiosk(GetKioskRequest) returns (Kiosk) {
option (google.api.http) =
{get: "/v1/kiosks/{id}" };
}
rpc DeleteKiosk(DeleteKioskRequest)
returns (google.protobuf.Empty) {
option (google.api.http) =
{ delete: "/v1/signs/{id}" };
}
[...]
}
Add REST annotations to your service
gRPC definitions with HTTP annotations
service Display {
rpc CreateKiosk(Kiosk) returns (Kiosk) {
}
rpc GetKiosk(GetKioskRequest) returns(Kiosk) {
}
rpc DeleteKiosk(DeleteKioskRequest)
returns (google.protobuf.Empty) {
}
[...]
}
gRPC definitions](https://image.slidesharecdn.com/implementingopenapiandgraphqlserviceswithgrpc-191015163736/75/Implementing-OpenAPI-and-GraphQL-services-with-gRPC-57-2048.jpg)
![# Call the CreateKiosk method
$ curl -d '{"name":"HTTP Kiosk", "size": { width: 1080, height: 720 } }'
localhost:8082/v1/kiosks?key=AIzaSy[...]bBo
# Call the GetKiosk method
$ curl localhost:8082/v1/kiosks/1?key=AIzaSy[...]bBo
Now you can use HTTP+JSON!](https://image.slidesharecdn.com/implementingopenapiandgraphqlserviceswithgrpc-191015163736/75/Implementing-OpenAPI-and-GraphQL-services-with-gRPC-58-2048.jpg)
Uploaded byTim Burks
Implementing OpenAPI and GraphQL services with gRPC
The document discusses the implementation of OpenAPI and GraphQL services using gRPC, highlighting the advantages and challenges of each technology. It covers how each API technology is built and called, emphasizing gRPC's high performance and cross-language compatibility. Additionally, it provides insights into service generation, error handling, and evolving APIs through examples and implementation strategies.
More Related Content
Similar to Implementing OpenAPI and GraphQL services with gRPC
Implementing OpenAPI and GraphQL services with gRPC
- 1. Implementing OpenAPI and GraphQLServices with gRPC Tim Burks ([email protected], @timburks)
- 2. 1. API technologies:OpenAPI, GraphQL, gRPC 2. Why implement your API with gRPC? 3. Implementing OpenAPI services with gRPC 4. Implementing GraphQL services with gRPC Topics
- 3. [email protected] @timburks Electronic Design Automation MobileApps APIs Hello!
- 4. Projects at Google gnostic (OpenAPICompiler) gRPC-Swift GAPICs for Google APIs
- 5.
- 6. APIs are Interfaces forApplication Programmers.
- 7.
- 8. Five Questions aboutAPI Systems 1. What problem is this API system addressing? 2. How are APIs called? 3. How are APIs built? 4. What’s great about this? 5. What’s challenging about this?
- 9. What problem isOpenAPI addressing? “The OpenAPI Specification, originally known as the Swagger Specification, is a specification for machine-readable interface files for describing, producing, consuming, and visualizing RESTful web services.” Wikipedia
- 10. How an OpenAPIservice is called: You might call it from your browser. In your applications, you call it from code. openapi-generator autorest APIMatic SwagGen (Swift) Paperclip (Rust)
- 11. How an OpenAPIservice is built: 🤞 Code-first Handwritten implementations. Handwritten API descriptions. ⚙ Code-first (better) Handwritten implementations. API descriptions generated from code. 📐 Spec-first Handwritten API descriptions. Implementations generated from API descriptions. ?🤖
- 12. What’s great aboutOpenAPI: ● Standards and tools make it easy to get started. ○ HTTP ○ JSON/YAML ● Very popular - large community.
- 13. What’s challenging aboutOpenAPI: REST? “Unfortunately, many of the APIs that claim to be RESTful layer a lot of proprietary concepts on top of HTTP. Essentially, they invent their own API and use HTTP as a lower-level transport layer, rather than using HTTP directly as it was designed. In fact, there’s so much variability in the way that people use the term REST in the context of APIs that it’s difficult to know what they mean by it unless you know them well.” Martin Nally
- 14. What problem isGraphQL addressing? “GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.” graphql.org
- 15. How a GraphQLservice is called: “With express-graphql, you can just send an HTTP POST request to the endpoint you mounted your GraphQL server on, passing the GraphQL query as the query field in a JSON payload.” https://graphql.org/graphql-js/graphql-clients/ Also GraphiQL!
- 16. How a GraphQLservice is built: ● Code-first, schema extracted from code ● Schema-first - code generated from SDL ● Generated from underlying APIs ○ Federated from other GraphQL APIs ○ OpenAPI-to-GraphQL ○ rejoiner
- 17. What’s great aboutGraphQL: ● Simple client-side interface ● Optimizes for client performance ● Layered on familiar technology ● Sharable BFF
- 18. What’s challenging aboutGraphQL: ● Complex implementation ● Performance challenges ● Versioning? ● Schema design Public GraphQL APIs are rare.
- 19. What problem isgRPC addressing? “gRPC is a modern open source high performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication. It is also applicable in last mile of distributed computing to connect devices, mobile applications and browsers to backend services.” grpc.io
- 20. How a gRPCservice is called: ALWAYS specification-first. Clients are generated from API descriptions written in the Protocol Buffer language.
- 21. How a gRPCservice is built: ALWAYS specification-first. Service code is generated. Typically this generated code declares an interface that is implemented in language-native data structures and patterns.
- 22. What’s great aboutgRPC: So much is built-in. High performance. Cross-language compatibility. Potential for very nice client integration.
- 23. What’s challenging aboutgRPC: Some learning and tooling is required!
- 24. 2 Why implement your APIwith gRPC?
- 25. APIs have anative language. Protocol Buffers
- 26. Protocol Buffers isa language-neutral, platform-neutral, extensible mechanism for serializing structured data.
- 27. “Protocol Buffers” meansseveral things 1. A serialization mechanism 2. An interface description language 3. A methodology
- 28. Protocol Buffer Serialization Amessage is just a stream of bytes [field_number<<3 + wire_type] [length if necessary][data]... $ hexdump /tmp/request.bin 0000000 0a 05 68 65 6c 6c 6f 0a is “0000 1010”, so field_number = 1 and wire_type = 2
- 29. Protocol Buffer IDL messageEchoMessage { string text = 1; // other fields... }
- 30. Protocol Buffer Methodology %protoc echo.proto --go_out=. # This runs the Protocol Buffer Compiler # (https://github.com/protocolbuffers/protobuf/releases) # # with a plugin called protoc-gen-go # # The plugin generates a Go source file that implements # the data structures defined in echo.proto and code # for reading and writing them as serialized bytes.
- 31. Try it! package main import( "fmt" "github.com/golang/protobuf/proto" echo "./generated" ) func main() { document := echo.EchoMessage{Text:"hello"} bytes, _ := proto.Marshal(&document) fmt.Printf("%+vn", bytes) }
- 32. gRPC scales tomultiple languages. Java Service Python Service Go Service C++ Service gRPC Server gRPC Stub gRPC Stub gRPC Stub gRPC Stub gRPC Server gRPC Server gRPC Server gRPC Stub
- 34. gRPC scales todistributed systems.
- 35. Load Balancer Users SQL Database Cluster NoSQL Database Cluster ETLPipeline Front End Business Logic Back End Business Logic Data Warehouse Invoicing Transcoding Illustration: Sandeep Dinesh, Google, Developer Advocate
- 36. Load Balancer Users SQL DatabaseCluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Load Balancer Users SQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Load Balancer Users SQL Database Cluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Load Balancer Users SQL Database Cluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Illustration: Sandeep Dinesh, Google, Developer Advocate
- 37. Load Balancer Users SQL DatabaseCluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Load Balancer Users SQL Database Cluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Load Balancer Users SQL Database Cluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic Load Balancer Users SQL Database Cluster NoSQL Database Cluster ETL Pipeline Front End Business Logic Back End Business Logic Data Warehouse Back End Business Logic Back End Business Logic API API
- 38. gRPC is openand extensible.
- 39. gRPC has astyle guide.
- 40. Generated API Clients Whatdo you get?
- 41. Trivial Getting-Started Experience GAPICclient: Channel channel = NettyChannelBuilder.forAddress(API_ENTRY, API_PORT).build(); List<ClientInterceptor> interceptors = new ArrayList<>(); GoogleCredentials credentials = GoogleCredentials.getApplicationDefault(); interceptors.add( ChannelFactory.credentialInterceptor(credentials)); LoggingService stub = LoggingServiceGrpc.newBlockingStub(channel, interceptors); gRPC: LoggingClient client = LoggingClient.create();
- 42. Flattening, Automatic Retry GAPICclient: LoggingService stub = ...; // Note: this only does a simple retry, exponential backoff // is more complex DeleteLogRequest request = DeleteLogRequest.newBuilder() .setLogName(LOG_NAME) .build(); for (int i = 0; i < MAX_RETRY; i++) { try { stub.deleteLog(request); } catch (RpcException e) { if (i == MAX_RETRY) throw e; } } gRPC: LoggingClient client = LoggingClient.create(); client.deleteLog(LOG_NAME);
- 43. Natural Pagination GAPIC client: LoggingServicestub = ...; ListLogsRequest request = ListLogsRequest.newBuilder() .setParentName("projects/" + PROJECT_ID) .build(); while (true) { ListLogsResponse response = stub.listLogs(request); for (Log log : response.getLogsList()) { System.out.printf("Log: %s%n", log.getDisplayName()); } String nextPageToken = response.getNextPageToken(); if (nextPageToken != null) { request = ListLogsRequest.newBuilder() .setPageToken(nextPageToken).build(); } else { break; } } gRPC: LoggingClient client = LoggingClient.create(); ParentNameOneOf parent = ParentNameOneOf.from(ProjectName.create(PROJECT_ID)); for (Log log : client.listLogs(PARENT).iterateAll()) { System.out.printf("Log: %s%n", log.getDisplayName()); }
- 44. Long-Running Operations SpeechClient speechClient= SpeechClient.create(); OperationFuture<LongRunningRecognizeResponse> recognizeOperation = speechClient.longRunningRecognizeAsync(config, audio); … LongRunningRecognizeResponse response = recognizeOperation.get(); ● Java: OperationFuture ○ Polls the service until the LRO is done ○ Provides metadata as the LRO is in progress ○ Provides the final result
- 45. Resource Name Types GAPICclient: PublisherService stub = ...; CreateTopicRequest request = CreateTopicRequest.newBuilder() .setTopic("projects/my-project/topics/my-topic") .build(); Topic response = stub.createTopic(request); gRPC: TopicAdminClient topicAdminClient = TopicAdminClient.create(); TopicName name = TopicName.create("my-project", "my-topic"); Topic response = topicAdminClient.createTopic(name);
- 46.
- 47. GAPIC Project Architecture(at github.com/googleapis) gapic-generator-python Wanted: Swift, Rust, Dart... gapic-generator (java, php, python, go, csharp, node, ruby) gapic-generator-ruby gapic-generator-go gapic-generator-typescript gapic-generator-csharp One API description (Annotated .proto files) Many language- specific client libraries
- 49.
- 50. protoc-gen-go_cli A protoc pluginthat generates CLIs in Go for APIs described with protocol buffers.
- 51. gRPC Fallback ● Goproxy ● Description and handwritten examples ● Node client (gax-nodejs)
- 52. gRPC Reflection ● ProtocolDiscussion ● Go implementation (example) ● Specification (protos)
- 53. Manage gRPC Serviceswith Cloud Endpoints
- 54. Cloud Endpoints Architecture GCE,GKE, Kubernetes or App Engine Flexible Environment Instance GCE, GKE, Kubernetes or App Engine Flexible Environment Instance (or off-GCP) Extensible Service Proxy Container API Container Google Service Management API User Code api calls gcloud Config deployment Google Cloud Console Google Service Control API config Runtime check & report Load balanci ng Stackdriver Metrics & logs Metrics & logs visualized
- 55.
- 56. Build a gRPCservice, get a free REST API! Originally defined in google/api/http.proto Now documented in AIP-127: HTTP and gRPC Transcoding
- 57. service Display { rpcCreateKiosk(Kiosk) returns (Kiosk) { option (google.api.http) = {post:"/v1/kiosks"}; } rpc GetKiosk(GetKioskRequest) returns (Kiosk) { option (google.api.http) = {get: "/v1/kiosks/{id}" }; } rpc DeleteKiosk(DeleteKioskRequest) returns (google.protobuf.Empty) { option (google.api.http) = { delete: "/v1/signs/{id}" }; } [...] } Add REST annotations to your service gRPC definitions with HTTP annotations service Display { rpc CreateKiosk(Kiosk) returns (Kiosk) { } rpc GetKiosk(GetKioskRequest) returns(Kiosk) { } rpc DeleteKiosk(DeleteKioskRequest) returns (google.protobuf.Empty) { } [...] } gRPC definitions
- 58. # Call theCreateKiosk method $ curl -d '{"name":"HTTP Kiosk", "size": { width: 1080, height: 720 } }' localhost:8082/v1/kiosks?key=AIzaSy[...]bBo # Call the GetKiosk method $ curl localhost:8082/v1/kiosks/1?key=AIzaSy[...]bBo Now you can use HTTP+JSON!
- 59.
- 60. gnostic-grpc ● 6 usefulthings I learned from GSoC ● How to build a REST API with gRPC and get the best of two worlds ● gnostic-grpc (end-to-end example) ● Envoy gRPC-JSON transcoding
- 61.
- 62. Code-first or Schema-first? ●Evolving the Graph (Jon Wong, Coursera) ● Hard-Learned GraphQL Lessons: Based on a True Story (Natalie Qabazard & Aditi Garg, Trulia) ● The Problems of "Schema-First" GraphQL Server Development (Nikolas Burk, Prisma)
- 63. Code-first GraphQL withRejoiner
- 64. Medallion (Rejoiner demoby Danny Baggett)
- 65. Sample query (Rejoiner+ Medallion) { events { id name olympics { year season } olympians { id gender last first } } }
- 66. Sample query (Rejoiner+ Medallion) { getEvents(input: {year: 2018, season: "WINTER"}) { id name } }
- 67. Let’s hand-wrap agRPC service (graphql-showcase)
- 68. Impressions / ProjectIdeas gRPC has a lot of API information in the .proto files and we can extend that with annotations. There are some gaps between GraphQL assumptions and gRPC practices that we would want to smooth with configuration or conventions. We could generate a GraphQL interface directly from gRPC reflection. Can we define a standard for gRPC-GraphQL transcoding?
- 69. resolver in-mem response schema GraphQL server on Apigee HostedTarget Google Apigee and GraphQL app Edge ● AuthZ ● security ● aggregate analytics Integrated Dev Portal ● GraphQL playgroundapp developer AX plug-in batch AX app GQL query response GQL query response Stackdriver ● query-level analytics on-board, use APIs