Download as PDF, PPTX
Uploaded byEberhard Wolff
Data Architecturen Not Just for Microservices
The document discusses data architecture in the context of microservices, emphasizing the importance of decoupling data and services to enhance scalability and robustness. It explores concepts such as bounded contexts, domain-driven design, data consistency, and replication strategies like event sourcing and command-query responsibility segregation (CQRS). The trade-offs between centralized databases and polyglot persistence are highlighted, along with the implications for performance and resilience in microservices architecture.
More Related Content
Similar to Data Architecturen Not Just for Microservices
![[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습](https://cdn.slidesharecdn.com/ss_thumbnails/cqrsesv201804221730publish-180422082848-180423040836-thumbnail.jpg?width=640&height=640&fit=bounds)
Data Architecturen Not Just for Microservices
- 1. Data Architecture – NotJust for Microservices Eberhard Wolff @ewolff Fellow
- 2.
- 3.
- 4.
- 5. Classic Data Architecture >Centralized databases > …or services that provide data > Ensures consistency across systems > …for data model > ...and updates to data > Reuse
- 6.
- 7. Who is usinga centralized database?
- 8. Who likes the centralizeddatabase?
- 9. Microservices: Definition > No consistentdefinition > Microservices are modules > Independent deployment units > E.g. processes, Docker container > Microservice owned by one team
- 10. Microservices: Definition Server / Container Server/ Container Micro Service Micro Service
- 11. Why Microservices? > Developa feature > …bring it into production > ...with no coordination > Independent scaling > Free choice of technology > Robustness > Security
- 12.
- 13. Microservices & Data Server/ Container Server / Container Micro Service Micro Service Order Schema
- 14. Microservices & Data Server/ Container Server / Container Micro Service Micro Service Order Schema
- 15. Microservices & Data >Decoupling for data, too > Separate data storage
- 16. Data Microservices Server / Container Server/ Container Micro Service Micro Service Order Data Microservice
- 17. Data Microservices Server / Container Micro Service OrderData Microservice Customer Data Microservice Remote calls influence performance availability No transaction across customer and order
- 18. Data Microservice > Changetwo microservices if new feature requires change to data schema > But: data in one place > No consistency issues
- 19.
- 20. Encapsulation > Information hiding >Hide the internal data structure > Provide access only through a well defined interface > Data and databases should not be exported
- 21.
- 22. Violates Encapsulation Server / Container Server/ Container Micro Service Micro Service Order Data Microservice Shared data model Logic
- 23. Separate Databases Server / Container Server/ Container Micro Service Micro Service Order Order
- 24. Different Databases Server / Container Server/ Container Micro Service Micro Service neo4j Oracle
- 25. Different Databases > “Polyglotpersistence” > Use the best tool for the job > Technology freedom – advantage of microservices > …but extra effort > Backup, disaster recovery etc. > Not as easy as e.g. different frameworks
- 26. Separate Schema Server / Container Server/ Container Micro Service Micro Service Oracle Schema Schema
- 27. Separate Schemas > Lesseffort > Decoupled data models > ...but limited independent scaling and robustness
- 28.
- 29.
- 32.
- 33. Domain-driven Design > 2004 >Still very relevant > By Eric Evans > Focus on part IV > Free reference: http://domainlanguage.com/ ddd/reference/
- 34. Order Shipping address Tracking # Items ItemCategories Priority shipping Customs # Account # ... Credit card # Order #
- 35. My Domain Model isa mess!
- 36. Bounded Context > Domainmodel is only valid for one context > There is no universal data model! > See all failed SOA attempts
- 37. Order Shipping address Tracking # Items ItemCategories Priority shipping Customs # Account # ... Credit card # Order # Customs Order Recommen- dations Order Tracking Order Shipping address Tracking # Item Categories Priority shipping Customs # Payment Order Account # Credit card #
- 38.
- 39. Bounded Context > Microservice= BOUNDED CONTEXTS > Changes for new features are local > …even if data models need to be changed
- 40.
- 41.
- 42.
- 43.
- 44. What about basic dataof an order?
- 45. Strategic Design > Howdo BOUNDED CONTEXTS relate to each other? > Context can have relationships > DDD defines several relationship patterns
- 46. Shared Kernel > Subsetof a model > …that two teams share > Eric Evans: Including code and database > Microservices: Just sharing a model
- 47. Anti-corruption Layer > Don’tlet e.g. a legacy model influence a new model > Isolate model by additional layer > No need to modify the old system
- 48. Context Relationships > Team= Deployment Unit = BOUNDED CONTEXT > Context Relationships define how BOUNDED CONTEXT are used… > ...and how much teams need to collaborate
- 49. Coordination Effort Shared BOUNDED CONTEXT SHAREDKERNEL CUSTOMER / SUPPLIER ANTICORRUPTION LAYER CONFORMIST SEPARATE WAYS
- 50.
- 51. Context Map > Showthe different BOUNDED CONTEXT > …and the relation to each other > BOUNDED CONTEXT might be microservices > ...or communication links
- 52.
- 53.
- 54. Centralized Shared Kernel > Ensuresconsistency > ...but needs to be called for a lot of operations > Resilience / performance / transactions > Have one master as the source of truth
- 55.
- 56. Decentralized Shared Kernel > Mightbe inconsistent > ...but all data for all requests is available in the local database > Better resilience… > ...and performance
- 57.
- 58. Database Replication > Builtinto the database > Replicate schema across database instances > But: Microservices have separated schemas > Every Microservice might have different data > …so database replication is not a good fit
- 59.
- 60. Events > Later additionto Domain-driven Design > Events with a business meaning > Decouple time: Asynchronous > Decouple logic: System can handle event as it pleases
- 61.
- 62. Events & DataReplication > Events lead to data replication > i.e. each system stores information it received in an event > Data stored in separate schema > Very decoupled > Hard to repair inconsistencies
- 63. More Fun WithEvents
- 64. Event Sourcing > InternalStructure for Microservice with events > Current state result of all events > Calculate state on the fly?
- 65.
- 66. Event Sourcing > Eventstore and snapshot help to repair inconsistencies > Event-based architecture in microservices
- 67. CQRS > Command –Query Responsibility Segregation > Commands change data > Query provide data > Implement in separate modules > …or even microservices > ...with potentially different BOUNDED CONTEXTS
- 68. Commands vs Events >Command: Change that data! > Event: Something has happened > Component decides if data should be changed
- 69.
- 70.
- 71. Batch > Get alldata > Provide API > …to decouple schema > Copy interesting data into local database
- 72.
- 73. Batch & DataReplication > Easy to repair inconsistencies > Batch run at specific points > i.e. updates take time > Data not consistent across microservices
- 74.
- 75. CAP Theorem > Consistency >All nodes see the same data > Availability > Node failures do not prevent survivors from operating > Partition Tolerance > System continues to operate despite arbitrary message loss C P A
- 76. CAP Theorem: P >Network partitions do occur > Even with highly available network hardware > Also: very slow response = partition > Need to deal with P
- 77. CAP Theorem: Cor A? > Node cannot access other nodes > Might have missed updates > A, not C: Answer with a potentially wrong answer > C, not A: Don’t answer – the answer might be wrong
- 78.
- 79.
- 80. Classic: Centralized Database Microservices: private database decoupling Data Microservices: Consistent but resilience/ performance / transactions / decoupling? Database per Microservice: Polyglot Persistence Schema per Microservice: Simple infrastructure
- 81. Redundant Data or BoundedContext? Batch Database Replication Events Redundancy? Context Map and Context Relations Replication CAP Event Sourcing CQRS e.g. Shared Kernel
- 82.