Downloaded 407 times
Uploaded byGunnar Hillert
S2GX 2012 - Introduction to Spring Integration and Spring Batch
The document provides an overview of Spring Integration and Spring Batch, detailing their purposes, history, components, and how they can work together in enterprise applications. It addresses various integration styles, messaging patterns, deployment options, and common batch processing practices, highlighting the benefits of both frameworks. Additionally, it includes resources for further learning and community engagement around these technologies.
More Related Content
Similar to S2GX 2012 - Introduction to Spring Integration and Spring Batch
S2GX 2012 - Introduction to Spring Integration and Spring Batch
- 1. Introduction to Spring Integration and Spring Batch Gunnar Hillert, Spring Integration Team Gary Russell, Spring Integration Team Twitter: @ghillert, @gprussell © 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.
- 2. What we willcover... • Spring Integration • Spring Batch • Using Spring Batch and Spring Integration together 2
- 3.
- 4. Integration Styles JVM JVM Core Messaging EAI • Business to Business Integration (B2B) • Inter Application Integration (EAI) • Intra Application Integration B2B External Business Partner 4
- 5. Integration Styles • File Transfer • Shared Database • Remoting • Messaging 5
- 6. Common Patterns Retrieve Parse Transform Transmit 6
- 7. Enterprise Integration Patterns • By Gregor Hohpe & Bobby Woolf • Published 2003 • Collection of well-known patterns • http://www.eaipatterns.com/eaipatterns.html • Icon library provided 7
- 8. “ Spring Integration provides an extension of the Spring programming model to support the well-known enterprise integration patterns. 8
- 9. History • Nov 13, 2007 - First commit: Nov 13, 2007 • Jan 23, 2008 - Spring Integration 1.0.0.M1 • Nov 26, 2008 - Spring Integration 1.0.0 • Nov 22, 2010 - Spring Integration 2.0.0 • Jan 6, 2012 - Spring Integration 2.1.0 • Sep 21, 2012 - Spring Integration 2.2.0.RC1 • 2013 - Spring Integration 3.0 9
- 10. What is SpringIntegration? • Light-weight messaging framework • Provides an adapter-based platform • Pipes and Filters at the core of Spring Integration’s architecture – Endpoint (Filter) – Channel (Pipe) – Message 10
- 11. Advantages • Provides buildingblocks to implement systems that are: – are loosely Coupled (Logically or Physically) – are Event Driven (EDA) – have a staged event-driven architecture (SEDA) • Sophisticated support for synchronous / asynchronous messaging 11
- 12. Configuration • XML NamespaceSupport • Annotation Support (e.g. @Transformer, @Router, @ServiceActivator) <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:int="http://www.springframework.org/schema/integration" xsi:schemaLocation=".."> ... <int:channel id="orders"/> <int:splitter input-channel="orders" expression="payload.items" output-channel="drinks"/> <int:channel id="drinks"/> <int:router input-channel="drinks" expression="payload.iced ? 'coldDrinks' : 'hotDrinks'"/> ... </beans> 12
- 13. Spring Integration DSLs •Scala val messageFlow = filter {payload: String => payload == "World"} --> transform { payload: String => "Hello " + payload} --> handle { payload: String => println(payload) } messageFlow.send("World") • Groovy def builder = new IntegrationBuilder() def flow = builder.messageFlow { transform {payload->payload.toUpperCase()} filter {payload-> payload =="HELLO"} handle {payload->payload} } assert flow.sendAndReceive("hello") == "HELLO" assert flow.sendAndReceive("world") == null 13
- 14. Deployment Options • Embeddedas a library • Stand-alone deployment • War deployment 14
- 15. What is ina Message? • Unit of information • Encapsulates data • Passed between endpoints • Consists of headers – contains data relevant to the messaging system • and a payload – actual data for the receiver – depending on use-cases: POJO instances or serialized data 15
- 16. What is ina Message? package org.springframework.integration; public interface Message<T> { MessageHeaders getHeaders(); T getPayload(); } 16
- 17. Message Headers • Message ID (automatically generated UUID) • Timestamp • Correlation Id • Reply Channel • Error Channel • Expiration Date • Priority • ... • Add your own headers using a Header Enricher 17
- 18. Function of aMessage • Command Message C • Event Message E • Document Message D 18
- 19. What is aChannel? • Channels connect producers and consumers (decoupling) • MessageChannel Interface: – PollableChannel (Polling Consumer) – SubscribableChannel (Event Driven) • Implementations: – DirectChannel – PublishSubscribeChannel – QueueChannel <int:channel id="input"> – PriorityChannel <int:queue capacity="10"/> </int:channel> – RendezvousChannel – ExecutorChannel 19
- 20. What is anEndpoint? • Polling or event-driven • Inbound or outbound • Unidirectional (Channel Adapter) or bidirectional (Gateway) • Internal or external (application context) <inbound-channel-adapter/> <outbound-channel-adapter/> <inbound-gateway/> <outbound-gateway/> <gateway/> <service-activator/> 20
- 21. Router • Message Router • Content-based router • Recipient list router (dynamic) • Payload type router • Header value router • Exception type router 21
- 22. Transformer • Delegating via ref/method • Spring Expression Language • Groovy, JRuby, Jython, JavaScript • Object-to-JSON / JSON-to-Object • Payload serializing/deserializing • File-to-bytes, File-to-String • JAXB, JibX, Castor, XMLBeans, Xstream • XPath, XSLT • Object XML Marshalling/Unmarshalling (Spring OXM) • ... 22
- 23. Spring Integration Components •Claim Check (In/Out) • Service Activator • Content Enricher • Scripting support (JSR 223) – Header Enricher – Ruby/JRuby, Javascript ... – Payload Enricher • Groovy • Control Bus • Message History • Delayer • Message Store • JMX Support – JDBC, Redis, MongoDB, • Message Handler Chain Gemfire • Messaging Bridge • Wire Tap • Resequencer • ... 23
- 24. Adapters • AMQP/RabbitMQ • MongoDB • Stored Procedures • AWS* • POP3/IMAP/SMTP • TCP/UDP • File/Resource • Print* • Twitter • FTP/FTPS/SFTP • Redis • Web Services • GemFire • RMI (SOAP or POX) • HTTP (REST) • RSS/Atom • XMPP • JDBC • SMB* • XPath • JMS • Splunk* • XQuery* • JMX • Spring Application • JPA Events • ... * Spring Integration Extensions Project 24
- 25. Tooling - SpringTool Suite (STS) • Namespace Support • Visualization • 4 Spring Integration specific STS Templates – Simple Template (Core Components only) – File Polling Template (File Adapter) – War Template (Uses Twitter Adapter) – Adapter Template (Create your own components) 25
- 26. Tooling - IntelliJIDEA • Spring Integration support since IDEA 10.5 • Namespace Support • Bean visualization • IntelliJ IDEA 12 (Preview) supports Spring Integration 2.1 & 2.2 26
- 27. Tooling - C24Integration Objects (C24iO) Studio • Supports a variety of standards: – SWIFT – ISO20022 – FIX • Namespace Support: <int-c24:transformer/> <int-c24:validating-header-enricher/> <int-c24:marshalling-transformer/> ... • http://www.c24.biz/io-spring.html 27
- 28.
- 29. Café Demo -Flow 29
- 30. Samples • https://github.com/SpringSource/spring-integration-samples • Contains50 Samples and Applications • Several Categories: – Basic – Intermediate – Advanced – Applications 30
- 31. Books • Just SpringIntegration • Pro Spring Integration • Spring Integration in Action 31
- 32. Source Code • https://github.com/SpringSource/spring-integration • https://github.com/SpringSource/spring-integration-samples • https://github.com/SpringSource/spring-integration-extensions • https://github.com/SpringSource/spring-integration-templates • https://github.com/SpringSource/spring-integration-dsl-groovy • https://github.com/SpringSource/spring-integration-dsl-scala 32
- 33. Contribute • Post Questionand Answers the Forums – http://forum.springsource.org/forumdisplay.php?42-Integration • Create Jiras – https://jira.springsource.org/browse/INT • Submit Pull Requests - Contributor Guidelines: – github.com/SpringSource/spring-integration/wiki/Contributor-Guidelines • New Spring Integration Extensions Repository 33
- 34.
- 35. Batch Jobs Differfrom online/real-time processing applications: • Long-running – Often outside office hours • Non-interactive – Often include logic for handling errors or restarts • Process large volumes of data – More than fits in memory or a single transaction 35
- 36. Batch and offlineprocessing • Close of business processing – Order processing – Business reporting – Account reconciliation • Import/export handling – a.k.a. ETL jobs (Extract-Transform-Load) – Instrument/position import – Data warehouse synchronization • Large-scale output jobs – Loyalty scheme emails – Bank statements • 36
- 37.
- 38. Chunk-Oriented Processing • Input-outputcan be grouped together • Input collects Items before outputting:Chunk-Oriented Processing • Optional ItemProcessor 38
- 39.
- 40. Simple File LoadJob 40
- 41. More Complex UseCases • It's very common to use an off-the-shelf reader and writer • More complex jobs often require custom readers or writers • ItemProcessor is often used if there's a need to delegate to existing business logic • Use a writer if it's more efficient to process a complete chunk 41
- 42. Job and Stepin Context 42
- 43. JobRepository and BatchMetadata 43
- 44. ExecutionContext • We needto know where a failure occurred to restart a batch process • Job Repository metadata is used to determine the step at which the failure occurred • Application Code (in reader/writer) needs to maintain state within a step (e.g. current chunk) • Spring Batch can supply that data during restart to facilitate repositioning 44
- 45. Common Batch Idioms •Batch jobs typically process large amounts of homogeneous input • Makes iteration a common concern: Repeat • Transient errors during processing may require a Retry of an input item • Some input may not be valid, may want to Skip it without failing • Some errors should fail the job execution, allowing one to fix the problem and Restart the job instance where it left off 45
- 46. Spring Batch • SpringBatch supports these common concerns • Abstracts them in the framework – Job business logic doesn't need to care about details • Allows for simple configuration with pluggable strategies 46
- 47. Business Logic Delegation– Spring Application 47
- 48. Business Logic Delegation– Spring Integration 48
- 49. Spring Batch Admin •Sub project of Spring Batch • Provides Web UI and ReSTFul interface to manage batch processes • Manager, Resources, Sample WAR – Deployed with batch job(s) as single app to be able to control & monitor jobs – Or monitors external jobs only via shared database 49
- 50. Scaling and ParallelProcessing • First Rule: – Use the simplest technique to get the job done in the required time – Do not optimize/parallelize unnecessarily • Options: – Multi-threaded Step (single process) – Parallel Steps (single process) – Remote Chunking of Step (multi process) – Partitioning a Step (single or multi process) 50
- 51. Using Spring Batch and Spring Integration together 51
- 52. Launching batch jobsthrough messages • Event-Driven execution of the JobLauncher • Spring Integration retrieves the data (e.g. file system, FTP, ...) • Easy to support separate input sources simultaneously Inbound Channel Adapter Transformer FTP D File C JobLaunchRequest JobLauncher 52
- 53. Providing feedback withinformational messages • Spring Batch provides support for listeners: – StepListener – ChunkListener – JobExecutionListener <batch:job id="importPayments"> ... <batch:listeners> <batch:listener ref="notificationExecutionsListener"/> </batch:listeners> </batch:job> <int:gateway id="notificationExecutionsListener" service-interface="o.s.batch.core.JobExecutionListener" default-request-channel="jobExecutions"/> 53
- 54. Externalizing batch processexecution • Use Spring Integration inside of Batch jobs, e.g.: – ItemProcessor – ItemWriter • Offload complex processing • Asynchronous processing support: – AsyncItemProcessor – AsyncItemWriter • Externalize chunk processing using ChunkMessageChannelItemWriter 54
- 55. Learn More. StayConnected. At SpringOne 2GX: • Building an Enterprise CRM with Grails and Spring Integration • What's New in Spring Integration • Spring Integration in the Wild • Batch Processing and Integration on Cloud Foundry • Building for Performance with Spring Integration & Spring Batch • Spring Integration, Batch, and Data: Future Directions • Managing and Monitoring Spring Integration Applications • Java Batch JSR-352 55
- 56. Learn More. StayConnected. • Web: www.springintegration.org • Twitter: @m_f_ @ghillert @z_oleg @gprussell Questions? Thank You!! 56
Editor's Notes
- #2 \n
- #3 \n
- #4 \n
- #5 \n
- #6 \n
- #7 \n
- #8 \n
- #9 \n
- #10 \n
- #11 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #12 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #13 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #14 \n
- #15 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #16 \n
- #17 Message<String> helloMessage = MessageBuilder.withPayload("Hello, world!")\n.setHeader("custom.header", "Value") .setHeaderIfAbsent("custom.header2", "Value2") .build();\n
- #18 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #19 In some applications it might be fine to send a reference to an\nobject over the channel, but in others it might be necessary to use a more interoperable representation like an identifier or a serialized version of the original data.\n
- #20 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #21 messaging framework that supports lightweight, event-driven interactions within an\napplication\n
- #22 \n
- #23 \n
- #24 \n
- #25 \n
- #26 \n
- #27 \n
- #28 \n
- #29 \n
- #30 \n
- #31 \n
- #32 \n
- #33 \n
- #34 \n
- #35 \n
- #36 \n
- #37 \n
- #38 \n
- #39 \n
- #40 \n
- #41 \n
- #42 \n
- #43 \n
- #44 \n
- #45 \n
- #46 \n
- #47 \n
- #48 \n
- #49 \n
- #50 \n
- #51 \n
- #52 \n
- #53 \n
- #54 \n
- #55 \n
- #56 \n
- #57 \n