Secure Big Data Analytics - Hadoop & Intel

Secure Big Data Analytics - Hadoop & Intel

• 1.
  Secure Big DataAnalytics Combining APIs, Security and Big Data + Data Center Software Division 1
• 2.
  Two Red HotTrends - How do they Intersect? API Management Big Data Analytics • Enterprise extending reach • Increased Volume, Variety, ? through APIs Velocity of unstructured data • API traffic overtaking web traffic • Drivers: mobile, cloud, social • Defacto communication for • Tremendous ROI mobile to server How does this effect application architecture to support growth? 2
• 3.
  Big Data Fundamentals Traditional Data Analysis Big Data Analysis Unstructured Cluster Relational Data Analyze Database Warehouse Organize Analyze Transaction Batch Streaming Devices (MapReduce) • Structured data • Unstructured, variety of data: “mashup” • Data ~ GBs to TBs • Data ~ TBs to PBs • Centralized: Data moves to analytics • Distributed: Analytics move to the data • Batch analytics • Streaming analytics Focus ventures in one of two “Only business model tech has left” areas: monetization of data or infrastructure to enable monetization of data March 12, 2012 3
• 4.
  Today’s Big DataTools & Hurdles New BI Tools “Big Data” includes tools like Hadoop, NOSQL technologies, massive parallel processing, and in-memory databases Existing Hurdles with Hadoop • Job Control - Enable clients to run jobs with security controls • Data On-ramping - Get data into Hadoop for processing, from internal sources, cloud services or network-connected devices • Data Off-ramping - Data availability to clients via APIs, suitable for mobile applications • Security and Compliance - Big Data processing provides PII protection, data security and PCI compliance 4
• 5.
  Connecting Data Movement: Back End to Device to ALL Departments 1 Problem: Today’s platforms are 2 Problem: Data and Potential value fragmented and not securely locked in fragmented solutions inhibit connected, limiting scale E2E analytics Dept Dept Dept Dept Dept Dept Dept Dept A B A B A B A B Retail platform Home Energy Platform Telco Service Provider Smart City 10k devices, 1M customers 300K home pilot in Germany Real-time CDR: 12TB/ day 3000+ cameras, 1PB/3mo API Control Point Analytics Edge Devices NB/ULT Phone Cameras Kiosk PoS DS API Control Point 5
• 6.
  API/Service Gateway Fundamentals Service API Data Mediation Management Transformation • Consistent policy enforcement for API CENTRALIZED across Service Gateway Central Proxy departments Enterprise • Use Models: CSB, ESB-light, Edge Security, API Gateway Monetization/Charge Back App Service Gov & Integration Security, Access, Compliance Developer Community • Meter usage • API management • Edge threat protection • Configuration not code • Throttle per SLAs • Policy creation & exe • Data Loss Protection • Discovery of aggregated • API Analytics • Legacy & SOA integration • Federated ID Brokering services from IT • Orchestrate & transform • PCI PII Data Tokenization • Meta data • Protocol translation Move from Line of Business to “Enterprise” Wide 6 API Mgt & Utilization of Analytics
• 7.
  Last Mile DeviceMobile Middleware • High Performance • Version Management • Content Optimization • Quality of Service • Ubiquitous Compatibility • External Cloud Service Support 7
• 8.
  Information Greed • Greedy Users: Instant response from touch-screens, context aware smart phones, etc • Greedy Business: Expect real time intelligence on the consumer derived from social, data warehouses, and data mining Addressing this greed requires new thinking for how to build Composite Applications 8
• 9.
  Composite Distributed Application Apps • Hybridized – New functionality with legacy code and data • Location Independent- 1-n clouds (private and public) and datacenters simultaneously • Knowledge Complete - Access to disparate “Big Data” warehouses owned by the business • Contextual – Produces just-in-time results based on client context, e.g. identity and location • Accessible & Performs – Produces data compatible with any client on any operating system, and does it instantaneously • Secure and Compliant - Meets compliance and security requirements for data in transit and data at rest Realizing composite apps can be done with a service gateway, which secures, brokers and mediates data for API access, and a Hadoop Cluster which provides data analysis and processing 9
• 10.
  “APIfication” of real-timeHadoop datasets PaaS Services Internal Client (Storage, RDMS) Users HTTP/REST Smartphone interactions with Network- & JSON Results Connected Tablet Clients Devices Partner Web Services Data On-ramping from the cloud with Types of Clients selective protection (FPE/Tokenization) Service Gateway Gateway Control Point DMZ Hadoop API Job Scheduler Legacy Apps and RDBMS IDM Web Services Metadata Server Existing Apps, Data and Infrastructure Node1 Node2 Node3 HDFS 10
• 11.
  Pulling it allTogether: Ref Arch for Composite Apps 11
• 12.
  Field Case Study Secure ‘Big Data’ Storage and REST API • Authenticate IP cameras based on IP address, 2-way SSL or message security • Codeless insertion and retrieval to and from HBase. Drag and drop with no Java coding • Expose ‘Big Data’ using a REST facade, ideal for native mobile applications and partner services • Provide a secure REST API with authentication and authorization based on OAuth and internal identity stores such as LDAP 12
• 13.
  Suggested Roadmap toComposite Apps & Big Data 13
• 14.
  More: www.cloudsecurity.intel.com Gartner Cloud Service Broker API Patterns Secure Big Data Hype Cycle White Paper Solution Brief 14