![What Does Splunk Really Do?
Into thisIt turns this
[Thu Sep 24 14:57:33 2009] [error] [client 10.2.1.44] ap_proxy: trying GET /petstore/
enter_order_information.screen at backend host '127.0.0.1/7001; got exception
'CONNECTION_REFUSED [os error=0, line 1739 of ../nsapi/URL.cpp]: Error connecting to host
127.0.0.1:7001', referer: http://10.2.1.223/petstore/cart.do?action= purchase&itemId=EST-14](https://image.slidesharecdn.com/gettingstartedwithsplunkbreakoutsession-151015125126-lva1-app6892/75/Getting-started-with-Splunk-Breakout-Session-7-2048.jpg)
Uploaded bySplunk
Getting started with Splunk Breakout Session
This document provides a summary of a presentation about Splunk. It discusses what Splunk is and how it works, including that Splunk is a platform for searching, monitoring, and analyzing machine-generated big data in real-time. It also covers key Splunk concepts like indexing, searching, reporting, alerting, and deployment options. The presentation demonstrates how to install Splunk, add sample data, perform searches, extract fields, create alerts and dashboards, and discusses integration, support resources, and the Splunk developer platform.
More Related Content
Similar to Getting started with Splunk Breakout Session
Getting started with Splunk Breakout Session
- 1. Copyright © 2014Splunk Inc. Getting Started Philipp Drieger Sales Engineer
- 2. During the courseof this presentation, we may make forward-looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward- looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward-looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk, Splunk>, Splunk Storm, Listen to Your Data, SPL and The Engine for Machine Data are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. ©2013 Splunk Inc. All rights reserved. Legal Notices 2
- 3. What is Splunk? GettingStarted with Splunk Search Alert Dashboard Deployment and Integration Community Help & Questions AGENDA
- 4. Spelunking: Splunking: to explore underground caves toexplore machine data 4
- 5. Log files Custom applications Webservers User clickstreams Social platforms Servers/hypervisors/virtual machines Configurations Telecom devices Storage devices Network devices Security devices, firewalls, IDS Databases Web services System metrics GPS DNS, DHCP AAA logs Proxy servers Errors Scripts Sensors What is MachineData?
- 6.
- 7. What Does SplunkReally Do? Into thisIt turns this [Thu Sep 24 14:57:33 2009] [error] [client 10.2.1.44] ap_proxy: trying GET /petstore/ enter_order_information.screen at backend host '127.0.0.1/7001; got exception 'CONNECTION_REFUSED [os error=0, line 1739 of ../nsapi/URL.cpp]: Error connecting to host 127.0.0.1:7001', referer: http://10.2.1.223/petstore/cart.do?action= purchase&itemId=EST-14
- 8. Industry Leading PlatformFor Machine Data Machine Data: Any Location, Type, Volume Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom Applications Messaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On- Premises Private Cloud Public Cloud Platform Support (Apps / API / SDKs) Enterprise Scalability Universal Indexing Answer Any Question Developer Platform Report and analyze Custom dashboards Monitor and alert Ad hoc search SCADA Automation and Control Systems 8
- 9. IT Operations Security and Compliance Mobile Intelligence App Dev and AppMgmt. Developer Platform (REST API, SDKs) Business Analytics Industrial Data and Internet of Things Small Data. Big Data. Huge Data. SplunkDeliversValueAcrossIT andtheBusiness
- 10. Getting Started In thisroom: 13:15 – 14:15 Getting Started with Splunk 14:15 – 15:15 Splunk for Security
- 11. InstallSplunk Splunk Home • WIN:Program FilesSplunk • Other: /opt/splunk (Applications/splunk) Start Splunk • WIN: Program FilesSplunkbinsplunk.exe start (services start) • *NIX: /opt/splunk/bin/splunk start www.splunk.com/download • 32 or 64 Bit? • Indexer or Universal Forwarder?
- 12.
- 13. Splunk Licenses Free DownloadLimits Indexing to 500MB/day • Enterprise Trial License expires after 60 days • Reverts to Free License Features Disabled in Free License • Multiple user accounts and role-based access controls • Distributed search • Forwarding to non-Splunk Instances • Deployment management • Scheduled saved searches and alerting • Summary indexing Other License Types • Enterprise, Forwarder, Trial
- 14. Default installation on:http://localhost:8000 Splunk Web Basics Browser Support • Firefox 10.x and latest • Internet Explorer 7, 8, 9 and 10 • Safari (latest) • Chrome (latest) Index data • Add data • Getting Started App • Install an App (Splunk for Windows, *NIX)
- 15. Add some data Downloadthesamplefile,followthislinkandsavethefiletoyourdesktop,then unzip:http://www.splunkbook.com(UsingSplunkBook) ToaddthefiletoSplunk: –FromtheWelcomescreen,clickAddData. – ClickFromfilesanddirectoriesonthebottomhalfofthescreen. – SelectSkippreview. – ClicktheradiobuttonnexttoUploadandindexafile. – ClickSave.
- 16. Best PracticeSuggestion: CreateanindividualIndexbasedon sourcetype. • Easiertore-indexdataifyoumakeamistake. •Easiertoremovedata. • Easiertodefinepermissionsanddataretention.
- 17.
- 18.
- 19. Search app –Summary view current view global stats app navigation time range picker Selecting Data Summary: • Host • Source • Sourcetype start search search box
- 20. Searching Search > * SelectTime Range • Historical, custom, or real-time Select Mode • Smart, Fast, Verbose Using the timeline • Click events and zoom in and out • Click and drag over events for a specific range
- 21. Everything is searchable Everythingis searchable • * wildcards supported • Search terms are case insensitive • Booleans AND, OR, NOT – Booleans must be uppercase – Implied AND between terms – Use () for complex searches • Quote phrases fail* fail* nfs error OR 404 error OR failed OR (sourcetype=access_*(500 OR 503)) "login failure"
- 22. Search Assistant Contextual Help -advanced type-ahead History - search - commands Search Reference - short/long description - examples suggests search terms updates as you type shows examples and help toggle off / on
- 23. Searches can bemanaged as asynchronous processes Jobs can be • Scheduled • Moved to background tasks • Paused, stopped, resumed, finalized • Managed • Archived • Cancelled Job Management Modify Job Settings pause finalize delete
- 24. Search Commands Search >error | head 1 Search results are “piped” to the command Commands for: • Manipulating fields • Formatting • Handling results • Reporting
- 25. Over 130 Commands! splunk.com> Documentation > Search Reference abstract accum addcoltotals addinfo addtotals af analyzefields anomalies anomalousvalue append appendcols ar associate audit autoregress bin bucket chart cluster collect common contingency convert correlate counttable crawl ctable dbinspect dedup delete delta diff discretize erex eval eventcount eventstats excerpt extract file fillnull folderize format gentimes head highlight iconify input inputcsv inputlookup iplocation join kmeans kv kvform loadjob localize localop lookup macro makecontinuous makemv maketable map metadata multikv mvcombine mvexpand nomv outlier outlierfilter outputcsv outputlookup outputtext overlap rangemap rare regex relevancy rename replace reverse run savedsearch savedsplunk script scrub selfjoin sendemail set sichart sirare sistats sitimechart sitop slc stash strcat streamstats sumindex summaryindex tail test timechart top transaction transam trendline typeahead typelearner typer uniq untable xmlkv xmlunescape xpath xyseries http://www.splunk.com/base/Documentation/latest/SearchReference/SearchCheatsheet
- 26.
- 27.
- 28. Fields Default fields • host,source, sourcetype, linecount, etc. • View on left panel in search results or all in field picker Where do fields come from? • Pre-defined by sourcetypes • Automatically extracted key-value pairs • User defined
- 29. Sources,Sourcetypes,Hosts • Host - hostname,IP address, or name of the network host from which the events originated • Source - the name of the file, stream, or other input • Sourcetype - a specific data type or data format
- 30. Tagging and EventTyping Eventtypes for more human-readable reports • to categorize and make sense of mountains of data • punctuation helps find events with similar patterns Search > eventtype=failed_login instead of Search > “failed login” OR “FAILED LOGIN” OR “Authentication failure” OR “Failed to ………………authenticate user” Tags are labels • apply ad-hoc knowledge • create logical divisions or groups • tag hosts, sources, fields, even eventtypes Search > tag=web_servers instead of Search > host=“apache1.splunk.com” OR host=“apache2.splunk.com” OR …………….host=“apache3.splunk.com”
- 31. Extract Fields Interactive FieldExtractor • generate PCRE • editable regex • preview/save
- 32.
- 33. Saved Search &Alert Basics
- 34. Saved Searches Leverage Searchesfor future Insights! • Reports • Dashboards • Alerts • Eventtypes Add a Time Range Picker • Preset • Relative • Real-time • Date-Range • Date & Time Range • Advanced
- 35. Create Alerts Scheduled orReal-Time • Define Time Ranges • Conditions • Thresholds
- 36. AlertingContinued… Searches run ona schedule and fire an alert • Example: Run a search for “Failed password” every 15 min over the last 15 min and alert if the number of events is greater than 10 Searches are running in real-time and fire an alert • Example: Run a search for “Failed password user=john.doe” in a 1 minute window and alert if an event is found
- 37. AlertingActions • Send email •RSS • Execute a script • Track Alert Details
- 38.
- 39.
- 40. Reporting results of anysearch Define your Search and set your time range, accelerate you search and more Choose the type of chart (line, area, column, etc) and other formatting options Build reports from
- 41. ReportingExamples • Use wizardor reporting commands (timechart, top, etc) • Build real-time reports with real-time searches • Save reports for use on dashboards
- 42. Dashboards Create dashboards fromsearch results
- 43.
- 44.
- 45.
- 46. Splunk Has FourPrimaryFunctions SearchingandReporting(SearchHead) IndexingandSearchServices(Indexer) LocalandDistributedManagement(DeploymentServer) DataCollectionandForwarding(Forwarder) A Splunk install can be one or all roles…
- 47. IngestsDataFromHeterogeneousDataSources Agent-lessandAgentApproachforFlexibilityandOptimization perf shell API Mounted File Systems hostnamemount syslog TCP/UDP EventLogs Performance Active Directory syslog hosts and network devices Unix, Linux and Windows hosts Local File Monitoring Splunk Forwarder virtual host Windows Scripted or Modular Inputs shell scripts API subscriptions Mainframes*nix Wire Data Splunk App for Stream
- 48. Understandingthe UniversalForwarder Forward datawithout negatively impacting production performance. Scripts Universal Forwarder Deployment Logs ConfigurationsMessages Metrics Central Deployment Management Monitor files, changes and the system registry; capture metrics and status. Universal Forwarder Regular (Heavy) Forwarder Monitor All Supported Inputs ✔ ✔ Routing, Filtering, Cloning ✔ ✔ Splunk Web ✔ Python Libraries ✔ Event Based Routing ✔ Scripted Inputs ✔
- 49. Horizontal Scaling Load balancedsearch and indexing for massive, linear scale out. Forwarder Auto Load Balancing Distributed Search
- 50. Multiple Datacenters Headquarters London HongKong Tokyo New York Distributed Search Index and store locally. Distribute searches to datacenters, networks & geographies.
- 51. HighAvailability,OnCommodityServersandStorage As Splunk collectsdata, it keeps multiple identical copies If indexer fails, incoming data continues to get indexed Indexed data continues to be searchable Easy setup and administration Data integrity and resilience without a SAN Index Replication Splunk Universal Forwarder Pool Constant Uptime
- 52. Service Desk SIEM Send Datato Other Systems Route raw data in real time or send alerts based on searches. Event Console
- 53. Integrate External Data LDAP,AD Watch Lists CRM/ER P CMDB Correlate IP addresses with locations, accounts with regions Extend search with lookups to external data sources.
- 54. Integrate Usersand Roles ProblemInvestigation Problem Investigation Problem Investigation Save Searches Share Searches LDAP, AD Users and Groups Splunk Flexible Roles Manage Users Manage Indexes Capabilities& Filters NOT tag=PCI App=ERP … Map LDAP & AD groups to flexible Splunk roles. Define any search as a filter. Integrate authentication with LDAP and Active Directory.
- 55. PowerfulDeveloperPlatform REST API Build SplunkApps Extend and Integrate Splunk Simple XML JavaScript HTML5 Web Framework Java JavaScript Python Ruby C# PHP Data Models Search Extensibility Modular Inputs SDKs
- 56.
- 57. SupportThroughthe Splunk Community Browseand share Apps from Splunk, Partners and the Community apps.splunk.com Community-driven knowledge exchange and Q&A answers.splunk.com 2015 -> more than 140 sessions conf.splunk.com .conf2015
- 58. Where to Gofor Help Documentation – http://www.splunk.com/base/Documentation TechnicalSupport – http://www.splunk.com/support Videos – http://www.splunk.com/videos Education – http://www.splunk.com/goto/education Community – http://answers.splunk.com – http://apps.splunk.com • SplunkBook – http://splunkbook.com
- 59. Thank You –Q&A
Editor's Notes
- #2 Welcome to SplunkLive [City]. Thank you for taking the time to attend today’s event.
- #22 1
- #31 Event types can help you automatically identify events based on a search. An event type is a field based on a search, it’s a way of classifying data for searching and reporting and it’s useful for user knowledge capture and sharing. Tags are different, in that they allow you to search for events with related field values. You can assign any field/value combination. So as an example, server names aren’t always helpful. Sometimes they contain ambiguous information. Using tags you can use a more meaningful term. The Splunk Manager allows you to enable/disable, copy, delete and edit tags that you’ve created.
- #32 Extracting fields that aren’t already pulled out at search time is a necessary step to doing more with your data like reporting. Show example of field extraction with IFX and an example using rex. Show other field extractor.
- #36 Real-time alerts always trigger immediately for every returned result Real-time monitored alerts monitor a real-time window and can trigger immediately, or you can define conditions Scheduled alerts run a search on a regular interval that you define and triggers based on conditions that you define
- #37 Run alert in Splunk. Splunk alerts are based on searches and can run either on a regular scheduled interval or in real-time. Alerts are triggered when the results of the search meet a specific condition that you define. Based on your needs, alerts can send emails, trigger scripts and write to RSS feeds.
- #38 Consider how you might use a scripted alert.
- #40 How can you leverage Splunk?
- #44 Show dashboard examples:
- #45 Why with the same settings is the shadow so dark?
- #59 Splunk deployments can grow to encompass thousands of Splunk instances, including forwarders, indexers, and search heads. Splunk offers a deployment monitor app that helps you to effectively manage medium- to large-scale deployments, keeping track of all your Splunk instances and providing early warning of unexpected or abnormal behavior. The deployment monitor provides chart-rich dashboards and drilldown pages that offer a wealth of information to help you monitor the health of your system. These are some of the things you can monitor: Index throughput over time Number of forwarders connecting to the indexer over time Indexer and forwarder abnormalities Details for individual forwarders and indexers, such as status and forwarding volume over time Source types being indexed by the system License usage
- #60 How can you leverage Splunk?
- #63 How can you leverage Splunk?