Practical Machine Learning and Rails Part1

Practical Machine Learning and Rails Part1

• 1.
  Practical Machine Learning andRails
• 2.
  Andrew Cantino VP Engineering, Mavenlink @tectonic Founder, Agile Productions @ryanstout
• 3.
  This talk will -introduce machine learning - make you ML-aware - have examples
• 4.
  This talk willnot - give you a PhD - implement algorithms - cover collaborative filtering, optimization, clustering, advanced statistics, genetic algorithms, classical AI, NLP, ...
• 5.
  What is MachineLearning? Many different algorithms that predict data from other data using applied statistics.
• 6.
  "Enhance and rotate20 degrees"
• 7.
  What data? The web is data. User decisions APIs A/B Tests Databases Logs Streams Browser versions Reviews Clicktrails
• 8.
  Okay. We havedata. What do we do with it? We classify it.
• 9.
  Classification
• 10.
  Classification OR
• 11.
  Classification :) OR :(
• 12.
  Classification • Documents o Sort email (Gmail's importance filter) o Route questions to appropriate expert (Aardvark) o Categorize reviews (Amazon) • Users o Expertise; interests; pro vs free; likelihood of paying; expected future karma • Events o Abnormal vs. normal
• 13.
  Algorithms: Decision Tree Learning
• 14.
  Algorithms: Decision Tree Learning Features Email contains word "viagra" no yes Email contains Email contains word "Ruby" attachment? no yes no yes P(Spam)=10% P(Spam)=5% P(Spam)=70% P(Spam)=95% Labels
• 15.
  Algorithms: Support Vector Machines (SVMs) Graphics from Wikipedia
• 16.
  Algorithms: Support Vector Machines (SVMs) Graphics from Wikipedia
• 17.
  Algorithms: Naive Bayes • Break documents into words and treat each word as an independent feature • Surprisingly effective on simple text and document classification • Works well when you have lots of data Graphics from Wikipedia
• 18.
  Algorithms: Naive Bayes You received 100 emails, 70 of which were spam. Word Spam with this word Ham with this word viagra 42 (60%) 1 (3.3%) ruby 7 (10%) 15 (50%) hello 35 (50%) 24 (80%) A new email contains hello and viagra. The probability that it is spam is: P(S|hello,viagra) = P(S) * P(hello,viagra|S) / P(hello,viagra) = 0.7 * (0.5 * 0.6) / (0.59 * 0.43) = 82% Graphics from Wikipedia
• 19.
  Algorithms: Neural Nets Hidden layer Input layer (features) Output layer (Classification) Graphics from Wikipedia
• 20.
  Curse of Dimensionality Themore features and labels that you have, the more data that you need. http://www.iro.umontreal.ca/~bengioy/yoshua_en/research_files/CurseDimensionality.jpg
• 21.
  Overfitting • With enough parameters, anything is possible. • We want our algorithms to generalize and infer, not memorize specific training examples. • Therefore, we test our algorithms on different data than we train them on.