Neural Networks

Neural Networks

• 1.
  NEURAL NETWORKS &Machine Learning Justin Chow Levon Mkrtchyan Eric Su Senior Project 5/16/07
• 2.
  What are NeuralNets? Refers to a computing paradigm that is modeled after the structure of the brain. Inspired by examination of the central nervous system and the neurons In Neuroscience, refers to physically collected neurons in our brains. Is a network because the function f(x) executed by a node is a composition of other functions, which are in turn defined as compositions of other function.
• 3.
  What is MachineLearning? Learning – Given a task to solve and a class of functions F, learning means using a set of observations to find an optimal solution that is an element of F Requires a cost function to determine how close we are to the optimal solution. Learning Paradigms Supervised learning Unsupervised learning Reinforcement learning Training employs many cutting edge mathematical theories The NN has a learning algorithm, which you train with thousands of examples.
• 4.
  Relations to A.I.Marvin Minsky, one of the founding fathers of A.I., built first neural network learning machine and wrote Perceptrons , foundational work of artificial neural networks. - Neural network is one of the main methods for developing computational intelligence. They often have very strong pattern recognition capabilities.
• 5.
  What is currentlybeing done? IBM is funding a four-year program called “systems neurocomputing” Developing neural networks to recognize patterns and avoid the “superposition catastrophe”. Is now using this research to recreate a person’s ability to perceive a broken line. Aston Martin, Daimler Chrysler, and other car companies are developing ANN models to detect cylinder misfires in engines. Georgia Tech introduced a neural network that combines living and robotic elements. uses neural networks of cultured rodent brain cells and robotic body Recent advances in VLSI circuits, optical computing, fuzzy logic, and protein-based computing have moved the field closer to realizing massively parallel hardware.
• 6.
  Learning Associative mapping– network learns to produce a particular pattern on the set of input units whenever another particular pattern is applied on the set of input units. Regularity detection - units learn to respond to particular properties of the input patterns.
• 7.
  How do theywork? The neuron Model biological neurons many inputs, one output have weights, a bias, and a threshold (activation) function The network architecture Three interconnected layers Input layer partitions processing (hidden) layer analyzes output layer … outputs programmer uses previous knowledge to ease training
• 8.
  How do theywork? Tissues networks like neural tissue output may be input to another network hidden layer may consist of a number of such tissues Training weight adjustments recognizing key part of input hard to see what the network “learned”
• 9.
  How do theywork? Firing rule – determines how one calculates whether a neuron should fire for an input Ex: take a collection of data, some which causes firing and some which don’t. If new data is inputted, elements most in common with firing data will then cause firing.
• 10.
  How do theywork? Feed forward architecture Signals traveling one way, from input to output (associates input with output) Feed backward architecture - Signals can travel both ways with loops. The state continually changes until equilibrium is reached
• 11.
  Successes Strengths ofneural networks: Pattern recognition Unclear algorithm No existing algorithm Large amounts of test data
• 12.
  Successes 20q Basedon a word game Learns from users Correct 80% of the time Image recognition Recognizing objects Categorizing images Rendering images searchable
• 13.
  Successes Signature analysisFirst large-scale use in US Compares with stored signatures 97% accuracy over old 83% old four-way classification more difficult Face recognition seeking to distinguish people takes 100-200 of training pictures per person average recognition rate of over 95% more training does not guarantee better recognition
• 14.
  Current Implementation Instantphysician Developed in 1980s, trained a NN to store a large amount of medical records. After training, could be presented with symptoms, and could then present the best diagnosis
• 15.
  Current implementations BusinessMarketing control of seat allocation on an airplane using feed-forward mechanism Credit models and mortgage screening boosted profitability of HNC by 27% Medicine NN used to model cardiovascular system. Build a NN of a patient and compare to actual patient. Can detect medical conditions before they happen.
• 16.
  Current implementations Userinterfaces Handwriting analysis tools, text-to-speech conversion (IBM, Babel) Industrial processes control machinery, adjust temperature settings, and diagnose malfunctions in robotic factories (Alyuda Research Factory)
• 17.
  Problems Encountered Applicationsusing neural networks have little or no data available for training on fault conditions, so fuzzy logic is used, based on an expert’s definition of certain rules. “ Neural network programs sometimes become unstable when applied to larger problems.” the larger the problem, the more neural networks must draw information from to obtain a solution, making neural networks very problem specific.
• 18.
  Problems Encountered Largerdatasets require more extensive training time to reach a predictive solution, and there is the possibility of overtraining, in which there was low training error but high actual testing error. Unknown data necessary for the solution will also cause a high error rate, sometimes by affecting the weighted values used in determining a solution.
• 19.
  Future Simple systemswhich have learned to recognize simple entities (e.g. walls looming, or simple commands like Go, or Stop) may have neural network chips implanted to help in decision-making. Japanese are already using fuzzy logic for this purpose. Use of neural networks to put labels on what is determined to be in the pictures, for use in medical searches User-specific systems for education and entertainment based on readings taken of the user.
• 20.
  Future Development ofintegration of man and machine, such as with retinal and cochlear implants Generally the development of use of neural networks in more everyday and diverse applications, such as in retail and manufacturing, to help make accurate decisions.
• 21.
  Questions?