Mohamed Loey, profile picture
Uploaded byMohamed Loey
PDF, PPTX661 views

Lecture 2: Artificial Neural Network

The document provides an overview of artificial neural networks (ANNs), detailing their structure and functionality, inspired by biological neurons. It discusses the perceptron model and its limitations, such as its inability to solve nonlinear problems like XOR, and presents multilayer perceptrons with backpropagation as a solution. Additionally, it outlines key concepts and methods related to neural networks, including forward propagation, backpropagation, and error minimization in training, while also mentioning a practical assignment related to neural network architectures.

Education◦
Related topics:
Deep Learning•

Embed presentation

Download as PDF, PPTX
Artificial Neural Network
 Artificial Neural Networks are computing systems inspired by the biological neural networks that is inspired by the way biological nervous systems, such as the brain, process information.
 Biological neuron has dendrites to receive signals, a cell body to process them, and an axon to send signals out to other neurons, the artificial neuron has a number of input channels, a processing stage, and one output that can fan out to multiple other artificial neurons.
 Rosenblatt proposed the earliest algorithm of artificial neural networks called Perceptron.
 The linear activation function determines the output of the unit.
x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1 x2 x1
 µ=0.3  w0=-.5, w1=1, w2=1  θ = 0 x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1 x2 x1
 µ=0.3  w0=-.5, w1=1, w2=1  θ = 0 θ(x)>0 =1 otherwise 0 x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1 x2 x1
 w0=-.5, w1=1, w2=1  net= w1x1+w2x2+w0x0 = x1+x2-0.5 x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
 net =w1x1+w2x2+w0x0 = x1+x2-0.5= 0+0-0.5=-0.5  yd= θ(net) = θ(-0.5)= 0  yd =ya correct x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
 net =w1x1+w2x2+w0x0 = x1+x2-0.5= 0+1-0.5 =+ 0.5  yd= θ(net) = θ(+0.5)= +1  yd != ya incorrect x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
 Update weight wi=wi - µ * xi  w0=-0.5-0.3*1=-0.8 & w1= 1-0.3*0=1 & w2=1-0.3*1=0.7 x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
 w0=-0.8 & w1= 1 & w2=0.7 x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+0.7x2-0.8 x2 x1
x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+0.7x2-0.8  net =w1x1+w2x2+w0x0 = x1+0.7x2-0.8= 1+0-0.8 =+0.2  yd= θ(net) = θ(+0.2)= +1  yd != ya incorrect x2 x1
x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+0.7x2-0.8  Update weight wi = wi - µ * xi  w0=-0.8-0.3*1=-1.1 & w1= 1-0.3*1=0.7 & w2=0.7-0.3*0=0.7 x2 x1
x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1  w0= -1.1 & w1= 0.7 & w2= 0.7 0.7x1+0.7x2-1.1 x1 x2
 Researchers discovered that Perceptron cannot approximate many nonlinear decision functions, for example, the XOR problem. x2 x1
 Researchers found a solution to that problem by stacking multiple layers of linear classifiers called multilayer perceptron to approximate nonlinear decision functions. Feedforward
 Werbos effectively solved the exclusive-or problem and more generally accelerated the training of multi-layer networks using Backpropagation algorithm. Backpropagation
 Forward propagation : is simply multiplying input with weights and add bias before applying activation function (sigmoid in here) at each node.  Backpropagation: is a method used in artificial neural networks to calculate the error contribution of each neuron after a batch of data Feedforward Backpropagation
Three concepts behind Backpropagation (From Calculus) 1) Derivative 2) Partial Derivative 3) Chain Rule
1)Derivative  =  ′ = 2
2)Partial Derivative  , = + 5  = 5  = 4 + 5
3)Chain Rule ℎ( ))′ = ( ( ))′ = ′( ( )) ∗ ′( )
 The linear and non linear activation function determines the output of the unit.
x1 x2 Net out Net out Net out Net out Input Layer Hidden Layer Output Layer k j i wkj wji
 Hidden Layer:  = ∑ ∗ ℎ  = x1 x2 Net out Net out Net out Net out k j i wkj wji
 Output Layer:  = ∑ ∗ ℎ ℎ  = x1 x2 Net out Net out Net out Net out k j i wkj wji
 Total Error:  = ∑ − ℎ ℎ x1 x2 Net out Net out Net out Net out k j i wkj wji t t
 Output Layer:  = + ∗ ∗ − ∗ ( − ) x1 x2 Net out Net out Net out Net out k j i wkj wji
 Hidden Layer:  = + ∗ ∗ − ∗  = ∑ − ∗ ( − ) ∗ x1 x2 Net out Net out Net out Net out k j i wkj wji
 Example i1 i2 i0 h1 h2 h0 o1 o2
i1 i2 i0 h1 h2 h0 o1 o2 w1 w3 w2 w4 w5 w7 w6 w8 w9 w10 w11 w12
 Assume = 0.5 t1 = 0.01 t2 = 0.99 .05 0.1 1 h1 h2 1 o1 o2 .15 .25 .20 .30 .40 .5 .45 .55 .35 .35 .60 .60
 Hidden Layers:  = ∗ + ∗ + ∗ = . ∗ . + . ∗ . + . ∗ = .  = = . = . .05 0.1 1 h1 h2 .15 .25 .20 .30 .35 .35
 Hidden Layers:  = ∗ + ∗ + ∗ = . ∗. + . ∗ . + . ∗ = .  = = . = . .05 0.1 1 h1 h2 .15 .25 .20 .30 .35 .35
 Output Layer:  = ∗ + ∗ + ∗ = . ∗ . + . ∗ . + . ∗ = .  = = . = . h1 h2 1 o1 o2 .40 .5 .45 .55 .60 .60
 Output Layer:  = ∗ + ∗ + ∗ = . ∗ . + . ∗ . + . ∗ = .  = = . = . h1 h2 1 o1 o2 .40 .5 .45 .55 .60 .60
 Total Error:  = ∑ −  = − = . − . = .  = − = . − . = .  = + = . + . = . o1 o2 t1 t2
 Our goal with back-propagation is to update each of the weights in the network so that they cause the actual output to be closer the target output by minimizing the error for each output neuron and the network as a whole.
 Output Layer  = + ∗ By applying the chain rule:  = ∗ ∗ h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Output Layer  = ∗ ∗  = ∗ + ∗ + ∗  = = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Output Layer  = ∗ ∗  =  = = ∗ = ( − ) = − = . − . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Output Layer  = ∗ ∗  = − + ( − )  = ∗ − ∗ − = −( − ) = −(. − . ) = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Output Layer  = ∗ ∗  = − ∗ ( − ) ∗ = . ∗ . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Update w5  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Update w6  = − ∗ ( − ) ∗ = . ∗ . ∗ . = .  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Update w7  = − ∗ ( − ) ∗ = . − . ∗ . ∗ − . ∗ . = .  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Update w8  = − ∗ ( − ) ∗ = . − . ∗ . ∗ − . ∗ . = .  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
 Hidden Layer  = + ∗ By applying the chain rule:  = ∗ ∗ i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
 Hidden Layer  = ∗ − ∗ ( − ) ∗ − ∗ + − ∗ − ∗  = 0.05 ∗ 0.5933 1 − 0.5933 ∗ [(0.7514(1 − 0.7514) ∗ .01 − 0.7514 ∗ 0.4) + (0.7729 1 − 0.7729 ∗ .99 − 0.7729 ∗ 0.5)]  = − . i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
 Hidden Layer  Update w1  = + = . − . ∗ . = . i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
 Hidden Layer  Update w2  = .  Update w3  = .  Update w4  = . i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
 MNIST is a large database of handwritten digits.  MNIST contains 60,000 training images and 10,000 testing images
Each student select one of neural network architecture from http://www.asimovinstitute.org/neural-network-zoo/ Download the paper that describe the network from Original Paper PDF Make two pages using word to summarize your selected neural network.
Use MS Word Send me e-mail to mloey@live.com with email subject “ Advanced Topics in CS2 – Task2 “ Put your Arabic name on word and email body Finally, press Send Deadline Next Lecture
 http://playground.tensorflow.org/
facebook.com/mloey mohamedloey@gmail.com twitter.com/mloey linkedin.com/in/mloey mloey@fci.bu.edu.eg mloey.github.io
www.YourCompany.com © 2020 Companyname PowerPoint Business Theme. All Rights Reserved. THANKS FOR YOUR TIME

More Related Content

PDF
Hands on machine learning with scikit-learn and tensor flow by ahmed yousry
byAhmed Yousry
 
PPTX
03 Single layer Perception Classifier
byTamer Ahmed Farrag, PhD
 
PDF
Deep Learning Frameworks slides
bySheamus McGovern
 
PPT
neural networks
byInstitute of Technology Telkom
 
PDF
Artificial Neural Networks Lect5: Multi-Layer Perceptron & Backpropagation
byMohammed Bennamoun
 
PPTX
Multi Layer Network
byInternational Islamic University
 
PDF
Introduction to Deep Learning, Keras, and TensorFlow
bySri Ambati
 
PDF
Lecture 5: Convolutional Neural Network Models
byMohamed Loey
 
Hands on machine learning with scikit-learn and tensor flow by ahmed yousry
byAhmed Yousry
 
03 Single layer Perception Classifier
byTamer Ahmed Farrag, PhD
 
Deep Learning Frameworks slides
bySheamus McGovern
 
neural networks
byInstitute of Technology Telkom
 
Artificial Neural Networks Lect5: Multi-Layer Perceptron & Backpropagation
byMohammed Bennamoun
 
Multi Layer Network
byInternational Islamic University
 
Introduction to Deep Learning, Keras, and TensorFlow
bySri Ambati
 
Lecture 5: Convolutional Neural Network Models
byMohamed Loey
 

What's hot

PDF
Dsp lab manual 15 11-2016
byGopinath.B.L Naidu
 
PPT
Adaline madaline
byNagarajan
 
PPTX
Reinforcement Learning : A Beginners Tutorial
byOmar Enayet
 
PPTX
Reinforcement Learning
bySalem-Kabbani
 
PPTX
Hopfield Networks
byKanchana Rani G
 
PPTX
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
bySimplilearn
 
PDF
Convolutional Neural Networks (CNN)
byGaurav Mittal
 
PPTX
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
PPT
Perceptron
byNagarajan
 
PDF
Deep learning - A Visual Introduction
byLukas Masuch
 
PPTX
Feedforward neural network
bySopheaktra YONG
 
PDF
Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorf...
byEdureka!
 
PDF
Recurrent neural networks rnn
byKuppusamy P
 
PPTX
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
bySimplilearn
 
PPTX
Regularization in deep learning
byKien Le
 
PPTX
An introduction to reinforcement learning
bySubrat Panda, PhD
 
PPTX
Autoencoders in Deep Learning
bymilad abbasi
 
PDF
Artificial Neural Network Lecture 6- Associative Memories & Discrete Hopfield...
byMohammed Bennamoun
 
PPTX
Activation functions
byPRATEEK SAHU
 
PDF
[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection
byTaegyun Jeon
 
Dsp lab manual 15 11-2016
byGopinath.B.L Naidu
 
Adaline madaline
byNagarajan
 
Reinforcement Learning : A Beginners Tutorial
byOmar Enayet
 
Reinforcement Learning
bySalem-Kabbani
 
Hopfield Networks
byKanchana Rani G
 
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
bySimplilearn
 
Convolutional Neural Networks (CNN)
byGaurav Mittal
 
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
Perceptron
byNagarajan
 
Deep learning - A Visual Introduction
byLukas Masuch
 
Feedforward neural network
bySopheaktra YONG
 
Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorf...
byEdureka!
 
Recurrent neural networks rnn
byKuppusamy P
 
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
bySimplilearn
 
Regularization in deep learning
byKien Le
 
An introduction to reinforcement learning
bySubrat Panda, PhD
 
Autoencoders in Deep Learning
bymilad abbasi
 
Artificial Neural Network Lecture 6- Associative Memories & Discrete Hopfield...
byMohammed Bennamoun
 
Activation functions
byPRATEEK SAHU
 
[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection
byTaegyun Jeon
 

Similar to Lecture 2: Artificial Neural Network

PPT
Neural networks,Single Layer Feed Forward
byRohiniRajaramPandian
 
PPTX
NNs5_6_MLhhbhbhhhbhjbhjhbjhhbhjP (1).pptx
bysyedtahaali4947
 
PPTX
Neural Networks
byMakerere Unversity School of Public Health, Victoria University
 
PPTX
10 Backpropagation Algorithm for Neural Networks (1).pptx
bySaifKhan703888
 
PDF
Nural Network ppt presentation which help about nural
bysayaleedeshmukh5
 
PPTX
Artificial Neural Networks ppt.pptx for final sem cse
byNaveenBhajantri1
 
PPTX
Deep neural networks & computational graphs
byRevanth Kumar
 
PPTX
Artificial neural networks
byShwethaShreeS
 
PPTX
Artificial neural network by arpit_sharma
byEr. Arpit Sharma
 
PDF
Machine Learning.pdf
bynikola_tesla1
 
PPT
Lecture2---Feed-Forward Neural Networks.ppt
byKassahunAwoke
 
PDF
Artificial Neural Networks
byStefano Dalla Palma
 
PPTX
Module1 (2).pptxvgybhunjimko,l.vgbyhnjmk;
byvallepubalaji66
 
PPTX
Classification using perceptron.pptx
bysomeyamohsen3
 
PPT
neural networking and factor analysis.ppt
byapsapssingh9
 
PPT
Artificial neural networks and deep learning.ppt
byjustjoking99yt
 
PPT
neural1Advanced Features of Neural Network.ppt
bydabeli2153
 
PPT
Data mining techniques power point presentation
byIDLEGamerz
 
PPT
neursgrdshdhfdhfdhdfhdfhdfhhfdal (3).ppt
bydineshkumarengg
 
PDF
Setting Artificial Neural Networks parameters
byMadhumita Tamhane
 
Neural networks,Single Layer Feed Forward
byRohiniRajaramPandian
 
NNs5_6_MLhhbhbhhhbhjbhjhbjhhbhjP (1).pptx
bysyedtahaali4947
 
Neural Networks
byMakerere Unversity School of Public Health, Victoria University
 
10 Backpropagation Algorithm for Neural Networks (1).pptx
bySaifKhan703888
 
Nural Network ppt presentation which help about nural
bysayaleedeshmukh5
 
Artificial Neural Networks ppt.pptx for final sem cse
byNaveenBhajantri1
 
Deep neural networks & computational graphs
byRevanth Kumar
 
Artificial neural networks
byShwethaShreeS
 
Artificial neural network by arpit_sharma
byEr. Arpit Sharma
 
Machine Learning.pdf
bynikola_tesla1
 
Lecture2---Feed-Forward Neural Networks.ppt
byKassahunAwoke
 
Artificial Neural Networks
byStefano Dalla Palma
 
Module1 (2).pptxvgybhunjimko,l.vgbyhnjmk;
byvallepubalaji66
 
Classification using perceptron.pptx
bysomeyamohsen3
 
neural networking and factor analysis.ppt
byapsapssingh9
 
Artificial neural networks and deep learning.ppt
byjustjoking99yt
 
neural1Advanced Features of Neural Network.ppt
bydabeli2153
 
Data mining techniques power point presentation
byIDLEGamerz
 
neursgrdshdhfdhfdhdfhdfhdfhhfdal (3).ppt
bydineshkumarengg
 
Setting Artificial Neural Networks parameters
byMadhumita Tamhane
 

More from Mohamed Loey

PDF
Lecture 6: Deep Learning Applications
byMohamed Loey
 
PDF
Lecture 4: Deep Learning Frameworks
byMohamed Loey
 
PDF
Lecture 4: How it Works: Convolutional Neural Networks
byMohamed Loey
 
PPTX
Lecture 3: Convolutional Neural Networks
byMohamed Loey
 
PDF
Lecture 1: Deep Learning for Computer Vision
byMohamed Loey
 
PDF
Design of an Intelligent System for Improving Classification of Cancer Diseases
byMohamed Loey
 
PDF
Computer Security - CCNA Security - Lecture 2
byMohamed Loey
 
PDF
Computer Security - CCNA Security - Lecture 1
byMohamed Loey
 
PDF
Algorithms Lecture 8: Pattern Algorithms
byMohamed Loey
 
PDF
Algorithms Lecture 7: Graph Algorithms
byMohamed Loey
 
PDF
Algorithms Lecture 6: Searching Algorithms
byMohamed Loey
 
PDF
Algorithms Lecture 5: Sorting Algorithms II
byMohamed Loey
 
PDF
Algorithms Lecture 4: Sorting Algorithms I
byMohamed Loey
 
PDF
Algorithms Lecture 3: Analysis of Algorithms II
byMohamed Loey
 
PDF
Algorithms Lecture 2: Analysis of Algorithms I
byMohamed Loey
 
PDF
Algorithms Lecture 1: Introduction to Algorithms
byMohamed Loey
 
PDF
Convolutional Neural Network Models - Deep Learning
byMohamed Loey
 
PDF
Deep Learning - Overview of my work II
byMohamed Loey
 
PDF
Computer Security Lecture 7: RSA
byMohamed Loey
 
PDF
Computer Security Lecture 5: Simplified Advanced Encryption Standard
byMohamed Loey
 
Lecture 6: Deep Learning Applications
byMohamed Loey
 
Lecture 4: Deep Learning Frameworks
byMohamed Loey
 
Lecture 4: How it Works: Convolutional Neural Networks
byMohamed Loey
 
Lecture 3: Convolutional Neural Networks
byMohamed Loey
 
Lecture 1: Deep Learning for Computer Vision
byMohamed Loey
 
Design of an Intelligent System for Improving Classification of Cancer Diseases
byMohamed Loey
 
Computer Security - CCNA Security - Lecture 2
byMohamed Loey
 
Computer Security - CCNA Security - Lecture 1
byMohamed Loey
 
Algorithms Lecture 8: Pattern Algorithms
byMohamed Loey
 
Algorithms Lecture 7: Graph Algorithms
byMohamed Loey
 
Algorithms Lecture 6: Searching Algorithms
byMohamed Loey
 
Algorithms Lecture 5: Sorting Algorithms II
byMohamed Loey
 
Algorithms Lecture 4: Sorting Algorithms I
byMohamed Loey
 
Algorithms Lecture 3: Analysis of Algorithms II
byMohamed Loey
 
Algorithms Lecture 2: Analysis of Algorithms I
byMohamed Loey
 
Algorithms Lecture 1: Introduction to Algorithms
byMohamed Loey
 
Convolutional Neural Network Models - Deep Learning
byMohamed Loey
 
Deep Learning - Overview of my work II
byMohamed Loey
 
Computer Security Lecture 7: RSA
byMohamed Loey
 
Computer Security Lecture 5: Simplified Advanced Encryption Standard
byMohamed Loey
 

Recently uploaded

PDF
Environmental Studies Module 4 BBA,BCCA Sem 1 NEP.pdf
byJayanti Pande
 
PDF
Advanced Process Flow and Micromachining Techniques for MEMS Fabrication: A D...
byGS Virdi
 
PPTX
Access partner report in Odoo 18.2 _ Odoo 19
byCeline George
 
PDF
The voice of the rain poem class 11th.pdf
byReeta Baral
 
PPTX
TAMIS & TEMS - HOW, WHY and THE STEPS IN PROCTOLOGY
byJohn Thanakumar
 
PPTX
GROWTH & DEVELOPMET MILESTONES (TODDLER).pptx
byAneetaSharma15
 
PDF
Madhumathi Endowment Quiz Finals by Sreejesh P S and Shyam Krishnan P - Keral...
bySreejesh P S
 
PDF
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
PPTX
UNIT 1: COMMUNICATION SKILLS, BARRIERS TO COMMUNICATION, PERSPECTIVES IN COMM...
byPOOJA KARVANDE
 
PDF
Cultivating Greatness Pune's Best Preschools and Schools.pdf
byWellington College
 
PPTX
B. Pharmacy Unit-3 (Fats and Oils Reaction and Analytical Constant)
bySandeshSul
 
PPTX
How to Access Revenue Report in Odoo 18 Events
byCeline George
 
PPTX
Isolation, Identification and Analysis of phytoconstituents.
byD. Y. Patil College of Pharmacy, Kolhapur.
 
PDF
International Men's Day Event: Breaking the Silence: Embracing Vulnerability ...
byAssociation for Project Management
 
PPTX
GROWTH & DEVELOPMENT MILESTONES (INFANTS).pptx
byAneetaSharma15
 
PDF
Comprehensive Principles, Design Techniques, and Applications of Electronic O...
byGS Virdi
 
PPTX
Multiple Linear Regression - Least Square Method X₁ on X₂ and X₃
bySundar B N
 
PPTX
How to Create a Spreadsheet in Odoo 18
byCeline George
 
PPTX
3 G8_Q3_L3_ (Cartoon as Representation in Opinion Editorial Article).pptx
byNorafeSahagun
 
PPTX
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 
Environmental Studies Module 4 BBA,BCCA Sem 1 NEP.pdf
byJayanti Pande
 
Advanced Process Flow and Micromachining Techniques for MEMS Fabrication: A D...
byGS Virdi
 
Access partner report in Odoo 18.2 _ Odoo 19
byCeline George
 
The voice of the rain poem class 11th.pdf
byReeta Baral
 
TAMIS & TEMS - HOW, WHY and THE STEPS IN PROCTOLOGY
byJohn Thanakumar
 
GROWTH & DEVELOPMET MILESTONES (TODDLER).pptx
byAneetaSharma15
 
Madhumathi Endowment Quiz Finals by Sreejesh P S and Shyam Krishnan P - Keral...
bySreejesh P S
 
M.Sc. Nonchordates Complete Syllabus PPT | All Important Topics Covered
byKNIPSS SULTANPUR
 
UNIT 1: COMMUNICATION SKILLS, BARRIERS TO COMMUNICATION, PERSPECTIVES IN COMM...
byPOOJA KARVANDE
 
Cultivating Greatness Pune's Best Preschools and Schools.pdf
byWellington College
 
B. Pharmacy Unit-3 (Fats and Oils Reaction and Analytical Constant)
bySandeshSul
 
How to Access Revenue Report in Odoo 18 Events
byCeline George
 
Isolation, Identification and Analysis of phytoconstituents.
byD. Y. Patil College of Pharmacy, Kolhapur.
 
International Men's Day Event: Breaking the Silence: Embracing Vulnerability ...
byAssociation for Project Management
 
GROWTH & DEVELOPMENT MILESTONES (INFANTS).pptx
byAneetaSharma15
 
Comprehensive Principles, Design Techniques, and Applications of Electronic O...
byGS Virdi
 
Multiple Linear Regression - Least Square Method X₁ on X₂ and X₃
bySundar B N
 
How to Create a Spreadsheet in Odoo 18
byCeline George
 
3 G8_Q3_L3_ (Cartoon as Representation in Opinion Editorial Article).pptx
byNorafeSahagun
 
How to Track a Link Using Odoo 18 SMS Marketing
byCeline George
 

Lecture 2: Artificial Neural Network

  • 1.
  • 2.
     Artificial NeuralNetworks are computing systems inspired by the biological neural networks that is inspired by the way biological nervous systems, such as the brain, process information.
  • 3.
     Biological neuronhas dendrites to receive signals, a cell body to process them, and an axon to send signals out to other neurons, the artificial neuron has a number of input channels, a processing stage, and one output that can fan out to multiple other artificial neurons.
  • 5.
     Rosenblatt proposedthe earliest algorithm of artificial neural networks called Perceptron.
  • 6.
     The linearactivation function determines the output of the unit.
  • 7.
    x1 x2 y 00 0 0 1 0 1 0 0 1 1 1 x2 x1
  • 8.
     µ=0.3  w0=-.5,w1=1, w2=1  θ = 0 x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1 x2 x1
  • 9.
     µ=0.3  w0=-.5,w1=1, w2=1  θ = 0 θ(x)>0 =1 otherwise 0 x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1 x2 x1
  • 10.
     w0=-.5, w1=1,w2=1  net= w1x1+w2x2+w0x0 = x1+x2-0.5 x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
  • 11.
     net =w1x1+w2x2+w0x0= x1+x2-0.5= 0+0-0.5=-0.5  yd= θ(net) = θ(-0.5)= 0  yd =ya correct x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
  • 12.
     net =w1x1+w2x2+w0x0= x1+x2-0.5= 0+1-0.5 =+ 0.5  yd= θ(net) = θ(+0.5)= +1  yd != ya incorrect x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
  • 13.
     Update weightwi=wi - µ * xi  w0=-0.5-0.3*1=-0.8 & w1= 1-0.3*0=1 & w2=1-0.3*1=0.7 x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+x2-0.5 x2 x1
  • 14.
     w0=-0.8 &w1= 1 & w2=0.7 x1 x2 ya 0 0 0 0 1 0 1 0 0 1 1 1 x1+0.7x2-0.8 x2 x1
  • 15.
    x1 x2 ya 00 0 0 1 0 1 0 0 1 1 1 x1+0.7x2-0.8  net =w1x1+w2x2+w0x0 = x1+0.7x2-0.8= 1+0-0.8 =+0.2  yd= θ(net) = θ(+0.2)= +1  yd != ya incorrect x2 x1
  • 16.
    x1 x2 ya 00 0 0 1 0 1 0 0 1 1 1 x1+0.7x2-0.8  Update weight wi = wi - µ * xi  w0=-0.8-0.3*1=-1.1 & w1= 1-0.3*1=0.7 & w2=0.7-0.3*0=0.7 x2 x1
  • 17.
    x1 x2 ya 00 0 0 1 0 1 0 0 1 1 1  w0= -1.1 & w1= 0.7 & w2= 0.7 0.7x1+0.7x2-1.1 x1 x2
  • 18.
     Researchers discoveredthat Perceptron cannot approximate many nonlinear decision functions, for example, the XOR problem. x2 x1
  • 19.
     Researchers founda solution to that problem by stacking multiple layers of linear classifiers called multilayer perceptron to approximate nonlinear decision functions. Feedforward
  • 20.
     Werbos effectivelysolved the exclusive-or problem and more generally accelerated the training of multi-layer networks using Backpropagation algorithm. Backpropagation
  • 21.
     Forward propagation: is simply multiplying input with weights and add bias before applying activation function (sigmoid in here) at each node.  Backpropagation: is a method used in artificial neural networks to calculate the error contribution of each neuron after a batch of data Feedforward Backpropagation
  • 22.
    Three concepts behindBackpropagation (From Calculus) 1) Derivative 2) Partial Derivative 3) Chain Rule
  • 23.
  • 24.
    2)Partial Derivative  ,= + 5  = 5  = 4 + 5
  • 25.
    3)Chain Rule ℎ( ))′= ( ( ))′ = ′( ( )) ∗ ′( )
  • 26.
     The linearand non linear activation function determines the output of the unit.
  • 28.
  • 29.
     Hidden Layer: = ∑ ∗ ℎ  = x1 x2 Net out Net out Net out Net out k j i wkj wji
  • 30.
     Output Layer: = ∑ ∗ ℎ ℎ  = x1 x2 Net out Net out Net out Net out k j i wkj wji
  • 31.
     Total Error: = ∑ − ℎ ℎ x1 x2 Net out Net out Net out Net out k j i wkj wji t t
  • 32.
     Output Layer: = + ∗ ∗ − ∗ ( − ) x1 x2 Net out Net out Net out Net out k j i wkj wji
  • 33.
     Hidden Layer: = + ∗ ∗ − ∗  = ∑ − ∗ ( − ) ∗ x1 x2 Net out Net out Net out Net out k j i wkj wji
  • 34.
  • 35.
  • 36.
     Assume = 0.5 t1= 0.01 t2 = 0.99 .05 0.1 1 h1 h2 1 o1 o2 .15 .25 .20 .30 .40 .5 .45 .55 .35 .35 .60 .60
  • 37.
     Hidden Layers: = ∗ + ∗ + ∗ = . ∗ . + . ∗ . + . ∗ = .  = = . = . .05 0.1 1 h1 h2 .15 .25 .20 .30 .35 .35
  • 38.
     Hidden Layers: = ∗ + ∗ + ∗ = . ∗. + . ∗ . + . ∗ = .  = = . = . .05 0.1 1 h1 h2 .15 .25 .20 .30 .35 .35
  • 39.
     Output Layer: = ∗ + ∗ + ∗ = . ∗ . + . ∗ . + . ∗ = .  = = . = . h1 h2 1 o1 o2 .40 .5 .45 .55 .60 .60
  • 40.
     Output Layer: = ∗ + ∗ + ∗ = . ∗ . + . ∗ . + . ∗ = .  = = . = . h1 h2 1 o1 o2 .40 .5 .45 .55 .60 .60
  • 41.
     Total Error: = ∑ −  = − = . − . = .  = − = . − . = .  = + = . + . = . o1 o2 t1 t2
  • 42.
     Our goalwith back-propagation is to update each of the weights in the network so that they cause the actual output to be closer the target output by minimizing the error for each output neuron and the network as a whole.
  • 43.
     Output Layer = + ∗ By applying the chain rule:  = ∗ ∗ h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 44.
     Output Layer = ∗ ∗  = ∗ + ∗ + ∗  = = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 45.
     Output Layer = ∗ ∗  =  = = ∗ = ( − ) = − = . − . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 46.
     Output Layer = ∗ ∗  = − + ( − )  = ∗ − ∗ − = −( − ) = −(. − . ) = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 47.
     Output Layer = ∗ ∗  = − ∗ ( − ) ∗ = . ∗ . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 48.
     Update w5 = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 49.
     Update w6 = − ∗ ( − ) ∗ = . ∗ . ∗ . = .  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 50.
     Update w7 = − ∗ ( − ) ∗ = . − . ∗ . ∗ − . ∗ . = .  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 51.
     Update w8 = − ∗ ( − ) ∗ = . − . ∗ . ∗ − . ∗ . = .  = + ∗ = . + . ∗ . = . h1 h2 h0 o1 o2 w5 w7 w6 w8 w11 w12 t1 t2
  • 52.
     Hidden Layer = + ∗ By applying the chain rule:  = ∗ ∗ i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
  • 53.
     Hidden Layer = ∗ − ∗ ( − ) ∗ − ∗ + − ∗ − ∗  = 0.05 ∗ 0.5933 1 − 0.5933 ∗ [(0.7514(1 − 0.7514) ∗ .01 − 0.7514 ∗ 0.4) + (0.7729 1 − 0.7729 ∗ .99 − 0.7729 ∗ 0.5)]  = − . i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
  • 54.
     Hidden Layer Update w1  = + = . − . ∗ . = . i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
  • 55.
     Hidden Layer Update w2  = .  Update w3  = .  Update w4  = . i1 i2 i0 h1 h2 w1 w3 w2 w4 w9 w10
  • 56.
     MNIST isa large database of handwritten digits.  MNIST contains 60,000 training images and 10,000 testing images
  • 69.
    Each student selectone of neural network architecture from http://www.asimovinstitute.org/neural-network-zoo/ Download the paper that describe the network from Original Paper PDF Make two pages using word to summarize your selected neural network.
  • 70.
    Use MS Word Sendme e-mail to [email protected] with email subject “ Advanced Topics in CS2 – Task2 “ Put your Arabic name on word and email body Finally, press Send Deadline Next Lecture
  • 71.
  • 72.
  • 73.
    www.YourCompany.com © 2020 CompanynamePowerPoint Business Theme. All Rights Reserved. THANKS FOR YOUR TIME