Downloaded 359 times
Uploaded bySopheaktra YONG
Feedforward neural network
The document discusses feedforward neural networks, specifically multilayer perceptrons (MLPs), explaining their structure, functioning, and the various types of activation functions used. It covers the perceptron model, its limitations with non-linearly separable data such as the XOR problem, and methods to prevent overfitting in MLPs. Additionally, it touches on techniques for adjusting weights and the backpropagation algorithm to improve learning accuracy.
In this document
Powered by AI
Introduction to feedforward neural networks, including multilayer perceptrons and key concepts in artificial neural networks.
Explanation of artificial neural networks (ANN), their biological inspiration, structure, and neuron functionalities.
Definition and functioning of a perceptron, modeling neurons for binary classification using weighted inputs.
Role of weights and biases in neural networks, and various activation functions such as Unit Step, Sigmoid, and Gaussian for output determination.
Importance of learning rate in training, weight updates, and the step-by-step process for training perceptrons.
Challenges faced by single-layer perceptrons with non-linearly separable data, exemplified by the XOR problem.
Structure and purpose of multi-layer perceptrons, including the stacking of logistic regression models for classification or regression tasks.
Description of the backpropagation process for error adjustment and weight modification across multiple neural network layers.
Overview of CNNs, designed for image pattern recognition, highlighting the significance of feature extraction.
Discussion on overfitting issues in MLPs, methods to mitigate it including regularization techniques and early stopping.
Resources and links for further learning in machine learning and neural networks.
More Related Content
Similar to Feedforward neural network
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Feedforward neural network
- 1. YONG Sopheaktra M1 Yoshikawa-Ma Laboratory 2015/07/26 Feedforwardneural networks 1 (multilayer perceptrons)
- 2. Kyoto University • ArtificialNeural Network • Perceptron Algorithm • Multi-layer perceptron (MLP) • Overfitting & Regularization Content 2
- 3. Kyoto University • AnArtificial Neural Network (ANN) is a system that is based on biological neural network (brain). ▫ The brain has approximately 100 billion neurons, which communicate through electro-chemical signals ▫ Each neuron receives thousands of connections (signals) ▫ If the resulting sum of signals surpasses certain threshold, the response is sent • The ANN attempts to recreate the computational mirror of the biological neural network … Artificial Neural Network 3
- 4.
- 5. Kyoto University What isPerceptron? 5 • A perceptron models a neuron • It receives n inputs (feature vector) • It sum those inputs , calculated, then output • Used for linear or binary classification
- 6. Kyoto University 6 Perceptron •The perceptron consists of weights, the summation processor, and an activation function • A perceptron takes a weighted sum of inputs and outputs:
- 7. Kyoto University Weight &Bias 7 • Bias can also be treated as another input ▫ The bias allow to shift the line • The weights determine the slope
- 8. Kyoto University Transfer orActivation Functions 8 • The transfer function translate the input signals to output signals • It uses a threshold to produce an output • Some examples are ▫ Unit Step (threshold) ▫ Sigmoid (logistic regression) ▫ Piecewise linear ▫ Gaussian
- 9. Kyoto University 9 UnitStep (Threshold) • The output is set depending on whether the total input is greater or less than some threshold value.
- 10. Kyoto University 10 PiecewiseLinear • The output is proportional to the total weighted output.
- 11. Kyoto University 11 Sigmoidfunction • It is used when the output is expected to be a positive number ▫ It generates outputs between 0 and 1
- 12. Kyoto University 12 Gaussian •Gaussian functions are bell-shaped curves that are continuous • It is used in radial basis function ANN (RBF kernel – Chapter 14) ▫ Output is real value
- 13. Kyoto University 13 Thelearning rate • To update the weights and bias to get smaller error • Help us control how much we change the weight and bias
- 14. Kyoto University 14 Howthe algorithm work? • Initialize the weights (zero or small random value) • Pick a learning rate (0 – 1) • For each training set • Compute the activation output ▫ Adjusting Error = differences between predicted and actual Update bias and weight • Repeating till the error is very small or zero • If the it is linear separable, we will found solution
- 15.
- 16. Kyoto University 16 Whatif the data is non-linearly separable? • Because SLP is a linear classifier and if the data are not linearly separable, the learning process will never find the solution • For example: XOR problem
- 17.
- 18. Kyoto University 18 XORClassification (Xor_classification.zip)
- 19. Kyoto University 19 •A series of logistic regression models stacked on top of each other, with the final layer being either another logistic regression or a linear regression model, depending on whether we are solving a classification or regression problem. Multi-layer perceptron (MLP)
- 20.
- 21. Kyoto University 21 Acloser look
- 22.
- 23. Kyoto University 23 •Use output error, to adjust the weights of inputs at the output layer • Calculate the error at the previous layer and use it to adjust the weights • Repeat this process of back-propagating errors through any number of layers • You may find mathematical equation of how to minimize cost function of neural network at 16.5.4 The backpropagation algorithm The Back Propagation Algorithm
- 24. Kyoto University 24 Convolutionalneural networks http://yann.lecun.com/exdb/lenet/index.html • Designed to recognize visual patterns directly from pixel images with minimal preprocessing. • The purpose of multiple hidden units are used to learn non-linear combination of the original inputs (feature extraction) ▫ Individual Informative ▫ Each pixel in an image is not very informative ▫ But the combination will tell
- 25.
- 26.
- 27.
- 28.
- 29. Kyoto University 29 Crossvalidation error
- 30. Kyoto University 30 •Simplifier the parameters/features ▫ Remove some unnecessary features • Regularization ▫ Adjusting the weight How to address it?
- 31. Kyoto University 31 •The MLP can overfit, esp. if the number of nodes is large • A simple way to prevent is early stopping ▫ Stopping the training procedure when the error on the validation set first start to increase • Techniques are ▫ Consistent Gaussian prior ▫ Weight pruning: smaller the parameters value ▫ Soft weight sharing: group of parameters value have similar value ▫ Semi-supervised embedding: used with deep learning NN ▫ Bayesian Inference Determine number of hidden units – faster than cross-validation Regularization
- 32.
- 33. Kyoto University • https://www.coursera.org/learn/machine-learning •https://www.youtube.com/playlist?list=PLea0WJq13cnCS4LLMeUuZmTx qsqlhwUoe • http://yann.lecun.com/exdb/lenet/index.html Reference 33
Editor's Notes
- #5 Dendrite: input Axon: output