Downloaded 25 times
Uploaded byMohamed Loey
Lecture 3: Convolutional Neural Networks
The document discusses convolutional neural networks (CNNs), highlighting their structure and applications in image and speech recognition while outlining the importance of deep learning in machine learning. It details the various layers of CNNs, prominent architectures such as AlexNet and VGGNet, and the datasets used for training and evaluation. Additionally, it covers strategies to mitigate overfitting in CNNs and emphasizes the significance of deep learning for complex tasks in data science.
More Related Content
Similar to Lecture 3: Convolutional Neural Networks
Lecture 3: Convolutional Neural Networks
- 1.
- 2.
- 5. Part ofthe machine learning field of learning representations of data. hierarchy of multiple layers that mimic the neural networks of our brain If you provide the system tons of information, it begins to understand it and respond in useful ways.
- 7. SuperIntelligent Devices Best Solution for image recognition speech recognition natural language processing Big Data
- 9. Geoffrey Hinton: Universityof Toronto & Google Yann LeCun: New York University & Facebook Andrew Ng: Stanford & Baidu
- 11. Deep learning(DL) is a hierarchical structure network which through simulates the human brain’s structure to extract the internal and external input data’s features
- 12. Large data setwith good quality Measurable and describable goals Enough computing power Neural Network (Brain of Human)
- 13. Deep Neural Networks DeepBelief Networks Convolutional Neural Networks Deep Boltzmann Machines Deep Stacking Networks
- 14. Convolution NeuralNetworks (CNN) is supervised learning and a family of multi-layer neural networks particularly designed for use on two dimensional data, such as images and videos. A CNN consists of a number of layers: Convolutional layers. Pooling Layers. Fully-Connected Layers.
- 18. Convolutional layeracts as a feature extractor that extracts features of the inputs such as edges, corners , endpoints.
- 22. The poolinglayer reduces the resolution of the image that reduce the precision of the translation (shift and distortion) effect.
- 25. fully connectedlayer have full connections to all activations in the previous layer. Fully connect layer act as classifier.
- 27. LeNet :The firstsuccessful applications of CNN AlexNet: The ILSVRC 2012 winner ZFNet: The ILSVRC 2013 winner GoogLeNet: The ILSVRC 2014 winner VGGNet: The runner-up in ILSVRC 2014 ResNet: The winner of ILSVRC 2015
- 31. MNIST Handwritten digits– 60000 Training + 10000 Test Data Google House Numbers from street view - 600,000 digit images CIFAR-10 60000 32x32 colour images in 10 classes IMAGENET 1.2 million images, >150 GB Tiny Images 80 Million tiny images Flickr Data 100 Million Yahoo dataset
- 32. The ImageNet LargeScale Visual Recognition Challenge (ILSVRC) evaluates algorithms for object detection and image classification at large scale.
- 34. 26.2 15.3 14.8 7.3 6.7 3.6 0 5 10 15 20 25 30 Before2012 AlexNet 2012 ZFNet 2013 VGGNet 2014 GoogleNet 2014 ResNet 2015
- 35. MNIST isa large database of handwritten digits. MNIST contains 60,000 training images and 10,000 testing images
- 36. CNN onMNIST Dataset
- 37. CIFAR-10 datasetconsists of 60000 32x32 colour images in 10 classes CIFAR-10 contains 50000 training images and 10000 test images
- 38. CNN onCIFAR-10 Dataset
- 39. Overfitting Problem Larger network have a lots of weights this lead to high model complexity Network do excellent on training data but very bad on validation data
- 40. CNN Optimizationused to reduce the overfitting problem in CNN by: 1) Dropout 2) L2 Regularization 3) Mini-batch 4) Gradient descent algorithm 5) Early stopping 6) Data augmentation
- 41. Dropout isa technique of reducing overfitting in CNN.
- 42. L2 Regularization:Adding a regularization term for the weights to the loss function is a way to reduce overfitting. where w is the weight vector, λ is the regularization factor (coefficient), and the regularization function, Ω(w) is:
- 43. Mini-batch isto divide the dataset into small batches of examples, compute the gradient using a single batch, make an update, then move to the next batch.
- 44. The gradientdescent algorithm updates the coefficients (weights and biases) so as to minimize the error function by taking small steps in the direction of the negative gradient of the loss function where i stands for the iteration number, α > 0 is the learning rate, P is the parameter vector, and E(Pi) is the loss function.
- 45. Early stopping monitoringthe deep learning process of the network from overfitting. If there is no more improvement, or worse, the performance on the test set degrades, then the learning process is aborted
- 46. Data augmentationmeans increasing the number of dataset.
- 47. Today NVidia Supportmy work with NVIDIA TITAN X THE MOST ADVANCED GPU EVER BUILT
- 48. TITAN X Specifications GPUArchitecture Pascal Standard Memory Config 12 GB GDDR5X Memory Speed 10 Gbps Boost Clock 1531 MHz NVIDIA CUDA® Cores 3584 Transistors 12,000 million
- 49. TITAN X InResearch Deep Learning Augmented Reality Machine Learning Image Recognition Computer Vision Data Science
- 50. Deep learningis a class of machine learning algorithms. Harder problems such as video understanding, image understanding , natural language processing and Big data will be successfully tackled by deep learning algorithms.
- 51. Each student selectone of Benchmark Dataset from https://goo.gl/DNQmtj
- 52. Download thepaper that describe the method used by the authors. Make one page using word to summarize your selected paper.
- 53. Use MS Word Sendme e-mail to [email protected] with email subject “ Advanced Topics in CS2 – Task3 “ Put your Arabic name on word and email body Finally, press Send Deadline Next Lecture
- 54.
- 55. www.YourCompany.com © 2020 CompanynamePowerPoint Business Theme. All Rights Reserved. THANKS FOR YOUR TIME
Editor's Notes
- #52 http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html