The human brain produces intelligence through the connection of millions of neurons. Can machines also mimic the structure of the human brain to produce their own intelligence for learning and operation?
The term "Deep Learning" may seem difficult to many people, but most people have come into contact with Deep Learning without realizing it. For example, chatbots, autonomous driving systems, voice assistants, and facial recognition are all examples of Deep Learning being applied in real-life. These tools may not be controlled by humans, but they have the same sensitive response-ability as humans. Today's article will take you to explore the mysteries of Deep Learning, and how machines can mimic human behavior through their own methods for making judgments and decisions. Let's learn about this powerful force in 5 minutes!
What is Deep Learning?
Deep learning is actually a functional formula. We give it a bunch of data, and it analyzes it through the layers of the neural network to come up with various answers and find the best solution closest to the standard answer.
The Basic Concept of Deep Learning
Deep Learning is composed of multi-layered neural networks that are similar in structure to the human brain. The 'depth' in Deep Learning refers to the many hidden layers in the model, and each hidden layer is capable of performing complex operations. Artificial neural networks are composed of three layers: the input layer, the hidden layer, and the output layer. This is because when a model is identifying an object, it needs to go through many stages and layers to make a judgment.
- Input layer: The first layer of a neural network, responsible for transmitting data to other layers without performing any calculations on the data.
- Hidden layer: Each model has a different number of hidden layers depending on the difficulty, the more hidden layers there are, the more complex the calculation and the more time it takes. A hidden layer is composed of several neurons, and learning begins here. The model primarily judges the data through a series of hidden layers and then obtains answers based on the different weights of each layer.
- Output layer: receives data from the last hidden layer, which gets the desired response.
How does Deep Learning work?
The operating model of Deep Learning is divided into three steps: Forward Propagation, calculation of intermediate loss, and Back Propagation.
- Forward Propagation: Data is input into the Deep Learning model and after each layer, a numerical response is obtained.
- Calculation of intermediate loss: Compare the model's answer with the standard answer to determine how much the intermediate deviation(loss) is.
- Back Propagation: In order to reduce the difference between the model's predicted answer and the correct answer, the predicted answer is pushed back into the model to adjust and optimize the proportion of hidden layers that can be used to predict the answer.
According to the DNN/MLP model architecture mentioned in this article, increasing the number of hidden layers can help solve problems of higher complexity and difficulty because it allows the model to better fit the data. At this point, you may be wondering if more layers in a neural network always mean more accurate judgments by the model, or if every model should try to increase its hidden layers as much as possible? The following will provide some answers.
- Is it true that the more layers a neural network has, the more accurate it will be?
Not necessarily, due to the vanishing gradient problem.
- What is a gradient?
The update value and error value of the weights in a neural network should be proportional to each other. We call this phenomenon a gradient. In the model, the gradient is used to optimize the weight ratios for each layer of the model. The training process usually occurs during the Back Propagation process.
- Why does the vanishing gradient problem occur?
In some cases, the gradient value almost disappears, usually when the number of layers in the neural network is increased, and when combined with certain activation functions, such as sigmoid and tanh, this can cause the vanishing gradient problem. (* An activation function is applied to the output of each layer in a neural network. The activation function allows the network to understand complex and nonlinear functions. Without an activation function, the network can only perform basic linear calculations, limiting the information it can learn and affecting the accuracy of the model.)
- What happens when the vanishing gradient problem occurs?
The model updates its weights based on the gradient to make more accurate predictions. However, when the model goes through more layers of judgment, the gradient becomes smaller and may even disappear, preventing the model from effectively updating the weights in each layer based on the gradient. If the model is unable to update its weights based on the gradient, it can result in a significant deviation between the predicted values and the corrected responses.
- How do we determine the optimal number of layers for our model?
In general, having more hidden layers typically leads to higher accuracy on the training data, but having too many layers can cause the model to overfit, resulting in high accuracy on the training data but low accuracy on actual predictions. Therefore, when determining the number of layers for a model, it is common to experiment with different numbers of layers and use the one that produces the highest accuracy on a test dataset.
What are the practical applications?
Deep Learning is most commonly used in speech and image recognition because speech and images are complex in terms of recognition and judgment. To illustrate the actual processes behind Deep Learning, we will use a model of how Deep Learning judges the items in an image.
When it comes to judging items in an image, Deep Learning will read the pixels in the image and convert those pixels into meaningful information through filters, which is ultimately the answer we want from the image processing algorithm. If this explanation is too abstract, Deep Learning is about mimicking the way the human brain works. When the brain reads an image, it starts by judging the outermost lines of the image, then processes the shape of the image, and finally derives the meaning of the image. For example, when judging a picture of a cat and a dog, the model uses filters to process the data from the image and extracts the edge lines, object contours, and prominent parts of the object. Based on these extracted features, the model can reasonably determine whether the probability of an object being a dog is higher or the probability of it being a cat is higher.
Reference:
https://www.simplilearn.com/tutorials/deep-learning-tutorial/what-is-deep-learning https://www.techtarget.com/searchenterpriseai/definition/deep-learning-deep-neural-network https://panx.asia/archives/53209