Search

Now that the history and math lessons are out of the way, we must be fair and give you a dessert, so let’s now prepare for the upcoming dirty work by setting our environment.

In this chapter, we will describe a very important topic in deep learning fundamentals, the basic functions that deep learning is built on. Then we will try to build layers from these functions and combine these layers together to get a more complex model that will help us solve more complex problems, and all that will be described by TensorFlow examples.

We’re still on the subject of gradient descent. But let’s now talk about gradient optimization, because of its importance to gradient descent. It is an optimization method for finding the minimum of a function, and it’s important in deep learning. It works to update the weights of the neural network through backpropagation.

In the previous chapters, we studied fully connected multilayer neural networks and their training, using backpropagation. In a typical multilayer neural network layer, with n input nodes and m neurons, we need to learn n × m parameters or weights. While a multilayer neural network may perform well in some cases—in particular, for those where the features of different dimensions are independent—there are some additional properties in the connection architecture that we might desire. For example, if it is known that the dimensions of the input data are strongly correlated or that the size of multilayer neural networks (both the number of layers and the number of neurons in each layer) must be limited for computational considerations, should there be any architectural changes introduced to a standard multilayer neural network to accommodate this additional constraint about the data or the network complexity?

Feature selection and engineering are important steps in a machine learning pipeline and involves all the techniques adopted to reduce their dimensionality. Most of the time, these steps come after cleaning the dataset.

After finishing Part 2 of Deep Learning Pipeline, you are ready now to build your pipeline. You now see the whole picture, but it's fair to tell you that you are missing some parts in each approach, and we will fill the gaps by giving you some advanced concepts in both natural language processing and computer vision. Then we will give you some examples, using different types of datasets to ensure that you can apply a deep learning pipeline correctly and easily.

If you have ever tried to read a deep learning or even machine learning book, you will find that these books define machine learning (ML) as the science that teaches machines how to carry out tasks by themselves. That’s a simple idea if you think of it this way, but the complexity is in the details of this mysterious science; it’s within the black art of how these machines can act like humans.

In the previous chapter, we answered important questions you have to know before continuing reading the book, such as: what is TensorFlow?, what makes it very valuable like this?, and is TensorFlow easy to learn? All these questions come to your mind and many developers too, when you see a TensorFlow word, but yes, TensorFlow is a very valuable package in deep learning. However, there are many packages like it that are compatible with deep learning.

In Chapter 1, we saw an introduction to the deep learning field, to the theories that are the basis of the field. We also discussed the evolution of deep learning and the needed mathematical notation to succeed while reading the book, and we showed you how to install the environment needed for the projects in this book.

In the previous chapter, we showed you some advanced concepts in computer vision such as state-of-art architectures and the transfer learning approach. It is important for you to understand these concepts when you are about to build a model to do a certain task for you.

In the previous chapter, we defined what data means; we also discussed types and levels of data. So, we are now just getting into action with data! In this chapter, you’ll learn how to understand and clean your dataset.