Search

We introduce the basic concepts of deep learning. We use TensorFlow 2.x, the Google cloud service, and Google Drive interaction to make the concepts come alive with Python coding examples.

We introduce you to TensorFlow input pipelines with the tf.data API, which enables you to build complex input pipelines from simple, reusable pieces. Input pipelines are the lifeblood of any deep learning experiment because learning models expect data in a TensorFlow consumable form. It is very easy to create high-performance pipelines with the tf.data.Dataset abstraction (a component of the tf.data API) because it represents a sequence of elements from a dataset in a simple format.

In the previous chapter, we demonstrated how to work with TFDS objects. In this chapter, we work through two end-to-end deep learning experiments with large and complex TFDS objects. The Fashion-MNIST and beans datasets are small with simple images.

Feedforward neural nets are generally great for classification and regression problems. CNNs are great for complex image classification. But activations for feedforward nets and CNNs flow only in one direction, from the input layers to the output layer. Since signals flow in only one direction, feedforward and convolutional nets are not ideal if patterns in data change over time. So we need a different network architecture to work with data impacted by time.

We’ve already leveraged RNNs for NLP. In this chapter, we create experiments to forecast with time series data. We use the famous Weather dataset to demonstrate both a univariate and a multivariate example.

We guide you in the creation of augmented data experiments to increase the diversity of a training set by applying random (but realistic) transformations. Data augmentation is very useful for small datasets because deep learning models crave a lot of data to perform well.

Reinforcement learning (RL) is an area of machine learning that focuses on teaching intelligent agents how to take actions in an environment in order to maximize cumulative reward. Cumulative reward in RL is the sum of all rewards as a function of the number of training steps.

Autoencoders don’t typically work well with images unless they are very small. But convolutional and variational autoencoders work much better than feedforward dense ones with large color images.

With feedforward neural networks, we achieved good training performance with MNIST and Fashion-MNIST datasets. But images in these datasets are simple and centered within the input space that contains them. That is, they are centered within the pixel matrix that holds them. Input space is all the possible inputs to a model.

We work through a complete deep learning example with Python’s TensorFlow 2.x library in the Google Colab cloud service. We also demonstrate how to link your Google Drive with the Colab cloud service.

Transfer learning is the process of creating new learning models by fine-tuning previously trained neural networks. Instead of training a network from scratch, we download a pre-trained open source learning model and fine-tune it for our own purpose. A pre-trained model is one that is created by someone else to solve a similar problem. We can use one of these instead of building our own model. A big advantage is that a pre-trained model has been crafted by experts, so we can be confident that it performs at a high level (in most cases). Another advantage is that we don’t have to have a lot of data to use a pre-trained model.

Generative modeling is an unsupervised learning technique that involves automatically discovering and learning the regularities (or patterns) in input data so that a trained model can generate new examples that plausibly could have been drawn from the original dataset. A popular type of generative model is a generative adversarial network. Generative adversarial networks (GANs) are generative models that create new data instances that resemble the training data.

We introduce you to Tensor Processing Units with code examples. A Tensor Processing Unit (TPU) is an application-specific integrated circuit (ASIC) designed to accelerate ML workloads. The TPUs available in TensorFlow are custom-developed from the ground up by the Google Brain team based on its plethora of experience and leadership in the ML community. Google Brain is a deep learning artificial intelligence (AI) research team at Google who research ways to make machines intelligent for the improvement of people’s lives.

Now that we have practiced tuning low-dimensional (or simple) data, we are ready to experiment tuning high-dimensional (or complex) data sets. Low-dimensional data consists of a limited number of features, whereas high-dimensional data consists of a very high number of features.

We combine the Anaconda distribution with the popular Scikit-Learn library to demonstrate a wide range of supervised machine learning algorithms supplemented with unsupervised learning algorithms where appropriate. With clear examples, all written in Python, we demonstrate how these algorithms work to solve machine learning problems.

Classification from complex data is handled exactly as with simple data. Data is loaded into feature set X and target y. X data is composed of a matrix of vectors where each vector represents a data element and y data is composed of a vector of targets. However, complex data is composed of a high number of features (hundreds to thousands). Such a data set is commonly referred to as one with a high-dimensional feature space. Text data is also complex because each document must be converted into vectors of numerical values suitable for machine learning algorithms.