Demystifying Machine Learning: An In-Depth Exploration

• Introduction

Today I'm going to try to answer the question, "What is machine learning?"

Have you guys ever wondered how Facebook is able to perform facial recognition or how companies are able to bombard you with their products all over social media? All of this is because of machine learning.

• Real-Life Applications of Machine Learning

So in this session, I'll be taking you through some of the important concepts of machine learning. First, we'll talk about some of the real-life applications where machine learning is applied, some of the challenges people faced before machine learning came about, how things change for machine learning over time, why you should be interested in using machine learning, what exactly is machine learning, some of the types of machine learning, some algorithms of machine learning with some simple examples, some of the breakthroughs that machine learning brought about, and what a career in machine learning entails.

You all know that machine learning plays an integral role in our day-to-day lives. Now let's talk about some of the places where it's applied.

I'm sure you guys need no introduction to Netflix. Netflix is where we get our daily fix for shows and movies. Now, say you watch a show like The Witcher. Now, as soon as you're done, Netflix recommends a movie like The Last Kingdom. How is Netflix able to make such a recommendation?

Say you uploaded a photo onto Facebook. Now, as soon as this photo is uploaded, Facebook is able to tell who's in the photo. How's Facebook able to do that? Or how is Facebook able to suggest friends on the basis of the profiles you've gone through, your mutual friends, or the locations you've been to?

Say you just made an online transaction using PayPal. Now, how does PayPal know that the transaction you made is the legitimate one? As you know, there are millions of transactions taking place at the same time. How does PayPal differentiate between what is real and what isn't?

• Life Before Machine Learning

So now that you know where machine learning is applied, you might be wondering how life was before machine learning.

To help you understand, I'll take you through some of the problems people faced before machine learning. Before machine learning became so widely used, AI systems were mostly rule-based, which means that these rules failed in real-life scenarios. Most of their state representations had to be manually coded, and this manually coded was hand-coded. As everyone knows, hand-coding is quite difficult. Now, the main disadvantage was that these problem scenarios or test scenarios would fail outside what their roles were coded for. That is, they would not be very useful in real-life applications, even though they work pretty well inside their controlled environments.

• Evolution of Machine Learning

So it's clear that things were pretty grim before machine learning. So what changed? Let's take a trip down memory lane to see how machine learning evolved over time.

Can you think of a time when machine learning wasn't so much a part of our lives? 50 years ago, that would have been the case. That's not to say that machine learning didn't exist back then; it was just that it wasn't so important to them as it is to us now.

So things began to change in the 1950s when Alan Turing came up with the Turing machine. Now, the Turing test determined if a computer had real intelligence of its own. In 1997, Garry Kasparov, the world chess champion, was beaten by IBM Deep Blue, which was their AI. In 2010, Microsoft Kinect was released, which was an accessory of the gaming console Xbox 360. People could interact with computers using their gestures. In 2014, Facebook's DeepFace or face recognition system was introduced, which could determine who was present in the photo. In 2016, Google's AlphaGo, which is their AI, beat the world champion at the game of Go.

• Importance of Machine Learning

Now let's talk about why machine learning is so important and why you should be interested in it.

Machine learning has changed our lives in several significant ways. Let's talk about why.

It allows powerful processing, which means it can process far more complicated data. Hence, the decisions that we make will be much more well-founded, and the predictions are much more accurate. Not only does it allow powerful processing but also quicker processing, that is, more work is done in less time. The outputs that we obtain are much more accurate.

Now when you think of big data, you think of a lot of data. Now this large amount of data has to be stored somewhere, and when this is stored, it also has to be managed. So with machine learning, you can perform affordable data management. A machine learning method is also considerably less expensive.

The most important part is, as we get more and more data, the more and more complex it gets, and machine learning allows us to analyze complex big data.

• What Is Machine Learning?

So what exactly is machine learning? Machine learning is a method that allows computers to imitate and adapt to human-like behavior. So the machine analyzes past data, learns from that data, and makes decisions or predictions. The computer is training itself to perform the task the right way based on everything it can learn from the past data. The computer is creating its own logic and solutions, all of which without any form of human assistance or intervention. These systems learn, grow, change, and develop themselves when exposed to new data.

• Types of Machine Learning

Now that you know what machine learning is, let's talk about some of its types. Machine learning is divided into three major types: supervised learning, unsupervised learning, and reinforcement learning.

Ever imagine how difficult email handling would be if Gmail didn't know what was spam and what wasn't? Your inbox would be quite the mess, wouldn't it? But this is where the first type of machine learning comes in: supervised machine learning. Now, email clients like Microsoft Outlook and Gmail use spam filtering methods to ensure that the users are kept safe from spam.

These spam filters are regularly kept updated with the help of supervised machine learning. Supervised machine learning is a method where a model, after sufficient training, is able to make predictions for the future. In a way, a question-answer pattern is formed: a question and answer are given as input, the machine learns from it, and when new data is encountered, it's able to make a prediction.

Now let's take for example a set of data shown as a box of squares and rectangles. When new data, which is a square and a rectangle, is encountered, the machine is able to determine that the square goes in the box of squares and the rectangle goes in the box of rectangles.

Now let's talk about unsupervised machine learning. Imagine you just added a photo onto Facebook. Now, as soon as that photo is uploaded, there's a good chance that Facebook is able to identify who's in the photo and recommend this person to be tagged in the photo. Now, this is done with the help of unsupervised learning. Unsupervised learning only deals with input data. It ensures that the data is more readable and organized. It analyzes the input data to find out patterns, similarities, or anomalies in them. This model is able to learn from observations, hence it's able to find structures and relationships among the input data.

Now, let's come back to Facebook. How is it able to identify people? It is simply by taking your friends' photos as the data set. It then compares the faces and the features of the people present in your photo, hence it's able to identify who's in it.

The best way to illustrate this will be you having to input a collection of pencils, rulers, and erasers arranged in no particular order. So the machine learns from it, it's able to identify the pattern, and then organizes data. The output is still the collection of pencils, rulers, and erasers, but now in a much more organized manner. Netflix, Amazon, and several other e-commerce websites use the same method.

Now let's talk about the last type of machine learning: reinforcement learning. What do you observe? You see that supervised learning is about making a prediction, and unsupervised learning is about finding a hidden pattern. But reinforcement learning works completely differently. It allows a computer to make a decision based on past rewards for its actions. Now, this type of machine learning is usually only to increase the efficiency of a tool or a program.

It should be noted that there are three stages involved in reinforcement learning: there's the decision, there's the feedback, and the learning.

Now, let me explain that with an example of a game of chess. The machine makes a move, which is the decision. Then it gets to know whether the move was a good one or a bad one, which is the feedback. Then it learns from the feedback. The mobile PC games that people play are also made on the basis of reinforcement learning. That's the reason why when you open up a game and replay a mission, the same enemies you see before aren't in the same place.

• Machine Learning Algorithms

Now let's talk about some machine learning algorithms. The main objective of a machine learning algorithm is to solve a certain kind of problem. So to solve each of these problems, we have a particular algorithm. So, under supervised machine learning, we have the classification problem and the regression problem, and under unsupervised machine learning, we have the clustering and the association problems. So let's talk about each of these problems in detail.

A classification problem comes under supervised machine learning. So what we require in a classification problem is a ‘yes or no’ prediction. Questions like whether something belongs to a particular category or not, whether something is broken, and so on. It can help determine what category a given observation belongs to. In the simplest of terms, classification enables the machine to classify something into a given class.

A very important use case of classification is spam filtering. Now, let's talk about how that works. To a given system, we provide spam and not spam emails. We talk about millions of emails. Now, how do they determine whether a given mail is spam or not? It's on the basis of how often these mails are received, how many of them have words like "lottery," which are usually associated with spam, and how many of them have been flagged as spam by users.

Now, this is given to the computer, and it is learning from it. After it has learned, whenever new mail strikes the inbox, it is able to separate them as spam or not spam. Classification of this sort is done with the help of the Naïve Bayes algorithm.

Now let's talk about the regression problem that comes under supervised machine learning. In a regression problem, we try to predict a value based on past data. Now, you might argue that regression and classification sound pretty similar to each other, and you're right.

Classification is also determined based on past data. Now, the difference is that in the case of classification, we are trying to classify an observation into a given category. In the case of regression, we are trying to predict a value based on past data. Also, a relation is created between the different variables. Looking into weather prediction.

We have past data like temperature, wind speed, humidity, and so on. And based on that, we can determine how much rain was obtained. Now, this is given to the system, and it learns from it. When new data is encountered, like temperature, wind speed, and humidity, the system will be able to determine how much rain we'll get. One of the most important things that you need to remember is that the output that is given at the end of a regression problem is always quantitative in nature. Also, the output that we obtain is always based on past data.

Moving on, clustering problems come under unsupervised machine learning. Clustering is a method where a set of observations are divided into subsets. Each of these subsets are known as clusters. The observations inside these clusters are similar to one another based on some parameter or the other. Clustering has found use in several fields, such as e-shopping, healthcare, and financial services. Clustering is different from regression or classification, as both of them have some sort of prior knowledge associated with them. But in the case of clusters, all the knowledge is available from inside the cluster.

Now, let me explain with an example. Let's assume a network provider wanted to set up towers in a particular area. What they would be taking into consideration would be tower range, local geography, and population. Now, this is given into the system, the system learns from it, and then is able to divide the area into clusters. The clusters are divided on the basis of optimal locations where towers can be located to ensure maximum connectivity for all the users. A popular algorithm that is used in clustering is K-Means, which is used to cluster data into K clusters based on some similarity measures.

In the case of the association problem, we try to identify patterns of association between different variables and items. This is commonly used in something known as market basket analysis. This is a theory that says that if you buy a certain group of items, you're also likely to buy another group of items. Now, the same concept could be applied to the e-commerce industry. Say you buy a set of books. The system learns from it and is able to suggest a new set of books the next time you try to do some shopping. This is also used in the healthcare industry. Before a new drug is launched, it is checked to determine what kind of DNA it is sensitive to.

• Breakthroughs in Machine Learning

Machine learning has brought about some extraordinary changes in the last few years. Let's talk about some of them in detail.

Ever since the dawn of AI, scientists have tried to ensure that artificial intelligence systems try to function like a human brain. Now, DeepMind Technologies, which is a British company, is one with a similar aim. The company now works on detecting eye diseases using machine learning, where a digital scan of the eye is created to determine if it has any diseases or not. When it was acquired by Google in 2014, they created a computer network based on the structure of the human brain.

They also created a neural network that could play video games on its own. AlphaGo is an artificial intelligence system that was created by DeepMind Technologies on the basis of state-of-the-art machine learning techniques. AlphaGo defeated the world's best player at the complicated game of Go. Now, the latest version of AlphaGo, which is called AlphaGo Zero, only requires scientists to input the rules of the game for the system to be able to play on its own. This is different from the previous situations where each of the move patterns had to be input into 40 processors for the system to be able to play the game.

Can you imagine how comfortable our lives would be if cars could travel from one point to another without any sort of human intervention? Thanks to machine learning and artificial intelligence this dream is becoming closer to reality. A self-driving car is one that travels from one point to another without any sort of human intervention. The self-driving cars have several components. Some of the most important ones are these self-driving sensors. These detect curbs in other cars while parking, there are lidar sensors that monitor the edges of roads and lanes, there are cameras that detect traffic lights, pedestrians, and other things on the road, there's a radar that monitors the position of nearby vehicles.

As I mentioned before, let's talk about the future of machine learning. In the future, you'd have deeper personalization which means that you'd have more direct marketing towards your interest and advertisements based on your personal preference. Self-driving cars, although it's a concept that's being worked out right now, it's not one that's reached its completion. In the future, you'd have cars that can run flawlessly even on the most crowded roads.

Smarter investment opportunities. Machine learning could ensure that your profit is maximized, taking into consideration your past purchases and the current market scenario to suggest smarter investment opportunities. Better medical diagnosis. Although healthcare is a field where machine learning is creating an identity slowly but steadily, it's not reached the point where personalized solutions can be given to patients. In the future, machine learning could change that.

• Building a Career in Machine Learning

I am sure many of you are interested in building a career in machine learning. Now let me tell you about some of the career prospects. Someone skilled in machine learning fits the shoes of a machine learning engineer. A machine learning engineer aims to create artificial intelligence. The programs they create take decisions on their own without being programmed to perform a particular task. What are the skills of a machine learning engineer?

Mathematics: The concepts of mathematics help deal with uncertainty and making reliable predictions with the help of the tools that they provide. The fundamentals of statistics: Since most machine learning algorithms are built on statistical models, knowledge in this field is very important.

Software design: A strong background in APIs of various kinds like web APIs, static and dynamic libraries are also important. Applying machine learning libraries and algorithms: Understanding models, their learning procedures, how packages, APIs, and other things work, their potential issues, and how they apply to each technology is very important.

Data modeling: It is critical that the concepts of data modeling be understood well to ensure that the algorithms that are created are sound in nature.

Programming languages: One should be acquainted with programming languages like C++, Java, Lisp and most importantly python since there's a large amount of computations to be performed on huge sets of data. it is necessary that concepts such as algorithms, data structures, computer architecture, and so on are understood well.

• What are the responsibilities of a good machine learning engineer?

Machine learning engineers basically are to ensure a good data flow between the database and backend systems. Using programming languages to run machine learning experiments with machine learning libraries. To deploy machine learning solutions into production. Implementing custom machine learning code and performing feature engineering, which is nothing but performing an analysis over the data preprocessing and several other processes.

• Job Market and Trends in Machine Learning

Statistics show there's a high trend of job postings in the field of machine learning. Over the last few years there has been a spike in hiring skilled personnel indicating that there's a strong interest in machine learning. Evidently, machine learning provides greater opportunities in terms of the job they can provide.

Google Trends show an increased interest in topics related to machine learning. The machine learning market size is also expected to grow from 26.03 billion to 225.91 billion dollars by the year 2030. This indicates a bright and clear future for the individuals skilled in the concept of machine learning.

• Summary

Now let's go through everything you learned. First, we spoke about what machine learning is, which is a method where computers imitate and adapt to human-like behavior. These systems make decisions on their own based on past data. We spoke about the types of machine learning: supervised, unsupervised, and reinforcement learning. Supervised learning, where a model is able to predict outputs based on past data. Unsupervised learning, where we detect hidden patterns in input data, and reinforcement learning, where a system's next course of action is based on past feedback.

We then went into machine learning algorithms. We spoke about some of the problems that these algorithms solve. Under supervised learning, we have classification, where we determine a ‘yes or no’ prediction, regression, where we predict a value based on past data. Under unsupervised learning, we saw clustering, where observations are divided into clusters, and association, where we determine patterns of association between different items or variables.

We spoke about some of the breakthroughs that machine learning brought about, like DeepMind, AlphaGo, and cars that drive themselves. Then finally, we spoke about the skills a machine learning engineer needs, which are mathematics, the fundamentals of statistics, software design, applying machine learning libraries and algorithms, data modeling, and programming.

And that brings us to the end of this article. I hope it was as you expected.