How to Choose a Machine Learning Technique

Why are there so many machine learning techniques? The thing is that different algorithms solve various problems. The results that you get directly depend on the model you choose. That is why it is so important to know how to match a machine learning algorithm to a particular problem.

In this post, we are going to talk about just that. Let’s get started.

Variety of machine learning techniques

First of all, to choose an algorithm for your project, you need to know about what kinds of them exist. Let’s brush up your knowledge of different classifications.

Algorithms grouped by learning style

It’s possible to group the algorithms by their learning style.

Supervised learning

In the case of supervised learning, machines need a “teacher” who “educates” them. In this case, a machine learning specialist collects a set of data and labels it. Then, they need to communicate the training set and the rules to the machine. The next step is to watch how the machine manages to process the testing data. If there are some mistakes made, the programmer corrects them and repeats the action until the algorithm works accurately.

Unsupervised learning

This type of machine learning doesn’t require an educator. A computer is given a set of unlabeled data. It is supposed to find the patterns and come up with insights by itself. People can slightly guide the machine along the process by providing a set of labeled training data as well. In this case, it is called semi-supervised learning.

Reinforcement learning

Reinforcement learning happens in an environment where the computer needs to operate. The environment acts as the teacher providing the machine with positive or negative feedback that is called reinforcement.

Machine learning techniques grouped by problem type

Another way to divide the techniques into groups is based on the issues they solve.

In this section, we will talk about classification, regression, optimization, and other groups of algorithms. We are also going to have a look at their use in industry. For more detailed information about every common machine learning algorithm, check out our post about machine learning algorithm classification.

Common algorithms

Here are the most popular ML algorithms. Sometimes they belong to more than one group because they are effective at solving more than one problem.

• Logistic Regression,
• Linear Regression
• Decision Tree
• SVM
• Naive Bayes
• k-NN
• K-Means
• Neural networks
• Random Forest
• Dimensionality Reduction Algorithms
• Gradient Boosting algorithms

Classification

Classification helps us to deal with a wide range of problems. It allows us to make more informed decisions, sort out spam, predict whether the borrower will return the loan, or tag friends in a Facebook picture.

These algorithms predict discrete variable labels. A discrete variable has a countable number of possible values and can be classified. The accuracy of the prediction depends on the model that you choose.

Imagine that you develop an algorithm that predicts whether a person has or does not have cancer. In this case, the model that you choose should be very precise in predicting the result.

Typical classification algorithms are logistic regression, Naive Bayes, and SVM. More information about them and other algorithms you can find in our blog.

Clustering

Sometimes you need to divide things into categories but you don’t know what these categories are. Classification uses predefined classes to assign to objects. On the other hand, clustering allows to identify similarities between objects, and then group them according to the characteristics they have in common. This is the mechanics that lays behind detecting fraud, analyzing documents, grouping clients, and more. Clustering is widely used in sales and marketing for customer segmentation and personalized communication.

K-NN, k-means clustering, Decision trees, Random forest can all be used for clustering tasks.

Prediction

Trying to find out the relationship between two or more continuous variables is a typical regression task.

Note: If a variable can take on any value between its minimum value and its maximum value, it is called a continuous variable.

An example of such a task is predicting housing prices based on their size and location. The price of the house in this case is a continuous numerical variable.

Linear regression is the most common algorithm in this field. Multivariate regression algorithms, Ridge Regression, and LASSO regression are used when you need to model a relationship between more than two variables.

Optimization

Machine learning software enables you to provide a data-driven approach to continuous improvement in practically any field. You can apply product usage analytics in order to discover how the new product features affect demand. Sophisticated software equipped with empirical data helps to uncover ineffective measures, allowing you to avoid unsuccessful decisions.

For example, it is possible to use a heterarchical manufacturing control system in order to improve the capacity for a dynamic manufacturing system to adapt and self-manage. Machine learning techniques uncover the best behavior in various situations in real-time – which leads to continuous improvement of the system.

Gradient descent algorithms are generally used in ML to work with optimization.

Anomaly detection

Financial institutions lose about 5% of revenue each year to fraud. By building models based on historical transactions, social network information, and other sources of data, it is possible to spot anomalies before it’s too late. This helps detect and prevent fraudulent transactions in real-time, even for previously unknown types of fraud.

Typical anomaly detection algorithms are SVM, LOF, k-NN, k-means.

Ranking

You can apply machine learning to build ranking models. Machine learning ranking (MLR) usually involves the application of supervised, semi-supervised, or reinforcement algorithms. An example of a ranking task is search engine systems like SearchWiki by Google.

Examples of ranking algorithms are RankNet, RankBoost, RankSVM, and others.

Recommendation

Recommender systems offer valuable suggestions to users. This method brings utility to users and also benefits the companies because it motivates their clients to buy more or explore more content.

Items are ranked according to their relevance. The most relevant ones are displayed to the user. The relevancy is determined based on historical data. You know how it works if you’ve ever watched anything on Youtube or Netflix. The systems offer you similar videos to what you have already watched.

The main algorithms used for recommender systems are collaborative filtering algorithms and content-based systems.

How to choose machine learning techniques to solve your problem

How to find the best machine learning algorithm for your problem? There are three basic approaches you can use.

Task-based learning

Categorize your problem. It’s possible to categorize tasks by input and by output.

By input:

• If you have a set of labeled data or can prepare such a set, it is the domain of supervised learning.
• If you still need to define a structure, it’s an unsupervised learning problem.
• If you need the model to interact with an environment, you will apply a reinforcement learning algorithm.

By output:

• If the output of the model is a number, it’s a regression problem.
• If the output of the model is a class and the number of expected classes is known, it’s a classification problem.
• If the output of the model is a class but the number of expected classes is unknown, it’s a clustering problem.
• If you need to improve performance, it’s optimization.
• If you want a system to offer options based on the history of actions, it’s a recommendation problem.
• If you want to obtain insights from data, apply pattern recognition models.
• If you want to detect problems, use anomaly detection algorithms.

Understand your data

The process of choosing the algorithm isn’t limited to categorizing the problem. You also need to have a closer look at your data because it plays an important role in the selection of the right algorithm for the problem. Some algorithms function normally with smaller sample sets while others require a huge number of samples. Certain algorithms work with categorical data while others only work with numerical input.

Understanding your data demands certain steps:

• Processing. The components of data processing are pre-processing, profiling, cleansing, pulling together data from different internal and external sources.
• Feature engineering. You need to transform raw data into features that can represent the underlying problem to the predictive models. It helps to improve accuracy and get the desired results faster.

Choosing the algorithm is a comprehensive task that demands the analysis of a variety of factors.

Other things that might affect the choice of a model:

• Accuracy of the model;
• Interpretability of the model;
• Complexity of the model;
• Scalability of the model;
• Time it takes to build, train, and test the model;
• Time it takes to make predictions using the model;
• If the model meets your business goals.

Trial and error approach

Sometimes the problem is too complex and you do not know where to start. More than one model seems like a good fit, and it is difficult to predict which one will turn out the most effective. In this case, you can test a couple of models and assess them.

Set up a machine learning pipeline. It will compare the performance of each algorithm on the dataset based on your evaluation criteria. Another approach is to divide your data into subsets and use the same algorithm on different groups. The best solution for this is to do it once or have a service running that does this in intervals when new data is added.

Neural networks

Finally, the majority of tasks ML has to solve today can be solved with the help of neural networks. So, the final approach to choosing an ML model is just to always go for artificial neural networks.

However, these models are expensive and time-consuming to build, which is why other models still exist. Neural networks need extremely large databases in order to be accurate. Other types of ML techniques might not be as universal but solve assigned tasks effectively even when working with small datasets.

Moreover, they tend to overfit and are also hard to interpret – neural networks are basically black boxes, and researchers don’t know what’s happening inside.

So if you have a small budget, a small data sample, or aspire to get valuable insights that are easy to understand, NNs are not for you.

Final thoughts

Your results depend on whether you manage to select and build a successful ML model. If you have a machine learning project in mind and are looking for solutions, Serokell provides machine learning development services that suit your business goals. Contact us to learn how we can assist you with your project.