Dimensionality Reduction for Machine Learning

What is dimensionality reduction?

Dimension Reduction refers to the process of converting a set of data having vast dimensions into data with lesser dimensions ensuring that it conveys similar information concisely. These dimensionality reduction techniques are typically used while solving machine learning problems to obtain better features for a classification or regression task.

What are the different dimensionality reduction techniques?

There are multiple dimensionality reduction techniques for machine learning. We will cover 12 such techniques in this course:

1. Missing Value Ratio
2. Low Variance Filter
3. High Correlation Filter
4. Random Forest
5. Backward Feature Elimination
6. Forward Feature Selection
7. Factor Analysis
8. Principal Component Analysis (PCA)
9. Independent Component Analysis
10. Methods Based on Projections
11. t-Distributed Stochastic Neighbor Embedding (t-SNE)
12. UMAP

Why should I learn dimensionality reduction?

That’s a fair question! Here are a few key reasons why every machine learning professional should know dimensionality reduction:

• Space required to store the data is reduced as the number of dimensions comes down
• Less dimensions lead to less computation/training time
• Some algorithms do not perform well when we have large dimensions. So reducing these dimensions needs to happen for the algorithm to be useful
• Dimensionality reduction also takes care of multicollinearity by removing redundant features.
• We can visualize high dimensional data thanks to the various dimensionality reduction techniques such as t-SNE

We could go on, but you get the point! Dimensionality reduction is a crucial cog in the machine learning project lifecycle.

I already know what Principal Component Analysis (PCA) is. Do I really need to learn more dimensionality reduction techniques?

Absolutely. Principal Component Analysis (PCA) is a powerful dimensionality reduction technique but it does have its challenges. You should consider this course as a Siwss Army Knife in your dimensionality reduction skill set!

Learning the various dimensionality reduction techniques will help you become a better machine learning practitioner and expand your horizons when you’re working on different machine learning problems.

Do I need to have huge computational power to apply dimensionality reduction?

The question on computational power varies from the volume of data we have. For a dataset containing millions of records and thousands of dimensions one needs to have huge computational power to reduce the dimensions and make the data suitable for building models. 

Where as if the dataset contains only less records and dimensions the computational power required will be very less. It all depends on the volume of data and how quickly you want the result.

What are the different applications of dimensionality reduction?

There are a plethora of applications of dimensionality reduction. Here’s just 3 of them:

• If the dataset has too many missing values, we use dimensionality reduction techniques  to reduce the number of variables
• We can find the importance of each feature and keep the top most features, resulting in dimensionality reduction
• We can use dimensionality reduction to find highly correlated features and drop them accordingly

There are a whole lot more as you’ll see inside the course.

What kind of machine learning projects can I apply dimensionality reduction on?

You can apply dimensionality reduction techniques on any dataset with a ton of variables. That does not mean you should do it without considering the challenges (more on that later)!

We suggest heading over to the DataHack platform and picking up a project to apply these dimensionality reduction techniques on. After all, practice makes perfect!

Will learning about dimensionality reduction and techniques like PCA help me in clearing machine learning interviews?

Of course! As a newcomer or fresher in machine learning, you’ll be asked about how you would deal with massive datasets. It’s a common interview question and your knowledge about dimensionality reduction techniques like PCA and Factor Analysis will hold you in good stead.

Is dimensionality reduction a supervised or unsupervised machine learning technique?

Dimensionality reduction can be both supervised and unsupervised. The unsupervised techniques of Dimensionality Reduction is used when the dataset you have is humungous and is done prior to supervised dimensionality reduction technique. They include-

• Principal Component Analysis
• Random Projections
• Feature Agglomeration

Supervised Dimensionality reduction techniques include methods like-

• Linear Discriminant Analysis

What are the challenges of applying dimensionality reduction techniques?

The major challenges of applying dimensionality reduction includes-

• Choosing the right variables from all the predictors is one of the most challenging tasks of dimensionality reduction. Wrong choice will lead to building a poor performing model.
• The second challenge is the choice of dimensionality reduction techniques. Out of all the techniques that exist, dimensions of a dataset can be reduced using multiple dimensionality reduction technique and choosing the right one is really important to get the perfect analysis of data.
• Ensuring possession of the right kind of system with appropriate computational power.