What are Eigenvalues and Eigenvectors?

Data Science, AI and ML
#1

Eigenvectors and Eigenvalues Introduction

Before we take a deep dive into calculating eigenvectors and eigenvalues, let’s understand what they really are.

Let’s consider that we want to build mathematical models (equations) where the input data is gathered from a large number of sources. As an instance, let’s assume that we want to forecast a complex financial variable, such as the behavior of interest rates over time. Let’s refer to interest rates as y.

The first step might involve finding the variables that y is dependent on. Let’s refer to these variables as x(i)

We will start our research by gathering data for variables that y is dependent on. Some of the data might be in textual format. The task would be to convert the non-numerical data into numerical data. As an instance, we often use one-hot encoding to transform values in textual features to separate numerical columns. If our input data is in images format then we would have to somehow convert the image into numerical matrices.

The second step would be to join the data into a tabular format where each column of the table is computed by 1 or more features. This will result in a large sparse matrix (table). At times, it can increase our dimension space to 100+ columns.

Now let’s understand this!

It introduces its own sets of problems such as the large sparse matrix can end up taking a significant amount of space on a disk. Plus, it becomes extremely time-consuming for the model to train itself on the data. Furthermore, it is difficult to understand and visualize data with more than 3 dimensions, let alone a dataset of over 100+ dimensions. Hence, it would be ideal to somehow compress/transform this data into a smaller dataset.

There is a solution. We can utilise Eigenvalues and Eigenvectors to reduce the dimension space. To elaborate, one of the key methodologies to improve efficiency in computationally intensive tasks is to reduce the dimensions after ensuring most of the key information is maintained.

Eigenvalues and Eigenvectors are the key tools to use in those scenarios