Pearson Correlation Coefficient in Data Science -- Part-2

[anish-arya]

If you are not following this conversation series here the previous post.

Correlation in Data Science -- An introduction

Correlation in Data Science – An introduction Correlation – a relationship or connection of two or more variables. It s essence is that when one variable’s value changes the other variable changes as well (decreases or increases). When calculating correlations, we try to determine whether there is a statistically significant relationship between two or more variables in one or more data samples. For instance, a relationship between height and weight, a relationship between performance and IQ t…

Pearson Correlation Coefficient in Data Science – Part-2

Let us assume that we have two processes, for each of which we measure some set of parameters. As a result, we get a set of pairs of numbers (process 1 and process 2 parameter values). Assuming that these processes are somehow related, we assume that this relationship should reveal itself numerically in the values of the parameters. Therefore, we can somehow get information about the presence or absence of the relation from the resulting number of pairs.

However, the relationship can be of different strength, so it is desirable to obtain not a binary “yes” or “no”, but a kind of continuous number that will characterise the degree (strength) of the relationship between the two variables. And now, meet the Pearson correlation coefficient.