Statistics and Probability - Correlation and Regression Analysis

Scatter Diagrams: This is a visual representation where pairs of bivariate data $(x, y)$ are plotted as individual points on a Cartesian plane. If the cluster of points tends to rise from the bottom-left to the top-right, it indicates a positive correlation. If the points cluster in a downward slope from top-left to bottom-right, it indicates a negative correlation. A random spread of points suggest zero correlation.

Pearson’s Coefficient of Correlation ($r$): This numerical value measures the strength and direction of a linear relationship between two variables. It ranges from $-1$ to $+1$. A value of $r = +1$ represents a perfect positive linear relationship (all points on a straight line rising), $r = -1$ represents a perfect negative linear relationship (all points on a straight line falling), and $r = 0$ indicates no linear correlation.

Regression Lines: These are 'lines of best fit' that minimize the square of the distances between the actual data points and the line. There are two lines: the regression line of $y$ on $x$ (used to predict $y$ when $x$ is known) and the regression line of $x$ on $y$ (used to predict $x$ when $y$ is known). Visually, these lines intersect at the point of the means $(\bar{x}, \bar{y})$.

Regression Coefficients ($b_{yx}$ and $b_{xy}$): These represent the slopes of the regression lines. $b_{yx}$ is the slope of the line $y$ on $x$, indicating the change in $y$ for a unit change in $x$. Similarly, $b_{xy}$ is the slope for $x$ on $y$. Both coefficients always have the same algebraic sign, which is also the sign of the correlation coefficient $r$.

Spearman’s Rank Correlation ($r_s$): This method is used when variables are qualitative (like beauty or intelligence) or when the data is ranked. It measures the degree of similarity between two sets of rankings. If $r_s = 1$, the ranks are identical; if $r_s = -1$, the ranks are in exactly opposite order.

Geometric Property of $r$: The correlation coefficient $r$ is the geometric mean of the two regression coefficients, expressed as $r = \pm \sqrt{b_{yx} \cdot b_{xy}}$. The sign of $r$ is chosen based on the sign of the coefficients. If both $b_{yx}$ and $b_{xy}$ are positive, $r$ is positive; if both are negative, $r$ is negative.

Angle Between Regression Lines: If $r = \pm 1$, the two regression lines coincide, forming an angle of $0^{\circ}$, indicating a perfect linear relationship. If $r = 0$, the lines are perpendicular to each other, intersecting at right angles at the point $(\bar{x}, \bar{y})$.