Statistics and Probability - Bivariate Statistics (Correlation and Linear Regression)

Scatter Diagrams: A visual representation of bivariate data where the independent variable ($x$) is plotted on the horizontal axis and the dependent variable ($y$) on the vertical axis. The pattern of dots reveals the nature of the relationship; a 'cigar-shaped' cluster suggests a linear correlation, while a random cloud of points suggests no correlation.

Pearson’s Product-Moment Correlation Coefficient ($r$): A numerical measure of the strength and direction of a linear relationship between two variables. It ranges from $-1$ to $+1$, where $+1$ indicates a perfect positive linear correlation (dots forming a line with a positive gradient), $-1$ indicates a perfect negative linear correlation (dots forming a line with a negative gradient), and $0$ indicates no linear correlation.

Interpretation of $r$ Strength: Generally, $|r| > 0.75$ is considered a strong correlation, $0.5 < |r| < 0.75$ is moderate, and $|r| < 0.5$ is weak. Visually, a strong correlation means the points in a scatter plot lie very close to a straight line, whereas a weak correlation shows points more widely dispersed.

The Mean Point: Every regression line of $y$ on $x$ must pass through the mean point $(\bar{x}, \bar{y})$. Visually, this point acts as a 'pivot' or 'centroid' for the data set, and plotting it on a scatter diagram helps verify if a calculated regression line is positioned correctly.

Least Squares Regression Line ($y$ on $x$): The line of best fit defined as $y = ax + b$ (or $y = mx + c$) that minimizes the sum of the squares of the vertical distances (residuals) between each data point and the line. It is used specifically to predict the value of $y$ for a given $x$.

Interpolation and Extrapolation: Interpolation is making a prediction within the range of the original $x$-values, which is generally reliable if $r$ is strong. Extrapolation is predicting values outside the range of data; this is risky and often unreliable because the linear trend may not continue indefinitely.

Correlation vs. Causation: A high correlation between two variables does not necessarily mean that changes in one variable cause changes in the other. There may be a 'lurking variable' influencing both, or the relationship may be purely coincidental.