Statistics and Probability - Bivariate Statistics (Correlation and Linear Regression)

Bivariate Data and Scatter Diagrams: Bivariate data consists of pairs of linked variables, $(x, y)$, where $x$ is the independent (explanatory) variable and $y$ is the dependent (response) variable. Visually, this is represented on a scatter diagram where $x$ is plotted on the horizontal axis and $y$ on the vertical axis. Each point $(x_i, y_i)$ represents an individual observation, allowing us to see patterns, trends, or clusters in the relationship between the two variables.

Correlation Direction and Strength: Correlation describes the nature of the linear relationship between variables. Visually, a positive correlation appears as a 'cloud' of points sloping upwards from the bottom-left to the top-right, indicating that as $x$ increases, $y$ tends to increase. A negative correlation slopes downwards from top-left to bottom-right. The 'strength' refers to how closely the points cluster around a straight line; a tight, narrow corridor of points indicates a strong correlation, while a widely dispersed cloud indicates a weak correlation.

Pearson’s Product-Moment Correlation Coefficient ($r$): This numerical value measures the strength and direction of a linear relationship, ranging from $-1$ to $+1$. A value of $r = 1$ indicates a perfect positive linear correlation (all points lie exactly on a line with a positive slope), $r = -1$ indicates a perfect negative linear correlation, and $r = 0$ indicates no linear correlation. Visually, as $r$ moves closer to $0$, the scatter plot appears more like a random 'blob' without a discernible linear path.

The Mean Point (Centroid): In any bivariate dataset, the mean point is defined by the coordinates $(\bar{x}, \bar{y})$. This point is the geometric center of the data. A key visual and mathematical property is that the least squares regression line must always pass through this point $(\bar{x}, \bar{y})$. On a graph, you can identify this as the 'balance point' of the scatter plot.

The Line of Best Fit (Least Squares Regression): This is the unique line that minimizes the sum of the squares of the vertical distances (known as residuals) between the data points and the line itself. The equation is typically written as $y = ax + b$ (or $y = mx + c$). Visually, it represents the 'trend line' that best averages out the distribution of points. It is specifically used to predict values of $y$ given values of $x$.

Interpretation of Regression Coefficients: In the regression equation $y = ax + b$, the gradient $a$ represents the predicted change in the dependent variable $y$ for every one-unit increase in the independent variable $x$. The $y$-intercept $b$ represents the predicted value of $y$ when $x = 0$. Visually, $a$ is the steepness of the line, and $b$ is where the line crosses the vertical axis.

Interpolation vs. Extrapolation: Interpolation is the process of predicting a $y$-value for an $x$-value that falls within the original range of the data; this is generally considered reliable. Extrapolation involves predicting $y$ for an $x$-value outside the data range. Visually, this means extending the regression line beyond the last known data point. Extrapolation is often unreliable as the linear trend may not continue indefinitely.