Statistics and Probability - Correlation and linear regression

Bivariate Data and Scatter Diagrams: Bivariate data involves the relationship between two variables, typically denoted as $x$ (the independent/explanatory variable) and $y$ (the dependent/response variable). This is visually represented on a scatter diagram where each data pair $(x, y)$ is plotted as a point. The pattern formed by these points helps identify the nature of the relationship.

Correlation (Direction and Strength): Correlation describes the linear relationship between two variables. Visually, a positive correlation shows points trending upwards from left to right, while a negative correlation shows points trending downwards. The 'strength' refers to how closely the points cluster around a straight line: 'strong' if they are tight together and 'weak' if they are widely scattered.

Pearson’s Correlation Coefficient ($r$): This is a numerical measure ranging from $-1 \leq r \leq 1$. An $r$ value of $+1$ indicates a perfect positive linear correlation (all points on a line with a positive gradient), $-1$ indicates a perfect negative linear correlation, and $0$ indicates no linear correlation at all.

Line of Best Fit (Regression Line): The regression line is a straight line drawn through the data points that best represents the trend. Visually, it should have an equal distribution of points above and below it. A key property is that the regression line of $y$ on $x$ must always pass through the mean point $(\bar{x}, \bar{y})$.

Linear Regression Equation: The relationship is modeled by the equation $y = mx + c$ (or $y = ax + b$), where $m$ is the gradient (the change in $y$ for every unit change in $x$) and $c$ is the $y$-intercept (the value of $y$ when $x = 0$).

Interpolation and Extrapolation: Interpolation is the process of predicting a value within the range of the given data set, which is generally reliable. Extrapolation is predicting values outside the range of the data set, which is visually represented by extending the line beyond the plotted points; this is often unreliable as the linear trend may not continue.

Causation vs. Correlation: It is crucial to remember that a strong correlation between two variables does not necessarily mean that one causes the other. There may be a 'lurking variable' influencing both, or the relationship may be coincidental.