Statistics

Analysis of variance (ANOVA) is a hypothesis-testing technique used to compare the means of three or more groups simultaneously, testing the null hypothesis that all group means are equal. Rather than running many pairwise t-tests (which inflates the chance of a false positive), one-way ANOVA partitions the total variability in the data into variability between groups and variability within groups, and compares these two sources via an F-statistic. A large F-statistic indicates that the variation between group means is large relative to the variation within groups, providing evidence that at least one group mean differs from the others. If ANOVA rejects the null hypothesis, follow-up post-hoc tests (like Tukey's HSD) are typically used to identify which specific groups differ.

The central limit theorem (CLT) states that if you draw sufficiently large independent random samples from any population with finite mean μ and finite variance σ², the distribution of the sample means approaches a normal distribution, regardless of the shape of the original population's distribution. This holds even if the underlying population is skewed, bimodal, or otherwise far from normal. As a practical rule of thumb, a sample size of n ≥ 30 is often considered large enough for the approximation to be good, though more skewed populations may require larger n. The CLT is the reason normal-distribution-based methods (confidence intervals, hypothesis tests) work so broadly across statistics, even when raw data isn't normally distributed.

The chi-square test is a hypothesis test used with categorical data to compare observed frequencies against frequencies expected under some null hypothesis. Its two most common forms are the chi-square goodness-of-fit test, which checks whether observed category counts match a hypothesized distribution, and the chi-square test of independence, which checks whether two categorical variables are associated (e.g., in a contingency table). The test statistic sums the squared difference between observed and expected counts, scaled by the expected counts, and compares the result to a chi-square distribution with an appropriate number of degrees of freedom. Larger discrepancies between observed and expected counts produce larger test statistics and smaller p-values, indicating stronger evidence against the null hypothesis.

The p-value is the probability, computed under the assumption that the null hypothesis is true, of observing a test statistic at least as extreme as the one actually observed. It is not the probability that the null hypothesis is true, nor the probability that the results happened by chance — a common and important misconception. A small p-value indicates the observed data would be unusual if the null hypothesis were true, providing evidence against it. Researchers typically compare the p-value to a pre-chosen significance level α (commonly 0.05): if p ≤ α, the result is called statistically significant and the null hypothesis is rejected.

A sampling distribution is the probability distribution of a statistic (such as the sample mean or sample proportion) obtained by repeatedly drawing random samples of a fixed size n from a population. Rather than describing individual data points, it describes how the statistic itself would vary across many hypothetical samples. The sampling distribution of the sample mean has the same mean as the population, μ, but a smaller standard deviation, σ/√n, called the standard error, which shrinks as sample size grows. Sampling distributions are the theoretical foundation for confidence intervals and hypothesis tests, since they tell us how much sample statistics fluctuate due to random sampling alone.

The t-distribution (Student's t-distribution) is a continuous probability distribution that arises when estimating the mean of a normally distributed population using a small sample and an unknown population standard deviation, which must be estimated from the sample itself. It looks similar to the standard normal distribution — symmetric and bell-shaped — but has heavier tails, reflecting the extra uncertainty introduced by estimating the standard deviation from limited data. The distribution's shape is governed by a single parameter, the degrees of freedom (df = n − 1 for a one-sample test), and as df increases, the t-distribution converges to the standard normal distribution. It underlies the t-test and t-based confidence intervals, which are the standard tools for inference on means when the population standard deviation is unknown and the sample is small.

In hypothesis testing, a Type I error occurs when the null hypothesis is rejected even though it is actually true — a 'false positive.' A Type II error occurs when the null hypothesis is not rejected even though it is actually false — a 'false negative.' The probability of a Type I error is denoted α (the significance level, chosen by the researcher, commonly 0.05), while the probability of a Type II error is denoted β. The power of a test, 1 − β, is the probability of correctly rejecting a false null hypothesis. There is an inherent tradeoff: for a fixed sample size, decreasing α (to reduce Type I errors) typically increases β (more Type II errors), and vice versa.

A statistical hypothesis test is a method of statistical inference used to decide whether the data provide sufficient evidence to reject a particular hypothesis. A statistical hypothesis test typically involves calculating a test statistic, then making a decision either by comparing the test statistic to a critical value or, equivalently, by evaluating a p-value computed from the test statistic. Roughly 100 specialized statistical tests are in use, covering means, proportions, variances, and more, but they all share this same underlying logic.

According to frequentist inference, a confidence interval (CI) is a range of values which is likely to contain the true value of an unknown statistical parameter, such as a population mean. Rather than reporting a single point estimate, a confidence interval provides a range, such as 2 to 4 hours, along with a specified confidence level, typically 95%. The confidence level describes the long-run reliability of the procedure used to construct the interval: if you repeated the sampling and interval-construction process many times, about that percentage of the resulting intervals would contain the true parameter.

The binomial distribution models the number of successes in a fixed number n of independent trials, each with the same probability p of success (a Bernoulli trial). It is characterized by two parameters: n (number of trials) and p (probability of success on each trial). Examples include the number of heads in 10 coin flips, or the number of defective items in a batch of 50 when each item independently has a 2% chance of being defective. The distribution's mean is np and its variance is np(1-p), reflecting how spread out the possible success counts are.

The exponential distribution models the waiting time between independent events that occur at a constant average rate λ — it is the continuous counterpart to the Poisson distribution, which counts the number of such events in a fixed interval. It is characterized by a single rate parameter λ > 0, and is famous for its 'memoryless' property: the probability of waiting an additional t units of time is the same regardless of how long you've already waited. Common applications include modeling the time until the next customer arrival, the lifetime of electronic components, or the time between radioactive decay events.

The Poisson distribution models the number of events occurring in a fixed interval of time or space, given that events happen independently and at a constant average rate λ (lambda). It's used for counts like the number of customer arrivals per hour, typos per page, or radioactive decays per second. Unlike the binomial distribution, the Poisson has a single parameter λ, which equals both its mean and its variance. It arises as the limiting case of the binomial distribution when the number of trials n grows large and the success probability p shrinks small, with np held constant at λ.

The normal (Gaussian) distribution is the iconic bell-shaped probability curve that appears throughout statistics, from measurement error to heights, test scores, and the distribution of sample means. Its ubiquity is not a coincidence — the Central Limit Theorem guarantees that averages of many independent random effects tend toward this exact shape, regardless of the underlying distribution.

A box plot (or box-and-whisker plot) is a graphical summary of a data set's distribution using five key numbers: the minimum, first quartile (Q1), median, third quartile (Q3), and maximum. The 'box' spans from Q1 to Q3 (the interquartile range, IQR), with a line marking the median; 'whiskers' extend to the smallest and largest values within 1.5×IQR of the box. Values falling beyond 1.5×IQR from the nearest quartile are commonly flagged as outliers and plotted as individual points rather than included in the whiskers. Box plots make it easy to compare the spread, skewness, and central tendency of several data sets side by side, and to visually flag potential outliers for further investigation.

Variance measures how spread out a set of numbers is by averaging the squared deviations from the mean. The standard deviation is the square root of the variance, which brings the measure back to the original units of the data, making it easier to interpret. A population's variance is denoted σ², and its standard deviation σ; sample statistics use s² and s, with the sample variance dividing by n − 1 (Bessel's correction) rather than n to give an unbiased estimate of the population variance. Both measures are foundational to statistics: larger values indicate data spread widely around the mean, while values near zero indicate data clustered tightly around it.

A z-score (or standard score) tells you how many standard deviations a data point lies above or below the mean of its distribution. It is computed by subtracting the mean from a value and dividing by the standard deviation, converting raw data into a common, unit-free scale. Z-scores let you compare values from different distributions (e.g., a test score and a height measurement) on equal footing, and they are the basis for looking up probabilities in the standard normal table. A z-score of 0 means the value equals the mean; positive z-scores lie above the mean, negative ones below.

The Pearson correlation coefficient, r, measures the strength and direction of the linear relationship between two quantitative variables. It ranges from −1 to +1: r = +1 indicates a perfect positive linear relationship, r = −1 a perfect negative linear relationship, and r = 0 indicates no linear relationship (though a nonlinear relationship may still exist). The sign of r matches the sign of the slope of the best-fit regression line, and r² (the coefficient of determination) gives the proportion of variance in one variable explained by the other. Correlation measures association, not causation — a strong correlation between two variables does not imply that one causes the other.

Multiple regression extends simple linear regression to model a response variable y as a linear function of two or more predictor variables x₁, x₂, ..., xₚ simultaneously. Each predictor gets its own coefficient, estimated (typically via least squares) to minimize the sum of squared residuals across all observations. The coefficient on each predictor represents its effect on y while holding all other predictors constant, which allows the model to separate out the individual contribution of each variable even when predictors are correlated with each other. Multiple regression is widely used to control for confounding variables, make predictions from several inputs at once, and quantify how much variance in y (measured by R²) is jointly explained by the full set of predictors.

A descriptive statistic is a summary statistic that quantitatively describes or summarizes features from a collection of information, while descriptive statistics is the process of using and analyzing those statistics. Descriptive statistics is distinguished from inferential statistics by its aim to summarize a sample, rather than use the data to learn about the population that the sample of data is thought to represent — it generally is not developed on the basis of probability theory. Even when a data analysis draws its main conclusions using inferential statistics, descriptive statistics are almost always presented alongside it, e.g. a table giving sample size, average age, and other characteristics.

In statistics, linear regression is a model that estimates the relationship between a scalar response and one or more explanatory variables. A model with exactly one explanatory variable is a simple linear regression; a model with two or more explanatory variables is a multiple linear regression. This term is distinct from multivariate linear regression, which predicts multiple correlated dependent variables rather than a single dependent variable. Linear regression is one of the most widely used statistical techniques, forming the basis for prediction, trend estimation, and causal-effect modeling across nearly every quantitative field.