Estimation
Overview
Estimation (推定, estimation) is a statistical procedure that reasonably infers the unknown parameter values of a population using observed sample data. Since it is mostly practically impossible to conduct a complete enumeration/census investigating the entire population, one infers values such as the population mean, population variance, and population proportion based on statistics calculated from a sample. Estimation, together with hypothesis testing, forms one of the two main pillars of inferential statistics, and is divided into point estimation, which presents a single value, and interval estimation, which presents a range under a given confidence level.
Main Content
Basic Concepts of Statistical Estimation
The starting point of estimation is the assumption that "a sample represents the population." A value calculated from data obtained through random sampling, that is, an estimator, is expected to be close to the true value of the parameter. For example, using the sample mean x̄ to estimate the population mean μ is typical. An estimator is a random variable, and a single actually calculated value is distinguished as an estimate.
Point Estimation and Interval Estimation
- Point estimation: Estimates a parameter with a single value. The sample mean, sample variance, sample proportion, etc. are representative point estimators.
- Interval estimation: Presents an interval expected to contain the parameter along with a confidence level. A 95% confidence interval has the frequentist meaning that if repeated sampling were done in the same way, about 95% of the intervals would contain the true parameter.
Properties of a Good Estimator
1. Unbiasedness: The expected value of the estimator matches the parameter.
2. Consistency: As the sample size increases, the estimator converges in probability to the parameter.
3. Efficiency: It has smaller variance than other unbiased estimators, making it more precise.
4. Sufficiency: It summarizes the information about the parameter contained in the sample without loss.
Major Estimation Methods
- Maximum likelihood estimation (MLE): Chooses the parameter value that maximizes the likelihood of observing the given sample. It has consistency and asymptotic normality, making it the most widely used.
- Ordinary least squares (OLS): Estimates regression coefficients by minimizing the sum of squared residuals.
- Method of moments: Finds parameters by equating sample moments with theoretical moments.
- Bayesian estimation: Estimates parameters through a posterior distribution that combines a prior distribution and likelihood, and uses the posterior mean or posterior median for point estimation.
Applications
Estimation is widely applied, from predicting approval ratings in opinion polls, estimating treatment effects in clinical trials, calculating economic indicators (unemployment rate, inflation rate), process capability analysis in quality control, and parameter learning in machine learning. In particular, sample size design, missing data handling, and bias correction are key practical issues that determine estimation accuracy.
Recent Trends
Entering 2024–2025, estimation methodology is evolving in the following directions.
- Popularization of Bayesian inference: With advances in computational techniques such as MCMC and variational inference and the spread of probabilistic programming languages (Stan, PyMC), hierarchical Bayesian models are rapidly spreading in industrial settings.
- Rise of causal inference: As demand grows to estimate causal effects of policies and treatments beyond simple correlation estimation, semiparametric estimation methods such as difference-in-differences (DID), instrumental variables, synthetic control, and targeted maximum likelihood (TMLE) are attracting attention.
- Convergence with machine learning: Neural network-based density estimation, normalizing flows, and distribution estimation using diffusion models are being combined with generative models.
- Uncertainty quantification: As the demand for presenting confidence intervals for deep learning predictions increases, conformal prediction is attracting attention.
- Sample bias problem: In estimation based on online data, the importance of securing representativeness and weight adjustment (post-stratification, raking) is being reexamined.
In short, estimation is establishing itself as a discipline that goes beyond simple formula application and demands assumptions about the data-generating process and honest reporting of uncertainty.
Related Topics
- [[Statistics]]
- [[Hypothesis testing]]
- [[Confidence interval]]
- [[Maximum likelihood estimation]]
- [[Bayes' theorem]]
- [[Sample survey]]
- [[Regression analysis]]
- [[Causal inference]]