Some sample statistics are NOT the best estimator for the population parameter.
depending on the optimality criterion
(unbiasedness, minimum variance among unbiased estimators, minimum mean squared error, etc.).
Examples:
Population standard deviation σ
** The sample standard deviation s is a biased downward estimator of σ. E[s] < σ
There is no simple closed-form unbiased estimator in general;
various corrected versions exist that reduce bias or MSE.
Population variance σ2 (under mean squared error)
Dividing by n-1 gives the unbiased estimator (MVUE under normality). -1 "correction"
Dividing by n is the maximum-likelihood estimator (biased).
Dividing by n+1 actually has smaller mean squared error than either of the above for normal data.
So the most common “sample variance” is not MSE-optimal.
Upper bound θ of a Uniform(0,θ) distribution
The sample maximum X_n is the maximum-likelihood estimator and the natural
sample statistic, but it is biased downward.
The unbiased (and better) estimator is (n+1)/n X_n.
Population mean of certain distributions
For a uniform distribution with unknown endpoints, the mid-range (average of sample
min and max) is the minimum-variance unbiased estimator of the mean,
while the ordinary sample mean is not optimal.
Sample median vs. sample mean
For a normal (and any symmetric) population the sample median is unbiased for the mean (and median),
but it is substantially less efficient (higher variance) than the sample mean.
For non-symmetric populations (e.g. skewed) the sample median is biased upward (positive bias)
but the bias shrinks as sample size n grows.
The idea that “the sample version of a parameter is automatically its best estimator”
is true in many textbook cases (especially the sample mean for μ), but it is not universally true.
Bias, variance, and mean-squared-error optimality frequently point to different estimators,
and the simple plug-in sample statistic is often suboptimal under one or more of these criteria.
***
A random sample of size n is unlikely to include the full extremes of the population.
The most extreme observations in the population are rare, so most samples under-represent the tails.
As a result, the sample tends to show less variation than the whole population.
This systematically pulls measures of spread (SD, MAD, range, IQR, etc.) downward on average.
The bias shrinks as n grows (all of these estimators are consistent),
but it is present for any finite sample size.