Distributions in mathematical statistics are characterized by many statistical parameters. Estimation of unknown distribution parameters based on various sample data allows one to construct distributions of a random variable.

Find a statistical estimate of an unknown distribution parameter - find a function of the observed random variables, which will give an approximate value of the estimated parameter.

Statistical estimates can be divided into unbiased, biased, efficient and consistent.

Definition 1

Unbiased estimator-- statistical estimate $Q^*$, which, for any value of the sample size, has a mathematical expectation equal to the estimated parameter, that is

Definition 2

Biased Estimation-- statistical estimate $Q^*$, which, for any value of the sample size, has a mathematical expectation that is not equal to the estimated parameter, that is

Definition 4

Consistent assessment-- statistical estimate, in which, with a sample size tending to infinity, tends in probability to the estimated parameter $Q.$

Definition 5

Consistent assessment-- a statistical estimate, in which, with a sample size tending to infinity, the variance of the unbiased estimate tends to zero.

General and sample means

Definition 6

General average-- average arithmetic values version of the general population.

Definition 7

Sample mean-- arithmetic mean of variant values sampling frame.

The values ​​of the general and sample mean can be found using the following formulas:

  1. If the values ​​of the variant $x_1,\ x_2,\dots ,x_k$ have, respectively, the frequencies $n_1,\ n_2,\dots ,n_k$, then
  1. If the values ​​of the variant $x_1,\ x_2,\dots ,x_k$ are distinct, then

Related to this concept is the concept of deviation from the mean. This value is found by the following formula:

The mean deviation has the following properties:

    $\sum(n_i\left(x_i-\overline(x)\right)=0)$

    The mean value of the deviation is zero.

General, sample and corrected variances

Another of the main parameters is the concept of general and sample variance:

General variance:

Sample variance:

These concepts are also associated with the general and sample standard deviations:

As an estimate of the general variance, the concept of corrected variance is introduced:

The concept of the corrected standard deviation is also introduced:

Problem solution example

Example 1

The population is given by the following distribution table:

Picture 1.

Find for it the general mean, general variance, general standard deviation, corrected variance and corrected standard deviation.

To solve this problem, first we will make a calculation table:

Figure 2.

The value of $\overline(x_v)$ (sample mean) is found by the formula:

\[\overline(x_in)=\frac(\sum\limits^k_(i=1)(x_in_i))(n)\]

\[\overline(x_in)=\frac(\sum\limits^k_(i=1)(x_in_i))(n)=\frac(87)(30)=2,9\]

Let's find the general variance using the formula:

General standard deviation:

\[(\sigma )_v=\sqrt(D_v)\approx 1.42\]

Corrected variance:

\[(S^2=\frac(n)(n-1)D)_in=\frac(30)(29)\cdot 2.023\approx 2.09\]

Corrected standard deviation.

The distribution of a random variable (general population distribution) is usually characterized by a number of numerical characteristics:

  • for a normal distribution, N(a, σ) is the mathematical expectation a and the standard deviation σ ;
  • for a uniform distribution R(a,b) are the boundaries of the interval in which the values ​​of this random variable are observed.
Such numerical characteristics, as a rule, unknown, are called population parameters . Parameter Estimation - the corresponding numerical characteristic calculated from the sample. Population parameter estimates are divided into two classes: point And interval.

When an estimate is defined by a single number, it is called point estimate. Point Estimation, as a function of the sample, is a random variable and varies from sample to sample during repeated experiments.
Point estimates are subject to requirements that they must satisfy in order to be "good" in any sense. This unbiasedness, efficiency And solvency.

Interval Estimates are determined by two numbers - the ends of the interval that covers the estimated parameter. Unlike point estimates, which do not give an idea of ​​how far the estimated parameter can be from them, interval estimates allow you to establish the accuracy and reliability of estimates.

As point estimates of the mathematical expectation, variance and standard deviation, sample characteristics are used, respectively, the sample mean, sample variance and sample standard deviation.

Estimation unbiased property.
A desirable requirement for estimation is the absence of systematic error, i.e. with repeated use, instead of the parameter θ of its estimate, the average value of the approximation error is zero - this is valuation unbiased property.

Definition. An estimate is called unbiased if its mathematical expectation is equal to the true value of the estimated parameter:

The sample arithmetic mean is an unbiased estimate of the mathematical expectation, and the sample variance - biased estimate of the general variance D. The unbiased estimate of the general variance is the estimate

Evaluation consistency property.
The second requirement for an estimate - its consistency - means an improvement in the estimate with an increase in the sample size.

Definition. Grade is called consistent if it converges in probability to the estimated parameter θ as n→∞.


Convergence in probability means that with a large sample size, the probability of large deviations of the estimate from true value small.

Efficient Estimation Property.
The third requirement allows you to choose best estimate from several estimates of the same parameter.

Definition. An unbiased estimator is efficient if it has the smallest variance among all unbiased estimators.

It means that effective evaluation has minimal dispersion relative to the true value of the parameter. Note that an efficient estimator does not always exist, but one can usually choose a more efficient estimator from two estimators, i.e., with less dispersion. For example, for an unknown parameter a of a normal general population N(a,σ), both the sample arithmetic mean and the sample median can be taken as an unbiased estimate. But the variance of the sample median is approximately 1.6 times greater than the variance of the arithmetic mean. Therefore, a more efficient estimate is the sample arithmetic mean.

Example #1. Find an unbiased estimate of the variance of measurements of some random variable by one device (without systematic errors), the measurement results of which (in mm): 13,15,17.
Solution. Table for calculating indicators.

x|x - x cf |(x - x sr) 2
13 2 4
15 0 0
17 2 4
45 4 8

simple arithmetic mean(unbiased expectation estimate)


Dispersion- characterizes the measure of spread around its mean value (measure of dispersion, i.e. deviation from the mean - a biased estimate).


Unbiased estimator of variance- consistent estimate of the variance (corrected variance).

Example #2. Find an unbiased estimate of the mathematical expectation of measurements of some random variable by one device (without systematic errors), the measurement results of which (in mm): 4,5,8,9,11.
Solution. m = (4+5+8+9+11)/5 = 7.4

Example #3. Find the corrected variance S 2 for a sample size of n=10 if the sample variance is D = 180.
Solution. S 2 \u003d n * D / (n-1) \u003d 10 * 180 / (10-1) \u003d 200

) problems of mathematical statistics.

Let us assume that there is a parametric family of probability distributions (for simplicity, we will consider the distribution of random variables and the case of one parameter). Here, is a numeric parameter whose value is unknown. It is required to estimate it by the available sample of values ​​generated by this distribution.

There are two main types of assessments: point estimates And confidence intervals.

Point Estimation

Point estimation is a type of statistical estimation in which the value of an unknown parameter is approximated by a single number. That is, you must specify the function of the sample (statistics)

,

whose value will be considered as an approximation to the unknown true value .

Common methods for constructing point estimates of parameters include: maximum likelihood method, method of moments, quantile method.

Below are some properties that point estimates may or may not have.

solvency

One of the most obvious requirements for a point estimate is that one can expect a reasonably good approximation to the true value of the parameter given enough large values sample size . This means that the estimate must converge to the true value at . This evaluation property is called solvency. Because the we are talking about random variables for which there are different types convergence, then this property can be precisely formulated in different ways:

When just using the term solvency, then we usually mean weak consistency, i.e., convergence in probability.

The consistency condition is practically obligatory for all estimates used in practice. Inconsistent estimates are rarely used.

Unbiasedness and asymptotic unbiasedness

The parameter estimate is called unbiased, if its mathematical expectation is equal to the true value of the estimated parameter:

.

The weaker condition is asymptotic unbiasedness, which means that the mathematical expectation of the estimate converges to the true value of the parameter with an increase in the sample size:

.

Unbiasedness is a recommended property of estimators. However, its importance should not be overestimated. Most often, unbiased parameter estimates exist, and then one tries to consider only them. However, there may be some statistical problems in which unbiased estimates do not exist. Most famous example is the following: consider a Poisson distribution with a parameter and set the problem of estimating the parameter . It can be proved that there is no unbiased estimator for this problem.

Grade Comparison and Efficiency

To compare different estimates of the same parameter with each other, the following method is used: choose some risk function, which measures the deviation of the estimate from the true value of the parameter, and the best one is considered to be the one for which this function takes a smaller value.

Most often, the mathematical expectation of the squared deviation of the estimate from the true value is considered as a risk function

For unbiased estimators, this is simply the variance.

There is a lower bound on this risk function called Cramer-Rao inequality.

(Unbiased) estimators for which this lower bound is met (i.e. having the smallest possible variance) are called efficient. However, the existence of an effective estimate is a rather strong requirement for the problem, which is by no means always the case.

The weaker condition is asymptotic efficiency, which means that the ratio of the variance of the unbiased estimate to lower border Cramer-Rao tends to unity at .

Note that under sufficiently broad assumptions about the distribution under study, the maximum likelihood method gives an asymptotically efficient estimate of the parameter, and if there is an effective estimate, then it gives an efficient estimate.

Sufficient statistics

The statistic is called sufficient for the parameter if the conditional distribution of the sample provided that , does not depend on the parameter for all .

The importance of the concept of sufficient statistics is due to the following approval. If is a sufficient statistic and is an unbiased estimate of the parameter , then the conditional expectation is also an unbiased estimate of the parameter , and its variance is less than or equal to the variance of the original estimate .

Recall that the conditional expectation is a random variable that is a function of . Thus, in the class of unbiased estimators, it suffices to consider only those that are functions of sufficient statistics (provided that such a statistic exists for the given problem).

The (unbiased) effective parameter estimate is always a sufficient statistic.

We can say that a sufficient statistic contains all the information about the estimated parameter that is contained in the sample.

Lecture plan:

    The concept of evaluation

    Properties of statistical estimates

    Methods for finding point estimates

    Interval parameter estimation

    Confidence interval for the mathematical expectation with a known variance of a normally distributed population.

    Chi-squared distribution and Student's distribution.

    Confidence interval for the mathematical expectation of a random variable that has a normal distribution with an unknown variance.

    Confidence interval for the standard deviation of the normal distribution.

Bibliography:

    Wentzel, E.S. Probability theory [Text] / E.S. Wentzel. – M.: graduate School, 2006. - 575 p.

    Gmurman, V.E. Probability theory and mathematical statistics [Text] / V.E. Gmurman. - M.: Higher school, 2007. - 480 p.

    Kremer, N.Sh. Probability theory and mathematical statistics [Text] / N.Sh. Kremer - M: UNITI, 2002. - 543 p.

P.1. The concept of evaluation

Distributions such as binomial, exponential, normal are families of distributions that depend on one or more parameters. For example, the exponential distribution with probability density , depends on one parameter λ, the normal distribution
- from two parameters m and σ. As a rule, it is clear from the conditions of the problem under study which family of distributions is being discussed. However, the specific values ​​of the parameters of this distribution, which are included in the expressions of the distribution characteristics that are of interest to us, remain unknown. Therefore, it is necessary to know at least an approximate value of these quantities.

Let the distribution law of the general population be defined up to the values ​​of the parameters included in its distribution
, some of which may be known. One of the tasks mathematical statistics is to find estimates of unknown parameters from a sample of observations
from the general population. Estimation of unknown parameters consists in constructing a function
from a random sample such that the value of this function is approximately equal to the estimated unknown parameter θ . Function called statistics parameter θ .

Statistical evaluation(hereinafter just evaluation) parameter θ theoretical distribution is called its approximate value, depending on the choice data.

Grade is a random variable, because is a function of independent random variables
; if you make a different sample, then the function will, generally speaking, take a different value.

There are two types of estimates - point and interval.

dotted is called an estimate determined by a single number. With a small number of observations, these estimates can lead to gross errors. To avoid them, interval estimates are used.

Interval is called an estimate, which is determined by two numbers - the ends of the interval, in which the estimated value is enclosed with a given probability θ .

P. 2 Properties of statistical estimates

the value
called assessment accuracy. The less
, the better, the more precisely the unknown parameter is determined.

A number of requirements are imposed on the estimation of any parameter, which it must satisfy in order to be “close” to the true value of the parameter, i.e. be in some sense a "benign" assessment. The quality of an estimate is determined by checking whether it has the properties of unbiasedness, efficiency, and consistency.

Grade parameter θ called unbiased(without systematic errors) if the mean of the estimate is the same as the true value θ :

. (1)

If equality (1) does not hold, then the estimate called displaced(with systematic errors). This bias may be due to errors in measurement, counting, or the non-random nature of the sample. Systematic errors lead to overestimation or underestimation.

For some problems of mathematical statistics, there may be several unbiased estimates. Usually preference is given to the one that has the least scattering (dispersion).

Grade called efficient if it has the smallest variance among all possible unbiased estimates of the parameter θ .

Let D() is the minimum variance, and
is the variance of any other unbiased estimator parameter θ . Then the efficiency of the estimate is equal to

. (2)

It's clear that
. The closer
to 1, the more efficient the evaluation . If
at
, then the estimate is called asymptotically efficient.

Comment: If score shifted, then the smallness of its dispersion does not mean the smallness of its error. Taking, for example, as an estimate of the parameter θ some number , we obtain an estimate even with zero variance. However, in this case, the error (error)
can be arbitrarily large.

Grade called wealthy, if with an increase in the sample size (
) the estimate converges in probability to the exact value of the parameter θ , i.e. if for any

. (3)

Consistency of the assessment parameter θ means that with growth n sample size evaluation quality is improving.

Theorem 1. The sample mean is an unbiased and consistent estimate of the expectation.

Theorem 2. The corrected sample variance is an unbiased and consistent estimate of the variance.

Theorem 3. The empirical distribution function of the sample is an unbiased and consistent estimate of the distribution function of a random variable.

Statistical estimates of the parameters of the general population. Statistical hypotheses

LECTURE 16

Let it be required to study the quantitative sign of the general population. Assume that, from theoretical considerations, it was possible to establish which distribution has a feature. This gives rise to the problem of estimating the parameters that determine this distribution. For example, if it is known that the trait under study is distributed in the general population according to the normal law, then it is necessary to estimate (approximately find) the mathematical expectation and standard deviation, since these two parameters completely determine the normal distribution. If there are reasons to believe that the feature has a Poisson distribution, then it is necessary to estimate the parameter , which determines this distribution.

Usually, in the distribution, the researcher has only sample data, for example, the values ​​of a quantitative trait obtained as a result of observations (hereinafter, the observations are assumed to be independent). Through these data and express the estimated parameter.

Considering as values ​​of independent random variables , we can say that to find a statistical estimate of an unknown parameter of a theoretical distribution means to find a function of the observed random variables, which gives an approximate value of the estimated parameter. For example, as will be shown below, to estimate the mathematical expectation of a normal distribution, the function (arithmetic mean of the observed values ​​of a feature) is used:

.

So, statistical evaluation unknown parameter of the theoretical distribution is called a function of the observed random variables. The statistical estimate of an unknown parameter of the general population, written as a single number, is called point. Consider the following point estimates: biased and unbiased, effective and consistent.

In order for statistical estimates to give “good” approximations of the estimated parameters, they must satisfy certain requirements. Let's specify these requirements.

Let there be a statistical estimate of the unknown parameter of the theoretical distribution. Assume that when sampling the volume, an estimate is found. Let's repeat the experiment, that is, we will extract another sample of the same size from the general population and, using its data, we will find an estimate, etc. Repeating the experiment many times, we get the numbers , which, generally speaking, will differ from each other. Thus, the evaluation can be considered as random variable, and the numbers as possible values.

It is clear that if the estimate gives an approximate value with an excess, then each number found from the data of the samples will be greater than the true value of . Therefore, in this case, the mathematical (mean value) of the random variable will be greater than , that is, . Obviously, if it gives an approximate value with a disadvantage, then .


Therefore, the use of a statistical estimate, the mathematical expectation of which is not equal to the estimated parameter, leads to systematic (one sign) errors. For this reason, it is natural to require that the mathematical expectation of the estimate be equal to the estimated parameter. Although compliance with this requirement will not, in general, eliminate errors (some values ​​are greater than and others less than ), errors of different signs will occur equally often. However, compliance with the requirement guarantees the impossibility of obtaining systematic errors, that is, eliminates systematic errors.

unbiased called a statistical estimate (error), the mathematical expectation of which is equal to the estimated parameter for any sample size, that is, .

Displaced called a statistical estimate, the mathematical expectation of which is not equal to the estimated parameter for any sample size, that is.

However, it would be erroneous to assume that an unbiased estimate always gives a good approximation of the estimated parameter. Indeed, the possible values ​​may be highly scattered around their mean, i.e. the variance may be significant. In this case, the estimate found from the data of one sample, for example, may turn out to be very remote from the average value , and hence from the estimated parameter itself. Thus, taking as an approximate value, we will make a big mistake. If, however, the variance is required to be small, then the possibility of making a large error will be excluded. For this reason, the requirement of efficiency is imposed on the statistical evaluation.

efficient called a statistical estimate, which (for a given sample size ) has the smallest possible variance.

Wealthy is called a statistical estimate, which tends in probability to the estimated parameter, that is, the equality is true:

.

For example, if the variance of the unbiased estimator at tends to zero, then such an estimator also turns out to be consistent.

Consider the question of which sample characteristics best estimate the general mean and variance in terms of unbiasedness, efficiency, and consistency.

Let a discrete general population be studied with respect to some quantitative attribute .

General secondary is called the arithmetic mean of the values ​​of the feature of the general population. It is calculated by the formula:

§ - if all values ​​of the sign of the general population of volume are different;

§ – if the values ​​of the sign of the general population have frequencies, respectively, and . That is, the general average is the weighted average of the trait values ​​with weights equal to the corresponding frequencies.

Comment: let the population of the volume contain objects with different values ​​of the attribute . Imagine that one object is randomly selected from this collection. The probability that an object with a feature value, for example , will be retrieved is obviously equal to . Any other object can be extracted with the same probability. Thus, the value of a feature can be considered as a random variable, the possible values ​​of which have the same probabilities equal to . It is not difficult, in this case, to find the mathematical expectation:

So, if we consider the examined sign of the general population as a random variable, then the mathematical expectation of the sign is equal to the general average of this sign: . We arrived at this conclusion by assuming that all objects in the general population have various meanings sign. The same result will be obtained if we assume that the general population contains several objects with the same value sign.

Generalizing the result obtained to the general population with a continuous distribution of the attribute , we define the general average as the mathematical expectation of the attribute: .

Let a sample of volume be extracted to study the general population with respect to a quantitative attribute.

Sample mean called the arithmetic mean of the values ​​of the feature of the sample population. It is calculated by the formula:

§ - if all values ​​of the sign of the sample population of volume are different;

§ – if the values ​​of the feature of the sampling set have, respectively, frequencies , and . That is, the sample mean is the weighted average of the trait values ​​with weights equal to the corresponding frequencies.

Comment: the sample mean found from the data of one sample is obviously a certain number. If we extract other samples of the same size from the same general population, then the sample mean will change from sample to sample. Thus, the sample mean can be considered as a random variable, and therefore, we can talk about the distributions (theoretical and empirical) of the sample mean and the numerical characteristics of this distribution, in particular, the mean and variance of the sample distribution.

Further, if the general mean is unknown and it is required to estimate it from the sample data, then the sample mean is taken as an estimate of the general mean, which is an unbiased and consistent estimate (we propose to prove this statement on our own). It follows from the foregoing that if several samples of a sufficiently large volume from the same general population are used to find sample means, then they will be approximately equal to each other. This is the property stability of sample means.

Note that if the variances of two populations are the same, then the proximity of the sample means to the general ones does not depend on the ratio of the sample size to the size of the general population. It depends on the sample size: the larger the sample size, the less the sample mean differs from the general one. For example, if 1% of objects are selected from one set, and 4% of objects are selected from another set, and the volume of the first sample turned out to be larger than the second, then the first sample mean will differ less from the corresponding general mean than the second.