Content:
We propose estimating density functions by means of a constrained optimization problem whose criterion function is the maximum likelihood function, and whose constraints model any (prior) information that might be available about the underlying stochastic phenomenon.
The asymptotic justification for such an approach relies on the theory of epi-convergence. A simple numerical example is used to signal the potential of such an approach.
Of course, the theory presented here doesn't depend on having prior information, so the `usual' framework is included as a special case. However, such additional information that often has been ignored, because statistical theory didn't validate its use, can now be part of the formulation of the estimation problem. The information can come in many forms: level of smoothness of the density functions, bounds on certain moments, shape, and so on.
The estimation problem is then one of finding a nonnegative function in a certain space, summing up to 1, satisfying some (functional) constraints, and maximizing the likelihood of observing a given sample. Our M-estimators are the solutions of these problems. As the sample size increases, it is shown that the estimation problems epi-converge to a limit problem whose (optimal) solution is the `true' density. Epi-convergence of the problems implies the convergence of the M-estimators to the `true' density.
The paper deals mostly with the theoretical foundations but in Section 8, a simple example illustrates the overall strategy that can be used in the numerical implementation. Samples (of size 20) are obtained from an exponential distributed random variable. Two estimation problems are formulated: one including the constraint that the density is a decreasing function, the other one without this constraint. We then make a comparison of the resulting estimates.
November '97