MLE is MAP with uniforma prior. NAP is MLE with regulation.

Maximum likelihood estimation (MLE)

• the observed data is most probable
• a dominant means of statistical inference
• Bayesian: a special case of maximum a posteriori estimation (MAP) that assumes a uniform prior distribution of the parameters.
• Frequentist: a special case of an extremum estimator, with the objective function being the likelihood.

Maximum a posteriori estimation (MAP)

• Bayesian: an MAP estimate is an estimate of an unknown quantity, that equals the mode of the posterior distribution. ( what is mode?)
• can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data.
• a regularization of maximum likelihood estimation (MLE). ( what is regulation?)

Bayes estimator ( Don't understand.)

I've solved this problem once forgot again, see here: Reading Notes | PRML (Bishop 2006)

All my confusions come from the different terms in different areas.

Christopher M. Bishop, Pattern recognition and machine learning, New York: Springer, 2006.
(p.165) This framework is known in the statistics literature as empirical Bayes (Bernardo and Smith, 1994; Gelman et al., 2004), or type 2 maximum likelihood (Berger, 1985), or generalized maximum likelihood (Wahba, 1975), and in the machine learning literature is also called the evidence approximation (Gull, 1989; MacKay, 1992a).

Selection properties of type II maximum likelihood (empirical Bayes) in linear models with individual variance components for predictors
* By marginalizing over w we obtain a marginal likelihood, also known as the type II likelihood or the evidence function (Bishop, 2006).
* maximizing the marginal likelihood (type II ML or empirical Bayes)

Christopher M. Bishop, Pattern recognition and machine learning, New York: Springer, 2006.
* Type-2 maximum likelihood, also known as the evidence approximation, in which we maximize the marginal likelihood function obtained by integrating out the weight parameters (iow, marginalized w.r.t. weights)