Statistical Pattern Recognition

Introduction

     A reviewer of the book Fundamentals of Pattern Recognition in Science suggested that a better title

might be Statistical Pattern Recognition. The author would agree that such is a good alternative title for the

book, and also that the name Statistical Pattern Recognition (SPR) well describes a discipline that has

evolved over the last 30+ years.

     Statistical Pattern Recognition involves:         

          Learning conditional probability density functions.

          Likelihood functions.

          Classical statistics.

          Decision making:

               Minimum probability of error.

               Minimum risk.

               Maximum likelihood.

               Nearest neighbor decision rule.

               Generalized Nearest Neighbor Rule.

          Bayes Theorem.

          Patrick's Theorem.           

          Clustering

          Supervised learning (learning with a teacher)

          Unsupervised learning (learning without a teacher)

          Expert knowledge.

          Feature Extraction:

               Basic functions.

          Feature Selection.         

 Decision Making

             Given that there are M mutually exclusive and exhaustive events B₁, B₂, ....,B_M considered

                    outcomes or decisions, Bayes Theorem computes the probability of any of these events B_i,

                    conditioned on (given) an event A called the findings. This conditional probability, p(B_i|A) is

                    called the a posteriori probability ("after the fact" or after the findings), and is computed in

                    terms of an a priori probability ("before the fact" or "before the findings") p(B_i), the

                    probability of the findings p(A), and an expression p(A|B_i) which also. is called a likelihood

                    function (the probability of the findings given the event Bi.). That is,

                                                          p(A/B_i) * p(B_i)                               p(A/B_i) * p(B_i)

                        p(B_i|A)      =           ------------------                     ---------------------------                                                                                                               M

                                                                 p(A)                                      Σ p(A|B_j)*p(B_j)

                                                                                                               j =1

                  Minimum Probability of Error Decision Rule

                       A classic decision rule in SPR is to make a decision that minimizes probability of error.

                  Given M categories B₁, B₂, ....,B_M and findings A, the decision that minimizes probability of

                  error in deciding the correct category is:

                       Decide category  B_j such that:

                            p(B_j|A)     =    Max{p(B₁|A), p(B₂|A),.....p(B_M|A)}

                   Maximum Likelihood Decision Rule

                        A classic decision rule in SPR is to make a decision that maximizes the likelihood.

                  Given M categories B₁, B₂, ....,B_M and findings A, the decision that maximizes likelihood

                  of deciding the correct category given findings A is:

                       Decide category  B_j such that:

                            p(A/B_j)     =    Max{p(A/B₁), p(A/B₂),.....p(A/B_M)}