Discriminant Analysis

Overview

Discriminant analysis employs a decision boundary to partition a feature space into classifications. We find a boundary by finding where \(\delta_1 = \delta_2\).

Details

Other notes

LDA has \(k \times p\) (from \(\mu_k\)) PLUS \(k - 1\) (from \(\pi_k\)) PLUS \(\frac{p}{p+1}/2\) from \(\Sigma\) (because the correlation matrix is symmetric)

QDA has \(k \times p\) (from \(\mu_k\)) PLUS \(k - 1\) (from \(\pi_k\)) PLUS \(k \times \frac{p}{p+1}/2\) from \(\Sigma_k\) (because the correlation matrix is symmetric)

QDA:

\[P(Y=k | X) = \frac{\pi(k) f_k(x)}{\sum_{i=1}^k \pi_k f_k(x)} = \pi_k f_k(x) \times C = \pi_k (2\pi)^{-p/2} \log |\Sigma_k|^{-1/2} \times \exp( -\frac{1}{2} (x - \mu_k)^T )\]

If take \(\log\) of this, it becomes the discriminant score. LDA cancels the first term. The squared term is used so a quadratic boundary is found.

\(\delta_k(x) = \ln p_k(x) + c\)