We consider the decomposition of a signal over an overcomplete set of
vectors. Minimization of the $\ell^1$-norm of the coefficient vector can often
retrieve the sparsest solution (so-called
"l1/l0-equivalence"),
a generally NP-hard task, and this fact has powered the field of compressed
sensing. Wright et al.'s sparse representation-based classification (SRC)
applies this relationship to machine learning, wherein the signal to be
decomposed represents the test sample and columns of the dictionary are
training samples. We investigate the relationships between
l1-minimization, sparsity, and classification
accuracy in SRC. After proving that the tractable, deterministic approach to
verifying l1/l0-equivalence fundamentally conflicts with the high coherence between same-class
training samples, we demonstrate that l1-minimization
can still recover the sparsest solution when the classes are well-separated.
Further, using a nonlinear transform so that sparse recovery conditions may be
satisfied, we demonstrate that approximate (not strict) equivalence is key to
the success of SRC.
Keywords: sparse representation, representation-based classification, mutual coherence, compressed sensing