A Tensor Based Isometric Projection Algorithm

Data manifold represented in the reality is intrinsically in tensor form and so tensor-based subspace algorithms can preserve the intrinsic spatial structure information. They are beneficial for data representation and classification and have been widely used in recent years. In this paper, a new algorithm called Tensor based Isometric Projection (TIsoProjection) is proposed. The proposed algorithm can naturally describe the spatial relationship between the column vectors and the row vectors. Also it solves the small sample size (SSS) problem. Experiments on the ORL and YaleB demonstrate that the proposed algorithm can achieve higher recognition rate.


Introduction
Face recognition has been a hot research topic of image area in recent years and subspace algorithms have been playing an important role. The typical subspace algorithms refer to Principle Component Analysis (PCA) [1], Linear Discriminant Analysis (LDA) [2]. Although these two algorithms achieve success in face recognition within certain environments, they both are feature extraction algorithms based on globality and the extracted features can not reflect the nonlinear manifold of high-dimensional data. When the illumination, poses and expressions of face images change, the recognition rate of both two algorithms will decrease significantly.
Recent studies show that an ensemble of face images may live on a nonlinear submanifold. Some manifold learning algorithms are used to discover the nonlinear structure of the data, such as Locally Linear Embedding (LLE) [3], Isometric Mapping (ISOMAP) [4], and Laplacian Eigenmaps (LE) [5]. There is a problem for these algorithms that projections only take place on the training data rather than the new test points directly, thus some linearization subspace algorithms for nonlinear manifolds, such as Locality Preserving Projection (LPP) [6,7], Neighborhood Preserving Embedding (NPE) [8], and Isometric Projection (IsoProjection) [9], are proposed. These improved subspace algorithms not only describe the nonlinear structure of the data well but also can project new data points directly. Experimental results show the effectiveness of subspace algorithms. The drawback is that, as a class of unsupervised algorithms, no discriminant information which is useful for classification is extracted in these algorithms.
In addition, the above algorithms solve the problem of projecting the image X in the high-dimensional vector space into the image Y in the low-dimensional vector space by using vector representation, that is, find a transformation matrix A which satisfies the projection equation The small sample size (SSS) problem of face recognition can cause the ill-pose problem for feature extraction for algorithms, such as LDA, LPP, NPE, and IsoProjection. Tensor based face recognition algorithms open up a new method for face recognition. The typical second-order tensor algorithms, Two-dimensional Principle the Component Analysis (2DPCA) [10] and Two-dimensional Linear Discriminant analysis (LDA) [11], are typical image representation technique. It is an efficient subspace algorithm to utilize two-dimensional matrix to represent the image directly. In spite of this, the algorithms, 2DPCA and 2DLDA, focus on the global structure preservation in Euclidean space and ignore the nonlinear manifold structure of the data.
In view of the above problems, the paper proposes a new algorithm called Tensor based Isometric Projection (TIsoProjection) by extending the original IsoProjection into the tensor space. The new algorithm uses the tensor instead of the traditional vector to represent face images and the singular value decomposition is done in tensor space to preserve the manifold structure of face samples. The face recognition using the new algorithm is much more convenient and accurate. Although this paper mainly discusses the second-order tensor images, the proposed algorithm can also be applied to the higher order tensor.

IsoProjection
The IsoProjection is a linear approximation of ISOMAP. Supposing D R represents the high-dimensional space of the original data, p Μ is a low-dimensional manifold embedded in D R , and p D ≤ , for a low-dimensional mapping function : , to maintain the intrisic geometrical structure of the data manifold, find a function f that meets the following condition (1) Design the following objective function In practical applications, can not be obtained directly because p Μ is unknown. The solution is to use the shortest path approximating. Assuming ij D is the shortest distance of sample points and . Let Y D be the Euclidean distance matrix of the subspace after dimensionality reduction and ( ) Y D τ be the corresponding inner product matrix, then the objective function in Eq. 2 is equivalent to the following formula The above objective function is deduced as solving the following problem

TIsoProjection
IsoProjection algorithms found on vector operations in which two-dimensional images are converted into one-dimensional vectors firstly. As a result, not only structure information of images is lost, but also T XX will be singular which lead to algorithm failure for such typical SSS problem of face recognition because the number of training images is very small compared to the dimension of image vectors. The common solution is SVD decomposition or PCA, i.e., SVD decomposition or PCA dimensionality reduction is done firstly, and then IsoProjection is used to handle the SSS problem to a certain extent. Because SVD decomposition or PCA dimensionality reduction loses a lot of classification information, the paper presents a new algorithm-TIsoProjection by extending the original IsoProjection to the tensor space.
Assume that the size of the original image is m n × . The goal of TIsoProjection algorithm is to execute linear approximation for face manifolds to obtain dimensionality reduction projection from the high-dimensional tensor space Assume the tensor X in the original space is projected into Y in low-dimensional space, and then the final goal of TIsoProjection is to find the m k × transformation matrix P and n l × transformation matrix Q such that the projection satisfies T Y P XQ = .
(5) Substitute objective function of Eq. 3 with Eq. 5 to obtain the following result tr(( (D ) P XQ Q X P)( (D ) P XQ Q X P)) tr( (D ) (D ) P XQ Q X P (D ) (D )P XQ Q X P P XQ Q X P) Add the condition ( ) 1 T tr YY = , then T T T T T T tr(P XQ (P XQ) ) tr(P XQ Q X P) 1 ⋅ = ⋅ = .
Because ( ) ( ) tr AB tr BA = , the third item of Eq. 6 is The objective function of Eq. 3 can be rewritten as Taking the condition ( ) 1 T tr YY = into account, the matrix P can be figured out by solving the following problem T T T T G XQ (D )Q X P XQQ X P τ = λ .
For ( ) ( ) tr Q X PP XQ = . Now the objective function of Eq. 9 is transformed into The matrix Q can be figured out by solving the following problem It is difficult to calculate P and Q directly through the Eq. 11 and Eq. 13, so the iterative algorithm, such as proposed by He [12], can be used. At first set P be the unit matrix and Q is gained by Eq. 13. Next, substitute Q into the Eq. 12 to obtain the matrix P and update. Repeat the above process till the desired iteration times and get the final matrixes P and Q .
In summary, the basic steps of the TIsoProjection method are as follows: Step1. Initialize the adjacency matrix of sample points using k nearest neighbor.
Step2. Calculate the shortest path between two sample points using Floyd algorithm.
Step3. Set P I = and the iteration times be d . Step4. Substitute P into the Eq. 13 to obtain the matrix Q ; substitute Q into the Eq. 12 to obtain the matrix P and update; repeat d times. Step5

Experiments and analysis
To verify the validity of the proposed algorithm, experiments on the sample images from ORL and YaleB face database were performed to compare the algorithm proposed in this paper with PCA, LDA, IsoProjection, 2DPCA, and 2DLDA. All the parameters for these recognition algorithms were adjust to the best state. The nearest neighbor classifier was employed to complete the final classification. All the experimental images were cut off and scaled to 32 × 32. The times of iterations was set to be 1 d = .
Experiments on the ORL face database. The ORL face database has 40 individuals, each 10 pictures, and the number of total pictures is 400. The original size of images is 112 × 92. The images are taken at different times. Face expressions and details vary in some degree (such as, eyes may open or close, laugh or be serious, wear glasses or not). Face poses may also change, that is, the range of depth rotation or plane rotation is up to 20°, and the change scale of faces is up to 10°. Fig. 1 shows some samples of one person from the ORL face database. 2, 3, 4, and 5 images were selected stochastically for each person from the ORL face database as a training set, and the remaining images as a testing set. Experimental results are shown in Table 1 (numbers in brackets represent the dimension of feature vectors). It can be seen from Table 1  To test the effect of illumination changes, 64 pictures were selected for experiments. 5, 10, 20, and 30 images were selected stochastically for each person from the YaleB face database as a training set, and the remaining images as a testing set. Experimental results are shown in Table 2. It can be seen that the TIsoProjection algorithm achieves the highest recognition rate, and the TIsoProjection algorithm is robust to illumination changes.

Analysis
According to the above experiments, the PCA algorithm is the worst one owing to the basis of the global Euclidean space. For LDA is a kind of supervision algorithms, it works better than PCA on recognition performance. The IsoProjection algorithm is able to describe the manifold structure of the data and extract useful information from the data for classification, so its recognition performance is better than PCA and LDA. Generally, tensor-based subspace algorithms preserve the row and column spatial structure information of samples, the overall recognition performance is superior to the vector-based algorithm. In addition, tensor-based algorithms solve the SSS problem faced by LDA, IsoProjection and so on effectively.

Conclusion
The paper proposes a Tensor based Isometric Projection algorithm for face recognition applicaton based on the IsoProjection algorithm. The new algorithm extracts feature directly from image matrixes to solve the SSS problem of the original algorithm and keeps row and column structure information of images to improve the computational efficiency. Experimental results show that this proposed method brings effective dimensionality reduction of face images, and is robust to the changes of illuminations and poses. Consequently, the recognition rate of the proposed algorithm is improved. However, the dimension of eigenvectors is much higher than the one in traditional one-dimensional algorithm. Therefore, more storage space and computing time for classification are needed for our algorithm. In addition, because the algorithm is linear, the intrinsic geometrical structure may not be detected for highly non-linear image manifolds of the manifold. It needs further studies on how to solve these problems.