Relevance Feedback Using RBF for Content Based Image Retrieval

Conventional approaches to image retrieval are based on the assumption that relevant images are physically near the query image in some feature space. However, semantically related images are often scattered across several visual clusters. This leads to adapting multiple queries to represent a query in the feature space. Therefore, it is necessary to handle disjunctive queries in the feature space. In this paper, a new content-based image retrieval method with relevance feedback technique using RBF neural network learning is proposed. The method transfers the process of relevance feedback into a learning problem of RBF neural network. RBFNN can describe the distribution of positive feedback sample images in feature space with a set of neighboring clusters produced through constructing neural network, for accurately reflecting their semantic relevance. The performance of the method using RBFNN is evaluated on a database of 10,000 images. Experimental results demonstrate the effectiveness of the proposed method.


Introduction
Content-based image retrieval (CBIR) has become one of the most active research areas in multimedia signal processing.It is well known that the performance of content-based image retrieval (CBIR) systems is mainly limited by the semantic gap between low-level features and high-level concepts.Retrieving images from a large database according to the semantic meanings is a challenging problem, due to the lack of mature artificial intelligence mapping semantic meanings to low-level descriptors.
Relevance feedback techniques have been widely used in CBIR to bridge the semantic gap between low-level features and high-level semantics [1].Relevance feedback (RF) is an interactive technique in the procedure of CBIR, which is originally used in the traditional text-based information retrieval systems.A relevance feedback-based approach allows the user to label the returned images as relevant or irrelevant.Such labeled examples are further used to refine retrieval results by short-term learning or long-term learning techniques.
Many relevance feedback algorithms such as query refinement [2], re-weighting [1], Bayesian or probabilistic learning [3], optimal learning over heuristic-based feature weighting [4], artificial neural networks [5], and kernel-based learning [6], etc., have been adopted in CBIR systems and demonstrated considerable performance improvement.Current approaches to CBIR assume that relevant images are physically near the query image in some feature space regardless of visual features.However, the similarity between images perceived by humans does not necessarily correlate with the distance between them in the feature space.That is, semantically relevant images might be spread out in the feature space and be scattered in several clusters rather than one.In this case, traditional relevant feedback approaches do not work well when shifting the query center by linear combination of the relevant images.
In this paper, we propose a relevance feedback technique based on NBFNN for content-based images, which transfers the process of relevance feedback into a neural network learning problem.By constructing neural network that is constituted of the neighboring clusters of sample images, the distribution of user feedback sample images is described in feature space, so as to reflect the relevance of sample images in semantic contents.In particular, constructing neural network is dynamic, that is, neural network constructed depends on which images are retrieved in response to the query.
The rest of this paper is organized as follows.In section II we describe the proposed RBF network and its discriminant function.The principal of the self-growing RBF neural network is introduced in section III.The proposed relevance feedback based on RBF is presented in section IV.Our experimental results are given in Section V. Finally, Section VI gives some concluding remarks of this study.

RBF Neural Network
A RBF network usually consists of three layers: the input layer, the RBF layer (hidden layer) and the output layer.The input layer simply transfers the input vector , , , through scalar weights to the next layer.
In most applications, the scalar weights are assigned unity values.Thus the whole input vector appears to each neuron in the hidden layer.Each hidden nodes perform the radial basis function over the incoming vector that appears at the input of each RBF neuron.The output layer yields a vector for k outputs by linear combination of the outputs of the hidden nodes to produce the final output.Fig. 1 presents the structure of a single output RBF network.The network output can be obtained by: where , , , is the input vector, ( ) φ denotes the radial basis function of the ith hidden node, i w presents the hidden-to-output weight corresponding to the ith hidden node, and k is the total number of hidden nodes.
A radial basis function is a multidimensional function that describes the distance between a given input vector and a pre-defined center vector.There are different types of radial basis function.A normalized Gaussian function usually used as the radial function, that is: Where The Self-Growing RBF Neural Network Since the number and location of centers in the hidden layer will influence the performance of the RBF network directly, it is very crucial to determine the parameters of Gaussian center.To solve this problem, the self-growing RBF neural network techniques have been developed.These approaches start with one or a few RBFs and generate new basis functions by recursively applying the input training data.The rationale of this learning is described as follows: the hidden layer starts with no hidden node.The first RBF node center 1 u is set by choosing one data randomly from input data sample.The value of Euclidean 2-norm distance between 1 u and the next input sample x , is compared withε .If it is greater, a new cluster which is centered at the location defined by x is created as 2 u , otherwise the elements of 1 u are updated as: Where .denotes the Euclidean distance, 0 1 η < < is the updating ratio.The criterion ensures that a new neuron is only added if input data is sufficiently far from the existing neurons.Thus, this procedure carried out to the remaining training sample.Consequently, the number of clusters grows, and RBF nodes center self-adjust continuously until all of the samples are processed.

Relevance Feedback Based on RBF
In the current approach, the query is modeled according to the local clustering distributed by the relevant samples.Instead of using a single modified query, the new alternative uses a series of prototypes as the models of relevant images.We propose learning algorithms which enable the RBF network to progressively model the notion of image similarity for effective searching.The image matching process is initiated when the user supplies a query image and the system retrieves the images in the databases which are closest to the query image.From these images the user selects those as relevant which are most similar to the current query image, while the rest are regarded as non-relevant.The feature vectors these images are incorporated as training data for the RBF network to modify the centers and widths.The re-estimated RBF model is then used to evaluate the perceptual similarity in a new search, and the above process is repeated until the user is satisfied with the retrieval results.The process of RBF neural network learning is divided into two stages.The first stage is to construct self-growing BRF neural network using relevance images, every neuron of which is named neighboring cluster.The second stage is to use the negative samples to modify the model.Network Growth Technique.Our objective is to split a single query into multiple clusters.Instead of modifying a single query we use a set of models to characterize the query.We utilize positive samples, to train the self-growing BRF neural network since these samples represent the desired images.The modification of the cluster centers is usually carried out by an iterative process.The algorithm can be summarized as follows: (1)Assuming that { } k i i u 1 = denote the set of k cluster centers, which are generated from previous iterations.Taking a new input positive sample ' n x R ∈ , where n corresponds the dimension of input spaces, to calculate the distances with each cluster (2) Then the best-matching cluster center is identified by the condition: u is created at the position of the sample point x.Otherwise it means this input vector belongs to the jth cluster and the cluster center j u is updated by: (4) Repeating the whole procedure while taking the next input positive sample until all of the relevant samples are processed.RBF Learning.Then we use the negative samples ''   n   x R ∈ to modify the model because they are the closest vectors to the query and the positive samples according to the distance calculation of the previous search.To achieve this, we adopt the anti-reinforced learning rule of learning vector quantization algorithm to move each i u slightly away from negative samples, so as to improve the quality of the classification regions.To be precise, let ''  x be the closest point to the model i u .The model is modified according to the following learning rule: Where 2 η is the learning constant which decreases with the number of iterations.Note that it is desirable for the process to move the models slightly away from negative samples.The similarity evaluation is then performed based on linear combination of all the responses from each Gaussian RBF units.The overall output f(x) for an input data x from the database functions as a similarity measure, namely, those images x in the database with higher values are more likely to be relevant to the user.Selection of RBF Parameters.Not all feature variables are equally important in similarity judgment.Thus it is important to indicate the relative importance of various feature variables by weighting them.Weight schemes need to be estimated based on judgments of relevance, and the nature of the query.Where j α in Eq.2 is a relevance weight which is derived from the variance of the positive samples as: Where p ξ is the standard deviation of the positive sample sequence .This weighting criterion is based on the assumption that if a particular feature is important, then all relevant samples should have a very similar value for this feature, i.e., the sample variance in the relevant set is small.Different positive images may have different degrees of relevance, thus contributing unequally to the user information need.At the feedback loop stage, the user specifies a score value i v for each relevant image i x .Later, after the RBF training stage of the current iteration, if x has become the kth point of j u , then the weight of j u is: Then j w is the normalized weight of the kth cluster: The determination of the RBF unit width is given by: min Where δ is a factor that controls the overlapping of different RBF units.

Experiment Results
The image database used in the experiment contains 10,000 color images of 100 different categories obtained from the Corel Gallery product.In our system, we use two types of visual features: color and texture.Color histogram, color moments, and color auto-correlogram are used as the representation for color features.Gabor wavelet and wavelet moments are used as the texture features representation.
In the experiments, we select 100 query images, one from each category, and evaluate the retrieval quality for a sequence of iterations starting with these initial queries.For each query, the top 20 images were retrieved to evaluate the retrieval performance.We perform 5 feedback iterations in addition to the initial query.All the measurements are averaged over 100 queries.Precision and recall curve is used to measure the retrieval performance.Precision is defined as the number of retrieved relevant images over the number of total retrieved images.Recall is defined as the number of retrieved relevant images over the total number of relevant images.
Fig. 2 illustrates the precision and recall graphs over the different iterations.From the graph, we can observe that the precision and the recall increase at each iteration.The precision and recall are 0.5 and 0.1 at first iteration and 0.8 and 0.16 at fifth iteration, respectively.The retrieval quality increases quickly in the initial stage.At the following iterations there are minor increases in the retrieval quality.This ensures that our method converges quickly to the user's true information need.
Fig. 3 illustrates the precision and recall graphs over the different number of retrieved images.According to Fig. 3, the precision decreases and the recall increases with increase of the number of retrieved images.This is reasonable, because with the increase of the number of retrieved images, more and more images that are irrelevant to the query are involved in the retrieved images, which have negative impact on the precision.When the value is 100, since the number of retrieved images is equal to the number of relevant images, the value of precision and recall are the same.
Fig.1The RBF neural network architecture the distance criterion parameter ε .If it is greater than threshold, then a new cluster center 1 + k

Fig. 2 Fig. 3
Fig.2 Precision and recall graphs over the number of iterations