Classification of Court Frame and Non-Court Frame for Tennis Video Based on Ontology

The paper proposes a method which can efficiently classify the court frame and non-court frame for tennis video. First, this paper extracts dominant color for each frame by calculating the histogram. And Then, constructs ontology by using protégé . In the end, uses swrl to construct inference rules which can classify the court frame and non-court frame. The experiment result proves that the method is executable.


Introduction
Tennis match is one of the most popular sport competitions.For the thousands of hours tennis match videos library, what users concern about is not how to transmit and play all the match videos by internet but how to find the highlights of the match quickly.During the research, it is found that all the highlights of the match are happened in footages of race and each frame of these footages almost contains court information.Therefore whether a frame contains a court information is playing an important role in the classification of court frame and non-court frame.In addition, in tennis video different matches differ in their match courts.For instance, the famous four open tennis tournament, the Wimbledon tennis open uses grass court, the French open uses red clay court, the US open and the Australian open use hard court.For this reason, it is important to tennis video retrieve to classify the grass court, red clay court and hard court for frames those contain court information.
Detection of court information in each frame is the main point of the classification of court frame and non-court frame.At present, detection of court has been the mature technique.For instance: the earlier method of court extraction was using simple threshold method to divide up the court [1].If the court information in video is obvious and light is less intense, this kind of method is simpler and can achieve better effect.Erkin [2] uses original color space and control of color space to control the process of learning dominant color and then through the dominant color detect the match court.Hayet [3] recognizes the football court and tennis court based on two dimensional calibration templates of vanishing points.
The paper determines whether a frame contains court information by using dominant color in combination of ontology and swrl rules, and infers the concrete class of the frame belongs to.The reason of introduction of ontology is that ontology is specification description which has definite domain knowledge and formalize and an effective tool of semantic modeling and knowledge representation.Therefore ontology can overcome semantic gap to some extent.Although ontology is a new research field, there still have related field research which can learn from.Paper [4,5] gives a detailed explanation of the ontology and its usage and OWL and the usage of protégé.In paper [6], Marco Bertini proposes semantic annotation of soccer videos by ontology, defines ontology by OWL and infers relationship between concept and instance of concept to improve the annotation quality.

Extraction of Dominant Color of Tennis Video Frame
Although match court occupies a main space in tennis video frame, detection of match court is always influenced by noise which appeared during the filming process and appearance of audience and billboard.To solve this problem, the paper needs to preprocess to each frame-only to read the center part of each frame.Particular way is that get the width and height of each frame and read the pixel from the 0.2*width*height to 0.8*width*height.In this way, we can cut off the top and bottom and reserve the center part.
The values of court pixel can not come in line well, even in the same match court, since there have influence from light, climate etc.To overcome this problem, the paper chooses HSV color space to analyze.Because HSV color space is close to intuitive sense.Therefore, next step, convert RGB color space into HSV color space for each frame.And then, quantify HSV color space and get 256-dimensional histogram.At last, get the dominant color for each frame through comparison of the values of 256-dimensional histogram.It is found, through a mess of experiments, that the value of dominant color of grass court is 65 or 95, dominant color of red clay court is 0 or 20, and dominant color of hard court is 205 or 235.

Establishment of an Ontology
Ontology is more and more popular in computer knowledge project since it can systematized describe existing of observable world and convenient for reusing and interactive of knowledge.Although there are various kinds of definition for ontology, in this paper, we follow the definition of Gabor which is most authority and is introduced most widely, that is, "an ontology is a formal, explicit specification of a shared conceptualization" [7].This definition is implicitly 4 level meaning: conceptualization, explicit, formal and share.
It is a general way to establish ontology by using ontology language.In common, ontology language provides basic modeling primitives of concepts, relationship between concepts and instances.Among these ontology language, OWL provides not only abundant but also practical modeling primitives for relationship of concepts.These abundant semantics support logic reasoning efficiently.
OWL divide into three entities TennisVideo3-hasDcolor, hasCourt, hasNCourt.hasDcolor is integer type used for getting dominant color value of a frame, hasCourt and hasNCourt is boolen tylpe used for making mark to a frame whether has court information.In order to classify a frame conveniently, we take a frame as an individual-tennisVideo.As seen in Fig1.

Construction of Rules Based on swrl
Besides abundant modeling primitives, ontology also can define rule for its concepts by using swrl.swrl editor is high interactivity open source swrl editor and tightly integrate with protégé to provide high interactivity rule editing interface and permit seamlessly alternate between swrl rule editor and OWL editor.User can cite entity of OWL conveniently in swrl.The main target of SWRL editor is interactive operation between SWRL and rule engine which already have.The main part of SWRL is the mechanism which is tightly integrated with rule engine provided by editor.The mechanism is supported by a subsystem called SWRL Factory.The system in this paper is developed by integrator of Jess rule engine and swrl rule editor [8].Specifically, establish an ontology named TennisVideo3 using protégé editor environment, and construct rules using SWRLTab which is a part of protégé.There use Jess rule engine to make the frames classify correctly.
swrl rule appears as formula of antecedent and consequent acting on instances in OWL and their relationship.General form is: Antecedent → Consequent.
The paper defines 15 swrl rules for TennisVideo3.Due to limitations of space, there only present a small portion of these rules.As shown in Fig. 2.

Fig.2 SWRL rules for TennisVideo3
Rule-1 and Rule-13 indicate that the frame will be classified to RedClayCourt class when value of hasDColor is 0 or 20, at the same time set hasCourt to 1.In this way, the frame will be classified to GrassCourt class and HardCourt class according to the value of hasDColor.In the end, the frames which the value of hasCourt is 1 are classified to PlayAction class.Frames which the value of hasNCourt is 1 are classified to Break class.

1222
Information Technology for Manufacturing Systems III

Results
Jess (java expert shell system) is CLIPS inference machine based on Java.In the integrated environment in protégé, it can integrate the SWRL rule and Jess rule engine by SWRLJessTab under SWRLTab.In the paper, we get the results through inferring swrl rules by using Jess.The subject of this experiment is three parts of videos which whole length of time is 1499s.To get video frame sequences, we sample the three videos in rate of 10 frames per second.And extract the dominant color for each frame and classify it.In these three videos, the time length of grass court video is 547s, the time length of red clay court is 482s and the time length of hard court is 470s.The results are shown in Table1, Table2, and Table3.From the Table1, we can know that the experiment can classify the video frames to the correct court class.Table2 is classification results of court frames in the video.The numbers of leakage classification frames represents the number of court frames missing in the classification results.The numbers of wrong classification frames represents the number of non-court frames which classified into court frame class.The reason of leakage classification frames is that some service actions shot in the players ' close-up, and this kind of frames seldom have court information.The reason of wrong classification frames is that some non-court frames contain court information or the color of billboard is closely like to the color of court.Table3 is classification results of non-court frames in the video.The reason of leakage classification frames and wrong classification frames is that we only use dominant color for classifying video frames.That is frames do not meet the conditions of court frame are classified to non-court frames class and do not make detailed constraints on the frame of the non-court.

Conclusion
The paper determine a frame whether contains court information by using dominant color in combination with ontology and swrl rules and infers the concrete class of the frame belongs to.Ontology learning is an emerging field of study, but its application prospects should not be underestimated.We want to accumulate experiences for later study through this attempt.The experimental results still have some errors because do not consider the lower-level characteristics like texture and spatial characteristics and high level semantic.For the future, want to use more lower-level characteristics and take advantage of ontology-can overcome semantic gap in certain extent-to improve the experimental effects.
[8]: ① Individual ( Instance ).Representation of an entity or a phenomenon in any interested domain.② Class.Set of individuals which have similarity features can organize to be a father-son level.③ Attribute.Representing two elements connection between two individuals or an individual and a data type.The paper defines 7 classes-{TennisVideo3, PlayAction,Break, MatchCourt, RedClayCourt, HardCourt, GrassCourt}.TennisVideo3 is sub-class of OWL: Thing.TennisVideo3 contains three sub-classes, that is, PlayAction，Break, MatchCourt.MatchCourt also contains three sub-classes, that is, RedClayCourt, HardCourt, GrassCourt.And then define a Datatype attributes for Advanced Engineering Forum Vols.6-7

Table 1
Classification results of specific court for video frames

Table 2
Classification results of court frames in the video

Advanced Engineering Forum Vols. 6-7
Table3 Classification results of non-court frames in the video