Key Posture Extraction from Object Manipulations Experiments

. In this work we present a novel concept for key posture by looking for grasping similarities among several grasping experiments. To highlight the nature of key posture idea, the object used in experiment had different sizes, although share the same type. Grouping the extracted data by joint, we search for time interval with high data concentration. If this time interval is shared by many joints in the same experiment we can extract key posture from that interval. The key posture can help a robotic hand system to grasp, control and manipulate the object through a specific task.


Introduction
One approach to grasping problem in robotics is to understand the human grasping system, and to try emulates it in a robotic hand.By analyzing data collected from tasks realized by human subjects, which is joint information extracted using a data glove, our main objective is to find grasping patterns between similar experiments.If a pattern can be found although different objects sizes and human hand characteristics, we can assume the pattern has a key importance in the realized task, not only for grasping but also for manipulation.Linking these key patterns (called key posture in this work) we can understand how the human hand grasps and manipulates the object to accomplish a specific task and implement that information in a robotic hand system, improving the controller or creating a learning system based on these patterns.Furthermore, many researches had been oriented to extract motion patterns [1,2], study how different grasps are performed [3], obtain simplified human hand models [4] and use learning methods to generalizing tasks [5].But the posture used along the task has not been well explored so far.
In this paper we present, in chapter "Method", the key posture definition and experiment detailed explanation.In chapter "Results" we show how we extracted the key posture and extracted ones.Finally, conclusions and future works are presented in Chapter Conclusion.

Method
Experiment Description.We used 10 humans subjects to collect the following joint information: 3 closure/flexure joints angles for each finger, 1 orientation angle for 2 fingers (index and ring), thumb cross over angle, hand tilt up/down angle (pronation) and hand tilt left/right angle (flexion), as shown in Fig. 1.The radial and ulmar deviation was not extracted in this work.To collect the data was used the data glove X-IST DataGlove HR3 from X-IST.
In order to investigate the effect of size and shape, each subject manipulates 3 different kinds of objects: 3 mugs, 3 pencils and 3 pet bottles (see Fig. 2).Each person realized the experiment 10 times for each object (30 times for each type).For example, for the mug experiment we had 3 types of mug, named as blue mug, green mug and white mug.For each mug, the subject realized the task 10 times (10 times for the blue mug, 10 times for the green mug and 10 times for the white mug).The realized task was the same for each type of object.The task for the mug experiment was to grab and empty the mug, like it was filled with some liquid (see Fig. 3).For the pencil experiment, it is to grab, press the pencil twice and release it.For the bottle experiment, it was to grab and try to open the cap of the bottle with two wrist movements.With a considerable amount of data, the extracted information was stored in a data base to improve further analysis.Analyzing data collected from tasks realized by human subjects, our main objective is to find key postures such that if connecting these postures the original manipulation can be generated.In other words, the key postures are the postures used in common for a certain number of joints among similar manipulations.To find these patterns we plot the motion trajectories for every joint, and search for the location with low data variation in common for a determined number of joints.Following this approach, we define the key posture as follows: . ( Advanced Engineering Forum Vols.2-3 . (2) The Eq. 1 represents the key posture, that we build based on the angle from all joints.In Eq. 2., Ji represents the set of joints that have data variance, in time interval , lower than .We define the time interval t as the time sequence between the time t2 and t1, where t1 is the start and t2 is the end of time interval.The value for Ji can be found using the average in this same time interval.In both Eq. 1 and Eq.2, i is the joint number and n is the quantity of all joints.In Eq.2 m is the number of data in time interval .Variance denotes the joint angle variation in time t.The symbol σ represents a positive constant that limits the data variance accepted to extract key posture.The value for changes according to the joints own angle limitation.
Each grasp use an unique set of joints in order to combine then to create the posture desired to realize the task [3,4].In other words, some grasp types do not employ all fingers, which means that potentially some fingers are not relevant for the grasp definition [3].With that in mind, we should consider only the most relevant joints to extract key posture.The variable in Eq. 2 represents a positive constant which stands for how much percentage of joints are important for the target grasp.For example, if we had an experiment that uses only joints from the thumb and index, is reasonable enough looking for data concentration only in these joins.So, in this case the variable = 8 (four joints for the thumb and four for the index).Both and are experiment related variables, although we used the same and values for all experiments realized in this word, once that our objective is not finding the optimal and .

Results
Key Posture Extraction.Fig. 4 shows the data for middle joint 2 (the red circle in Fig. 1).Each series represents one different experiment realized by the same person.To see how the data concentration changes among the time, we can look at Fig. 5.According to Eq. 2 and using a = 100, we classify the highlighted time interval in Fig. 4 as key posture interval candidates.According to [4] one hand with 12 DOF is able to fill the desired grasp requirements for our experiments.Based on that, we defined the (60% of all joints that can be collected by our data glove).In other words, if one key posture candidate interval is shared for at least 60% of the joints, we can say that the data average in that time interval can express a key posture for that experiment.Along the key posture candidate interval we can extract as much as the time interval ( ) allows.In this work we determined this time interval as , using t2 = t1+1.This represents an entire unit/frame/step of our experiment time sample.As larger

310
Mechatronics and Information Technology is the time interval t, more difficult to identify unique postures.For example, if a specific posture is used only during step 10, we can extract this posture taking the data average between the time interval t, where .But using t, as , we can misplace information from others postures not related with that unique posture used only at step 10.Doing so, probably we extract many similar key postures in intervals where we have both concentration and unchanging data, but we do not have loss of information.Fig. 5. Data Variance for Fig. 4 Doing this analysis for the mug experiment, we extracted 3 different key postures, as show in Fig. 6.We used the grasping simulation GraspIT [7] to build the models with extracted key postures.
The key posture 1 represents the pre-shaping posture, which is the action to prepare grasping action, probably without force contact.The key posture 2 shows us the initial grasping posture.The key posture 3 shows the releasing posture.These postures correspond to time intervals where variance is locally minimized.

Fig. 6. Example of Extracted Key Postures for the Mug Experiment
Analyzing the results, we realized that in some specific time interval in the experiment time line, a considerable number of joints don't change their values.
Furthermore, other joints at the same time interval, showed different behavior compared to the majority and other locations with key posture candidates, not related to those presented in most of the cases.Taking into consideration the experiment, one reasonable reason to that is while some joints hold the object, other ones execute the manipulation.Although the joints can't take roles on the task realization, in that time interval we can classify then as manipulation and supporting joints.The supporting joints don't change their values, holding the object while other joints manipulate it.Hence, from this observation we presume that the manipulation to accomplish the task occurs in that time interval.
To find key posture related to the manipulation, we can limit the joints we are looking for data concentration for those ones that changes their values within in specific time interval where probably is happening the manipulation.For example, in the pencil experiment, while almost all the finger hold the pencil, the thumb is used to press it.Searching for data concentration in thumb joints, Advanced Engineering Forum Vols.2-3 fixing the value for the other (once their values don't change) we can extract the key posture used for manipulation, show in Fig. 7.The obtained key postures correspond to the postures which intuitively needs for completing the manipulation.As the objective of this work is implementation in robotic hand, we adapted these key postures to be fitted into the computational model of kanazawa hand (a dexterous hand present in our laboratory).

Conclusion
We proposed the novel concept of key postures to expresses human hand object manipulation, which would be useful for robot manipulation and human motion analysis.From grasping experiment, collected from human hand subjects, we successfully extracted key posture from those experiments.Furthermore, after detailed observation among the experiment, we realized that in manipulation time interval, we can extract specific key posture for some joints while fixing other ones to find postures used during the manipulation.
As future work we planning to take use the key posture information in hand system controllers, to improve the system response.Nevertheless we want to extend the idea of to create a learning system based on the fusion between the concept of key posture and EigenGrasp, presented in [6].

Fig. 1 .
Fig. 1.Data that can be collected from data glove