Previous Research Projects

3D Sensing Platform

Although 3D perception is a very simple task for us, mankind, it is very difficult to implement a system that can understand 3D environments, artificially. Partly, the reason is that visual perception and cognition do not depend on an outstanding function of physical eyes but depend on a function of the brain which can integrate our experiences and knowledge. Our team has developed a 3D robot camera that senses environments using active method like bats or dolphins. The 3D sensor emits infrared patterns to objects and environments using a DMD (digital mirror device, which is used for a projection TV or a projector), and reconstructs a 3D shape of the objects and environments. This active stereo is a very practical method so that it provides depth images even in texture-less environments, in contrast a passive vision system could not. Our 3D sensor can be used to medical area, service robots, and biometrics.

We presented an original approach to coding the light patterns for robust depth imaging based on structured light. We have discovered that the degradation of precision and robustness, seen in most conventional approaches to structured light, comes mainly from the overlapping of multiple codes in the signal received at a camera pixel, where the overlapped codes are from the neighboring and/or, even, distant pixels of the projecting mirror array. Considering the criticality of separating the overlapped codes to precision and robustness, we propose a novel signal separation code, referred to here as “Hierarchically Orthogonal Code (HOC),” for depth imaging. HOC provides not only the separation of overlapped codes, but also a robust decision on pixel correspondence with error correction based on a contextual likelihood among the sets of separated codes from neighboring camera pixels. The experimental results have shown that the proposed HOC significantly enhances the robustness and precision in depth imaging, compared to the best known conventional approaches. The proposed approach opens a greater feasibility of applying structured light based depth imaging to a 3D modeling of cluttered workspace for home service robots.

Fig. Since 2003, Korea Government has been supporting a research on intelligent robotics (10 years, 130 million dollars) as one of 21C Frontier Programs. We are developing service robots who will be suppose to help elder peoples. Especially, our group (ISRC) is working on the topic "3D Object/Environment Recognition and Modeling for Manipulation." Based on our technology (sensing, modeling and recognition), T-bot can serve in the Robotic Cafe.

(a) (b) (c) (d)

Fig. Design of optical system of the sensor (a), implemented projection module ((1) LED module, (2) collimate lens, (3) reflector, (4) DMD module, (5) projection lens), T-bot with another 3D sensor (with a commercial projector, camera and hardware control board (c), the implemented 3D sensor (d).

Moreover, the proposed HOC algorithm was implemented by using hardware platform which applies the Xilinx XC2V6000 FPGA to perform a real time 3D modeling and the invisible IR (Infrared) pattern lights to eliminate any inconveniences for the home environment. The experimental results have shown that the proposed HOC algorithm significantly enhances the robustness and precision in depth imaging, compared to the best known conventional approaches. Furthermore, after we processed the HOC algorithm implemented on our hardware platform, the results showed that it required 34ms of time to generate one 3D image. This processing time is about 24 times faster than the same implementation of HOC algorithm using software, and the real-time processing is realized.

Facial Image Representation by Statistical Feature Extraction

Facial Image Representation by locally salient ICA: The performance of face recognition methods using subspace projection is directly related to the characteristics of their basis images, especially in the cases of local distortion or partial occlusion. In order for a subspace projection method to be robust to local distortion and partial occlusion, the basis images generated by the method should exhibit a part-based local representation. We propose an effective part-based local representation method named locally salient ICA (LS-ICA) method for face recognition that is robust to local distortion and partial occlusion. The LS-ICA method only employs locally salient information from important facial parts in order to maximize the benefit of applying the idea of “recognition by parts.” It creates part-based local basis images by imposing additional localization constraint in the process of computing ICA architecture I basis images. We have contrasted the LS-ICA method with other part-based representations such as LNMF (Localized Non-negative Matrix Factorization) and LFA (Local Feature Analysis). Experimental results show that the LS-ICA method performs better than PCA, ICA architectureⅠ, ICA architectureⅡ, LFA, and LNMF methods, especially in the cases of partial occlusions and local distortions.

Fig. Facial image representations using (a) PCA, (b) ICA architecture I, (c) ICA architecture II, (d) proposed LS-ICA, (e) LNMF and (f) LFA basis images: A face is represented as a linear combination of basis images. The basis images were computed from a set of images randomly selected from the AR database. Using Proposed LS-ICA basis images, the concept of “recognition by parts” can be effectively implemented for face recognition.

Related Papers (including other subspace methods):

l J. Kim, J. Choi, J. Yi, and M. Turk, Effective Representation Using ICA for Face Recognition Robust to Local Distortion and Partial Occlusion, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1977-1981, 2005.

l J. M. Choi, J. H. Yi, A Two-Stage Dimensional Reduction Approach to Low-Dimensional Representation of Facial Images, International Conference on Biometric Authentication, (LNCS 3072) pp.131--137, 2004.

l l J. S. Kim, J. M. Choi, J. H. Yi, ICA Based Face Recognition Robust to Partial Occlusions and Local Distortions, International Conference on Biometric Authentication, (LNCS 3072), pp 147—154, 2004.

l Jongsun Kim, Jongmoo Choi, Juneho Yi, Face Recognition Based on Locally Salient ICA Information, BioAW2004 (Biometric Authentication Workshop in conjunction with ECCV2004), Prague, May 15 2004, (LNCS 3087), pp.1~9, 2004.

l J. M. Choi, J. H. Yi, Low-Dimensional Image Representation for Face Recognition, IEEE Workshop on Multimodal User Authentication, Dec. 2003.

l J. M. Choi, J. H. Yi, Low-Dimensional Data Representation for Pattern Recognition, Proceedings of International Conference on Cognitive Science, pp. 84-87, July 2003.

l Juneho Yi, Jongsun Kim, Jongmoo Choi, Junghyun Han, and Eunseok Lee, Face Recognition Based on ICA Combined with FLD, International ECCV 2002 Workshop Copenhagen, Denmark, June 1, 2002, (LNCS 2359), pp.10~17. 2002.6.2.

l J. M. Choi, S. H. Lee, C. G. Lee, J. H. Yi, A Real-Time Face Recognition System using Multiple Mean Faces and Dual Mode Fisherfaces, IEEE International Symposium on Industrial Electronics, pp. 1686 ~ 1689, 2001.

l J. M. Choi, S. H. Lee, C. G. Lee, J. H. Yi, A Real-Time Face Detection and Recognition System Robust to Illumination Changes, 3rd International Conference on AUDIO- and VIDEO-BASED BIOMETRIC PERSON AUTHENTICATION, (LNCS 2091), pp.360~365, 2001.6.6.

Cognitive Robotic Engine

The dependability of robotic perception and action may not come from "the perfection of individual components for perception and action," but from "the integration of individual components into dependable system behaviors, no matter how imperfect and uncertain individual components may be." We proposed a novel robotic architecture, referred to here as "Cognitive Robotic Engine (CRE)," that provides such dependable system behaviors, especially, for service robots. CRE features 1) the spontaneous establishment of ad-hoc missions in connection to sensing, 2) the construction of asynchronous concurrent architecture of perceptual building blocks or processes as an in-situ process plan with concurrent processing and fusion of multiple evidences, in such a way as to reach a fast resolution of ad-hoc mission under resource constraints. 3) the incorporation of action processes into the asynchronous concurrent architecture of perceptual processes for proactively collecting additional sensing data of less uncertainty or new evidence, and 4) the optimality in selecting a particular asynchronous and concurrent architecture of perceptual building blocks as well as the choice of particular action blocks to be invoked. CRE is applied to the case of a robot identifying a caller dependably in a crowed and noisy environment.