We have developed a novel fiber-optic vibration sensors and applied commercially available strain and temperature sensors to health monitoring of composite structures. In this study, we constructed an optical fiber network integrating four types of optical fiber sensor into a carbon reinforced plastic (CFRP) panel. These four sensors were the vibration sensor developed by our laboratory, two distributed sensors based on Brillouin and Raman backscattering and Fiber Bragg Grating (FBG) sensors. By dealing the data obtained from the measurement systems corresponding to these four sensors, strain/stress and temperature distributions throughout the panel can be monitored. Vibration and elastic waves transmitting on the panel are also detected at several sensing points. Furthermore, we will be able to determine damage locations and modes by processing the wave signals. To make the panel with the optical fiber sensor network more sensitive and smarter, we are developing some techniques that can improve the performance of the sensors and can assess the structural integrity by analyzing measurement results. In this paper, the development of the first generation of our smart composite panel with the optical fiber sensors is described and the techniques making the panel more sensitive and smarter are also described.