A novel nanofiller containing layered organo-modifed montmorillonite (oM) and mesoporous silica micro-sphere (MS) was prepared by in situ deposition method, and the microstructures and morphologies of this nanofiller were characterized by Fourier transform infrared spectrometry (FTIR) X-ray diffraction (XRD) and transmission electronic microscopy (TEM). EP/oM-MS nanocomposite was obtained by adding oM-MS to EP matrix. Morphologies and mechanical of the new ternary nanocomposite were investigated. For purpose of comparison, the corresponding binary nanocomposites, i.e., EP modified with either oM or MS, were tested as well. The test results of mechanical properties show that oM obviously improves the strength of EP and MS enhances the toughness of EP, but oM-MS exhibits synergistic effect on toughening and reinforcing of EP.