BT resins composed of 4,4’-bismaleimidodiphenylmethane (BMI) and 2,2’-bis-(4-cyanatophenyl)propane (BCE) were modified by octa(maleimidophenyl) silsesquioxane (OMPS). It was found that the curing reaction of BCE were accelerated by OMPS, and the onset temperature of the cyclotrimerization was reduced up to 95.5°C (by DSC). As demonstrated by DSC and FT-IR, there was no evidence indicated the co-reaction between maleimide and cyanate ester. 2,2’-diallyl bisphenol A (DBA) and diglycidyl ether of bisphenol A (E-51) were also used to enhance the toughness of BT resins, and the formulated BTA and BTE resins were obtained. The results of DMA and TG show that the BT, BTA, and BTE resins containing 1wt% of OMPS exhibit enhanced thermal properties in comparison with pristine BT0, BTA0, and BTE0 resins, while more content of OMPS may impair the polymer matrix, though the effect of OMPS was slight. The dielectric constant of these hybrid materials were reduced by incorporation of OMPS, while overmuch contents of OMPS were disadvantageous for dielectric constant due to the aggregation of OMPS.