For a continuum structure filled with many solid phases with different stiffness, the layout of the materials should be optimized to perform an excellent structural mechanical behaviour. The differences between the materials’ moduli affect the final layout of materials in structure. To investigate the effects, a numerical approach is adopted to find the optimal topology of a continuum with multiple materials. In the method, the layout of the material is determined by the local strain energy density (SED). Briefly, in design domain, a material point with higher SED should have higher modulus. To show the effects obviously, a continuum with three phases is used for experiment. In the structure, the modulus of the stiffest material and the volumes of materials are fixed. The other materials’ moduli change alternately. Numerical results demonstrate the layout of materials in structure depends on the differences between the moduli greatly, and the topology of the stiffest material is clear if its modulus is far more than the others.