Masonry building with frame structure (hereinafter referred to as MBFS), rigid upper and flexible lower, is a special kind of structure appropriate for Chinese economic conditions in current years. Although this kind of structure is proven unstable in earthquake, its significant functional advantages and economic competitiveness make it popular in roadside buildings in small cities. In recent years, MBFS damaged seriously in earthquakes. This paper expounds different models and reasons of seismic damage of this kind of structures. This paper also discusses measures to reduce seismic damage of such structure. In the end, this paper proposes an ideal failure model of MBFS after analyzing the essence of damage. The ideals in this paper can help engineers reducing the seismic damage of MBFS as much as possible.