In the former research, the Damper control system, TMD control system and the Mixed Control of the active control and passive control were all optimized alone. It is found that less research of the optimization is on the Mixed Control of the Damper system and TMD system. In order to achieve better control effect, we venture to propose a Mixed Control system of third-generation three-dimensional 20-storey Benchmark model under the multi-directional earthquake in this paper. By means of Rayleigh damping and D-value, dynamic equations including the damping matrixes, stiffness matrixes of the structure are obtained. Then in the process of the optimization, the displacement reducing coefficient (DRF) is used as the objective function to optimize the placement of damping devices and control parameters based on Genetic Algorithm (GA). Numerical results in this paper show that the optimal design method proposed in this paper is effective and flexible. It can obviously reduce the seismic responses of building structure.