Virtual manufacturing technology has become an effective method for decision and planning in manufacturing. Due to ergonomics problems are widely concerned in assembly design planning, a human-centered virtual assembly system framework is proposed for ergonomics analysis for assembly operation in this paper. The six-layer framework integrates virtual human modeling, motion capture and recognition, ergonomics evaluation and virtual assembly process planning as an organic whole. Data exchanging and system function are discussed based on this framework. The work in virtual reality (VR) technology, motion capture technology, ergonomics method and optimization method for implementing the system is also described. The framework would provide a new approach for the combination of virtual manufacturing and ergonomics analysis in the future.