Finite element model updating of structures usually ends up with a nonlinear optimization problem. An efficient optimization technique is proposed firstly, which draws together the global searching capability of chaos-based optimization technique and high searching efficiency of trust-region Newton method. This hybrid approach is demonstrated to be more efficient and prone to global minimum than conventional gradient search methods and random search methods by testifying with three test functions. The optimization problem for model updating using modal frequencies and modal shapes is formulated, and a procedure to update the boundary support parameters is presented. A modal test was conducted on a beam structure, and the identified mode frequencies are employed to formulate the optimization problem with the support parameters as the updating parameters. The discrepancy between the mode frequencies of the finite element models before and after updating is greatly reduced, and the updated support condition meet quite well with the insight to the devices that form the supports.