Electro-magneto-rheological (EMR) fluids, which exhibit Newtonian behavior in the absence of a magnetic field, are abruptly transformed within milliseconds into a Bingham plastic under an applied magnetic field, called the EMR effect. Based on this effect, the particle-dispersed EMR fluid is used as a special instantaneous bond to cohere abrasive particles and magnetic particles together so as to form a dynamical, flexible tiny-grinding wheel to machine micro-groove on the surface of optical glass. Experiments were conducted to reveal the effects of process parameters, such as the feed rate of the horizontal worktable, feeding of the Z axis, machining time and machining gap, on material removal rate of glass. The results indicate that the feed rate of the worktable at horizontal direction has less effect on material removal rate, which shows a fluctuation phenomenon within a certain range. The feed rate of the Z axis directly influences the machining gap and leads to a remarkable change on material removal rate. Larger material removal rate can be obtained when the feeding frequency of Z direction is one time per processing. With the increase of rotation speed of the tool, material removal rate increases firstly and decreases afterwards, and it gets the maximum value with the rotation speed of 4800 rev/min. The machining time is directly proportional to material removal amount, but inversely proportional to material removal rate. Furthermore, material removal rate decreases with the increase of the machining gap between the tool and the workpiece. On the basis of above, the machining mode with the tiny-grinding wheel based on the EMR effect is presented.