The development of a digital material framework is presented, allowing to build virtual microstructures in agreement with experimental data. The construction of the virtual material consists in building a multi-level Voronoï tessellation. A polycrystalline microstructure made of grains and sub-grains can be obtained in a random or deterministic way. A corresponding finite element mesh can be generated automatically in 3D, and used for the simulation of mechanical testing under large strain. In the examples shown in this work, the initial mesh was non uniform and anisotropic, taking into account the presence of interfaces between grains and sub-grains. Automatic remeshing was performed due to the large strains, and maintained the non uniform and anisotropic character of the mesh. A level set approach was used to follow the grain boundaries during the deformation. The grain constitutive law was either a viscoplastic power law, or a crystallographic formulation based on crystal plasticity. Stored energies and precise grain boundary network geometries were obtained directly from the deformed digital sample. This information was used for subsequent modelling of grain growth with the level set approach, on the same mesh.