Integrating and querying data from heterogeneous sources is a hot research topic in database research field. The emergence of the Semantic Web brings new paradigm shift of computing in data integration research where data is heterogeneous and distributed. To solve the problem this paper proposes a semantic-based approach. A Semantic-based Heterogeneous Relational Data Integration System, called SHRDIS, is presented. In this system, ontology is used as the mediated schema for the representation of the data source semantics. SPARQL queries over global schema are rewritten into local SQL queries that can be executed on heterogeneous relational databases. The architecture and implementation of SHRDIS is illustrated in detail. The experiment results show that the SHRDIS system has nice performance and scalability.