The mosque is the largest and most complete mosque in China. Built in 742 during the Tang Dynasty (618-907), the mosque was expanded often, reaching a total area of over 12,000 square meters. Most of its present constructions were built during the Ming Dynasty (1368-1644).

Sitting in the west and facing east, the mosque opens in the direction facing the Islamic Holy Land of Mecca. The mosque has four rows of compounds, with a total length of 245 meters and 47 meters in width. The first and second rows of the compounds have wooden and stone memorial archways and brick-carved tablets. Sitting at the center of the third courtyard is the Heart-examining Building, used to summon followers. The three-storied building has an octagonal pyramidal roof. Its south and north side rooms are the guest room and scripture hall respectively, where precious handwritten copies of the Koran and the Mecca Map of Mohammed's birthplace are kept. The fourth compound is the center construction with three connecting halls. The center hall's Yizhen Pavilion is shaped like a dancing phoenix (also known as the Phoenix Pavilion). The pavilion has two halls built on both sides with precious stone tablets inside. A big platform is located in the front of the pavilion; to its west is the grand main hall that is seven bays wide and nine bays deep. The hall comprises a front corridor, service hall and back hall. The service hall has a capacity of over 1,000 people. The main hall has exquisitely carved beams and columns giving it a splendid and imposing demeanor. It also has over 600 pieces of caisson ceilings decorated with various Arabic patterns. The whole temple harmoniously combines Islamic and traditional Han architectural styles, creating its own features.