The temple is one of the famous Eight Outer Temples of Chengde, which were constructed in succession between the 52nd year (1713) of Emperor Kangxi's reign and the 45th year (1780) of Emperor Qianlong's reign of the Qing Dynasty (1644-1911). Originally, there were 11 temples, but only seven of them and one site remain. The Eight Outer Temples include the Puren Temple, the Pule Temple, the Anyuan Temple and the ruins of the Pushan Temple to the east of the Chengde Mountain Summer Resort, and the Puning Temple, the Sumeru Fushou Temple, the Putuozongsheng Temple and the Shuxiang Temple to the north.

In the 26th year (1761) of Emperor Qianlong when the empress dowager celebrated her 70th birthday, the emperor accompanied his mother to Wutai Mountain to offer incense to Buddha. According to legend, Wutai Mountain is where Manjusri Bodhisattva performed Buddhist rites, and he once appeared at the Shuxiang Temple. Having seen the statue of Manjusri Bodhisattva , its image standing in the temple lingered in the empress' mind. In the 39^th year (1774) of Emperor Qianlong's reign, he ordered the erection of a temple at the Chengde Summer Resort Manor to imitate the Shuxiang Temple in Wutai Mountain. He named it Shuxiang Temple. The temple had stretches of forests and rocks and grand pavilions, but only small sections of its broken walls remain.