With one side facing the land, the city's other three sides face the sea. A military institute was established in the city in 1079 during the Northern Song Dynasty (960-1127). Its present name, Chongwu, (to emphasize military affairs) came from the early Ming Dynasty (1368-1644). The city walls, moats and coast defense were built beginning in 1387 in the Ming. Chongwu City is one of the best-preserved coastal stone cities of ancient China.

The city walls were built with layers of granite with a perimeter of 2,456.7 meters. Its base is 5 meters wide and 7 meters high, and the arched gateways are open on all four sides. Semi-circular enclosures were built between the outer and inner city gates on the east, north and west sides, and each of the east, south and north walls have a beacon tower on it. The height of city walls was increased by more than 1 meter in 1417.

Over 100 years after Chongwu City was built, Japanese pirates failed to invade the country and the coastal area was kept away from intrusions. Renowned General Qi Jiguang, who led the war against Japan, trained his troops there in April 1567. In April 1651, Zheng Chenggong took advantage of the strategic barrier to resist troops sent by the Qing (1644-1911) regime.