When the Kanamori clan was transferred to Dewa Province in 1692, the castle came under the control of the Maeda clan. Three years later Hida Province became tenryō territory, under the direct control of the Tokugawa shogunate, and the buildings of the castle were pulled down, with a jin'ya built at the foot of Shiroyama.
