A third legend linked the fort to the well known king of Rajasthan. Prithviraj brought this settlement under his rule and also many neighbouring villages. Subsequent to his demise, his descendants sold the land to Khair-ud-din Subuktagin. An inscription (not very legible) in Sanskrit on the gateway of the fort mentions that the place had been taken over by Swami Raja, who had established a stone tablet recording the event. Later, the fort was acquired by Sahab-ud-din, who in turn appointed a sanidi, an African, and a Bahelia, and gave them titles as hazari (meaning "governors of the fort"), and also conferred a jagir. It is said that the Bahelia family were in possession of the land holdings till the fort was finally ceded to the British in 1772.
