Mr. Cornelis Speelman became the next lord in 1740, because he had married Jacob's sister Agatha van der Hoeven in 1716. Cornelis was a grandson of Cornelis Speelman (1628-1684), Governor-General of the Dutch East Indies. Cornelis became schepen in 's-Hertogenbosch, and died in 1746. He was succeeded by mr. Jacob Speelman (1722-1787), who made an impressive career in Leiden. In 1788 he was succeeded by his son mr. Cornelis Jacob Speelman (1747-1825). During his tenure the French general Pichegru used the castle as his headquarters. The French period also meant the abolition of the feudal rights. Cornelis Jacob was succeeded by his son Abraham Florentius (1784-1840), but in 1834 the family sold their property in Heeswijk and Dinther.
