The building was built for Voivode of the Brześć Kujawski Voivodeship, Rafał Leszczyński. The castle was predominantly used for defensive purposes with keeps in the corners of the structure. The subsequent owners expanded the residence – making the residence into a magnate Renaissance stronghold. In 1853, the partially run-down castle was bought by Count Tytus Działyński, for his son Jan Kantega and his wife Princess Izabela Czartoryska.