The construction of the Bütow castle began with the Teutonic Order in 1390, during the reign of pfleger Jakob von Reinach. The construction works completed in 1405 were managed by an outstanding Teutonic builder Nicholas Fellenstein, brought from Malbork. He probably was the author of the concept deviating from the typical pattern of the conventual Teutonic castle and allowing the use of firearms for defense. At the end of the fourteenth century, it was an innovative solution, probably introduced due to the great importance of the castle, located in the border areas, near an important trade and communication route connecting the Teutonic Knights state with ethnically German territories, from which military and material help for the order flowed.
After the defeat of the order in the Battle of Grunwald in 1410, the castle and the Bytów land briefly came under Polish sovereignty, but after signing the First Peace of Toruń in 1411, the Teutonic garrison returned. In the following years, due to the financial problems of the order, the castle declined, especially since in 1433 it was damaged during the siege by the Hussite army. This was mentioned by the letter of the Bytów keeper to the Grand Master of the Teutonic Knights from 1451. Three years later, after the outbreak of the anti-Teutonic uprising and the thirteen years Polish-Teutonic war, the castle surrendered without a fight to the Gdańsk burghers, and in 1455 they handed it over to King Kazimierz Jagiellończyk. In exchange the Polish ruler gave Bytów to the prince of Słupsk Eric II for his services, but due to weak defense, in 1460 the castle fell into the hands of the Teutonic Knights again. Finally, only under the Second Peace of Toruń of 1466, the entire Bytów land together with the castle was incorporated into the Polish kingdom, although Kazimierz Jagiellończyk once again handed it over to fief of Eric II.
From 1466, the castle belonged to Pomeranian dukes and was the seat of their castellans. The successor of Eric II, Bogusław X, began the Renaissance transformation of the stronghold. It received earth fortifications, corner bastions and a transformed entrance. Around 1570, new economic and residential buildings began to be erected along the walls: the so-called Ducal House on the south-eastern side, and the office on the south-west side. In addition, in 1623, Bogusław XIV founded a house for princely widows at the northern corner. Being an important seat of Gryfit dynasty, the castle transformed into a comfortable, early modern residence.
The first losses castle suffered during the Thirty Years War, when the imperial army devastated Bytów. In 1638, after the childless death of the last Gryfit ruler, Bytów together with the castle passed into Polish hands, being the seat of starosts. During the Polish-Swedish wars the castle burned down, and a corner four-sided tower was blown up. Then, partly rebuilt, it served as a court and prison, and after World War II it was finally renovated with the seat of the West Kashubian Museum.