Located in the high part of the town, the castle is the residence of the Duke and Duchess of Frías, constituting a private property that is not open to visits.
Martín Alonso de Córdoba commenced construction in around 1340, and the castle, which represented the foundational nucleus of the new town, underwent successive extensions and reformation, which explain its current aspect: it consists of three towers in a triangular layout around the patio de armas.
The towers are referred to as Mocha, the Torre de las Palomas and the Torre del Homenaje, which is the tallest and most magnificent.