Most people, including me, identify castles with the Middle Ages, even though, as I discussed earlier, castles were really only a feature of the period of the eleventh through fourteenth centuries.
The earliest castles were largely wooden, but very quickly they began to be built instead in stone. A wooden castle could be set on fire, but with a stone castle, all that would burn were the various stables, kitchens, barracks and the like inside the walls--and even those were soon being built of stone.
Originally castle towers were square and built of field stone, that is stone picked up and brought to the site. They still look grim and forbidding, and they certainly did at the time. Early castles were generally built by the very wealthy and powerful, such as the counts of Anjou, who built dozens of castles in the early eleventh century. (These were however plastered and whitewashed, which might have made them look less forbidding, though rooms were still dark inside.)
The keep, the original big square tower with the great hall on its main floor, remained as castles were rebuilt in subsequent generations. Both advances in military technology and changes in fashion meant that any castellan lord with any sense of self-esteem (or self-preservation) was always rebuilding. New towers were often round rather than square in the twelfth and thirteenth centuries, both because they thus would not have a corner that could be undermined, and because it was easier to see in all directions. These towers were pierced with numerous arrow-slits, narrow on the outside to present a small target, but wide on the inside, so an archer could shoot in either direction.
By the middle of the twelfth century, castles were being built of quarried stone rather than field stone. Limestone and sandstone were preferred as relatively easy to quarry. Walls were built very thick, and passages and stairways were often built into the walls.
No two castles were alike. Each one was suited to its site. Natural rock outcroppings were incorporated if possible. A sharp bend in a river was often chosen, as providing a natural moat. A steep hill looked to medieval lords like the obvious place to put a castle. The Auvergne region of central France still has a certain number of castles built (somewhat precariously) on the sharp cones of extinct volcanos.
Once gunpowder came in, toward the end of the Middle Ages, the dynamics of castle building changed, because castles no longer were the impregnable fortresses they had been earlier. As long as walls could be taken down with cannons anyway, those rebuilding decided they might as well have nice windows and let in some light.
Most medieval castles are now in ruins. They have been done in not just by time but also by deliberate efforts in the early modern period. Louis XIV in the seventeenth century did not want castles where rebellious nobles might try to defy him, so he sent his minister, Vauban, around France with a number of cannons to systematically break down castle walls. The English Civil War of roughly the same period had the same result.
Some European towns and villages now have a big empty plaza where a castle used to be. In other cases, the castle is still there and has been incorporated into the local architecture.