Medieval society was violent--but not anarchical. Efforts were constantly being made, some more successful than others, to rein in violence. When you have a number of muscular, well-trained fighting men carrying swords, it was, not surprisingly, a constant concern.
In the modern US, we tend as a society to fear those of lower socio-economic status. Poorer parts of town are considered dangerous. In the Middle Ages, in contrast, the powerful, the wealthy, the elite were considered the most frightening. On the other hand, poor people's only defense against dangerous warriors was found in other wealthy and powerful men.
The reason villages grew up around castles was because the villagers saw them as sources of protection. A lord would certainly defend his own peasants, even if he ran roughshod over someone else's peasants' fields (or even the peasants themselves).
Knights and nobles were trained to be fighters. From the twelfth century on, nobles defined themselves in part by their military prowess. A touchy sense of honor, where every insult had to be met with violence, was a constant issue.
Of course, this violence was not unmitigated. For one thing, it is much harder to kill someone from a distance with a sword than with a gun, so until gunpowder was invented in the fourteenth century, fighting had to be up-close. (Nobles used bows for hunting, but not for attacking each other.)
Also, "peace" was considered an extremely important value. If two knights insulted each other and then sprang at each other, their friends were supposed to pull them apart, not egg them on. Everyone knew all too well that a fight with knives and swords would turn deadly all too fast.
At exactly the same time as knights and castles first appeared in France, in the years around 1000, bishops started holding Peace of God councils. They would bring knights together and make them swear mighty oaths, on relics, not to harm the harmless--churchmen, women, merchants, peasants. This was reasonably successful, at least in getting the knights to consider that killing the weak was wrong and might even send them to hell.
Two generations later, the bishops followed up on their initial success by establishing the Truce of God, where they persuaded knights that they shouldn't even kill each other on Sundays, great saints' feast days, Advent, or Lent. This was not nearly as effective, but it certainly made the knights think. By the twelfth century, everyone more or less agreed (whether or not they always acted on it) that Christians should not kill Christians.
Churchmen instead urged the knights to go on Crusade, to use their warrior skills to kill Muslims. The knights themselves preferred tournaments, mock battles where people (theoretically) were not killed, but they were able to show off their prowess.
There were of course psychopaths in the Middle Ages, just as there are now, people who enjoyed inflicting pain and suffering. The chroniclers always described such acts of violence as "unheard of," indicating that they were far from the norm.
See more here on medieval conflict resolution.