In the pub there will be a person called "Pub visitor". He may be in three states: Drunk, Tipsy, and Sober. If he is drunk, he will accept any fight you provoke him into. If he is Tipsy, he may or may not fight, and he may refuse to drink with you. If he is sober, he will refuse to fight or drink unless you insult him.
Talking to the Pub Visitor gives you a few options:
If you insult him he will go outside with you and fight you together with his friends with their swords. This is not always very easy because you have no armour and only have a sabre. If you beat him and his friends, you will get a greater thaler reward than if you had a fistfight with him. However, your relationship with the town you are in will decrease.
You can ask him to measure your strength with him for a bet from maximum 25 denars (50 denars winning reward), if you win you get 2 experience from the fight, the double amount you betted + 50 experience. There is no penalty to your relationship with the town, but this kind of fight is the hardest to get the Pub Visitor to accept.
You can raise your glass for a leader of a faction. However, if you drink to the leader of a faction other than the faction that the town belongs to, the Pub Visitor will attack you.
Note: If you fight him multiple times, he will say he is fatigued for awhile.