You need magical weapons, Fluent. I believe +1 will kill him the first time, but you might need +2 the next time you meet him. Go to the shop of the lady you rescued and she'll have magical weapons. You can use the crown to get out of the dungeon, temporarily, or just use the encounter button and speak home. I hope you're not fighting him in hopes of good loot. He's a rat

He's also not the main baddie you have to kill to get the reward from the inn. Thought I should add that….