It’s easy to judge people by the way they look, because he’s pretty sure that’s how people look at him too. So it allows him to be as insistent as he wants, because he’s pretty sure he knows better than the other guy, even though he knows he doesn’t look like he does.
People say you don’t really know a person until you’ve had dinner with them, but it takes three dinners, a lot of wine, and more than a day’s journey to finally discover who’s been hiding underneath the hood all along. And then he’s just ashamed of himself, because even he isn’t everything he wanted to be.
As penance, he hides in the safety of what he doesn’t know, and learns to be content with that.