fucker

That would be the epithet you apply to someone you'd like to smack in the mouth, but can't / don't because you're not a physically violent person.