The thing with privilege is one who has it needs to learn privilege exists and is real. In this usage I refer to privilege as having qualities congruent with the English speaking world's most desired qualities. I'll refer to this as the societal default state.
I'll also posit that the English speaking world's default is an athletic white male. Male privilege is hardly restricted to anglophones, but for the purpose of this musing I'll draw my lasso around the English world and concentrate there.
You're white? White privilege. You're male? Male privilege. You match up with het male sex (for women) or power (for men) fantasies? That's another privilege.
Straight privilege, wealth privilege, power by election/appointment privilege. The list goes on.
The default is largely invisible to those who fit into these defaults.
The default is much like wearing motocross armour wherever you go. Bump into someone else with the same armour, neither of you are likely to notice. Bump into someone without the armour, they will notice while the one fitting the default might not.
Bump into someone and they say, "Hey, dude, watch the elbows?" a reasonable response would be, "Oh, sorry, you okay?" "S'cool." And life goes on, a little more aware of elbows for awhile until they're forgotten about and then another bump in a crowd. Rinse, repeat.
Replay that in the context of race/gender/other privilege:
Bump into someone and they say, "Hey, dude, watch the elbows?" "No I didn't." "Uh, yeah, you did. Just watch the elbows, k?" "I did not, I didn't feel a thing." "With the armour you're wearing, it's no wonder. Just watch the elbows. Please." "I am not an elbowist, how dare you accuse me of such a thing." "wtf..."
Now, with sane reasonable people in a social context:
Say a borderline offensive joke. Someone with or without the privilege says, "Dude, that was kinda rasist/sexist/otherist." A reasonable reply would be, "Oh. Sorry about that." Another reasonable reply might be, "Oh, what? Sorry I'm a bit unclear here, elaborate?" An unreasonable response might be, "No it wasn't, you're just being a wuss." Casual dismissal. Cute.
One of the things that led to the motocross armour analogy:
Please watch that, it'll make you a better person.
And remember, they're just elbows. They can be used with intent, or by accident. And it's up to us to behave like reasonable adults.
Edit to add: The job of the people with 'privilege', then becomes to cultivate a safe enough place that when the elbows give some bumping, that the bumpee can feel secure that their protest will be taken seriously and not dismissed out of hand, ridiculed, or attacked.