I have always gave the gil back when they sent me a /tell. If it was a message than I'd reply saying I'm willing to give the gil back if /trade not delivery box. Most of the time they'll just tip ~10% for being kind.
I've heard of people that do the same, but depends on the gil amount of the overbid. if it's 100k or more than it's a loss on the mistake. Still rude to do since that person may have worked months to get that much gil to spend.
Now if it was someone I deemed as a fool and treats people disrespectfully, I'd keep it. Ignore the /tell or message. Hopefully they'll change their attitude after that.