If a player is going to take a quick tap, the emphasis is on "quick", so he may well have to guess where the correct spot is. Unless you think the player is cheating in some sense, then calling him back will lose the element of surprise, so the benefit is lost. That seems fair for an understandable misapprehension. Once is happenstance, twice is coincidence, the third time it's enemy action. A bit like repeat infringements ...
It may also depend on the circumstances. Any scrumhalf should know that if a PK is awarded at a scrum, he cannot take the tap from the side of the scrum.