The only reason to pull him now is because the mound is in his head. He's not that unsafe.
It was in his head. He's been checking his landing spot for a while now. That slip may not have tweaked anything, but it is the kind of thing that can make you think that you could if it is happened again. That's in his head. Riggs gave him a chance to refocus and he couldn't.