You don't choose a target for Slumbering Walker's ability at the time it triggers. Rather, a second "reflexive" ability triggers when you remove a counter from Slumbering Walker this way. You choose a target for that ability as it goes on the stack. Each player may respond to this triggered ability as normal.