When you activate Rubble Rouser's mana ability, it won't deal damage to each opponent right away. Instead, a second "reflexive" ability triggers whenever you activate that ability. If you activated the mana ability during the process of casting a spell or activating an ability or during a window to activate mana abilities when you're asked to pay a cost during the resolution of another spell or ability, that ability won't go on the stack until you're done casting, activating, or resolving that spell or ability. Each player may respond to that triggered ability as normal.

Rubble Rouser's last ability is a mana ability. It doesn't use the stack and can't be responded to.