I recall that this configuration has been necessary with Ubuntu+Unity+Compiz for years. I just had to figure it out again, so I want to document the solution.
Install CompizConfig Settings Manager, then run it.
Then click General Options then Focus & Raise Behavior then click the + icon next to Focus Prevention Windows and set the following:
Click Add, and you’re done. Your Focus Prevention Windows field now looks something like