Let's say Dragons and his immediate family are immune from his warging.
After manipulating everyone to kill the Night King (who was trying to save everyone), he manipulated Dany to destroy King's Landing.
He then 'pushed' Jon to kill Dany and Tyrion to nominate him. Everyone voted for him apart from Sansa. Due to her reistance, he was glad to let her go, but blocked the Iron Islands and Dorne from leaving.
He was glad to see Arya leave, as she couldn't be controlled. He's very concerned about Drogon, as he is a complete wild card in his plans.
No, this twist is even more out there than what we got. If Bran was that powerful, then he wouldn't have to go to all this trouble. He could just warg into anyone close by Dany or Cersei or anyone else and have that person kill them out of the blue.