Chasing 91, Rohit scored 46 off 20 balls which included four boundaries and as many sixes to take India over the line with four balls to spare in the rain-affected game.
Earlier, wicketkeeper Matthew Wade scored an unbeaten 43 to power Australia to a challenging 90 for 5.
The match began two-and-a-half hours after its designated time due to a wet outfield and was reduced to an eight-over-a-side contest.
Axar Patel (2/13) bowled a fiery spell, snaring two wickets.
Skipper Aaron Finch, who scored a 15-ball 31, and Wade (43 off 20) were the top contributors for Australia.
Australia: 90 for 5 in 8 overs (Matthew Wade 43 not out, Aaron Finch 31; Axar Patel 2/13)
India: 92 for 4 in 7.2 overs (Rohit Sharma 46 not out; Adam Zampa 3/16)