The substitute ended Leicester's resistance within four minutes of coming off the bench before Marouane Fellaini, also a late replacement, made sure of the win.
After a frustrating first half for United, who had a Juan Mata strike ruled out for offside, they were denied again shortly after the break.
Kasper Schmeichel stopped Romelu Lukaku from becoming the first player to score in each of his first three United appearances by saving his penalty after Danny Simpson had handled Anthony Martial's cross.
But Rashford struck from Henrikh Mkhitaryan's 70th-minute corner, the Armenian's fifth assist of the season, to Old Trafford's relief.
Fellaini then converted Jesse Lingard's cross with eight minutes remaining to take Jose Mourinho's side back to the top.
Man of the match: Paul Pogba (Man Utd)