With interim first-team manager Michael Appleton watching on, Muskwe's header opened the scoring on 13 minutes.
Swansea's Jack Bainbridge was dismissed for a high challenge just before half-time, and Muskwe doubled the lead in the 59th minute from Liandro Martis's cross.
Leicester, unbeaten in six matches across all competitions, visit Liverpool on 29 October. Swansea host Everton on 27 October.
Watch Muskwe's goals
(Photo courtesy of Plumb Images/LCFC)