On a day in which Palace's Roy Hodgson became the oldest manager in Premier League history, it was Leicester who started strongly, with Jamie Vardy's header going narrowly over.
But Palace made the breakthrough in the 40th minute when Batshuayi diverted James McArthur's wayward shot into the net.
On 64 minutes, Leicester were level when Jonny Evans fired in Harvey Barnes' low ball into the box.
Palace were back in front six minutes later, after a clever cross from McArthur found Zaha and he slotted into the bottom corner.
Luka Milivojevic made it 3-1 with an 81st-minute penalty after Jeffrey Schlupp had been fouled, before Zaha wrapped up the win with a goal on the counter-attack in stoppage time.
Palace move up to 13th on 30 points, six points above the relegation zone.
Leicester are two points and a place above Palace.