The visiting team had lost four in a row and not won in seven, but started well.
After Milivojevic's free-kick went just wide, they led on 23 minutes when Tomkins prodded home from close range at the second attempt.
In the second half, Tomkins had a shot cleared off the line while Huddersfield Town striker Steve Mounie fired just wide on the turn.
Milivojevic made sure of the points on 68 minutes with a penalty after Andros Townsend had been fouled by Zanka.
Palace rise to 16th, two points clear of the bottom three, a place and a point behind Huddersfield Town.
Huddersfield Town: 31 Mar v Newcastle (A)
Crystal Palace: 31 Mar v Liverpool (H)