Needing a win to go above Liverpool at the top of the table, Man City's first clear opening came on 32 minutes when Sergio Aguero's header was brilliantly clawed off the line by Leicester goalkeeper Kasper Schmeichel after it hit the woodwork.
Ilkay Gundogan twice went close either side of half-time, before Leroy Sane's introduction gave fresh impetus to the defending champions.
Aguero was denied superbly by Schmeichel again before Kompany hammered a 25-yard shot into the top corner on 70 minutes.
Former Man City striker Kelechi Iheanacho spurned a great opportunity to level for Leicester, but Man City held on to go top on 95 points, one point above Liverpool with one match left.
Leicester City stay ninth on 51 points.
Man City: 12 May v Brighton (A)
Leicester: 12 May v Chelsea (H)