Gwargis, 19, had rescued a point against Wolverhampton Wanderers earlier in the week and the Swedish midfielder struck a powerful long-range finish past goalkeeper Matt Macey on 53 minutes.
Even though Gunners captain Robbie Burton was sent off for a second bookable offence on the hour, they nearly equalised in stoppage time when James Olayinka headed against the crossbar.
The result means Arsenal stay second, level on points with leaders Chelsea having played two matches more. Brighton are four points behind in fifth.
Rovers inflicted Leicester's first loss since September with strikes from Dominic Samuel, Daniel Butterworth and Brad Lyons.
Admiral Muskwe got the goal for a Leicester side who remain one point behind the top two. Blackburn are up to eighth.