Byers had put the Swans ahead early on and both he and Oli McBurnie hit the woodwork before half-time.
Charlie Gilmour equalised for Arsenal with a 64th-minute penalty, after Vlad Dragomir had been fouled.
But Byers restored the lead when he turned in Gorre's cross from close range on 85 minutes, before Gorre netted the third in the final minute.
The Swans are seventh in Division 1, five points and two places behind the Gunners.
Swansea: 26 Nov v Sunderland (A), PL2
Arsenal: 24 Nov v West Ham (H), PL2