Matt Targett's curling cross enabled Danny Ings to head home the opener after 20 minutes but eight minutes later, Henrikh Mkhitaryan guided in a Nacho Monreal centre.
Southampton were back ahead just before half-time with Ings rising above the Arsenal defence to loop a header into the top corner.
Substitute Alexandre Lacazette's backheel teed up Mkhitaryan to fire in a deflected shot off Jannik Vestergaard on 53 minutes.
But Saints had the final say, Austin heading in with five minutes to go to halt Arsenal's 14-match unbeaten run in the Premier League.
Southampton are on 12 points and move out of the bottom three on goal difference while Arsenal stay fifth, three points behind Chelsea.
Southampton: 22 Dec v Huddersfield (A)
Arsenal: 22 Dec v Burnley (H)