Chelsea took the lead when Eden Hazard scored a 76th-minute penalty on his first start of the season, after Fabian Schar was adjudged to have fouled Marcos Alonso.
Newcastle levelled on 83 minutes when Yedlin crossed for Joselu to head in.
But Chelsea earned their third successive Premier League win under Maurizio Sarri when Alonso's shot was turned into his own net by Yedlin.
The victory puts Chelsea second in the table, level on points with leaders Liverpool and Watford. Newcastle have one point from three matches.
Newcastle: 1 Sep v Man City (A)
Chelsea: 1 Sep v AFC Bournemouth (H)