The visitors began brightly at Aldershot and took a deserved lead in the 26th minute through Jack Roles.
Chelsea had a chance to level on 65 minutes when they were awarded a penalty for handball. But Billy Gilmour saw his effort saved by Spurs' American goalkeeper Brandon Austin.
The home team were not to be denied a second chance as Brown, a substitute, scored his 13th goal of the season on 89 minutes.
The point takes Chelsea to third on 18 points in Division 1 while Spurs drop to ninth on 12.
The Foxes goalkeeper made a number of saves to claim a point, not least from Nabil Touaizi's second-half penalty.