Zero Kilometre Stone in Clark Adam Square. Buda side of the Chain Bridge, at the bottom of the castle hill.
Alternatively, if you wish you can meet the group at the top, at the "Fountain of the Fishing Children", situated next to big bird statue Turul.
Then the tour guide can escort you back to the starting point.