The location of the Diogenes Club near The Mall may actually help explain why Mycroft has a strange bunker office. The building is right by an area with a number of known underground government citadels, tunnels, etc. The Admiralty Citadel, for example, is just on the other side of The Mall. And the building’s so close to the known portion of the Q-Whitehall facility that the writers wouldn’t even have to try hard to claim the Diogenes bunkers were part of the expansion of those tunnels that’s still locked up as a secret in the National Archives for another decade or so. (And all this reminds me I should really finish that post series…)
TL;DR version: Sherlock asked to be taken to The Mall because it’s a convenient drop-off point for the Diogenes Club, home of Mycroft’s office.