Because the city is already a popular tourist destination, Rosentraub said London shouldn't expect much of an economic boost from tourism.

"Some cities really buy this tourism argument," Rosentraub said. "But we have studies up the wazoo showing that it will never happen."

And costs are already mounting. The British government has raised its initial $4 billion cost estimate to nearly $15 billion. Some estimates project an even higher cost.

But London is also accomplishing some major infrastructure goals, including projects designed to give the long-suffering East End neighborhood a facelift.

The final verdict? "London probably won't lose too much," Rosentraub said.

Szymanski, meanwhile, expressed dismay that politicians continue to tout the Olympics as an economic boost, and not just a great sporting event.

"I think the Olympics is and should be a great sporting event, but it is not and should not be considered a major economic event," Szymanski said.

"It's a lot like having a party," he added. "It's a good time but it doesn't make you rich."