While soliciting prostitution is legal in certain areas of Colombia, it is considered a breach of the agency's conduct code, the government sources said. High-level officials in the Secret Service and the Department of Homeland Security were outraged over the incident, the sources said, noting that the investigation indicated the prostitutes were brought back to a hotel that had been secured for the summit.

The president arrived in Cartagena on Friday, the same day he first learned about the incident, White House press secretary Jay Carney said from the coastal resort city. Obama will spend more time in Colombia, where security concerns had limited previous presidential trips, than any other U.S. president.

Two nearly back-to-back explosions in Cartagena -- which caused only minor damage and no casualties, Colombia National Police spokesman Cantihho Toncell said -- were a reminder of the violence that has gripped Colombia as its government has battled powerful drug cartels. Violence has significantly fallen off in recent years as the Bogota-based government, aided by U.S. extradition efforts, has successfully picked apart the cartels.

Still, there is an extensive security presence in the walled colonial city of Cartagena for the summit -- including more than 7,600 police officers and thousands more troops.