"Furious would be an understatement," Boylan told ABC's "Good Morning America." The couple has two adult children, including a son who led an infantry platoon in Afghanistan as an Army lieutenant.
Broadwell is married with two young sons and lives in Charlotte, N.C. She has not returned phone calls or emails seeking comment.
As the criminal investigation continued into the emails to Kelley, FBI Director Robert Mueller and eventually Attorney General Eric Holder were notified that agents had uncovered what appeared to be an extramarital affair involving Petraeus, said one of the law enforcement officials.
Broadwell and Petraeus have each been questioned by FBI agents twice in recent weeks, with both acknowledging the affair in separate interviews. The FBI's most recent interviews with Broadwell and with Petraeus both occurred during the week of Oct. 29, days before the election, one of the law enforcement officials said. The FBI notified Obama's director of national intelligence, James Clapper, of the investigation on Tuesday Nov. 6, Election Day.
Clapper called Petraeus that night and urged him to resign. Clapper informed the White House late Wednesday, and aides informed the president Thursday morning, before Petraeus came to personally hand in his resignation letter.
Some members of Congress are questioning why they weren't told sooner. Democratic Sen. Dianne Feinstein of California, who heads the Senate Intelligence Committee, said she wants to investigate why she had to find out from news reports Friday.
But there were at least a couple of members of Congress who heard inklings of the affair before the election. Republican Rep. Dave Reichert of Washington state received a tip from an FBI source that the CIA director was involved in an affair in late October. Reichert arranged for an associate of his source at the FBI to call House Majority Leader Eric Cantor on Saturday, Oct. 27, according to Cantor spokesman Rory Cooper.
The FBI agent who contacted Reichert was the same one who first received the allegations from Kelley, a federal law enforcement official said Monday night. That agent's role in the case consisted simply of passing along information from Kelley to the FBI agents who conducted the investigation, but that agent was subsequently told by his superiors to steer clear of the case because they grew concerned that the agent had become obsessed with the investigation, the official said. The agent was a friend of Kelley and long before the case involving Petraeus got under way, the agent had sent Kelley shirtless photos of himself, according to this official. The Wall Street Journal first reported that this FBI agent was kept away from the case.
Cooper told The Associated Press Monday that Cantor notified the FBI's chief of staff of the conversation but did not tell anyone else because he did not know whether the information from a person he didn't know was credible.
"Two weeks ago, you don't want to start spreading something you can't confirm," Cooper said.
The FBI responded by telling Cantor's office that it could not confirm or deny an investigation, but assured the leader's office it was acting to protect national security. Cooper said Cantor believed that if the information was accurate and national security was affected, the FBI would, as obligated, inform the congressional intelligence committees and others, including House Speaker John Boehner.
One of the law enforcement officials who spoke to the AP said long-standing Justice Department policy and practice is not to share information from an ongoing criminal investigation with anyone outside the department, including the White House and Congress. The official said national security must be involved to notify Capitol Hill, and that was not the case in the Petraeus matter.
Petraeus' affair with Broadwell will be the subject of meetings Wednesday involving congressional intelligence committee leaders, FBI deputy director Sean Joyce and CIA Deputy Director Michael Morell.
Petraeus had been scheduled to appear before congressional committees on Thursday to testify about the Benghazi attack that killed four Americans, including U.S. Ambassador Chris Stevens. Morell is expected to testify in place of Petraeus.
Feinstein and others didn't rule out the possibility that Congress will try to compel Petraeus to testify about Benghazi at a later date, even though he's relinquished his job.