"I’m assuming that the Clinesmith indictment will be the only one," said Gowdy, who also once led the House Oversight committee