From reionization until now: The history of the ionizing background