"Last year we were confronted with the consequences of decades of haphazard policymaking and underinvestment in the criminal justice system - bringing it to the brink of collapse," said review chief Mr Gauke.
The Independent Sentencing Review also said there's been a "knee jerk" and "unstrategic" reaction over the decades towards longer jail terms, despite an overall fall in crime since the mid-1990s.
Successive governments trying to look "tough on crime" have driven a rising prison population that's overwhelmed the system, according to a report.
"It is clear that in order to address the capacity issues we face, we must have an honest conversation about who we send to prison, and for how long," said Mr Gauke.
It also said the rising numbers locked up had been "catastrophic" for prisons and the probation service - with resources diverted from reoffending and safety to emergency measures to cope with demand.
