The Sunni Muslim coalition, which has been battling the Iran-aligned Houthis for over four years in Yemen, said it destroyed a site storing drones and missiles in Dhamar.
Franz Rauchenstein, head of the International Committee of the Red Cross delegation in Yemen, said after visiting the prison complex and hospitals on Sunday that a “safe presumption is that over 100 had been killed”.
The Houthi health ministry earlier said at least 60 bodies were pulled from the rubble at the detention centre, which officials said housed 170 prisoners.
“There are three buildings hit and the building where the detainees were located, most of them or the majority has been killed,” Rauchenstein told Reuters by telephone.
“The prisoners in that facility were prisoners that we had visited in relation to the conflict.”
He said the Yemeni Red Crescent Society was still trying to retrieve bodies and that around 50 injured people had been taken to hospital.
The Yemen UN Office of the High Commissioner for Human Rights said 52 detainees were among the dead. At least 68 detainees are still missing.
“I hope the coalition will launch an inquiry into this incident. Accountability needs to prevail.” Martin Griffiths, Special Envoy of the Secretary-General for Yemen, said in a statement.
Residents told Reuters there had been six air strikes.
“The explosions were strong and shook the city,” a resident said. “Afterwards ambulance sirens could be heard until dawn.”
The coalition, which has come under criticism by international rights groups for air strikes that have killed civilians, said it had taken measures to protect civilians in Dhamar and the assault complied with international law.
The Western-backed alliance intervened in Yemen in March 2015 against the Houthis after they ousted the internationally recognised government from power in the capital, Sanaa, in late 2014.
The movement, which holds most major population centres in the Arabian Peninsula nation, has stepped up cross-border missile and drone attacks on Saudi Arabia in recent months. The Saudi-led alliance has responded with strikes on Houthi targets.
“The crime of the aggressors against prisoners is additional proof that they are ready to violate all Yemenis, even those loyal to them,” the Houthi-run Al Masirah TV quoted the group’s leader Abdul Malik al-Houthi as saying.
The head of the Houthis’ national committee for prisoner affairs, Abdul Qader al-Mortada, said many of those held were due to be released in a local deal to exchange prisoners of war.
The United Nations is trying to ease tension in Yemen to prepare for political negotiations to end the war that has killed tens of thousands and pushed the long-impoverished country to the brink of famine.
The conflict is widely seen in the region as a proxy war between Saudi Arabia and Iran. The Houthis, who deny being puppets of Tehran, say they are fighting a corrupt system.