The bombing hit the centre in the Haradh district in the northwestern province of Hajja on Saturday, according to the UN humanitarian coordinator for Yemen, Lisa Grande. She did not name the party behind the attack.
"Any attack on a civilian site is unconscionable and a clear violation of international humanitarian law," Grande said in a statement.
"An attack like this cannot be justified -- ever."
The centre is in an area under government control in Hajja, but the Huthi rebels control other parts of the province.
The Iran-backed Huthis have battled a pro-government military coalition, led by Saudi Arabia and the UAE, for control of the impoverished country for four years, triggering what the UN calls the world's worst humanitarian crisis.
Both parties are accused of failing to protect civilians. The Saudi-led alliance, which controls Yemeni airspace, has been criticised by the UN for the killing and maiming of children in air raids.
Saudi Arabia's state-run King Salman Humanitarian Aid and Relief Center (KSRelief) on Saturday accused the rebels of the attack. This organisation also said eight had been killed and 30 wounded.
Grande said dozens of civilians have been killed in Hajja over the past two months and hundreds of families displaced. More than one million people are food insecure in the province, according to the UN official.
The World Health Organisation says the war has killed an estimated 10,000 people since 2015, when Saudi Arabia and its allies joined the government's fight against the rebels.
24 million Yemenis -- more than three quarters of the country's population -- are now dependent on some form of aid for survival, and the UN estimates more than 12 million are at risk of starvation.
More than 1.13 million Yemenis received emergency food aid in December and aid agencies aim to reach an additional 230,000 people in January.