Two people were killed in Sunday's attack during a mass at a Tanzanian church, officials said Monday, as President Jakaya Kikwete called the explosion an "act of terrorism".
Six people have been arrested, including four from Saudi Arabia, officials said.
"This is an act of terrorism perpetrated by a cruel person or group who are enemies of the country," Kikwete said in a statement, condemning the bombing in the northern Tanzanian town of Arusha in which at least 30 people were also wounded.
Such a large scale attack targeting a church is one of the first such incidents in Tanzania.
Arusha's commissioner Magesa Mulongo confirmed that two people had died and that six people had been arrested, two from Tanzania and four from Saudi Arabia.
"Investigations are ongoing," Mulongo said, adding that the four Saudis had arrived at Arusha airport on Saturday.
The two Tanzanians arrested were Christian, he added, but gave no further details.
The blast took place outside a Roman Catholic church in Arusha, a town popular with tourists visiting the famous Serengeti national park and snowcapped Mount Kilimanjaro.
Officials have given no indication as to who might have carried out the attack, but tensions have been high between Tanzania's Christian and Muslim communities in recent months.
The newly built church, in the Olasti district on the outskirts of Arusha, was celebrating its first ever mass when the blast occurred, and people were squeezed into the church building as well as sitting on benches outside.
The Vatican's ambassador to Tanzania, Archbishop Francisco Montecillo Padilla, was attending mass at the church but was not harmed, officials said.
Kikwete, who said he was "shocked and deeply saddened" by the reports of the explosion, called on people to remain calm while police investigated the attacks.
"We are ready to deal with all criminals including terrorists and their agents who are based in the country or externally," Kikwete said.
After the attack, worshippers angrily accused the police and the government of failing to properly protect them.
In February, a Catholic priest was shot dead outside his church on the largely Muslim archipelago of Zanzibar, the second such killing in recent months. A church was also set on fire on Zanzibar in February.
Last month, in the far south of Tanzania, police fired tear gas to disperse around 200 Christian rioters attempting to torch a mosque over an argument over who should be allowed to slaughter animals.