Two people were killed in Sunday's attack during a mass at a Tanzanian church, officials said Monday, as President Jakaya Kikwete called the explosion an "act of terrorism".
Six people have been arrested, including four from Saudi Arabia, officials said.
"This is an act of terrorism perpetrated by a cruel person or group who are enemies of the country," Kikwete said in a statement, condemning the bombing in the northern Tanzanian town of Arusha in which at least 30 people were also wounded.
The deadly attack on the church is one of the first such incidents to hit Tanzania.
Officials have given no indication as to who might have carried out the attack, but tensions have been high between Tanzania's Christian and Muslim communities in recent months.
Arusha's commissioner Magesa Mulongo confirmed that two people had died and that six people had been arrested, two from Tanzania and four from Saudi Arabia.
"Investigations are ongoing," Mulongo said, adding that the four Saudis had arrived at Arusha airport on Saturday.
The two Tanzanians arrested were Christian, he added, but gave no further details.
The blast occurred outside Saint Joseph's Roman Catholic church in Arusha, a town popular with tourists visiting the popular Serengeti national park and snowcapped Mount Kilimanjaro.
The newly built church, in the Olasti district on the outskirts of Arusha, was celebrating its first ever mass when the blast occurred, and people were squeezed into the church building as well as sitting on benches outside.
The Vatican's ambassador to Tanzania, Archbishop Francisco Montecillo Padilla, was attending mass at the church but was not harmed, officials said.
Kikwete, who said he was "shocked and deeply saddened" by the reports of the explosion, called on people to remain calm while police investigated the attacks.
"We are ready to deal with all criminals including terrorists and their agents who are based in the country or externally," Kikwete said.
After the attack, worshippers accused the police and the government of failing to properly protect them.
In February, a Catholic priest was shot dead outside his church on the largely Muslim archipelago of Zanzibar, the second such killing in recent months. A church was also set on fire on Zanzibar in February.
In March, 52 followers of controversial Muslim cleric Sheikh Ponda Issa Ponda were jailed for a year for violent riots in October in the commercial capital of Dar es Salaam, sparked by rumours that a 12-year-old boy at a Christian school had urinated on a copy of the Koran.
Ponda is the head of Jumuiya ya wa Islamu, or the "community of Islam", a group not recognised by the Tanzanian government.
Last month, in the far south of Tanzania, police fired tear gas to disperse around 200 Christian rioters attempting to torch a mosque over an argument over who should be allowed to slaughter animals.
Around half of Tanzanians are believed to be Christian, and around a third of the population to be Muslim, although there are no official figures.
In neighbouring Kenya -- whose troops invaded southern Somalia in 2011, prompting warnings of revenge by the Al-Qaeda linked Shebab insurgents -- several churches have been targeted in attacks similar to the Arusha blast.
While Tanzania does not have troops in Somalia, it is home to Islamist groups connected to radical groups in the wider region including the Shebab, according to United Nations experts.