"It's a dangerous drug that we're on with this idea of zero interest rates," O'Neill said. "When you loan money to the government at these rates, you're paying them to hold your money. It's not a good thing for our society."
He compared scaling back asset purchases to moving to methadone, and he said he worries that it won't be easy to unravel the quantitative easing the Fed has done since the financial crisis.
O'Neill, who served as Treasury secretary in George W. Bush's first term, said that a look at economic history for the past 150 years suggests a normal real interest rate is about 3 percent for a no-risk loan. "We need to start there," he said. "We need to get back to a point where we have 3 percent real interest return on short-term investments."
But going there immediately would cause calamity as asset prices get marked down to reflect higher rates, warned O'Neill, adding that the biggest long-term threat to the country is Washington's lack of action to deal with our fiscal problems.
He said that the country needs to revert to a basic principle: The American people must pay for the things they want and need.
"Tax systems should be used to raise the revenue we need to pay for agreed public purposes," O'Neill said. "They should not be used to redistribute income and wealth and favor interest groups or induce people to do things."
O'Neill advocates replacing the entire tax system with a progressive value-added tax.
"If we cleaned up our tax act, we could do so much good," O'Neill said.
More from our partner, CNBC: