ALBANY – The gulf between the haves and the have nots is growing wider in Dutchess County and around New York.

The number of millionaires in Dutchess County jumped by 33 percent from 2008 to 2012, according to records from state Department of Taxation and Finance. The number of people making $200,000 or more jumped as well, by 17.9 percent, to 4,932 in 2012.

In Ulster County, the number of millionaires grew by 2 percent from 2008 to 2012 to 96 people. In 2012, there were 1,726 people making more than 200,000 in Ulster, according to the state.

However, the average household income in Dutchess County dropped over the same period, according to U.S. Census Bureau estimates.

In 2008, the average household income in Dutchess County was $88,891, according to the census estimates. In 2012, it was $86,119, a 3 percent drop.

In Ulster County, the average household income increased slightly, from $76,488 to $77,105.

A local expert attributed the increase in millionaires to the influx of downstate residents to the mid-Hudson Valley, as well as investment gains tied to the recovery of the stock market since 2008.

“We are basically seeing a relationship in Dutchess County that is similar to what is going on nationwide with respect to the increase in the number of millionaires,” said Christy Huebner Caridi, director of Marist College’s Bureau of Economic Research.

The rising number of millionaires has renewed debate on tax rates for the most affluent to help pay for education, social services and assistance for those in need. However, some have argued that higher taxes on the wealthy would make people and businesses flee the state.

The number of millionaires in New York increased 16 percent between 2008 and 2012, reaching the highest point since 2008, state data showed.

New Yorkers making more than $1 million taxable income in 2012 jumped to 41,414 filers, up from 35,705 in 2011 and exceeding the 35,788 millionaires in 2008, according to records from state Department of Taxation and Finance.

Westchester County had the most millionaires outside Manhattan, with about 7,300, up 10 percent, between 2008 and 2012, the most recent figures available. Some counties also had large increases: up 38 percent in Erie County; 33 percent in Monroe and Dutchess counties; and 22 percent in Broome County.

It’s the latest rebound for the state’s wealthiest residents amid ongoing pressure from liberal groups and unions for New York leaders to increase taxes on the rich. Those groups said the growth in millionaires is another reason why the rich should pay more in taxes to fund schools, infrastructure and services for the needy.

“I think the evidence of the growing number of millionaires shows that a surcharge on millionaires doesn’t drive them out of the state,” said Billy Easton, executive director of the union-backed Alliance for Quality Education. “So asking them to pay a little more, it is a totally reasonable approach.”

But conservative groups said the increase in millionaires in New York doesn’t tell the full picture.

New York had the slowest growth in the country in millionaires between 2011 and 2012, said E.J. McMahon, president of the Empire Center for State Policy.

He cited IRS records that showed the number of millionaires nationwide grew by 29 percent, from 304,000 to 392,000, between 2011 and 2012. In New York, it was up 15 percent, ranking last in the nation.

McMahon said the figures are exacerbated because wealthy people boosted their income reporting as much as possible in 2012 to avoid new, higher federal tax rates in 2013.

“We have a tax code that’s dependent on high-wealth people and in a crucial year, we were at the bottom of the pile,” McMahon said.

Between 2010 and 2011, the number of millionaires in New York grew by 8.5 percent and was up 19 percent between 2009 and 2010, state records show. During the recession, millionaires in New York dropped 22 percent in 2009 compared to 2008.

Gov. Andrew Cuomo has argued that taxing the rich hurts the state’s competitiveness. In 2011, he and the Legislature agreed to keep higher income taxes on millionaires, but he lowered them slightly from where the rates had been previously and also lowered them for the middle class. Still, the higher rate on the rich raises about $2 billion for the state’s coffers, helping to stave off budget deficits.

In March, Cuomo rejected New York City Mayor Bill de Blasio’s call to raise taxes to pay for pre-kindergarten in the city. Instead, Cuomo added more aid to the state budget for the program.

Cuomo said Thursday in a speech in Albany that people and businesses will simply leave the state if taxes are increased. New York is already among the highest taxed states in the nation, and it’s likely to be surpassed by low-tax Florida next year as the third most populous state.

About 1.2 million people left to other states between 2000 and 2009, the Empire Center said.

“We’re in a competition with those other states,” Cuomo said. ” Our friends who say we should use the tax code to address social inequities and raise the taxes on the rich: If that’s going to happen, that has to happen at the federal level.”

The reason, he said, is because, “We are in a foot race with every other state, and our level of taxation and our regulations are going to say whether we come in first, second, third or fourth.”

The highest tax bracket in New York is a marginal rate of 8.82 percent for individuals who earn $1 million or more and married couples above $2 million.

More millionaires help the state’s finances. This year, the state budget estimates that millionaires will make up about 1 percent of the state’s total income tax filers, but account for about 41 percent of the taxes paid.

Cuomo has been criticized for relying heavily on the wealthy to bankroll his re-election last month. Of the $45 million he raised, 81 percent came from people who gave $10,000 or more, a review from the New York Public Interest Research Group found in October.

“He’s been the governor of the wealthy on economic issues,” said former Assemblyman Richard Brodsky, D-Greenburgh, Westchester County.

Brodsky criticized Cuomo for agreements with the Legislature this year to cut bank and estate taxes.

“America is a place where people can make a lot of money. That’s good,” Brodsky said. “What’s unfortunate is the notion that they need special tax treatment. It raises the question of fair share.”

Advocates for education and health care will again ramp up their push for more state spending after Cuomo releases his 2015-16 budget in late January for the fiscal year that starts April 1. He will lay out his agenda for 2015 in his State of the State address Jan. 7.

Business groups will be pressing for more tax cuts, saying New York remains an expensive state.

Mike Durant, state director for the National Federation of Independent Business, said Cuomo should focus not Wall Street, but on small businesses, which represent more than half the state’s private-sector workforce.

“For all the superlatives the last four years, Albany has been a lot kinder to big business or Wall Street than Main Street,” Durant said.