Perhaps it's their inability to truly marginalize the tea party. They've attempted to portray them as any number of things but have failed. How irritating that the tea party has real power and is not going anywhere. Also in the back of their minds there's this scratchy feeling that there's something else here. That something else is the sizable and energized electorate that, in repudiation of this administration, set a new record for going to the polls and turning out the bums. And those folks are still there.
p.s. But of course, the tea party is a real challenge for the Republican old guard too.