Winter riding can be tough. I night right off-road until it gets down to the mid 20's or so. Helmet liner...multiple layers, sweats or tights, and maybe even a multi-layer ski glove.
Fingers and toes will get the coldest. You can get a 30-pack of toe warmers for $12 at Costco. A little ghetto trick I do is to leave the ends of a couple of plastic shopping bags in my pack. Put 'em over your sock and it'll help a lot- especially if your shoe is ventilated. You can get full-on shoe warmers, too, but I find them a little inconvenient and tough to fit.
Unless you're REALLY good at handling, you won't want to ride on pavement in the snow, or on any surface covered in ice.