mine does it too...i never could find any info when id search the forums. like roleeze mentioned, i think it's related to the seat memory too. The mirror seems too hard to move on its own..plus, every now and then i need to move the thing upwards, i dont imagine vibrations and potholes can cause that. A slight change in the seat seems about right.
i even have the phantom mirror issue on the driver door, where the mirror moves on its own.. but that seems to be a pretty common thing.