I was having a similar issue, but I would limp when I let out of it. A few other symptoms that I noticed were rough idle, loss of power at lower RPM's (below 3K), and poor fuel economy.
I took the car into BMW and told them my symptoms. They first changed the exhaust VANOS. I picked up the car, and before I got home car limped. Took the car back and they changed out the cam shaft (worn ledges).