It could a boost leak, or boost control. Start with the obvious. Tighten hoses, clamps whatever you have.
If the problem persists then move on to boost control, vacuum line, solenoids could be the issue. But they cost 200$ so you don;t want to swap them in before you are sure. If you know someone with an N54 you can swap them with him and see if the issue goes away.
'10 E92 335i , COBB