Upgrading the intercooler will be an all around good upgrade, no matter if you are adding boost or not. The stock intercooler is not well suited for running the car hard for longer periods of time given the small size. Your car will thank you for going with a larger intercooler.
The downpipes are certainly something you can pick to do or not, they will allow the turbos to spool more freely, as well as having less back pressure is good for the longevity of them. At stock boost levels, you may not need to ever do it, but if you are planning on getting a tune down the road, you may want to start the look for something you like.
Charge pipe really isn't something you need to worry about unless you start to have issues with it popping off, or want to run methanol.