I removed my rear seatback - this thread has good info:
http://www.m3post.com/forums/showthread.php?t=363108
I didn't remove the other pieces because it would look ugly and even altogether they weigh almost nothing compared to the seatback itself.
I'd discourage you from removing the rear seat for several reasons.
A) It makes the cabin a noticeably noisier environment
B) It limits the car's practicality (two seating places, not four)
C) Safety - you will have to find a way to secure items placed in the trunk because nothing is stopping them from sliding forward
I removed my rear seatback primarily because I have fixed race seats which make the rear seat inaccessible, so I ditched the weight. My car is not street driven much anyway. The sole benefit from rear seat removal is the ~45 lbs weight reduction which is unnoticeable on the street and only faintly noticeable on the track.