There isn't a hard number.
I bet it's a rough ride in your car without five people. The springs are going to be compressed even more = less available damper travel, so you'll be constantly rebounding off the bumpstops. It's more than unpleasant. I agree it's probably not a good idea to take five people in your car.
2011 BMW E92 335i, 6MT, M Sport, Le Mans blue - Michelin PSS, Quaife LSD, JRZ RS Pro, JRZ camber plates, Eibach ERS, full M3 susp, custom toe arms, Powergrid end-links, brake ducting, SS brake lines, UUC SSK, ETS FMIC, ER CP, Perf. Exhaust, work by VAC Motorsports