i think its attached to the trim

yes, it is attached to the trim, so take off the long piece of trim it is attached too and there are clips on the back

When you take the trim off the dash, take the start button off. Once you have the start button off, just pull on the ring around it and it should just pop/snap right off. It has 4 clip points so you should probably start at an angle and work your way around.
It's okay to man-handle it a little bit - it's not going to break on you.

I took mine off when i did my custom start button.

In addition, the actual "Start/Stop Engine" piece is also a cover - it has two snap points. With a dental tool or very thin wire (smaller than a clothes hanger), you can pop this off.