IIRC the 'real' way, if you're looking to replace them with new, is to remove the whole bumper. Two bolts in the centre, and two towards the end looking from the chassis towards the back, IYSWIM. Then the caps are held onto a small bendy metal plate with self tappers.
Easiest way I've found is to just go a little harder and deeper whilst off-roading ...