aup mate the shims are there to determine the tension on the bearings and are set by using a spring balance onto the arm where the front track rod (drag link) fits, so that needs removing along with the main track rod, the cv joint and stub axle.
more shims = less tension
less shims = more...