I wouldn't use a screwdriver, I only have experience in removing the older type (3.5) and that came out with the spring pressure.
Would think if you carefully drill a hole in it making sure not to go straight through and screw a s/tapper or similar in it and try pulling it out with that.