Camshafts lobes - like big end journals - are lubricated by the hydrodynamic wedge principle, the relative movement of the two surfaces forms a wedge of oil which the cam-lobe "surfs" on, when the nose of the camshaft lobe reaches the follower, this wedge starts to break down as it spills off the nose of the lobe, also if the lobe is moving too slowly the wedge isn't properly formed, at low RPM the window of opportunity where the spillage and therefore breakdown of the wedge will increase possibly leading to the nose of the cam being dragged slowly over the follower instead of on a wedge of oil. This is why a slower moving lobe will increase wear. The removal of zinc from modern oil (to protect catalytic converters) doesn't exactly help matters either as this was the last line of defence against metal to metal contact, and there's nothing remotely as good as it to provide the same characteristic.
Depending on clearances and amount of wear, the big-ends may suffer too, oil pressure and flow isn't exactly generous at idle with Rover V8s, at very low rpm oil spillage from the big-ends may outstrip supply, the reduced rotational speed will also inhibit the formation of the hydrodynamic wedge as described above encouraging metal to metal contact.