When you make the variator boss shorter it causes the retaining nut and washer to tighten down against the splines on the shaft versus the outer half of the variator pulley (nut is tight, but variator has wobble and teeth get into belt, will also destroy splines on shaft and outer pulley half). You could likely gain the speed back with Dr. Pulley sliders as they will have a higher high, and lower low due to their shape. Your rollers apparently were at the end of their ramp, before the pulley was closed. (unless you have changed the rollers to really light ones, which would require really high rpm to close the pulley all the way).