The easiest way is to set scaling_governor to userspace. Then, only setting scaling_setspeed changes the frequency.

It is important to note that writing to vdd1_opps_vsel does not actually change voltage level. The new levels only take when frequency is changed. Therefore if you want to change voltage level when using userspace governor, you should set a different frequency after writing to _vsel file.