Whenever you apply an adjustment layer for altering exposure, set its blend mode to Luminosity.
For adjustment layers that adapt colour, such as Color Balance, set its blend mode to Color.
By using these two blend modes for certain types of adjustments, you’ll only affect the light values or colour information in an image.
