Sure, but even sedentary people only lose a pound of muscle a year. It is really easy to maintain once you have spent that 2-3 years building a foundation, as you did, with weights.