Gwendolyn Hardeman had a difficult childhood