Text this: From Halos to Galaxies. X. Decoding Galaxy SEDs with Physical Priors and Accurate Star Formation History Reconstruction