Text this: Modeling crystal defects using defect informed neural networks