Text this: Enhanced quantum state swapping via environmental memory