Text this: Measuring quantum relative entropy with finite-size effect