Text this: Inconsistency of point-particle dynamics on higher-spin backgrounds