We might be dealing with the same custodian. We frequently deal with issues like this due the custodian accepting rollover money before it can be confirmed the rollover is acceptable. We have been unsuccessful in getting them to change their procedures.
The receiving plan has been holding the assets ever since the check proceeds were placed in the plan's cash account. If it has been determined by the Plan Administrator to be an invalid rollover due to terms of the document, the correction is to distribute it from the plan as soon as possible.
See https://www.irs.gov/retirement-plans/verifying-rollover-contributions-to-plans
We have the participant complete a distribution form instructing us where to send the money. We issue a new 1099R as it is a new distribution. As Bri stated, this participant just needs to setup a Roth IRA.