Text::CSV_XS versions before 1.65 for Perl return a 0xFF byte that is not in the input for an empty last field followed by a lone CR in parse.
When the input to parse ends in a separator and a lone CR, the parser treats the end of the input as the next character and stores the end-of-file marker in the field as the byte 0xFF. With binary set, parse("a,\r") returns the fields "a" and "\xFF". On UTF-8 flagged input the last field is a UTF-8 flagged string holding that lone byte, which is malformed UTF-8. The same input with a non-empty last field is rejected as an error.
A CRLF file read line by line and chomped ends this way when its last column is empty, so the application receives a byte that is not in its input.
Upgrade to Text-CSV_XS 1.65 or later.