UTF-8 problem in CS-PackageOutput output

[jamesmayfield]

From Ben Roth:

hi,

the CS-PackageOutput.pl script crashes at some input:

Wide character in subroutine entry at /iesl/canvas/beroth/workspace/relationfactory_iesl/evaluation/bin/CS-PackageOutput.pl line 98, <$infile> line 1442.

This is the line in the input file, where the error occurs (it is the only line out of ~2000 where an error occurrs):

CS14_ENG_203_9fd8f06e-c852-306f-9038-784b263dbc11 org:employees_or_members lsv eng-NG-31-142692-10074714:3009-3158 Michael O’Hanlon eng-NG-3 1-142692-10074714:3076-3091 0.7375

It has a special character, but this character should be correctly encoded in UTF-8.
The "LANG" variable is set to "en_US.UTF-8" .

Any ideas why this may occur?

thank you,

Ben

[arunchaganty]

I have also faced this problem, even with the latest version of the scripts (downloaded 6/19/17). I found that I could fix this problem using the Encode package see (https://stackoverflow.com/questions/32285925/wide-character-in-subroutine-entry-utf-8-encoded-cyrillic-words-as-sequence).