Line Input crashes when importing Unicode

We have a facility for staff that allows them to download an Excel marking scheme, enter student marks and then to upload it as a tab delimited file (or xml file) back into the system. Last night a member of staff using the facility crashed both servers in our cluster.

On further inspection (and a lot of development server crashes!) I finally narrowed the problem down to a single line of code

Line Input #fileID, strLine

On further investigation it appears that the user exported their marking scheme from Excel using the ‘Unicode (.txt)’ rather than ‘ Text (.txt)’ and it’s the import of Unicode that was causing Domino to crash. My quick solution to stop the server crash is to use ‘Dos2Unix’ convert the file before importing. Although it seems to alter the format so that it no long imports correctly, it’s stopped the server from crashing.

8 Comments Add yours

  1. Jerry Carter says:

    If you have excel installed on the server, you can use it to open the file and saveas in the preferred format.

  2. Paul says:

    Thanks. I should have said we’re running on Solaris, so we can’t use OLE integration.

  3. Jerry Carter says:

    Ah – I should have caught that from “Dos2Unix” which my dyslexic brain read as “Uni2Dos”, thinking Unicode transformation. If memory servers, line breaks / carriage returns differ from Dos to Unix, which might break the line input End Of Line detection. There may be some hope in using a Stream instead of line input, but I’ve found (some releases) have problems after you put so much text in the stream. Good luck getting it sorted out!

  4. Paul says:

    Thanks anyway, much appreciated. I think I’ll either have to rewrite the agent in Java or find a command line app that will do the conversion.

  5. Erik Brooks says:

    Agh – don’t open an Excel spreadsheet on your server. You’re potentially asking for any number of Microsoft’s exploitable vulnerabilities to infect your box.

    If you use the NotesStream class (fully supported in LotusScript as of ND 6.5+) then you should be good-to-go.

  6. Jerry Carter says:

    ooh – hadn’t thought about ‘sploitz. Good call, Erik.

    Here’s a note about using NotesStream – I think we were using R 7 at the time… should have noted it but by the time frame I believe that’s accurate. http://datatribesoftwerks.com/A55C1C/DatatribeBlog.nsf/archive/20070626-0BA6E8

  7. Paul says:

    Eric, thanks. I’d forgot all about NotesStream.

    Jerry, thanks for the warning. Was the problem the 64kb print buffer? I’ve ran into similar issues where you print very big strings but they get truncated and you have to loop through the string splitting it into smaller chunks?

  8. Kerry Sittig says:

    Another Excellent post, I will bookmark this post in my Newsvine account. Have a great day.

Leave a Reply