[#SVC-1836] Mono Beta: Exception thrown when mono gets ISO-8859-1 when expecting UTF-8

2. Second Life Service - SVC

Mono Beta: Exception thrown when mono gets ISO-8859-1 when expecting UTF-8

Created: 14/Mar/08 12:29 PM Updated: 10/Apr/08 01:41 PM

Return to search

Component/s:

Scripts

Affects Version/s:

Mono Beta

Fix Version/s:

Mono Beta

Environment:

« Hide

Second Life 1.18.6 (77968) Jan 29 2008 20:06:25 (Second Life Release)

You are at 255856.7, 255341.9, 21.7 in Sandbox Cordova MONO located at sim3000.aditi.lindenlab.com (8.2.33.226:12035)
Mono 1.19.1.82025

CPU: Intel Pentium 4 (Unknown model) (3000 MHz)
Memory: 2047 MB
OS Version: Microsoft Windows XP Service Pack 2 (Build 2600)
Graphics Card Vendor: NVIDIA Corporation
Graphics Card: GeForce 8800 GT/PCI/SSE2
OpenGL Version: 2.1.2
LLMozLib Version: 1.1.0 (Mozilla GRE:1.8.0.12_0000000000)
Packets Lost: 27/51419 (0.1%)
Viewer Digest: 3bb91f71-43e8-7bd9-be82-8ffb1b7bdf10

Issue Links:

Relates

This issue Relates to:

~~SVC-1483~~ MONO Beta : POST is broken :( http_request seems to do something, but http_response is crashing as described when using POST

This issue is related to by:

~~SVC-1276~~ Mono issues

Linden Lab Issue ID:	DEV-13088
Linden Lab Internal Branch:	mono-based-on-1-19-1-server-82811

Description

« Hide

Whenever using some non English characters coming from a blog, got very strange exception from Mono runtime. Error is well documented by ~~SVC-1483~~ bug where it was erroneously considered part of HTTPPOST handling, but it was really an internal handling of UTF8 characters. Scouse told that there are "invalid" utf-8 characters, but in such case which characters are invalid, and why they are considered good characters from the lsl2 version?

Character flow coming from http://www.salahzar.info/lsl/feedburner.php?feed=http://www.beppegrillo.it/index.xml is apparently giving this problem. See ~~SVC-1483~~ for comple program and error returned.

Here again the exception:

[16:41] RSS HTTPPOST: DEBUG(main): http://www.salahzar.info/lsl/feedburner.php? feed=http://www.beppegrillo.it/index.xml&start=0
[16:41] RSS HTTPPOST: Script run-time error
[16:41] RSS HTTPPOST: System.NullReferenceException: Object reference not set to an instance of an object
at LindenLab.SecondLife.LslUserScript.Add (System.String rhs, System.String lhs) [0x00000]
at LSL_ea6230e4_e156_d593_88f0_7bfea3f0e21c.defaulthttp_response (Key reqid, Int32 status, System.Collections.ArrayList meta, System.String body) [0x00000]
at LSL_ea6230e4_e156_d593_88f0_7bfea3f0e21cdefaulthttp_responseFrame.ResumeVoid () [0x00000]
at LindenLab.SecondLife.UThread.UThread.Run () [0x00000]
at LindenLab.SecondLife.Script.Run (ScriptEvent evt) [0x00000]

All

Comments

Change History

Sort Order:

[ Permlink | « Hide ]

Salahzar Stenvaag added a comment - 14/Mar/08 12:33 PM

see 1483 for where the problem was spotted.
UTF-8 handling is quite important for European market (and for my whiteboards)

[ Permlink | « Hide ]

Scouse Linden added a comment - 18/Mar/08 02:12 AM - edited

The character which are bad are the ones with accents. The first character is an E8, this is not UTF-8. The character set your website is returning is Latin 1 despite of the fact that you request UTF-8. There are more rigorous checks under mono.

I captured the output of the website on the 14th.

We support UTF-8 request and Latin 1 requests. We don't support UTF-8 requests when you give us Latin 1. This 'feature' may disappear from LSL.

My browser doesnt support the characters it displays � instead.
I attempted this in both Safari and Firefox and got the same result. You can make the page appear correct by changing the character encoding in the View menu.

[ Permlink | « Hide ]

Salahzar Stenvaag added a comment - 18/Mar/08 04:25 AM

Umm... if that is really the problem, you should catch that exception and not letting it flow up, since it is absolutely not informative and will surely provide chaos. I understand that having it in the stacktrace can help you in understand the problem, so you might leave it.

I'm not an expert in c#, but I'm a professional Senior programmer in Java, for IBM, and tracing, bug reporting and fixing is our common day work, and we really know that such errors might appear in the traces, but must be prefixed with a meaningful messages something like:
"Errors in decoding UTF8 parameters (are they really UTF-8?), invalid character E8 found" and then the rest of the stack dump, maybe even the actual buffer in hexadecimal. (What about enabling an undocumented Verbose mode to enable mono exceptions to be viewed, otherwise having some remapping like "Internal error found #015 (UTF8 Handling), please launch with Verbose to have detailed trace to be sent to our support"?).

This way people can find out where to fix and finding workarounds.
In any case is unacceptable that an lsl script (we are not writing in c# yet) is giving exotic internal error without hints.

Another question: you said that ancient lsl did support this mixture between UTF-8 and latin1, but officially the position of linden was to have Mono supporting 1:1 whatever was before done by lsl2. If this is NOT the case, the community would greatly appreciate to have a listing of all you are not meaning to support. I can understand that mono is less leniant than lsl2, just need to know where and how
After all this is exactly what the "Beta" stage of mono is for... To understand how to deploy it in official grid reducing the complaints and maximizing the benefits.

Thanks anyway for addressing this and giving me the hint that might allow to solve the problem.

[ Show » ]

Salahzar Stenvaag added a comment - 18/Mar/08 04:25 AM Umm... if that is really the problem, you should catch that exception and not letting it flow up, since it is absolutely not informative and will surely provide chaos. I understand that having it in the stacktrace can help you in understand the problem, so you might leave it. I'm not an expert in c#, but I'm a professional Senior programmer in Java, for IBM, and tracing, bug reporting and fixing is our common day work, and we really know that such errors might appear in the traces, but must be prefixed with a meaningful messages something like: "Errors in decoding UTF8 parameters (are they really UTF-8?), invalid character E8 found" and then the rest of the stack dump, maybe even the actual buffer in hexadecimal. (What about enabling an undocumented Verbose mode to enable mono exceptions to be viewed, otherwise having some remapping like "Internal error found #015 (UTF8 Handling), please launch with Verbose to have detailed trace to be sent to our support"?). This way people can find out where to fix and finding workarounds. In any case is unacceptable that an lsl script (we are not writing in c# yet) is giving exotic internal error without hints. Another question: you said that ancient lsl did support this mixture between UTF-8 and latin1, but officially the position of linden was to have Mono supporting 1:1 whatever was before done by lsl2. If this is NOT the case, the community would greatly appreciate to have a listing of all you are not meaning to support. I can understand that mono is less leniant than lsl2, just need to know where and how

After all this is exactly what the "Beta" stage of mono is for... To understand how to deploy it in official grid reducing the complaints and maximizing the benefits. Thanks anyway for addressing this and giving me the hint that might allow to solve the problem.

[ Permlink | « Hide ]

Scouse Linden added a comment - 18/Mar/08 05:40 AM - edited

It doesnt matter who you are, if you have a bug with a repro we will consider it on its own merits.
I have looked at this bug and we're considering how to fix it. I'm leaning towards replacing the broken characters with � but I want to consult with the engineers who originally wrote the LSL HTTP functionality.

Mono and LSL languages should be compatible. We are not aiming for 100% but to some very high level.

I consider this to be a problem with the API and the fix will probably effect both LSL and Mono.

[ Permlink | « Hide ]

Vektor Linden added a comment - 10/Apr/08 01:41 PM

Ths looks good, no more exceptions, and a simple "?" replacing unknowns.