History | Log In     View a printable version of the current page.  
  • All submissions to this site are governed by Second Life Project Contribution Agreement. By submitting patches and other information using this site, you acknowledge that you've read, understood, and agreed to those terms.
Issue Details (XML | Word | Printable)

Key: VWR-2404
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Qarl Linden
Reporter: SignpostMarv Martin
Votes: 175
Watchers: 63
Operations

If you were logged in you would be able to see more operations.
1. Second Life Viewer - VWR

lossless texture compression on small textures not lossless

Created: 10/Sep/07 03:57 PM   Updated: 24/Jul/08 02:32 PM
Component/s: Graphics
Affects Version/s: 1.18.3 Release Candidate, 1.18.3.5, 1.19.1 Release Candidate
Fix Version/s: 1.20 Release Candidate

File Attachments: 1. File 2404.tga (997 kb)
2. File BoxSculptieExact2.bmp (12 kb)
3. File buckle-004-003.bmp (12 kb)
4. File da2d7853-8fc7-4d80-e0dc-b791f28f85a8.j2c (3 kb)
5. File dodecsculpt2.tga (3 kb)
6. Text File lossless.patch (1 kb)
7. File newmesh.tga (3.00 Mb)
8. File rings.bmp (48 kb)
9. File rings2.bmp (48 kb)
10. File sculpt-snapshot.bmp (2.87 Mb)

Image Attachments:

1. 8x8 gradient borked_001.jpg
(67 kb)

2. 8x8 RGBYWGGK upload still borked.png
(332 kb)

3. Borked_jet_engine.jpg
(71 kb)

4. BoxSculptieExact2Picture.jpg
(21 kb)

5. BuckleView1.jpg
(39 kb)

6. BuckleView2.jpg
(39 kb)

7. goodstairsbadstairs.jpg
(288 kb)

8. hilbert_sculpty_bad_compare.jpg
(93 kb)

9. lossless-compression-bug-002.jpg
(81 kb)

10. lossless-compression-bug.jpg
(113 kb)

11. Lossless_Comparison_63RC.jpg
(231 kb)

12. lossless_issue.jpg
(161 kb)

13. newRC_1_18_4_still_problematic.jpg
(65 kb)

14. partially_fixed.jpg
(96 kb)

15. sculptiefailure_1.20.0.3-failed.png
(219 kb)

16. sculptiefailure_1.20.0.3-loaded.png
(222 kb)

17. sculptiefailure_1.20.0.3.jpeg
(56 kb)

18. sculpties_after_fix.jpg
(279 kb)

19. sculpties_before_fix.jpg
(303 kb)
Issue Links:
Duplicate
 
Relates

Linden Lab Issue ID: SL-54632
Linden Lab Internal Branch: 1.20.7.87883 RELEASECANDIDATE

Sub-Tasks  All   Open   
 Sub-Task Progress: 

 Description  « Hide
Despite claiming to be lossless, the "lossless compression" mode for small textures results in uploads appearing different than the source image.

Lossless compression either needs to be fixed, or it's claims of being lossless need to be adjusted to match the reality of output.

 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
Lex Neva - 11/Sep/07 08:46 AM
Okay... I think some interesting things are happening here with respect to the way the texture preview window works. It LOOKS like what we're seeing is wrapping around/fuzzing of the edges of the texture in the texture preview window after uploading. This is normal, because textures on objects in SL behave this way. The borders always wrap around and bleed a little (which can cause some other annoying problems with textures!) I think this is because of the way textures are handled in 3d graphics libraries/cards. Are you sure this wrapping around is actually causing errors in the mesh itself, or just in the visual comparison of the texture that you've shown us?

Qarl Linden - 11/Sep/07 12:46 PM
FYI - yes there is a bug here, we're looking into it. (oddly, it seems to be a download bug, not an upload bug. should be fixed soon.)

Lex Neva - 12/Sep/07 09:12 AM
I take it back!

DanielFox Abernathy - 16/Sep/07 07:07 AM
Example of bitmap which does not upload lossless properly in SL RC 1.18.3(4)

DanielFox Abernathy - 16/Sep/07 07:17 AM
Example of 128x128 which doesn't seem to upload losslessly

DanielFox Abernathy - 16/Sep/07 07:18 AM
Example of lossless upload issue; same texture uploaded with SLImageUpload vs "upload lossless" with RC 1.18.3(4)

DanielFox Abernathy - 16/Sep/07 07:24 AM
I've run into the same problem so hopefully this is the right place to post my findings. This is not an isolated case that I have noticed but I have had this happen several times. Although the new lossless upload functionality seems to work okay most of the time, I've had many sculpties that were corrupted until I went back and used SLImageUpload to upload them, whereupon they look fine. The attached "rings.bmp" did not upload correctly with the 'lossless' checkbox checked when I attempted it, using Second Life client version 1.18.3 (4) Sep 14 2007 ; I uploaded it twice just to make sure. You can see in the provided lossless_issue.jpg ; this is the SAME sculpt bmp "rings.bmp" the only difference is one was uploaded through the client and the other through SLImageUpload.

However, the attached "rings2.bmp" uploads fine through the SecondLife client; I am not sure what triggers the failure in the SecondLife uploading. This would lead me to believe there is in fact an issue with uploading. If it was just a downloading issue, I would not expect to be able to remedy the problem with SLImageUpload.

camilla Yosuke - 19/Sep/07 03:04 PM
 ok, I just made a test : when I upload ( linux RC 1.18.3.2 ) a texture in 64*64 with losless compression on, it ends VERY BAD.
Now, the same texture, sized in GIMP to 128*128 without interpolation, uploaded in 128*128 with losless on rezzes perfectly.

uchi desmoulins - 23/Sep/07 01:47 PM
I have several sculptmaps that become corrupt after the initial download of them. Meaning, if I reload SL, they become corrupt, distorted, ugly and bad, and they remain so, the second time I view them..

THE FIX? - Clearing my cache and starting SL over again.

Then, they are fine. But only temporarily. So, is image data not being cached correctly?

And of course, this 'fix' doesn't apply to ALL sculptmaps, just a few that for mysterious reasons look good initially, then later appear distorted. ARGH!

SignpostMarv Martin - 24/Sep/07 04:52 AM
that would seem to suggest that the lossless sculptie maps are being re-encoded.

Funk Schnook - 29/Sep/07 04:06 AM
There is definitely a bug here. I have started using opensim to test sculpties and noticed once I upload to SL's main grid, the sculpties appear lumpier (I can see this in wireframe mode). So to test this, I did the following on LL's servers and on my own local opensim server:

1. uploaded a 64x64 tga using lossless compression
2. Save the image back to your HD (save image as...)
3. Load your original image and the newly saved tga into photoshop.
4. Copy and paste the newly saved image into a new layer on top of your original sculp map and set its blending mode to DIFFERENCE

This should produce a pure black image. You can take the color picker and swipe over pixels, while looking at the RGB values in the INFO panel. It should always be 0,0,0

Now the interesting part. If I save the tga from opensim, everything works. The color picker shows pure black on every pixel of the 64x64 image.

If I use the image saved from LL's server, the color picker shows some small differences using the color picker (eg, 0,1,0 or 1,0,0 etc etc). It's not pure black

So, according to this test, the SL client is actually uploading the image losslessly to the server (otherwise opensim's test would fail), but something is going wrong on LL's server side. Either the image data is not being stored correctly, or the sim is corrupting the data when it's sending it back to the client.

Qarl Linden - 29/Sep/07 09:22 AM
Funk - yes. the bug is not in the image encoding - it's in the networking system. i'm still trying to completely undestand it - it has something to do with assumptions about the datasize of the images. it's interesting that you're seeing success using the opensim code - have you checked a LOT of textures? (whatever i try, i'm finding that some textures will work in some cases, and not in others.)

Funk Schnook - 30/Sep/07 01:34 AM
The plot thickens.

1. As I said, opensim displays the sculpt map correctly after uploaded then applied to the prim.
2. BUT... once I relog and view the mesh its lumpier (looks just like the version on LL's grid)
3. If I clear my cache and relog, the sculpt map is fine again.
4. If I relog again WITHOUT clearing cache the sculpt map is lumpy again.

So at least on opensim, clearing cache and relogging fixes it. On main grid a cache clear doesnt fix it.

Now its starting to look like there is (also??) a client/cache bug, since opensim will display it correctly after a cache clear/relog.

SignpostMarv Martin - 30/Sep/07 07:16 AM
Funk Schnook, can you do me a favour and run this diff test (possibly repeatedly) on textures of all sizes ?

I'd be interested to know whether the textures people have uploaded are gradually losing bits :-P

Funk Schnook - 01/Oct/07 04:39 AM
I have attached an image (lossless-compression-bug.jpg) which shows the effect of the bug in opensim and secondlife.

The first image is my 3d model. The second shows you how the sculptie looks in open sim when you first upload it (this is the way it also looks after you relog and clear your cache).

The next 2 images display the buggy sculptie in both opensim and SL.

Clearing the cache fixes opensim but not SL as far as my testing goes.

This is starting to look more and more like a client side bug to me, since a cache clear will refetch the correct sculptie in opensim.

Funk Schnook - 01/Oct/07 05:07 AM
I have uploaded another screenshot showing LOSSY compression on both opensim and SL. It always looks like this whether you clear cache or not.

So, if you want your sculpties to look consistent, stick to LOSSY compression right now.

Funk Schnook - 01/Oct/07 05:11 AM
I have now attached the original sculptie 64x64 image (buckle-004-003.bmp) used in the tests above

Qarl Linden - 01/Oct/07 12:07 PM
here's the problem:

our SL clients and servers assume that all of our images are encoded the same way - and use this to optimize image transfer over the network. (it thinks to itself - "i need a 64x64 image - that should be about 64x64x3 bytes, compressed at 8:1, so i only need to download 1536 bytes.") this sort of heuristic occurs in the sim code, the viewer code, and the cache code, in different ways, which causes a variety of failure modes.

the assets on the server are in-fact intact and not degrading over time. you just can't see them.

short answer - listen to Funk: "So, if you want your sculpties to look consistent, stick to LOSSY compression right now."

Funk Schnook - 01/Oct/07 01:51 PM
@Qarl: Well I was hoping this issue would be sorted out before releasing another stable viewer. So basically what your saying is, you added this option, but it doesnt work.

I have heard some reports about slimageupload not displaying this problem. I havent run any tests yet but will try to soon. (ugh... now Im spending time beta testing instead of working and running my business)

Qarl Linden - 01/Oct/07 02:06 PM
heh - no. i'm saying that sometimes it works, and sometimes it doesn't, and we decided to leave it for the cases when it is useful.

Funk Schnook - 01/Oct/07 02:13 PM
@Qarl: Ah ok :) Well I have been testing a bunch of differnt sculpt maps over the past few days and the phtoshop difference test always showed some data loss (even though some of the sculpties actually looked quite good and usable anyway).

Id like to narrow down why some of your tests come up perfectly lossless. It must have something to do with file sizes?

Qarl Linden - 01/Oct/07 03:04 PM
what i'm seeing is that 64x64 images upsampled to 128x128 seem to work well - probably because they has a lot of redundancy (the upsampling) and so compress better (back within the assumed 8:1 ratio.)

Funk Schnook - 01/Oct/07 07:56 PM
@Qarl I didnt do a great deal of testing with 128x128, and the ones I tried werent upsampled from a 64x64 so wouldnt have compressed as well. They showed the same flaws as the 64x64s.

I'm thinking "nearest neighbor" upsampling in PS would be best for this so we arent introducing additional color info into the sculpt map.

If this works, using upsampled 128x128 images will be a good work around. Considering we are usually using bigger images for our prim textures, I guess it doesnt really matter about having a bigger sculpt map. Can't be any worse than the guy applying a 1024x1024 on his 0.01m prim jewelry ;)



Qarl Linden - 02/Oct/07 10:41 AM
"I'm thinking "nearest neighbor" upsampling in PS would be best for this so we arent introducing additional color info into the sculpt map"

yes, exactly so.

meanwhile, one of the other lindens here suggested a good way to fix the problem - perhaps we'll have a full solution soon.

Funk Schnook - 02/Oct/07 12:01 PM
OK Qarl I'm having some success with this method:

1. render a 64x64 sculpt map
2. copy the top row of pixels into the clipboard (the top pole - I actually clean it up as a solid color too so my pole is nice and centered)
3. downsample (nearest neighbour) to 32x32 (I clean up the bottom pole here too as a solid color).. youll also notice your top line/pole went missing. Thats why I copied it to clipboard
4. upsample (nearest neighbour) to 64x64 and paste back the top row of pixels
5. upsample to (nearest neighbour) to 128x128
6. I add a black alpha mask just so my sculpt maps dont show up in the object panel (this is optional)

I havent downloaded the file back from sl to do a difference test in PS though because sl renders the sculptie without those huge jaggies seen in my screenshots. It renders the same in sl and opensim regardless of a cache clear.

Thanks for that final little clue which helped me work this out :)

You could even make this a photoshop action quite easily (except for the pole fixing?) so it's automated

Qarl Linden - 02/Oct/07 03:14 PM
kk guys - i think we got this one solved. patch is being tested by QA now.

DanielFox Abernathy - 02/Oct/07 07:11 PM
Great to hear Qarl. Eager to test it out :)

Acru Joliat - 03/Oct/07 07:36 PM
Will the fix for this be included with the upcoming rolling restart to fix SVC-750? I realy hope it will...

Qarl Linden - 04/Oct/07 09:56 AM
Acru - no. rolling restarts update software on our servers - this bug is in the viewer - so you'll need to download a new viewer. i wouldn't expect us to release a new viewer for a couple weeks.