• All submissions to this site are governed by Second Life Project Contribution Agreement. By submitting patches and other information using this site, you acknowledge that you have read, understood, and agreed to those terms.
Issue Details (XML | Word | Printable)

Key: MISC-975
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Critical Critical
Assignee: Soft Linden
Reporter: Zimzim Yongbo
Votes: 49
Watchers: 9
Operations

If you were logged in you would be able to see more operations.
4. Second Life Misc Issues - MISC

Noticeable sim performance decrease from Second Life Server 1.19.1.80913

Created: 02/Mar/08 02:49 PM   Updated: 05/Jun/08 12:25 PM
Return to search
Component/s: Miscellaneous
Affects Version/s: None
Fix Version/s: None

Environment: Second Life Server 1.19.1.80913
Issue Links:
Duplicate
 

Linden Lab Issue ID: DEV-11427
Linden Lab Internal Branch: Branch_1-19-1-Server


 Description  « Hide
Following the installation of Second Life Server 1.19.1.80913 at Tech Expo we noticed a serious performance decrease. This was obvious to us from delays in the execution of scripts (such as Skidz Primz, various types of sensor based radar etc), and was confirmed by Thomas Conover's excellent Advanced Border & Performance-Scanner v4.5a.

The last time we experienced something as bad as this it was when an upgrade has been compiled with the wrong flags checked in the compiler resulting in poor performance (in that case a near 50% increase in script execution times).

I have spoken to concierge about this and was asked to file a JIRA report.

Please note that this isn't some vague "feeling" i have, it's as plain as night and day - the sim I've been working on day after day for over a year is lagging to hell since the second it came back up with 1.19.1.80913 freshly installed - we all noticed it within minutes.



 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
Janine Cummings added a comment - 03/Mar/08 05:26 PM
Same for me today. Both Region where i have parcels in where updated today. and i can realize a performance drop of around 50%.

tangletwigs fairymeadow added a comment - 04/Mar/08 06:32 AM
I can confirm this issue from my own perspective of TPing regularly between an 80913 sim, and the previous revision (whose version number I dont have recorded). Performance of scripted attachments for example, especially those with touch activated drop down menus were severely reduced in the newer sim version. Such menus were taking upwards of 10 seconds to open on 80913, previous version managed <2 seconds for the exact same operation.

Since the grid wide deployment of 80913 yesterday both sims display the same poor performance issue.


hok wakawaka added a comment - 04/Mar/08 08:18 AM
I am experiencing a similar decrease in Sim performance....My land is spread across three adjacent mainland sims.....During the passed 8 hours, performance on all of these sims has dropped dramatically.......It is very apparent just from looking.....It is also clearly evident in the sim stats......I have used Conover's Sim Performance Meter forever...I have found it to be extremely accurate.....it measures sim performance largley on the basis of script execution times.....Performance scores range from 1 to 5,000.....Each of these sims have consistently EVERY DAY FOR MORE THAN A YEAR had performance ratings of 4,000 ++ ....And that performance rating was very apparent....THIS MORNING EACH OF THESE SIMS NOW HAVE RATINGS OF 1,000 - 1,300.....!!!!!!!!!!! .. None o f these sims have ever performed anywhere near as poorly as they are this AM...These are not just numbers ....The poor performance is very apparent.......

I have also visited other sims that i know to also always have performed extremely well......All of them show a similar decline in perfromance

..FIX THIS PLEASE...


Farallon Greyskin added a comment - 04/Mar/08 10:17 AM
I also can confirm that there has been some kind of change. It seems to be affecting link messages which are going very slow or being missed entirely!

I have a small 7 prim animated "pet" I created and it's body parts now no longer move in synch AT ALL on even relatively unloaded sims where it worked perfectly till very recently.

Also, the time used by the pet during a reposition event used to be a very short spike to 0.5ms it is now spiking at 1.5!

So on my class 4 sim it seems like scripts are now 3 times slower than before


CodeBastard Redgrave added a comment - 04/Mar/08 10:19 AM
Anything that degrades script performance is a huge problem, hence why I voted on this issue.

markbyron falta added a comment - 04/Mar/08 01:06 PM
This should be a critical priority at minimum, and probably a showstopper. Linden needs to fix this issue yesterday and more importantly, fix the process that allows this to happen on a regular basis.

Ilana Debevec added a comment - 04/Mar/08 01:53 PM
Conformed, performance in our sim is down at least 50%, and all others I've been to since the update.

Soft Linden added a comment - 04/Mar/08 04:21 PM
Found & fixed. For the curious, it was a too-aggressive use of strncpy resulting in repeatedly zeroing out a large holding buffer for strings. I also cleaned up some related code and in the tests where we were seeing close to a 50% increased cost on string operations, operations are now about 10% cheaper than when it all started.

hok wakawaka added a comment - 04/Mar/08 04:26 PM
TY Soft

That's great news

Looking forward to seeing this fix implemented across the grid

Guess I can take down my For Sale signs now


Rascal Ratelle added a comment - 04/Mar/08 06:58 PM

Glad to see this is being reported and worked on.

Ever since the upgrade to 1.19 I have noticed a serious increase in sim lag, a tremendous increase in script errors, I already had to replace two items due to script corruption, this didn't happen before the 1.19 upgrade, Serious texture latency issues, Rendering issues,very slow and delayed rendering, Including the following issue that only happened after the 1.19 upgrade: https://jira.secondlife.com/browse/VWR-5177, increased ruthing issues/ av rendering issues,
Friends list stuck on "Waiting/hippos",
increased sim crossing issues,
Very low FPS, failed logins and failed teleports.


Eirynne Sieyes added a comment - 05/Mar/08 03:12 AM
Same drastic performance decrease on PrimPlay Island following 1.19 upgrade as per Conover's Advanced Border and Performance meter. Sim previously reporting near 4000 now in the area of 1700. Please fix this.

Esmee Isbell added a comment - 05/Mar/08 09:26 AM
I noticed this right away just by perception, and then laggy script response and unusually eratic sim stats led me to compare stats for adjacent sims – one having received the update, and the others which had not. So thankful to the OP for writing this up, and bless you Soft Linden and team for finding the issue. Looking forward to the fix being rolled out.

DoC Eldritch added a comment - 05/Mar/08 03:27 PM
I'm noticing huge pending download numbers as well. This hasn't happened on my sim in months. If I remember correct, there was something put in the code months ago whereas the "stuck" pending would be bypassed and not allow others to stack up.

Duckling Kwak added a comment - 06/Mar/08 09:16 PM
Just got 1.19.1.81484 deployed to my SIM.

Pre-1.19.1.81132 I was seeing ABP in the 4500s. With 1.19.1.81132 I dropped to 1700s. Glad to report that I am now seeing 4000 - 4800 (presumably all the adjacent server upgrades/reboots are impacting performance at this time).

All indications are that we're definitely moving back to a more normal state!


eren padar added a comment - 07/Mar/08 12:36 AM - edited
This issue is confirmed in VWR-4987. The situation has not improved and has not been resolved as stated in the above Linden post (continued degradation of performance confirmed by several posts after that).

On our sim we noticed a sudden "doubling" in the number of supposed scripts on the sim, as well as TFT jumping from 10.5 to 20+ overnight... supposedly all script related. Overnight we went from about 1700 scripts to supposedly 4000 scripts (seriously) without a corresponding increase in prim use. Search for the alledged extra scripts did not point to any new source. Estate search for unauthorized builds and scripting produced no results.

Like the reporter, the last time we noticed a serious sudden degradation of performance like this was in early 2005 when Linden Lab went from dedicated servers to stacking sims on a single server box. Our sims went at that time from lightning fast to snail slow... and never recovered. At a later time when we cut scripting and other sim contents to the bone we received performance similar to the constant-lag-state that is a standard of SL today, but even at 10.5 TFT sims do not perform as they did in 2004.

Like the poster of this report, the recent drop in performance was significant, not of one's imagination, was definite and measurable... and it was overnight. The reported number of "scripts" reported by the system is highly questionable. Think about it... 4000 scripts is more than 62 scripts per 1024. 1/4 of our sim is sandbox, which would bring the figure to 83+ scripts per 1024m. I'm not sure I could pack 83 scripts to every 1024m if I intentionally tried.

Interestingly enough, when we reported this to Linden Lab, the number of supposed scripts on our sim suddenly jumped DOWN from 4000+ to 3200... with no alteration of sim content on our part (and yes, we keep VERY close watch on sandbox. The fluxuations were not due to sandbox contents).

We run a very tight sim. We watch scripts, watch builds, watch prim counts regularly. This sudden degradation in performance was exactly as described in this JIRA posting-- and was not attributable to customer content.

Since the initial jump from 10.5 TFT to 20+TFT and 1700 scripts to (supposedly) 4000, our sim has dropped back down to (supposedly) 3200 scripts and 14.9 TFT average. It still has not improved to the former 10.5.

Since that time we have installed a rather large and scripty build that increased sim performance to around 18.5. That jump is attributable to Customer Content and traceable. But considering the current situation, we are not convinced that the new build in actuality increased our TFT by some 4 points. Most likely the 18.5 TFT is an exaggerated statement on what performance actually should be.


Soft Linden added a comment - 07/Mar/08 06:06 AM
@eren - The changes in this JIRA would not affect the number of active scripts. Your best bet would be to file a support ticket, being sure to name the region, and pointing out that you believe more scripts are listed as running than actually exist. Support can dig into this for you and forward it to an engineer if they find a discrepancy, or discuss how they estimated with you if they don't share your findings.

nya Linden added a comment - 05/Jun/08 12:25 PM
fixed in Second Life Server 1.19.1.81484