|
History
| Log In
|
|
|
|
|
|
Added MISC-64 which is precisely the kind of IM restriction that would make this change workable
I don't know how much this really affects things. Or what is the more to blame, central server retrieving the information, the information being sent or the permissions and IMs to be processed by sim. But at this point I am taking your word for it.
Anyway, I wanted to say that perhaps an approach where sim and server would talk and negotiate what to send over might not be the best idea. Too much latency lag and possibly just additional traffic and processing. So it might be better that: A - server would do some optimizations when sending groups, for example do not send information for certain roles when they do not apply, like when a group owns no land. B - rarely used roles could be sent on demand, that is, if there are any such roles that the sim needs to know about, I don't know the architecture C - sim of course could notify back that for example for a certain group it has no members that are listening to the IMs and that it does not need to receive them. D - sim could filter the roles for a group that it determines it owns no land on it. Some of it is maybe already done, some of it may not be feasible for whatever reason, but this my 2 cent. I have come up with a simpler variant of this:
Allow users to "suspend" group membership. When you suspend membership in a group, unless it's your active group, you're not treated as a member of that group in any way. You don't get IMs, you don't get to donate tier to the group, you don't get group access to land, and so on. The only time the table with your group membership info for that group is accessed is when: (1) You make the group active, or you unsuspend it, and you're re-entered into the group in the roles you had when you suspended it. (2) You make the group inactive, or suspend a group, when your membership and role information is copied to the suspended group table and you're removed from the group. (3) A group administrator checks the "show suspended members" box on the "Members & Roles" tab of the group. There would be no limit to the number of groups you could be a suspended member of. As has already been noted in another issue, permissions are the real bottleneck here... so there may be no reason to restrict messaging-related components from using both the group-members table and the suspended-group-members table, and it would be practical to treat inactive groups as "chat only" groups.
This would allow many people to suspend most of their groups immediately, and reduce group-related performance issues right away. It's not clear that the database impact is improved, and that's where the Lindens have said the real issue is.
Right now the sim does the equivalent of this when you enter a sim: SELECT group FROM groups WHERE user = '$agent_uuid'; What you're suggesting is more like doing this on every new parcel crossing: SELECT group FROM groups WHERE user = '$agent_uuid' AND group IN (SELECT group FROM parcel); (though of course SELECT group FROM parcel is a local request on the sim, the list of UUIDs would be sent to the server) The second request by itself has a significant proportion of the database impact of the first request, even with an index on (user,group), and you're making it more often (particularly if you're flying). The impact on the database and sim could well be greater. What I'm suggesting is keeping 'groups' from getting bigger, while letting people join an effectively unlimited set of groups. The simulator only needs to request information about a group if a user tries to do something that is forbidden.
For example: - I try to rez an object on group-owned land that restricts object creation to group-members only - Simulator requests permissions information for me relating to that group - Simulator refuses my rez attempt because I'm not a member of the group, it caches this information in my avatar entry that's passed around - I try to rez an object on the same parcel (or a parcel owned by the same group) - Simulator uses the cached result to immediately refuse my request After a while the simulator will drop off groups that I haven't need a permissions check for, i.e. - if after an hour I haven't tried to rez another object, then it can assume I'm unlikely to try again, perhaps I've left the parcel that needed it and don't intend to come back? This way only groups that are actively needed are retained, also has the nice side-effect that you don't actually need to have a group active to use it. If they decide to stick with only the active group being able to affect permissions then all that needs done is to fetch that and nothing else, as no other groups are of any use unless the user switches (in which case the new group can be fetched, with the old one perhaps being cached a little while). I think you're making an assumption about where the bottleneck is that isn't actually what the Lindens said the bottleneck is: the database load on the group table, not the amount of data the simulator needs to know about you. Every fetch from the group table has an impact, even if it's only fetching one record. If there is an index on "user", then the database impact of "WHERE user = '$uuid'" and "WHERE user = '$uuid' AND group = '$uuid'" are similar... it's going to walk the index for "user" in both cases: on the average it will terminate half way through the second walk, but you'd be doing it more often.
And I think you misunderstood what I was referring to. I didn't mean that only the active group would have an effect, rather that *suspended* groups would only have an effect when they were active. Other groups would have an effect all the time. And I really think you're underestimating the impact of this latency. When you need to kick someone out of a group you want them out *then*, not when the sim group cache expires. When you're a member of a group and you're walking across the parcel boundary, your scheme would make you wait when crossing that parcel boundary the first time. And if I had to wait an hour for the "not a member of the vendor group" notation on my agent to go away after I joined a vendor group because I'd absent-mindedly tried to build there I think I'd have to send the angry postal workers after you. The issue of how to check for expired group data is not within the scope of this debate, it's a problem that presumably is already solved somehow, or if not is a problem that this proposal does not aim to fix. Yours would suffer from the same problem as well.
I'm still also completely at a loss as to what you're suggesting, there seems to be no distinction in what a suspended or non-suspended group even does or is, there seems no advantage to "suspending" a group. Only the active one needs any data retained for permissions checks, all other data is the domain of the IM system (however that works) or the client (for looking up group data if you want it). I don't think this suggestion would greatly add to group-database load anyway, as it only fetches group data as it is required. Currently it seems that the system fetches all group data even if you never use any of it, and does this every time you log-in, which is a lot more strain than should be needed. Besides, the primary aim of this proposal is in reducing the payload of data that is passed between simulators when avatars teleport and region-cross, as currently it is too high, and with many people (unnecessarily) wanting MORE groups then this payload will only get higher should LL cave to those demands. Anyway, if we stick to only active groups being valid for land-access etc., then there is very little work required. When the group is selected you do a simple look-up for the user's permissions using a query like: SELECT * FROM roles WHERE group = '$group' AND user = '$user' It fetches a single row, describing what the user can do. It then caches this, attaching it to the avatar's data in the simulator (which will also be passed on to other simulators), so that checks for permission are instantaneous. If the user changes group, then the new group can be looked up and cached as well, with no more than say 3 groups being cached in total. This allows the user to switch between groups without any cost unless they switch to a new one. Cache expiration as stated must already be solved somehow, probably by somehow noting which simulators last asked for information on a group, and notifying them of any change so that they can expire cached entries or do a new look-up. If you're at a loss to what I'm suggesting, I've gone into more detail in MISC-208. The bottom line is that the suspended groups would be stored in a separate table unless they were active, and that that table wouldn't be used by the permission system. In addition, users would only have the current limit on non-suspended groups, so the incentive would be there to suspend groups that aren't needed for active permissions.
This means that even if you're a member of 36 vendor groups, you could suspend them all and go and update your stuff in all 36 malls and never have to make ANY of them active unless you actually needed to rez a new vendor. So instead of having 36 database requests, as in your scheme, there would be none. My point is that the Lindens have said that the bottleneck on groups is the database load caused by the size of the group table. Any solution needs to reduce the size of the group table or at least keep it from growing. The problem is that your SELECT statement costs about as much (within a factor of two, on average) whether the constraint is "user" or "user, group" unless there's an index on (user, group). But indexes aren't free, either: they need to be updated every change and the cost increases with the size of the table. I've improved the performance of a database system by *removing* indexes. In that case I'd say to implement both. My concern isn't with the group database, I don't want to make it any worse, and don't believe this would. All I want is to reduce the amount of data being shunted about with avatars as they move around.
If manually or automatically "suspending" infrequently used groups can reduce database size or load for the commonly accessed group data-base then go for it, MISC-208 doesn't look like a proposal for it though, please post it somewhere in it's own right. Also, in your example; updating all 36 vendor spots wouldn't require any queries under my proposal either, it would only require a query if you actually try to do something that requires it, either when you try to rez a new vendor, or if you switch to one of the groups for the plots of land. If you're just editing the contents of a vendor you own then it won't affect anything as you don't need to switch group to do that. If the vendor is deeded to group then you will always need to retrieve group data to validate that as not everyone can modify group-deeded objects. So when I go to rez the vendor I get a delay while the sim does the database check?
The advantage to the current system with or without suspended groups is that the group fetches are asynchronous. When you need to know what groups you are in the sim already knows, there's no delay. What your proposal does is put the database lag in the critical path for anything that involves permissions, to optimize something that isn't a bottleneck. There's definitely an advantage to limiting the *amount* of group information that gets passed around by default, and only tracking the UUIDs of groups you're actually a member of (16 bytes per group... negligible even for hundreds of groups). But not tracking the group membership itself is a false economy. As I've already stated the check is either performed when you go to rez something (the latency of which is tiny unless LL's internal network is dreadful) OR it can be done when you activate a group. Either way I'm talking about replacing a single big query that fetches data for all groups whether you need it or not, with a single query for a single group when you need it;either when the data comes to be used (access check, rezzing etc.), or it is indicated by the user (through activation of a group) that the data is soon to be needed.
It's not the *network* latency that's an issue, it's the *database* latency.
And you're not replacing a big query with a little one, you're replacing a big query with one that's only slightly smaller. * Database time is dominated by searches and updates far more than by the size of the results... fetching one row over two keys is, on average, half as expensive as fetching 20 rows over one of the keys because on average the database needs to examine half the results to find the match. You can speed it up by creating an index on both keys but then you increase the cost of updates. * Database time for searches and updates is dominated by table size and competing activity. Reducing the size of the table and reducing the number of queries helps. Reducing the size of the queries and increasing their number hurts. * Increasing the number of available groups without doing anything to prevent a proportionate increase in the table size is going to make things worse no matter what you do about the sim-sim communication. * Reducing the sim-sim communication by only transferring the information that the sims need for the common permissions checks (which is basically group membership and a small number of ability bits) will solve the sim-sim communication problem without increasing the number of database queries. See http://nwn.blogs.com/nwn/2008/01/robin-linden-on.html
Created proposal for suspending groups to alleviate the database load, SVC-1173. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
New Feature
Open
Normal
Only track required groups
It actually occurs to me that one issue (besides the slight pauses during high-traffic to download a new group) is that with a very large number of groups there can be a large amount of traffic generated by group IMs.
This of course isn't desirable by either the system or the users, so it may make sense then for this to go in tandem with a 'Receive group IMs' option (similar to receiving group notices). This would allow a user to opt-in to receiving group IMs, otherwise they must explicitly join the group IM session (through the IM button) in order to receive other user's IMs. This a much sought after feature already, if someone knows a proposal already asking for it then please note it here, otherwise I'll search for it/post one when I have time.