|
|
|
[
Permlink
| « Hide
]
Sean Linden added a comment - 05/Nov/07 04:58 PM
Actually we don't need even the central HTTP proxy for this. This could be implemented directly on the simulator, with the URLs being either caps or pointing to a web server in the simulator itself, i.e. http://sim###.agni.lindenlab.com:simulatorport/uuid/foo/bar/baz
 . If we do this with caps, it's possible the llHTTPServer call would not be able to return the URL directly, but would generate an event when the URL came back. Silly question, but do we want this to be prim specific like email or do we want script specific like http?
Is the uuid in the URI the prims or the scripts? (my last comment was in response to Sean's)
I suppose it could be random like the request above asks for. A replacement for xml rpc? Yay.
Are you working on this, Sean, or just reporting for someone else? Still all kinda in the works as things progress, can't wait though. =)
llHTTPServer LSL Portal Page: Added the rest of some more functions to start completing the base set. As of now the function set is following a prim based approach from general LSL community feedback. Easily changable though. =)
XML-RPC is the only script based comm. Both email & HTTP are prim based on the responses. So should this be? That would also utilize the objects per sim max as an HTTP server max per sim, instead of the script max. Can a sim even sustain over 15,000+ HTTP servers? Looks more like every prim will get its own subdir on the existing sim's http server rather then get its own dedicated server.
It makes more sense and yes, I do believe an http server can host 15,000 different directories. 
Makes me wonder how one will handle a moving attachment that may cross simulators though.. will need a way to report the new domain for the prim once it moves. Hopefully, not via email. I'd guess doing something like the following:
no_http() Consider that on average most of the heavy uses for this wouldn't be in moving items Wow, this sure would rock. I can't wait to see this. A couple of things spring into my mind, though.
First, having a proxy for all incoming HTTP requests might not be a bad idea. That way you have a single point where you can easily filter incoming requests if you want to ban an IP or somesuch. Also, allowing us to host random HTTP servers on your equipment might be a little dangerous. This is the hypothetical-evil part of my mind talking, but consider that someone might see this as an opportunity to use your bandwidth and resources to host HTTP content that's completely unrelated to SL. Maybe they'll set up a filesharing network that grabs files in nibbles from a bunch of scripts. Maybe they'll host illicit files relatively anonymously on your servers... such as small exploit executables, shellcode, etc. You might find yourself hosting stuff you don't want to host. Remember that llBase64ToString function, which might let someone server up arbitrary binary files via a script. Of course, that depends on what exactly we can include in our HTTP responses from the script. Will we be allowed a response at all (please!!)? Will it be limited to 2kb like llHTTPRequest() responses? Will we be able to specify arbitrary response headers? Content type? Caching/expiry information? And one more thing... it'd be SO SO COOL if llHTTPRequest() could direct a request toward these servers. I know that currently, for example, llHTTPRequest() can't be used to send an XMLRPC request to another script, because the proxy lives at the border of LL's network and doesn't want to direct traffic back in. I'd just like to beg you to please reconsider this policy and reconsider the technical limitations around this to see if there might be some kind of solution that allows requests to go out and loop around back in, without the cooperation of a forwarding server out on the net. This would be a great stopgap measure for script->script communication and/or shared memory while we wait for something else more reliable than email. I would LOVE this. I've been trying to use XML-RPC but it has so many damned problems it's almost completely useless, when it does work it's incredibly slow!
Having it go down to a simulator level also makes a lot more sense. Wouldn't it make more sense to treat this like XML-RPC for an extra level of security? Just off the top of my head real quick:
llOpenHTTPServer() as the function to open the server. Then an event: http_server(type, key channel, string source_uri, method, string data) type in [HTTP_OPEN_REQUEST | HTTP_INBOUND_DATA] Then call it by something like: [channel]&data=[your data here]">http://sim###.agni.secondlife.com/http/?channel=[channel]&data=[your data here] Otherwise, people will be able to grab the key of your object and send messages to it unless you check again source_uri - not pretty. Just a thought. I believe the discussions the lindens had moved more towards using the CAP system to provide the URL's
ie: https://cap.secondlife.com/cap/0/12345678-abcd-ef12-1234567812345 One big benefit of using the CAP system is that llHTTPRequest can make calls against the cap.secondlife.com URL. This would mean that the HTTPServer and HTTPRequest pair could be used not only for external -> internal communications, but also for object -> object communication. I thought of a good solution to my hypothetical problem of people abusing LL's kindness in order to serve arbitrary non-SL-related crap anonymously in SL. It's pretty simple: limit opening an HTTP server to people with payment info on file.
This is normally NOT something I would suggest as a way to prevent griefing. I know there are a lot of people clamoring to lock down LSL "to fight griefers", making it so that newbies under a certain age, or newbies without payment info on file, or even non-premiums, can't use LSL, or can't use X, Y, Z functions of LSL, etc. Normally I hate the idea and vehemently argue against it. The entire reason I'm here and contributing to the SL community right now is because I was able to get into SL and start scripting without having to pay monthly. I'll storngly oppose all measures to limit "griefing functions" such as llRezObject(), llGiveInventory(), etc to non-newbies or people with payment info on file. However, in this one specific case, I think it's okay. First of all, there ARE options for people without payment info on file to use, and they do work, albeit somewhat slowly: XML-RPC and llEmail. Allowing people completely anonymous access to hosting a HTTP server is a pretty bad idea, because we could see the entire grid subverted into one big cog in a botnet, and I'd really like to see LL not banned from the entire rest of the internet. Requiring payment info on file (or even age or identity verification, so long as the information can be subpoenaed by a court) requires that the account creating an HTTP server is NOT anonymous, which will mean that anyone performing illegal content can be subpoenaed. In short, people won't be jerks as often if they're unable to hide behind anonymity. I do want to be sure that this restriction doesn't unfairly prevent non-US folks from playing with HTTP servers. This'll take some more looking into. Also, I imagine that the check on whether a given script's owner has payment info on file is probably not instantaneous. That'd mean that the URL for the server (or the denial) would probably not come immediately. I think that'd be fine; just make it an event like llOpenRemoteDataChannel(). To be extra crystal clear: This is a design proposal and suggestion. It is not a linden-sanctioned-official-feature. No one is currently implementing this, and no ETA for even when development on this or any similar feature might start.
I will see if I can answer some questions that have come up though: Central Proxy: Probably not gonna happen. Scalability issues make this difficult. Yes, of course you could offer a reply (that is a large part of the point of a 'server'), and yes the size of the reply would probably be limited. Also due to how the cap system works while we could allow different methods (PUT/GET/DELETE/POST etc) we may have to restrict url params (?foo=bar) which would mean using POST data to pass data in. In the end probably just a minor inconvenience. And yes, flipper is right - the llHTTPServerStart or llHTTPServerRequestURL or whatever will have to trigger an event with the url, which may be the same event used for normal responses (aka xmlrpc) or a different event. This isn't really for security though. It is because the cap server (whether central or on the same host) is not part of the simulator process. This means that the requests are asynchronous and we can't stall the sim or script waiting for a response. In other words, for the same reason that dataserver events work how they work. The security part if covered by using the cap system which has unguessable urls. The 'no_http' or similar is a bit tricky because of the ways a server may be lost. I'd like opinions on polling mechanisms for this (llRegistHTTPServer returns the same URL if one already exists, or something) - those are easier to implement, although the no_http may not actually be that hard. I am just prattling off the top of my head here. Also, since this is a cap system we would probably want a method to revoke a cap and generate a new one. Aw, don't go bursting our bubble
 Still, I think this kind of proposal means more when it's a Linden making the proposal, not just a resident.
As to the central proxy: yeah, I kinda figured. Didn't hurt to suggest it, though. And you DO use one for llHTTPRequest... I think the no_http thingis pretty darn important, or at least give us information on when we can expect the server to close, or even barring THAT, give us ideas of when we can expect it NOT to close. I think the no_http suggestion would actually be much easier and more future-proof, since it allows me to write a script that I can expect to work even if the conditions in which my server closes change in the future. As to revoking the cap, I think this could be handled via some kind of no_http event as well. It seems to me that it'd be much better for the stability and reliability of our scripts if this is done via an asynchronous event, rather than with the script polling to see if it's lost its server. You don't want us polling 60 times a second, do you? This is a replacement for the ailing XML-RPC system, and if big systems like SLExchange are going to use it, they need to know they can rely on it. I don't pretend to understand the backroom intricacies of this but my project is inoperable until a fix or replacement is available for the XMLRPC.
I already had the option of HTTP polling but the idea of 1,000+ objects polling every 2 or 3 seconds (which would work for me) to see if there were any messages just seemed to be immoral. You are incorrect in one thing Lex: there is no central proxy or gateway for llHTTPRequest. This is one of the primary reasons this feature is viewed as such a success by us.
AnnMarie Otoole: It is understandable that at large scales in SL verses smaller scales outside SL, all the polling can be hard. However if your servers can handle it, thousands or tens of thousands of objects polling every few seconds is much nicer on SL's central resources and will be much more reliable than XMLRPC. Crazy but true! ....oh! I could have sworn I remembered reading that there was a proxy for llHTTPRequests(), and I got the distinct impression it was a central one. I guess maybe I misunderstood and it's a proxy on each simulator or something. Darn, I guess that makes my plea for the ability to do script->script HTTP not so easy to implement, doesn't it
llHTTPRequest goes through a squid running on the simulator host itself.
I'd love having this feature, however, I would point out that you could achieve similar results with the existing xml-rpc system but changing the channel key returned to llOpenRemoteDataChannel to be the region_id/unique_id/path_id string mentioned above (since keys and strings are largely interchangeable and you could return this value in remote_data casted to a key). You'd still have a incoming dispatch server, but it would be far more lightweight than the existing one and completely stateless, so it would be orders of magnitude more reliable. Existing scripts would work except in cases where people expected the contents of the channel key to be an actual guid.
I agree that a way of streamlining XML-RPC as Emma describes may be better in the short term, especially if we can't get this HTTP server thing done soon. XML-RPC for me has become completely unusable, I am frequently having my scripts time out without the XML-RPC ever getting through which is making a networked vendor extremely difficult =(
However, if (as Emma proposes) the XML-RPC channel were changed to identify simulators in the first part then they could at least be routed at high-speed to the simulator to deal with. The huge delays and the delays involved in all XML-RPC functions (llSendRemoteReply()) are making networked vendor systems unreliable at best, or require huge numbers of llHTTPRequest() calls =( It is not feasible to change xmlrpc in the way Emma describes, for exactly the reason in her last sentence - some content would be likely to break.
There are ways the xmlrpc path could be optimized, but none are easy. At this time I believe development time would be better spent implementing http_server (or some variant) which will always be more scalable and longer lasting than xmlrpc. I have posted my design for this feature at
.
Work has not been started on this project. My design differs in some significant ways from Sean's initial proposal. I created a new issue because I wanted to keep the comments clean and about the new design. I was torn between resolving as 'won't finish' and 'duplicate'. I chose 'duplicate' because it better indicates that progress continues in the spirit of this issue.
For those still only watching this issue, I have commented on
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
New Feature
Resolved
Normal
Allow LSL scripts to act as HTTP servers in order to replace XMLRPC with something scalable
