Monday, 26 May 2008

Touring the Open Source Grids

After about 6 months in Second Life I mustered the courage to try one of the alternative open source grids. Having seen the term OpenSim on various Second Life blogs, I Googled that, and found the OpenSim website. At the top of the page was a link to the Grid List, which names a surprising number of public grids, listed in order of their initial appearance. The oldest was DeepGrid, which began operations in June 2007, followed by Open Source Grid (OSGrid) which began in July 2007, and OpenLife which began in October 2007.

I decided to start with OpenLife. I connected to the OpenLife website, and after a bit of searching found the OpenLife Wiki and a page that tells how to get started.

The wiki says that you can use either the RealXtend Viewer or the SecondLife Viewer, but the RealXtend Viewer failed to start for me. So I followed the instructions on how to create a shortcut to OpenLife for the SecondLife Viewer. I opened an account with the avatar name Danton Sideways, and launched the viewer.

I found myself on a rather barren island, in the default female avatar called "Ruth." I tried to edit my appearance, but nothing happened. I looked into my inventory, and found it to be completely empty. I then flew over a few adjoining sims, which were vast stretches of terraformed land with nothing on them. I also tried to use Search to find people, but the Search function gave no results whatsoever. My initial reaction was: "Is this a joke?"

I came away from that initial OpenSim experience thinking that all I could do was fly over an empty terraformed wasteland in the Ruth avatar, looking for other avatars who were not there. On the next try, however, things got better. But before I go on with my story, let us take a glance backwards, to survey the origins of the open source alternatives to Second Life.

How Second Life went Open Source

Back in 2005 Gwyneth Llewelyn presented in her blog post on Expanding Second Life an early vision of how Second Life could be spread across a system of decentralized servers. Gwyneth suggested that Linden Lab could set up a system whereby anyone could run their own sim with their own asset server (which implied making the software open source), but would need to pay Linden Lab for an encrypted certificate in order to connect to the main grid. Gwyneth envisaged a system with many sim operators, each having their own users, as follows:

"When 'your' user logs in at your own sim, the UUID which is generated is encrypted with a key which has been provided by LL. If you don’t use that key, well, you may register with an 'isolated sim' and have your fun there. But you won’t be able to access any content on the main grid, nor be able to export your content there."

Gwyneth thus proposed a way for Linden Labs to go open source but still remain profitable by using encrypted keys. Linden Labs was slow to respond to this challenge. Meanwhile, a low-profile group of non-Linden programmers had already begun reverse engineering the Second Life protocol.

On May 12, 2006 John Hurliman registered, on the Gna! free software development platform, a new open source project called libsecondlife. The registration post defined libsecondlife as "an open source library to allow software to access the Second Life world." Hurliman announced that "we have a complete protocol description", and explained:

"Our goal is to create a multi-threaded, object-oriented library to write a Second Life client of any complexity. We currently do not have any source code, as it's still in the design and planning phase."

Libsecondlife has been described as "a BSD open source library for creating custom clients that could connect to Second Life". A technical description of the libsecondlife project is given on the Second Life Wikia. Development of libsecondlife is still ongoing, as can be seen on the libsecondlife wiki, where most of the articles are signed by Baba Yamamoto.

Libsecondlife however soon generated a major scandal with its derivative CopyBot programme. CopyBot was originally created as a debugging tool by the libsecondlife development team, who had in mind using it to be able to backup one's own created content on one's own hard drive, or to be able to export one's own intellectual property to other environments. But since it was open source, CopyBot was quickly modified by unscrupulous hackers who recompiled it as a tool to replicate objects and avatar appearances without permission. Angry Second Life residents met with Linden Lab on November 13, 2006 to discuss the Copybot problem, and then organized a series of virtual protests, as described by Reuters and Hamlet Au. The outcry led Linden Lab on 14 November 2007 to declare that the use of CopyBot was a TOS violation.

Less than two months after the CopyBot scandal, on 8 January 2007, Phoenix Linden announced on the official Second Life Blog that the source code would be released for the Second Life viewer. It thus seems that libsecondlife and CopyBot were the catalysts for prompting Linden Lab to go open source themselves, in order to maintain control over a development which appeared inevitable.

However, in January 2007 Linden Lab only released the source code for the Second Life viewer, also called the "client," which is the software that the user installs on their own computer in order to access Second Life. There remained the software that runs Second Life itself on the Linden Lab servers, storing and retrieving all of the data about the sims and the users' assets, and implementing the virtual world in real time.

Various attempts had already been made to create an open source virtual world server, but had floundered in face of the massive task of writing both the server and the client at the same time. It was now no longer necessary to write the client, since Linden Lab had released the source code for the official Second Life viewer, and the libsecondlife library was available for creating other custom clients that could connect to Second Life. So in that same month of January 2007, Darren Guard (also known as MV) founded the OpenSim project, to concentrate on developing on open source virtual world server that the Second Life client could connect to. As explained in the Wikipedia OpenSim article, OpenSim uses libsecondlife to handle communication between the client and the server.

Then in April 2007 Linden Lab announced that they planned to release the Second Life server source code (or at least parts of it). But while Linden Lab was still just thinking of releasing parts of the server source code, the first public open source grids, based on OpenSim, started operating. DeepGrid began in June 2007, followed by OSGrid in July. One of the participants gives this description of the first day of DeepGrid:

"There was a sim, its name was "Mortville" and a half-dozen crazed programmers were jumping around on a little island, flying into the ground, all chat was showing up twice, IM crashed the sim, all objects were phantom, no trees, no nothing, but it was wondrous. Within a couple of weeks there were a dozen sims on Deepgrid."

Linden Lab struggled to keep up. On 13 September 2007, as explained in a page on the Second Life Grid website, Linden Lab kicked off a public planning process for "designing protocols for virtual world software interoperability," which led to the formation of the Architecture Working Group (AWG). One of the stated goals of the AWG is to "enable third parties to run servers that connect to the Second Life Grid platform." In other words, since the development of open source grids is inevitable, Linden Lab wants to enable third parties to connect their own sims to Second Life, for a fee. Which is pretty much what Gwyneth Llewelyn was talking about back in 2005.

The latest development in this direction is an announced plan for IBM to hold Second Life grid servers on their private corporate network, behind the firewall, but with enough network access to communicate with the central services on the Second Life grid. This project is further described on the blog of Zha Ewry (David Levine) of IBM. Zha Ewry is also the head of the Linden-sponsored Architecture Working Group, and IBM runs its own private sims using OpenSim software. One of these is the Shengri La sim, built by Shenlei Winkler of the Fashion Research Institute, as can be seen here.

My Further Adventures on the Open Source Grids

As I've told above, my initial experience in OpenLife was a total disappointment. I did nothing but fly over barren terraformed sims in the Ruth avatar, with no inventory and no search function. So I decided to try CentralGrid to see if that was any better.

On the CentralGrid website I found a link to the CentralGrid forum, with a page that tells how to log on. Following these instructions, I created a shortcut to the Second Life viewer, in exactly the same way as for OpenLife. Once logged in, I found myself in an identical Ruth avatar with an identically empty inventory. The only difference was that the entry sim was somewhat more built up, with a few blocks of empty streets and wide-spaced, sterile buildings all in the same modern style. And my feet didn't sink a few inches into the earth, as they tended to do in OpenLife. But the search function was equally unusable. What I understood later is that all of these open source grids are just picking up the latest version of the OpenSim software, and tweaking it, so they can only offer the functions which have already been developed for OpenSim.

In both OpenLife and CentralGrid, the main activity seems to be selling entire sims. Some of the open source grids allow users to connect their own servers, but these two seem to require you to rent the servers from them. There is no official way to rent just a plot of land (though one could always rent a plot by making a private deal with a sim owner). Few of the grids have any plans to create an in-world currency, though CentralGrid is reportedly working on it. People buy these sims knowing that the software is still under development, and that there is no in-world economy, thus no possible return-on-investment, so of course the prices are lower than for sims on Second Life.

Some days later I decided to try again. I went back to the OpenLife website and logged onto the chatroom. One nifty feature of OpenLife is this chatroom, which can be joined easily by anyone, without having to know how to use IRC channels. There is nearly always someone on line to whom you can ask questions. I found online Sakai OpenLife himself, the founder of OpenLife, and asked why I was unable to edit my appearance. He replied that it was explained on the wiki. So I went back to the wiki and found that there was indeed a page about editing appearance. You must go into your inventory, click on Body Parts, right click to get "create new," and then wear the new object to edit it. I thus created and edited four body parts (body, skin, hair, eyes) and three pieces of clothing (shirt, trousers and shoes). I tried to get the same look as for Danton Sideways in Second Life, but the result was rather far from the mark. And I was unable to grow a beard, because while my chin puffed out into a beard shape, I had no appropriate texture colour to add, so I reverted to a clean-shaven look.

Remaining in-world on the dreary welcome island, I changed windows to go to the chat room, and asked where I might find a more lively sim. Someone suggested I try Electron, which is the sim of Gobi Greenwood (correction: Electron is actually owned by Gumby Roffo). Electron is rather nicely built up, including a stage and screen that streams a video clip. On my next visit I went to Quartz, another of Gobi's sims, where he is setting up a freebie store. It is a several story building which for the moment is totally empty. I also went to the Renegade sim, where I met Josh Noonan inside a fancy temple build. He proposed to rent me a plot via an external payment using paypall. I said I was interested, but I have to go back to discuss this further.

I then returned to CentralGrid. I set the destination sim to CentralGrid Home, but had a problem logging in. The error message said something about waiting for a handshake, and the forum instructions for dealing with this were rather complicated. So I just tried a different destination (Central Grid Sandbox), and that worked. I edited my avatar exactly as on OpenLife – the underlying OpenSim software is identical – but the result was again disappointing. You would have trouble recognizing that it is supposed to the same Danton Sideways as is in Second Life.

I tried teleporting in CentralGrid, and arrived in the Moonbeam sim, where I was surprised to actually see another avatar on line, hanging in the air. Then I crashed. When I logged back in I was Ruth again, and had to open my inventory and put on all of my edited body parts and clothing items one by one. When I was dressed I looked for the avatar I had seen, and found Happi Homewood, working on his multi-story shopping center in the Moonbeam sim. He explained that he was getting ready for when CentralGrid would implement an in-world currency. I sent him a friend request, so that I could find him easily in-world, and he accepted it. That was my last act so far in CentralGrid.

And last but not least, I made one in-world visit to the Open Source Grid. I found the OS Grid website, and a link on the site to the OS Grid forum, which is obviously used by a fairly large and active community. Creating as usual a new shortcut for the Second Life viewer, I logged in as Danton Sideways, and landed in Wright Plaza. I took a moment to quickly run through the habitual OpenSim avatar editing process, putting on my standard 4 body parts and 3 pieces of clothing, and then explored the sim

I was pleased to find a fairly intricate, colorful build, with lots of vegetation, typical of what one finds in Second Life. And there were several avatars roaming around in just that one sim. I went up to a certain Paulie Femto, and chatted a while. He said I could use the sandbox if I wanted to on his Cameo Beach sim, which is actually run from his home computer, connected to OS Grid. He said he tried to keep it operating 24/7. He also told me about the freenode IRC channel for OS Grid. That definitely got my interest. After all these years of hearing about them, I've never actually had the occasion to use an IRC channel. This may be my big chance!


IYan Writer said...

Great writeup of OS sims history and current status - thanks!

Gwyneth Llewelyn said...

Thanks for this article! I think you overestimate my claims back in 2005; several others not only said the same, but a few actually implemented something (in spite of LL's initial lack of support; however, LL quickly reverted their position and are now active participants both in the libsecondlife and the OpenSim projects), which I found deeply encouraging :)

Yes, we're a few years away from having an OpenSim-based "clone" of SL as it was in, say, early 2003 or so, when most of the functionality we have today was definitely there, at least on the server side. It baffles me how people are actually willing to engage the "Land business" in the OpenSim grids — talk about speculation! — when it's clear that anyone except a high level programmer won't have patience for a world without permissions, without a payment mechanism, and just voluntary support. Then again, the same could be said about SL in 2002, and it definitely attracted 500 eager users to do the beta-testing for almost two years — which is the time I expect the OpenSim grids to become minimally acceptable for regular power users (not programming gurus, but power users nevertheless).

And when that happens, hopefully LL times everything "just right" so that the interconnection mechanisms with their grid are also ready by 2010, which ought to be a reasonable timeframe, taking into account the way they're working on it for over a year and a half right now, and which has been slow and painful work — as we all feel in our virtual bones every time the Texan grid loses the connection with the Californian grid.

Desmond Shang said...

Just for clarity - in the mention above, use of CopyBot itself was never against service terms. Just its use for unlawful activities.

So if you choose to CopyBot your own material, or material for which you have permission to do so, there is no crime.

Of course, CopyBot goes hand in hand with piracy the way bittorrent goes hand in hand with unlawfully ripped mp3's and movies.

Danton Sideways said...

I'm glad you all found this post to be useful. The new OpenSim grids are an amazing development, and one which came faster than anyone expected. Thus I wonder if the additional functions will take a full two more years to be developed, as Gwyneth suggests, or rather less.

OpenSim is at the crossroads of two of the outstanding achievements of "Massively Distributed Collaboration", which are open source software development and wikis (Second Life is a sort of 3D wiki). It will be interesting to see what new sorts of cultural behaviour and social systems will grow up with the new Open Source Grids.

And thank you, Desmond, for your observation about the legality of using CopyBot-type programs to save personal inventory to one's own hard disk. What do the Terms of Service say about exporting one's own intellectual property from Second Life onto a different platform?

Consiliera said...

Thanks for documenting the OpenSim history in so much detail! I had similar experiences when trying out CG and OpenSim some time ago and I am eagerly watching the process.

Regarding the porting of your own items to other grids: As far as I know that is okay with LL's TOS because that's exactly what the professional software "Second Inventory" ( lets you do (officially, across grids) and so far I haven't heard anything negative from LL regarding the product. It's widely used and even runs on my Mac using Parallels :-)
Furthermore I read about a new feature LL is planning to introduce in the near future but I can't find the source, it goes like this: They're planning a new permission setting, something like "portable" (in addition to copy/mod/transfer) in order to allow creators to allow or disallow the portability of their object to Open Sims. Maybe somebody else know more specifics on this.

100nada said...

Thanks for reminding me that I have a sim in Openlife :DDDD (it's my partners' sim, actually): Been there a couple of entusiastic times, then I gave up, when all the stuff I had built went away from my inventory. Still, is a nice and quiet place. Very good article you have here.

Cat Magellan

Connected-Worlds said...

Remaining in-world on the dreary welcome island, I changed windows to go to the chat room, and asked where I might find a more lively sim. Someone suggested I try Electron, which is the sim of Gobi Greenwood.

Electron is actually owned by Gumby Roffo, not Gobi Greenwood

gumby37 said...

G'day and thanks for a nice article on the current grids . As the real owner/renter of Electron thanks for visiting. I have been in SL since March last year, Im using Openlife for the beauty of having lots of prims to build with. Yes its a tad lumpy some days but I didn't step in unaware of its development. As a result Im looking forward to the improvements that happen each week or month. it is behind SL by about 2 or 3 years...but is still a nice addition. I will look forward to inter grid hopping when the protocol has been worked out and every ones happy.
Feel free to look me up as Gumby Roffo in either grid.

Loki Popinjay said...

Great post. I think however the question in the long run will be if LL can maintain the momentum to develop into the heart of a growing open grid or whether others will take over. I wrote about Sun's "project wonderland" a while back and I think it might turn out to be stiff competition for SL.

Prokofy said...

Danton, I'm glad you characterized the use of Copybot as something done by "unscrupulous hackers". That's correct. But you've failed to report a very important fact about all this: the "unscrupulous hacker" is John Hurliman (Eddy Stryker in SL) *himself*, who was the maker -- and seller! -- of Copybot. You forgot to mention that not only did this cynical manipulator laugh in the IRC channel with his buddies about how this device would harass and humiliate people, he sold it for a high price, and it did a brisk business until the Lindens shut it down.

He went from having his main and alt suspended, to eventually having the Lindens invite him to the select FIC formation at the Lab and first meeting of the Architectural Working Group. So, you can see where the "unscrupulous hackers" are all located...

I'm also thinking there's some sequencing of dates not quite right here, as my recollection is that the CentralGrid announcement and webpage came BEFORE the Lindens' announced the intention to open their server code, but I'd have to research this.

Prokofy Neva

Danton Sideways said...

Hi Prokofy. I was wondering how long it would be before you showed up to chastise me for my interest in OpenSim. I am aware of the importance of Intellectual Property concerns, but I think the Open Source movement can coexist with IP rights. It is true that OpenSim was made by reverse engineering of Second Life, but the Lindens seem to recognize that it could be to their advantage to have their platform serve as the basis for a new universal standard.

What you say about John Hurliman is disturbing. In addition to being Eddy Stryker in SL, does Hurliman also go by the name of Baba Yamamoto, or is that someone else?