Lightmaps

For a long time I wanted to have two things in the game:
1. Night time shadows.
2. Different visibility based on your location, ambient light, nearby light sources, what you wear, what your attributes are. For example, someone sitting in a shadow at night time, with no armor on, would not be visible by someone sitting a few meters away. However, the person in the shadow would see someone that is in open space, under a light. Wearing shiny (metal) armors would make you more visible. Having a higher perception skill would make it easier to see people hiding in dark places.

Last night I spent about 8 hours (going to bed near 6 AM) to write a quick and dirty server side light map generator. The idea was to take each walking tile, see what the nearby lights are, trace a ray from the tile to the light, and if it isn't blocked add an attenuation base done the distance to the light, then store the info.

The generation was relatively easy an painless, due to the fact that I used the collision detection routines that were already there (same as for the arrows). It was fast too, it took only about 12 seconds to generate it for Whitestone, which is the biggest map size we have in the game.
I wasn't pleased at all with the results though. First problem was that most of the lights are placed inside lamppost or fireplaces and so on. Which meant that the ray collided with the object and the tile was marked as not illuminated. I fixed this by checking the end collision point, and if it was less than 30 cm away from where the light was, I assumed the collision was with the lamp post (or whatever other object is used to represent a light source) and marked it as illuminated.

The next problem was that even if a small tree branch is occluding the light source, a whole tile is marked as shadowed. Not good. This could be solved by adding about 10-20 rays with a small displacement, then computing an average value to see how many made it to the light source.
Then I realized we have two more problems that can't be solved with the current algorithm: The collision thing does not take the texture into account, so many trees obscure a light source even if the leaves would allow the light to pass. Now, this could be solved by taking the texture into account, but then some function would need to be severely modified. The second problem, which is far harder to solve, is that the client does not have this information, as we have no light maps. So even if a tree obscures the light, if a tile or object is close to a light, it will be fully illuminated. This would create a lot of confusion for the players, as they wouldn't know where are the good spots to hide, or where they can expect some hidden player to be.

There are quite a few other problems which I won't even go into, such as the fact that to do it right, we'd need like 6 bytes for each walking tile, to take the light intensity at different heights.

I talked with Schmurk about it, and we decided that the best solution to this problem would be to use lightmaps on the client.
Now, there are a few problems with the lightmaps, especially with high quality light maps:

a. They can take many GB of hard disk space, for one map alone.
b. They will take many MBs of RAM, possibly up to 512 or so per map.
c. They can take forever and ever to generate, possibly a day or two for a big map, on a fast computer.
d. Given their big size, they can't be included with the client download. Even compressed they might add up to 1GB or more for the whole game. So we will have to use p2p networks to distribute them, and maybe even sell a DVD with the prerendered maps.
e. They do not work with dynamic maps.
f. They do not work with moving things (players, animals, etc.)
g. If you make a change to a map, you will have to regenerate the whole lightmap for that map.

Now, for the advantages of using them:
1. Once created, the server can use them as well (with some modifications).
2. We can have true shadows during the night and in indoors areas, we can have as many light sources as we want, and a light can be visible from as far as we want it. Additionally, we can have soft shadows, and shadows from multiple lights (so there won't be only a shadow or no shadow).
3. The game will look so much better with them.

I already contacted a former developer, who is very good with math and OpenGL, and he will implement them for us (we'll pay him for it).

Until then, I am going to ignore all the collision with the lights, and use only a linear attenuation on the server, to determine how visible a player is. This is a good way to test the system, and when we have the light maps, we'll use them on the server too.

Collision detection

During the past week, I spent most of my time working with the collision detection code provided by Schmurk.
He did a really good job, so there were very few problems with it. The first test, on the test server, started friday. There were very few issues, mostly due to the fact that I was setting the collision height a little bit too high (150 CM). Another problem was that I forgot about some 'ghost objects', which are deleted objects that are still in the maps in order to preserve the ID sequence for some other objects. So in a few cases ghosts objects were interfering with the collisions.
After fixing those two problems, and after Schmurk adjusted a few settings in his code, everything works fine now.
Now we still need client support for the arrows, which will be a lot of work as well, but then we can have arrows flying through the game, and a new skill (Range).

I am pretty exciting about all the things we can do in the near future. Things such as having some monsters attack you only if they see you. Or some magic spells where you must to have a line of sight with the enemy. Or, maybe, even for some quests.

What we are working on

Learner tried (with pretty good results) to make use of the zlib library that is linked in the Windows version to the binary (it is needed for the XML parser). He tested compressing the maps and 3d objects with it, and it seems that now the map loading time is faster.
He didn't do extensive tests yet, but it only makes sense for compressed files to load faster, after all the HDD is the slowest thing in a computer, and with today's fast CPUs the decompression is almost instantaneous.

Then we have our 'special effects guy', KarenRei who has almost completed integrating his new special effects in the client. He posted some screenshots here.

I am already working at integrating Schmurk's collision detection code in the server (should take a few more days, plus a lot of testing), and then Learner will add the gzip support to the server as well, so that the maps will load faster, which will mean reduced start up time, which will hopefully compensate the reduced start up time from the collision detection initialization stuff.

All in all, things are pretty good for, and we hope that we will have a new client update around April or May with the new special effects and some speed ups. The arrows are not going to be in the next client though, there is still a lot of work to be done.

Windows Vista

Vista is out for almost two weeks now, and EL doesn't seem to work on it. We don't really know why, no developer has Vista, nor do we plan on switching to Vista any time soon. In fact, I am thinking about moving to Linux or FreeBSD in a few years, when they will be more mature for desktop use, and when the percentage of XP computers will drop under 30% (or when the majority of hardware manufacturers will stop releasing drivers for XP).

It seems that the majority of our players don't care too much about it either. Now, that poll is not the most accurate reflection of their will, since many didn't care to vote, but nevertheless it is good to see that about one third of our players use non MS OSes.

I am aware that many people buying new computers (that come preinstalled with Vista) might not be able to play EL, and this doesn't concern me for the time being. We are right now in a 'comfort zone' where the amount of players is just right. 500+ more players and then it will become kind of crowded for one server, then we'd have to look into more servers, which is a lot of work (see my previous entry).
Of course, sooner or later someone will address those issues so our game will work fine on Vista eventually. But there is no rush.

One other thing I am curious about is the number of people that will say no to MS and go with alternative OSes. If things go as I believe they will, in 2 years the percent of non MS OSes will be somewhere around 10%. Right now the percentage of non MS operating systems it's about 7%, divided mainly between OSX and Linux (4% OSX and 3% Linux). It is difficult to find an accurate source for those numbers, some sites reporting Linux to around 3% and OSX to around 4%, others reporting more or less than that.

867 concurent players

Last saterday, we had a new record: 867 players, and counting the player run bots, about 950 connections.
I think that this new record is a little bit inflated, it was Stakhanov's day (*3 harvesting experience) and some people also brought their alts on, to level them up faster.
Nevertheless, it was an interesting experience from a development standpoint; the CPU usage almost doubled compared to just 800 connections (it went up to 22%).
A big part of the increase was due to the operating system itself, such as the time spent dealing with intrerupts (hdd access, networking).
Now, 22% CPU is not a lot, but it seemed that the increase was almost exponential, which means that we might not be able to accomodate more than 2K players on a single machine. So in a year or two, we might have to have another server.
This is a logistic nightmare in itself (we would need new moderators for that server, new forums subsections for each server, more time spent to update the servers, and so on).

On the other hand, we need more data, especially to see what happens when we reach 1K connections on a daily basis. We are currently not using full optimizations during the server compilation (we use O1, not O3). We can also try a few more tricks that would reduce the server load a little bit, but it is really hard to predict how the CPU usage will look like with 2K connections. If we are lucky, we might be able to push the limit to 3K players, especially because we ony use a P4 @3Ghz, which is an OLD CPU (about 2 years or so). A newer CPU with more cache might make a big difference.

But one thing is clear, in the future we will have to have more than one server, and need for that is an indication of our success. And by the time we will need that, we hope our financial status will allow us to hire a few full time team members to take care of the moderation issues associated with our future growth.