Instances

I decided to add some instancing to EL, to spice up the fight.
This is how it will work (although some things might change).
There will be a few new maps that are instanced, and the following conditions will apply:
1. One player can go there every so often, which is defined by each instance type. The time between going there again will be something like 100-200 hours.
2. Most, if not all of the instances will have minimum and maximum a/d requirements, so for example a pr0 player won't be able to go and spoil the fun for the newbies.
3. Each instance type will require a certain number of players to go in (a minimum and maximum number).
4. There might be a cost associated with going in an instance.
5. Most (or all) of the instance monsters will be unique monsters, with unique drops.
6. To join an instance, a group of players will have to join a secret channel that they chose. Then they will all go to an NPC which will teleport them to the desired instance. To prevent abuse, all the players from that channel must stay near the NPC.
7. If you die in an instance, and you have a rostogol stone, you will be placed at the beginning of the instance, so that you can rejoin your friends.
8. There will be 5 (or more) monster waves, that is, if you kill a wave, the next will spawn shortly after. Those waves will be somewhat random, from a predefined set of possibilities. The final wave will be a 'boss' type of monster.
9. The drops will be unique items that can not be obtained in any other way. My current plan is to make some really powerful items, but they will have a high chance to degrade, so the game won't be full of them.

That's pretty much it. I don't know exactly when they will be in the game, probably in a month, if everything goes well.

Kqueue implementation done

Last night I fixed the last remaining bug in my Kqueue, and now everything works great.
Right now, there are 823 connections, and the server is taking less than 7% CPU. Before implementing this, it would have taken close to 20%.
And there are some other, non network related optimizations I can do (especially the stuff that trashes the CPU cache), so it's a good thing.
But more about them later.

Kqueue working

Sometime last week I finished the Kqueue implementation, and after a failed deployment (where it crashed the server an hour after the restart) I did some final touches and now it works great. There is still a bug to fix where sometimes a socket attempts to send data to an nonexistent client, but this is pretty rare and will only cause someone to be disconnected from the server.
I am, however, trying to see what causes that problem.

And now for the performance results:
Using SDL_net to check which sockets had data, the server was taking about 15% of the CPU with 700 connections. Now, with Kqueue, it rarely goes over 6%. The nice thing about Kqueue is that it scales very well. For example, if it takes 0.01 ms to check a socket, it will take 1 ms to check 100 sockets. With select() or poll() it might take 0.01 ms to check a socket, but 5 ms to check 100 sockets, and 20 ms to check 200 sockets (all the data is made up, just for an illustration purpose).
What this means for those who do not understand technical terms is that before this change, the server couldn't have handled more than, say, 1500 connections (with enough spare CPU for an invasion). Now, I think it can handle up to 5K connections or more.
It also means that the server should be more responsive than before, although there is not much of a difference if the server responds in 130 rather than 140 ms (most of it being the network latency).

The conclusion of working with Kqueue is that it is well worth the effort to implement it, and it is also pretty easy to do so, once you understand how it works. However, the documentation is not very good, and it is wrong, or at least misleading in some areas.
For example, if you try to change an event in the same call where you also get the pending events, your pending event will not be changed until after, which can cause problems.
And there is a nasty bug where if you add a listening socket to a kqueue, the sockets added after that via accept() will become, for kevent, listening sockets as well, so you won't get any notifications from them. The way I did it was to create two kqueues, one for the listening sockets and one for the normal sockets.

Last week's work

There has been something in the last week that significantly reduced the time I spend working at EL. That something is Fallout 3, a game I've been waiting for almost 10 years.
I ordered it off Amazon and got it delivered on 28th, and ever since then I am playing it like a maniac. Usually, I don't play video games, maybe I spend 50 hours a year playing video games. But Fallout 3 is way too addictive, and I think it is the best game I played so far.

Anyway, this past week I almost finished the server programming for horses. We tested them on the test server a few days ago, and there are very few bugs left, so if everything goes well, we'll have the client update sometime in December. And as expected, everyone who tested the horses loved them.

Regarding my optimizations, I finally finished setting up a local FreeBSD machine for my tests. This test machine is an old laptop, a Pentium M 1.7 Ghz with 1 GB of RAM. I had some problems with the standard FreeBSD install, so I used PCBSD instead.
So today I ran the server for the first time on my FreeBSD test machine, and I've noticed something perplexing: It is orders of magnitude faster than the machine we currently use for the main server!
That machine uses an older Xeon @3Ghz (made in 2004 I think), and runs FreeBSD 6.x and gcc version 3.4.6 [FreeBSD] 20060305
My laptop test machine thingy runs FreeBSD 7.x and gcc version 4.2.1 20070719 [FreeBSD]
On the main server machine, an empty (just the AI) server takes about 1% CPU. On my laptop, it takes 0.00% (!) CPU. Now, you might think that an empty server shouldn't take more than 0.00%, but the thing is, there is a lot of AI (over 1.3K entities, most of them moving around).

This is very strange, and I am not sure why my laptop is so much faster. It could be that GCC 4 does a much better job, or the new FreeBSD is much better at handling stuff, or maybe the main server machine is not configured properly? I'll have to ask my host to do a few hardware tests, and possibly update the OS as well.