Master Server Source Code

EQ² · Mar 4, 2014

Not sure if anyone will even remotely give a monkeys' about this, but my thoughts are that maybe other struggling Unreal Engine 2.5 titles may one day get some extra life out of this like XMP has.

After Omen's original replacement XMP master server became too unstable to maintain I wrote this based on my original Ethereal logs of the XMP master server query exchanges. However I also went a little further and implemented as much of the protocol as I was subsequently able to sniff by experimenting with server settings and such. This code is all my own work and I'm releasing it under GPLv3 on the basis that it might be useful to someone, somewhere and possibly educational to others.

I can't say it's the most beautiful code out there, largely because big chunks of it evolved as I was in the process of reverse engineering the protocol. But it works and is pretty stable (it's been running as the current XMP master server for the last 4 years) so take from that what you will.

Code is available on my GitHub.

Here's to another 10 years of life support for our beloved XMP.
Peace, everybody.

(Incidentally, this doesn't mean I'm taking my current XMP servers down and is not an invitation for people to start up their own master servers for U2XMP. I hope that people appreciate that I've been maintaining the XMP master servers for 8 years now at my own expense and will be respectful enough to not fragment the tiny remaining coherence this game has left by running a "competing" server).

dutch_gecko · Mar 5, 2014

Thanks for this. I'll forward this on to some Tribes: Vengeance players as they've been without a master server for a while. From what I can tell they have some ad-hoc implementations but at the very least they can compare code with your version.

EQ² · Mar 5, 2014

Sure thing! That's exactly the kind of thing I had in mind as I imagine there are many UE2.5 games from the same era suffering a similar fate. If it helps anyone then making it open source was a good move as far as I'm concerned.

Also: hey Gecko

fireant · Mar 10, 2014

Hey, i come from above Tribes community mentioned above. First - thanks for this..even if it wont work for us it will be nice learning opportunity. Second - how do i forward the game to this new master? I remember that i saw some .ini configurations for some unreal games, but if i recall correctly they didnt do anything..our current "ad-hoc" solution relies on edited .dll files with removing master server address (editing .dlls with notepad kinda sucks..). So i wonder is there some ini configuration that i'v missed? And third - C# ftw, also im too lazy to go thru code but are you using some library for the cli ui? Like curses for windows or something, because it looks really pretty!

Edit: dont mind the last question, i saw the code..very impressive! How do you make your self comment everywhere

?

EQ² · Mar 10, 2014

fireant said:
Hey, i come from above Tribes community mentioned above. First - thanks for this..even if it wont work for us it will be nice learning opportunity.

That's what I thought, the code itself actually documents the protocol rather better than any document you can find, and I also included my "DevTools" project in the commit which can be used to experiment with the protocol.

fireant said:
Second - how do i forward the game to this new master? I remember that i saw some .ini configurations for some unreal games, but if i recall correctly they didnt do anything..our current "ad-hoc" solution relies on edited .dll files with removing master server address (editing .dlls with notepad kinda sucks..)

Well there are a number of solutions if you don't want to edit hard-coded addresses in a DLL. Firstly (and most simply), as long as it's a host name that's hard coded and not a raw IP then you can put an entry in the hosts file for the system with your new server IP in it, this is the simplest but not very user friendly. Secondly you could write a "wrapper" DLL which intercepts the winsock connects and forces connections to a new server, but this is pretty complex and you'd need a good solid understanding of windows, MFC and the codebase you're interacting with (I recommend IDA for reading DLL logic, it's awesome). Thirdly, you could write an intentional DNS hijack daemon, although this probably won't make you any friends and anti-virus will probably stomp on your face

But hey it's an option.

fireant said:
So i wonder is there some ini configuration that i'v missed?

Maybe, again IDA is a good tool for working out whether something is read from a config file at some point, just because it's not in the default ini's doesn't mean it's hard coded.

fireant said:
And third - C# ftw, also im too lazy to go thru code but are you using some library for the cli ui? Like curses for windows or something, because it looks really pretty!

I personally love C#, Omen's original master server replacement was written in VB and I didn't get the source code, so when I decided to write my own I naturally chose C#. I wrote the console stuff myself. You probably already noticed (if you read the code) that the interface is actually a pluggable module and there are actually several UI's bundled in the code, namely a native windows UI, a console UI and a telnet UI. I was going to write a web UI but I lost interest in the project.

fireant said:
Edit: dont mind the last question, i saw the code..very impressive! How do you make your self comment everywhere ?

Partly through 20+ years of coding and knowing what it's like to go back to uncommented code, and partly due to coding being my full-time occupation so it's just a habit. Glad it helps though.

fireant · Mar 11, 2014

So, I did setup my hosts file, managed to forward game server to the master and set breakpoint in UDPConnection.Receive method (after data is received), then the program flow goes to HearthbeatListener.ListenThreadProc where it checks packet validity and game and packet version (i changed Protocol.UDP_PROTOCOL_VERSION so the condition returns true).
Then the flow goes to UDPConnection.OnReceivedHeartbeat which triggers ReceivedHeartbeat event, correct? This triggers ServerList.ReceivedHeartbeat which, i think, is supposed trigger some hearthbeat event for concrete server? Well it does not, because whenever the program flow enters this method, "servers" list is empty.

I suppose that somewhere in between receiving udp packet and calling "concrete server event" went something wrong, and no concrete server was created. Any ideas what might be the problem?

EQ² · Mar 12, 2014

fireant said:
So, I did setup my hosts file, managed to forward game server to the master and set breakpoint in UDPConnection.Receive method (after data is received), then the program flow goes to HearthbeatListener.ListenThreadProc where it checks packet validity and game and packet version (i changed Protocol.UDP_PROTOCOL_VERSION so the condition returns true).
Then the flow goes to UDPConnection.OnReceivedHeartbeat which triggers ReceivedHeartbeat event, correct? This triggers ServerList.ReceivedHeartbeat which, i think, is supposed trigger some hearthbeat event for concrete server? Well it does not, because whenever the program flow enters this method, "servers" list is empty.

I suppose that somewhere in between receiving udp packet and calling "concrete server event" went something wrong, and no concrete server was created. Any ideas what might be the problem?

Wow, well I wrote the code 4 years ago so this is going to be a bit of a mind dump for me, I did start writing some documentation of the protocol at the time but I never finished it, you can read that here (PDF) but it describes the protocol rather than the master server behaviour.

In a nutshell, inbound connections spawn a Connection instance whose job it is to identify whether the remote host is a CLIENT or SERVER and that the remote host is a supported version, etc. This class then delegates to a ClientConnection or ServerConnection which handles the rest of the handshake and communications from that point onwards.

The concrete Server instance is created in the constructor of the ServerConnection, which is the thing which actually requests the heartbeat, so basically by the time the heartbeat is requested the Server instance should have already been created and added to the ServerList meaning it'll be there when the heartbeat is received.

From this we can deduce several possibilities:

The heartbeat is being sent unsolicited from the remote server, independently of being requested by the master server, this is a deviation from the original protocol since a "stock" 2225 or 2226 build only sends heartbeat packets when requested as part of the protocol handshake.
The heartbeat is sent concurrently with the uplink, meaning that the remote host isn't waiting for the heartbeats to be requested but instead just fires them at the master server regardless, this might be easy to deal with since you can just change the behaviour to store and forward the heartbeat packets so that there's a common pool which new connections can inspect to see if a heartbeat was already received.
The heartbeats are coming in as expected, as the response to the initial HELLO, but the ServerConnection is being closed because of a protocol error (meaning it's removed from the ServerList) before the UDP heartbeats actually arrive. This is pretty likely if the inbound data from the remote server are in an unsupported format or contain extra values that aren't part of the original protocol

These possibilities give you a pretty clear plan of attack: determine a clear order of operations, specifically: is the heartbeat arriving before HELLO (eg. unsolicited, scenario 1 or 2), you can determine this by setting breakpoints in the HandleHello method and in the heartbeat received handlers and seeing which gets hit first (oh, and now you'll find out about how complex the threading is in this server

) or alternatively is the HELLO being sent as expected but the connection closing before the heartbeats arrive (so set breakpoints in Abort or maybe just some Debug.WriteLine calls (or just check the logs, they're pretty verbose).

Now if you find that it's scenario 3, you're going to have to do some detective work, the DevTools package will help you here because it lets you experiment with taking a packet apart using a simple string syntax, so basically if you suspect that a packet is comprised of two bytes and a string, you could use the string "bbs" (one character per line) to deconstruct it in the PacketAnalyser and see whether you get sane data. If you don't then you can experiment until you get the right format.

To load the PacketAnalyser, make sure the DevTools.dll is in the master server directory and then issue the command

Code:

module add DevTools

and then stop the master server and start it again, you'll get a window come up which will show you all inbound packets and let you inspect them, here's an example with a simple query:

As you can see I've just made a simple deconstruction of the packet (the text following the pop type being the name to assign to that variable for your reference), the output updates in real time and the top window shows the "remainder" of the packet which is not assigned via the structure. Hopefully you should find it fairly easy to experiment and hopefully work out how to adapt to any differences in the packet structure coming from the server.

Hope this helps, good luck getting it working.

fireant · Mar 12, 2014

Well..UCPConnection.Receive is triggered twice, HandleHello is never triggered and HearthbeatListener.OnReceivedHearthbeat is triggered twice..

No logs are created (i mean, last logged item is "[NET] Port 27900 bound successfully") and Packet Analyzer does not seem to do anything (i indeed loaded it with module add DevTools and restarting) - it does not show any packets, connections or anything...

Also, there are only 13 bytes in the udp packet received - [9,0,0,0,0,116,114,105,98,101,115,118,0] (same bytes in both requests)..i am quite confused with your paper, so i ask, what does mean what? I suppose that first byte is protocol version (maybe first 4 bytes for int?) but the rest is a mistery for me..also 13 bytes is not enough to pass even cd-key.

EQ² · Mar 12, 2014

fireant said:
Well..UCPConnection.Receive is triggered twice, HandleHello is never triggered and HearthbeatListener.OnReceivedHearthbeat is triggered twice..

No logs are created (i mean, last logged item is "[NET] Port 27900 bound successfully") and Packet Analyzer does not seem to do anything (i indeed loaded it with module add DevTools and restarting) - it does not show any packets, connections or anything...

Which likely means that the TCP connection itself is never made and that heartbeats are somehow arriving unsolicited. You will need to find out what port the uplink is happening on and ensure that you're listening on the correct port. If you are listening on the right port then set a breakpoint in the constructor of Connection and trace the code through from there. If it's never fired then you're not getting any packets come in for some reason.

fireant · Mar 12, 2014

Oh wait, there is both udp and tcp happening

When i fire up wireshark and filter by dest ip of old master server, i see only udp communication going on..

EQ² · Mar 12, 2014

fireant said:
Oh wait, there is both udp and tcp happening
When i fire up wireshark and filter by dest ip of old master server, i see only udp communication going on..

Interesting, then it may be an entirely different protocol or it may be that they use UDP as the transport for some reason, care to send me the wireshark files?

fireant · Mar 12, 2014

I think that there is no need..I was testing with wireshark without host file redirection (so i could filter more easily), than i tried with Microsoft netmon (which can filter by process) and i tried with hosts file enabled. So i found tcp connection to port 29920, problem now is that i dont know how to configure master to listen on this port. I replaced 27900 in whole solution by 29920 but it still seems to be listening on 27900.

EQ² · Mar 13, 2014

fireant said:
I think that there is no need..I was testing with wireshark without host file redirection (so i could filter more easily), than i tried with Microsoft netmon (which can filter by process) and i tried with hosts file enabled. So i found tcp connection to port 29920, problem now is that i dont know how to configure master to listen on this port. I replaced 27900 in whole solution by 29920 but it still seems to be listening on 27900.

Issue the commands at the command line

Code:

port bind 29920
port unbind 29920

fireant · Mar 13, 2014

So, there is some connection opened, but it shows <NULL> in the packet analyzer. I also experimented little with host file and managed to redirect another set of udp packets, i think that those are actual hearthbeat becuse they happen periodically and are 487 bytes big, problem is that they always have different protocol version, sometimes even negative. Any ideas?

EQ² · Mar 14, 2014

fireant said:
So, there is some connection opened, but it shows <NULL> in the packet analyzer. I also experimented little with host file and managed to redirect another set of udp packets, i think that those are actual hearthbeat becuse they happen periodically and are 487 bytes big, problem is that they always have different protocol version, sometimes even negative. Any ideas?

Not a clue, it'll just need some very patient detective work. In all honesty the contents of heartbeat packets is not really that crucial, it's the source port that we actually care about. I would focus on getting the TCP part to work though, that's where the main handling occurs and where the bulk of the protocol takes place.

fireant · Mar 15, 2014

So i figured out that tribes protocol is so different from other games that it will be good idea to start from scratch, i made simple program http://pastebin.com/hrngUNBU that will listen for clients asking for server list.

Now i get client request which when ascii encoded contains

mapname\numplayers\maxplayers\hostname\hostport\gametype\gamever\password\gamename\gamemode\gamevariant\trackingstats\dedicated\minver

I suppose that tribes does not do any cd key checking or anything when it starts conversation with this, so i tried with respoding as

\mapname\abc\numplayers\7\maxplayers\32\hostname\87.98.194.26\hostport\7777\gametype\tribesv\gamever\63446\password\0\gamename\tribesv\gamemode\CTF\gamevariant\CTF\trackingstats\0\dedicated\1\minver\63446\

and just ascii encoding this and sending over to client (as shown in the program)

but nothing shows in the game, i also tried to look into unrealscript source if i can debug this somehow but most of the gamespy logic happens behind the scenes in some dll (functions are native) so i have no way of knowing how the game reacts internally. So i wonder, am i missing something in the response? Some initial bytes with msg length, or game name or something? Maybe some list formating (if there is any in gamespy protocol)?

Manticore · May 31, 2020

Bump.

Search

Search

Master Server Source Code

EQ²

Code Monkey

dutch_gecko

Think Pink

EQ²

Code Monkey

fireant

New Member

EQ²

Code Monkey

fireant

New Member

EQ²

Code Monkey

fireant

New Member

EQ²

Code Monkey

fireant

New Member

EQ²

Code Monkey

fireant

New Member

EQ²

Code Monkey

fireant

New Member

EQ²

Code Monkey

fireant

New Member

Manticore

Official BUF Angel of Death (also Birthdays)