FA Networking - Centralized p2p.

[Ze_PilOt]

I won't explain how FA networking is working. It's peer to peer, nothing fancy.

But I will explain how Blizzard does it.
It's peer to peer with a centralized server.

Like FA, the sim is computed in each client. But instead of sending your commands to all others players, it send your command a single time to the server, and that server dispatch them to all other client.

Meaning that for a 4v4 game, in FA you need to send your commands to 7 players, in starcraft2 you only need to send it to one.
In both cases, you receive 7 commands.

Blizzard way of doing thing is solving the upload problem (download is rarely the limitation here).

[Ze_PilOt]

The proxy server

You should know that FAF also has a centralized server - The proxy server.

What it does is, when two players can't connect to each other, they go through that server, via FAF client.

But it still means that if you have to connect to 4 players through the proxy, you are still sending 4 commands.

That is kind of stupid, because these 4 commands are exactly the same, only the destination is different. But as they are sent to the same destination (the server), it's redundant.

[Ze_PilOt]

What can be done?

In a perfect case, we have access to the source and change the way FA works.
But it's not an option.

So in the best possible scenario, you send only one command to the proxy, and the proxy send it to all players.

How to do that?

Sadly not easily.
But not impossible either.

FAF client act as a relay. Each packet goes through it when you are using the Proxy. This is already done.

So, first thing would be to decode these packets, encapsulate them and send them to the proxy.

The encapsulation part is already done and working.

What need to be done is the decoding, so it avoid sending redundant commands to the server (only one is needed for each tick).

Then, on the proxy, instead of sending one packet to one client, need to re-encode the packets so it looks like the original ones (the one the client dismissed) and send it to all clients.

This is not impossible to do, but requires some reverse-engineering.

First, analysis the FA packet protocol, and detect what is making a packet tied to a client. That would be the very first step.
Wireshark and some networking knowledge are required.

I need your help with that. I know how to do it, I just don't have the time to do it.
So I need you to do it, not just tell how it must be achieve

Once it's done, we will advance to the next step.

[Ze_PilOt]

What we already know about the FA protocol?

Quite a lot actually.
Here is what Neruz did:

The header is 15 bytes, the structure is:

Code: Select all

(header)
byte (type of the packet)
int (mask, I've no clue what it's used for)
short (Serial, used for numbering the packets)
short (InReplyTo, used to indicate which packet the acknowledgement packet was sent for)
short (Sequence, I don't really know what is this, seems like it indicates connection status)
short (Expected, the sequence expected for ack? )
short (Length, length of the data in the packet)

Here's the python code I use to parse the header:

Code: Select all

# -*- coding: utf-8 -*-
# the code above is needed for some of the chars in the packets
import struct
import socket
from UserDict import UserDict

# Type flags
MP_CNT = 0 # Connect Type
MP_ANS = 1 # Answer Type
MP_DAT = 4 # Data Type
MP_ACK = 5 # Acknowledgement Type
MP_KPA = 6 # KeepAlive Type
MP_GBY = 7 # Goodbye Type
MP_NAT = 8 # NAT Type

_packetDefs = {}

_typeDefaults = {"q":0,"d":0,"x":"","B":0,"b": 0, "h": 0, "H": 0, "s": "", "f": 0.0, "i": 0, "I": 0, "c": " ","L":0, "l": 0}
_keyDefaults = {"Type":"B", "Mask":"I", "Serial":"H", "InReplyTo":"H", "Sequence":"H", "Expected":"H", "Length":"H"} # TODO: use this table instead defining each in the packet Definitions

_packetDefs[MP_CNT] = [["Type", "B", 0],
                        ["Mask", "I"],
                        ["Serial", "H"],
                        ["InReplyTo", "H"],
                        ["Sequence", "H"],
                        ["Expected","H"],
                        ["Length", "H", 45],
                        ["Protocol","I", 2],
                        ["Timeout", "I", 1],
                        ["Unknown1", "4s"], # maybe playername?
                        ["Compression", "B", 0]]

_packetDefs[MP_ANS] = [["Type", "B", 2],
                        ["Mask", "I"],
                        ["Serial", "H"],
                        ["InReplyTo", "H"],
                        ["Sequence", "H"],
                        ["Expected","H"],
                        ["Length", "H", 77],
                        ["Protocol","I", 2],
                        ["Unknown1","4s"], # changing ( '\x93Q\x01\x0b' ), needed in the reply packet, some kind of checksum?
                        ["Unknown2","3s"]] # constant ( '\xec\xb8-' ), game name, mapname, player name, version name?

_packetDefs[MP_DAT] = [["Type", "B", 2],
                        ["Mask", "I"],
                        ["Serial", "H"],
                        ["InReplyTo", "H"],
                        ["Sequence", "H"],
                        ["Expected","H"],
                        ["Length", "H", 14],
                        ["LobbyMessage", "B", 100],
                        ["EOF", "B", 14],
                        ["Pad1", "B", 0],
                        ["PlayerName","6s", "hello2"],
                        ["Pad2", "B", 0]]

_packetDefs[MP_JOIN] = [["Type", "B", 2],
                        ["Mask", "I"],
                        ["Serial", "H"],
                        ["InReplyTo", "H"],
                        ["Sequence", "H"],
                        ["Expected","H"],
                        ["Length", "H", 0]]

_packetDefs[MP_ACK] = [["Type", "B", 2],
                        ["Mask", "I"],
                        ["Serial", "H"],
                        ["InReplyTo", "H"],
                        ["Sequence", "H"],
                        ["Expected","H"],
                        ["Length", "H", 0]]

_packetDefs[MP_KPA] = [["Type", "B", 2],
                        ["Mask", "I"],
                        ["Serial", "H"],
                        ["InReplyTo", "H"],
                        ["Sequence", "H"],
                        ["Expected","H"],
                        ["Length", "H", 0]]

_PacketType = lambda data: struct.unpack("B", data[0:1])[0]

# The packet is built from the _packetDef[Type]
# the first val in the list is the key, second is the packing string, and the third is the value, if there isnt any
# it defaults the vals found in _typeDefaults
# passed **values overwrite anything

class MPacket(UserDict):
    def __init__(self, Type=None, data=None, **values):
        UserDict.__init__(self, values)

        if data:
            Type = _PacketType(data)

        self["Length"] = _packetDefs[Type][6][2]
        self["Type"] = Type

        self._packStr = "="
        for pair in _packetDefs[Type]:
            self._packStr = self._packStr + pair[1]
            if not self.has_key(pair[0]):
                try:
                    self[pair[0]] = pair[2]
                except:
                    self[pair[0]] = _typeDefaults[pair[1][-1:]]

        self["Padding"] = []
        for padN in range(struct.calcsize(self._packStr), _packetDefs[self["Type"]][6][2] + 15):
            self._packStr += "B"
            self["Padding"].append(0)

        if data:
            data = struct.unpack(self._packStr, data)
            for i in range(0, len(_packetDefs[Type])):
                tokens = _packetDefs[Type][i]
                if type(data[i]) == str:
                    self[tokens[0]] = data[i]
                else:
                    self[tokens[0]] = data[i]

    def Unpack(self, data):
        pass

    def Pack(self):
        values = []
        for packetDef in _packetDefs[self["Type"]]:
            values.append(self[packetDef[0]])
        values.extend(self["Padding"])
        return struct.pack(self._packStr, *values)


You can try parsing the header, it's 15 bytes so you'll want to cut it off from the data.
The server will expect a data packet that contains your name and such after connecting, and several other data I couldn't figure out so far.
You'll have to send back an MPacket(MP_ACK) after each packet received ( and you'll have to set the InReplyTo value too) or supcom will go crazy.

Also you'll want to run the following console commands, to make stuff easier.

Code: Select all

ConExecute('net_compressionmethod 0') # turns off deflate compression
ConExecute('net_debuglevel 3') # I don't really know whats the max debug level, 3 seems to be enough :p

[Ze_PilOt]

I don't think we actually to go further than the header if it's the only thing that vary in for a command packet.

From there, we can filter and rewrite stuff. But all the content of the packet itself must be similar (otherwise it means the whole packet need some rewrite), that need to be checked.

[Vmcsnekke]

upload bandwidth is no problem for most of us,
and sc-fa does not use that much bandwidth anyway,
I believe it uses approx. 1kbyte/sec per player.

so why on earth do we need this ?

[Kekouse]

Upload IS a huge Bottleneck for people with an average ISP connection. For many many years I couldn't play games above 6 players because of my Upload limitation.
And it's way more than 1kbyte/s per players

[eXcalibur]

i have a friend, who has this upload problem. he can play 1v1 (or 2v2 with 2 AIs) without lag. a third person i a game makes the game lag (after a couple of seconds the sim pauses for about half a second). this lag gets worse the more real people are connected to him...

when you want to do a centralized server, it would be great - under one condition: this behavior of faf, will NOT be default.
there are 2 reasons for this:
- the maintenance cost of the server would rise further. in a future (let's say: 10 years), when there is not enough money for the server anymore, because this centralized server and the proxy server function increased the maintenance costs (bandwidth, power, and upgrading) in the past. when no money gets donated anymore, the server should be able to function the longest time possible.
- in a future after the server had to be shut down, it will be impossible to use faf without the server, when all p2p connections require a server.

[Ze_PilOt]

We are not talking of changing FA here. It couldn't be done.

[Myromax]

If someone could get this to work, would every connection go through the proxy server?
I have an idea how to do this, but i have never used python before... can't even get the source to run...