Patching system.

[Ze_PilOt]

FAF is currently using bsdiff4 for patching.

It is really efficient at creating patch, but also very slow to process them, and more importantly memory hungry.
Ie. a Nomads patch can take up to 1.3go of ram and several minutes.
The resulting patch is 2mb.

I'm currently testing alternative.
Right now, XDelta seems quite nice.
Low memory footprint, (256 mb for the same patch), and blazing fast (2-3 seconds instead of minutes)
Of course, it produce bigger patches. 14mb instead of 2 mb (for a 147mb file).

Do you have any experience to share with other patching utility we can use?

[ALL_Sys_Nominal]

I donno anything about it but i shall contribute my utmost mental support

[Brute51]

I found this webpage which could be interesting. Also, a tool called "ebsdiff" says it can have a limit on the used memory so that could be interesting aswell.

An alternative to making the patches on the FAF server is to do it on the computer of the person who is updating a mod. It would become their problem and not the servers problem to generate the files. I realise there are several problems with this idea but I think making a seperate utility program to create "FAF patch files" could solve quite a few of those problems, especially if its mostly automated. I'm thinking about a program that would connect to FAF, download the latest 5 versions of the mod, create 5 patches and then uploads them to FAF. The tool needs access to the FAF database and ofcourse the new mod file (and time). For this to work FAF should support multiple updates (if you are 7 versions behind then you'll need to update twice).

[Ze_PilOt]

The problem is that you must produce patch from and to any version. The server does it on the fly, but if you have to create them locally when updating the mod, it would be.. too much.

[Brute51]

there is, i think, no real need to make patches for every version to every version (forward only). how often does it happen that someone has a version more than 2-3 updates old? and in such cases do you really need to update from very old to new with one patch? what i tried to say earlier is that the system would be easier if it could process more than 1 patch sequentially. then you can limit creating patches to the last 5 versions for example. fewer patches per update necessary (harddisk space), quicker update and perhaps less memory use aswell. also the user side creation of patch files becomes feasible, meaning potentially getting rid of the patching system problem and the exponential increasing number of patch files.

[Ze_PilOt]

I happens a lot when people are trying to see older replays
Space & Generation aren't really a problem anymore with xdelta, so why not keeping it?

The google thing seems nice, but it apply on executables, and we are mostly updating zip files.

[Brute51]

Do you also make patches to go from a new version to an older one? Cause.. wow! I seem to remember that FAF would just redownload the full old version.

Anyway, yeah if xdelta does a good enough job then there is no problem

[Ze_PilOt]

Do you also make patches to go from a new version to an older one? Cause.. wow! I seem to remember that FAF would just redownload the full old version.

The first time yes. If you do it twice (or another player downgrade too), he will get the patch.