As far as I know, this survey was the largest non-automated collection of data for FAF. We managed to get nearly 550 responses overall, which is drastically higher than the participation in just about any other attempt at FAF data collection. The 2nd highest I was able to find was the ancient 2016-2017 feedback page in which people were able to create issues they wanted FAF to focus on and vote up any issue they felt was pertinent. While there was no way of seeing the precise level of participation in that survey as people could vote multiple times, the most popular options (fixing connection problems, galactic war, team matchmaker) did not breach 250 votes. If you are a FAF player, chances are you view at least one of these issues as a good idea to address and would have voted for them to be addressed by the development team. Regardless, I think it’s pretty reasonable to say that this ladder survey vastly superseded this feedback form.

So why was this survey so successful? Well, it certainly had little to do with time. The feedback form itself is still able to be voted upon and has existed for several years. I closed the ladder form after 10 days. The main reason came down to the ways in which the survey was advertised and thanks to google recording the precise time the survey was filled out, I can generally gauge the impact of a variety of communication tools available to FAF.

General Timetable:
5/14 - 35 Results (Introduced, I personally advertised in the FAF Discord and made a QAI message)
5/15 - 44 Results
5/16 - 13 Results
5/17 - 3 Results
5/18 - 7 Results
5/19 - 105 Results (Put on MikZZ Russian FAF website, the Russian FAF Discord, and the vk.com website)
5/20 - 8 Results
5/21 - 6 Results
5/22 - 260 Results (Public Broadcast message to all players online on FAF)
5/23 - 44 Results (News Post on the Survey)
5/24 - 8 Results

The survey was heavily biased towards the upper level of FAF until 5/19. For the first day, 1800+ data was half of my results, by 5/18 it was still a quarter. By 5/19, I had a huge surge of data points in the 0-800 range. Likewise, 5/22 saw a massive spike in players at that level on FAF as well. What does this teach us? Well, basically that the FAF Discord is a terrible way to communicate with new players generally. Our old news site saw a nice level of participation from news players itself.

However, absolutely nothing compares to the public broadcast message I had Brutus use to put the survey in front of people. It’s even more impressive that it resulted in HALF of my results for the survey when it has a biased sample pool that has already had a lot of active FAF players removed. This is generally something to keep in mind. You would expect depreciating effectiveness from every tool as time went on due to the fact the pool of “willing participants” is decreasing.

Essentially, if we well and truly want feedback from the general mass of FAF on issues, we need to force the survey in front of their clients if we want to have a remotely unbiased sample. Merely showcasing it in aeolus, on the forums, and on the Discord gives a general result of the 1500+ opinion on issues but not the actual population. The News page alongside the variety of Russian news sources gives a more unbiased picture of the general FAF opinion, but it still fails to garner the depth of datapoints of the public broadcast.

The data is pretty self explanatory, there wasn’t really much utility in seeing the overall results aside from confirming the reality that a good general ladder system is impossible. I did decide to increase the interval of our current system to 1 month based on these results, though.



This one’s a bit more spicy. Essentially I curated the data and looked at the percentage of people in each specific rating category that voted in line with specific broad statements. For instance, if you recall, we had an option for many 5x5, some 10x10, and no 20x20 as well as an option for many 5x5, some 10x10, and some 20x20. Regardless of which option you chose, you generally could be considered a person that favors 5x5 and so you were part of the “most 5x5” trend.

If we ignore the noise of the curves themselves and look at the trend lines, we can see the sort of guidelines we should follow when structuring the pools in relation to one another. What do I mean by that? Well, I’ll explain it better in the conclusion, but generally speaking there needs to be an intuitive level of communication between the various pools. We cannot have a pool of say 12 maps at the <300 level that has mostly 10x10 and then the <500 segment has 9 maps with mostly 5x5 maps. It’s terrible for the system as a whole regardless of what player survey results say they prefer. Instead, we would structure the pools looking at these factors:
Rating increase correlates with an increase in desire for 10x10s
Rating increase correlates with a decrease in desire for 5x5s
Rating increase has a slight correlation with a desire to eliminate 5x5s, but it rarely crosses a boundary significant enough to warrant removing 5x5 in any bracket.
Rating decrease correlates with a desire to eliminate 20x20s, but it is hardly large enough to warrant removing 20x20s.

This might seem like a strange placement for a topic like this. However, it’s necessary to understand my decisions related to how to interact with other development projects in order to get a more focused look at the upcoming data from the survey.

Long story short, FAF is developing a division system similar to a variety of other games such as League of Legends or Starcraft in order to provide a sense of reward and progress for lower rated players while also eliminating a lot of the confusion of trueskill being a rank rather than a rating estimator.

So why is this related to ladder? Well, because of the way that the leagues will be divided! When the developers and I came up with a concept for the league divisions, we decided to utilize the ladder month system as a way to test out the feasibility of the rating categories we decided upon. We came up with the division of <300, 300-800, 800-1300, 1300-1800, and 1800+. This essentially made a funnel in which every group above was twice the size as the group below it. In order to split the massive <300 group, we decided to create an additional division that would include players that fell below -200 shown rating. This also creates league sizes in which, assuming you were the precise median, you would almost guarantee to only play players within your league on the ladder.

What this means is that, for the sake of intuition and clarity for players that would be utilizing this new division system and the new tmm, we need the player brackets between both systems to be coherent with one another. We can’t have a 600-1000 map pool that goes into two different leagues, it would be immensely confusing for the players that consider themselves part of a uniform “league” organization. So instead, I decided to make the rating divisions for the ladder pools operate under the same rating ranges as what we will use for the league system itself. I will however end up grouping the two bottom categories into the same pool, so we will keep <300 as a singular entity in the map pool division.

Key information for looking at the Data:
NewInterval = 0 rated data
ThreeInterval = <300 rated data
.
.
.
EighteenInterval = <1800 rated data
StrongInterval = 1800+ rated data

The left side column is the quantity of weeks for a pool, so 4 weeks is a month and so on.









This data is the simplest of all potential elements of the survey. Generally, the vast majority were satisfied with a pool between 3 weeks and a month. I decided to ignore any subtle variations in the length, once again, for the sake of making everything intuitive for both the players and the people that will be managing these systems (read: me). If you think about how interval and pool size relate, they kind of attain the same goal. A small pool rotated often is essentially trying to assuage the pains of monotony that come with a small pool and progress it into the benefits of a larger pool. A larger pool rotated very rarely, or never, is likewise attempting to remove the additional variability that comes through the inclusion of new maps and progress it towards the benefits of a smaller pool.

In addition to those factors, I also felt that the difference between 3 weeks and a month are absolutely marginal in the broad scope of things. Having to recall that I need to rotate pools B and D every 3rd week and A, C, E at the end of the month is just a recipe asking for failure. Likewise, it will be chaos in making sure that all the map divisions communicate with one another. What happens if you have the <300 map pool rotate on the 3rd week and now have entirely different maps from the pool above it? Now the people at the cusp between both rating divisions are going to have a horrible time transitioning between the two pools and you likely discourage their participation in the ladder until their league has caught up with the others.

For these reasons, I decided to make a uniform one month interval across all pools.

For this data, Bracket1 is <300, Bracket2 is 300-800, etc.
The map sizes are on the far left column.







Alright, this is very easily the most complicated part of the survey. It’s also the one most open to disagreements with my decision as, well, going off the data it’s a stupid decision and looks like “classic FtX thinking he knows what’s better for people than they know for themselves why even make a survey blah blah.” What do I mean? Well, let me present my solution first and then work through the process of getting to that solution.

I decided to utilize an incremental pool system. This means that:
<300 will have 7 maps.
300-800 will have 9 maps.
800-1300 will have 12 maps.
1300-1800 will have 15 maps.
1800+ will have 20 maps.

If you decided to just go by mode:
<300 will have 15 maps.
300-800 will have 15 maps.
800-1300 will have 15 maps.
1300-1800 will have 20 maps.
1800+ will have 20 maps.

If you decided to just go by median:
<300 will have 15 maps.
300-800 will have 12 maps.
800-1300 will have 15 maps.
1300-1800 will have 12-15 maps.
1800+ will have 12 maps.

My logic for ignoring the “obvious” conclusion mainly stems from qualitative analysis and a look at the trends of the specific brackets compared to general trends.

The first thing I noticed was Bracket2 and their significantly larger minority in the 9 or smaller map category. In particular, 9 maps was almost large enough to be the most popular option within the group. I decided that the large difference between this group’s distribution and the general distribution of pool size preferences was justification to have this group utilize 9 maps. This also meant that the <300 bracket would have to utilize some level of maps at or below 9 maps as it would be incredibly strange to go from a map pool with more variation to one with less for new players. I ultimately settled on 7 due to the fact that a very large segment of the <300 population desired 5 maps relative to the norm, but it wasn’t large enough to make me feel that it was a good idea to go for such an extreme option. 7 maps to me was a less extreme option while shaping the pool around the higher distribution of 5 map preference at that level.

In addition, beyond the data, I felt that smaller pools would serve as a much better introduction to FAF. As the logic of this rating division for ladder will be extended into the matchmaker, it’s imperative that it be made as conducive to a good gameplay experience early on as possible. I felt that the data we had for the lower rated pools was, well, biased towards people that would like the general structure of the current system. I wanted the lower pools to consist of the classic maps in a rotation which are often played so that new players would have clear resources in the form of both guides and replays to improve and they wouldn’t be overwhelmed with the quantity of strategies needed to succeed.

Now for Bracket 3, 4, 5 I followed a similar train of logic. I noticed Bracket 3 seemed particularly favored towards 15 maps in comparison to the rest of the brackets. However, I also felt that a 15 map jump from a 9 map jump was much too extreme. I instead opted for a 12 and 15 map sequence for Bracket 3 and 4 as both seemed to generally converge to those map sizes, which is pretty obvious as all the brackets do. Now for Bracket5 I went for 20 maps because of the huge desire for 20 maps in the bracket below it. As I said before, suddenly seeing the map pool size go down is quite odd for players so even though Bracket5 had a bias towards 9-12 maps, I cannot really ignore the desires of the group right below it. Both Bracket4 and Bracket5 had 20 maps as their most favored option, and so I decided to make it the pool for the top bracket.

While the data does not fit the conclusion entirely, I used it to shape the decisions within the rating division model. I do believe another entirely viable map split is 9/12/15/(15)20/20 or anything similar to that, but I’ll gauge the need for those slight adjustments later on and I felt this initial breakdown would be the simplest one for everyone to get used to how a rating division pool should ideally work.

Recall NewPref = 0 rated, threePref = <300, etc.
On the left side column, you have the map preference options. They are numbered.
1 = m/s/n
2 = m/s/s
3 = n/m/s
4 = n/s/m
5 = s/m/n
6 = s/m/s

The first letter represents the answer for 5x5, the second is 10x10, and finally 20x20. “S” means some, “M” means mostly, and “N” means none.







This one is pretty basic, everyone wants S/M/S except with the “some” meaning different things based on the trends within specific brackets. Long story short, look at the graph in section II to see the trends that I based my decisions on.

Results:
<300 3:4:0
300-800 4:4:1
800-1300 4:6:2
1300-1800 4:8:3
1800+ 4:10:6

While the <300 participants did want to see 20x20 maps, I couldn’t exactly include them in a ratio that wouldn’t absolutely ruin a 7 map pool. This is part of the reason why a 9 map pool for <300 might be a viable alternative. If you decide to go, for example, 3:3:1 for the <300 pool, they would be getting 20x20 maps more often than the 300-800 pool on average, which is just extremely counterintuitive. I was convinced to just drop the <300 maps regardless of what the <300 participants desired after being told to watch some <300 games on 20x20 maps and seeing what kind of gameplay it results in. These players do not need to be introduced to the depth of responsibilities that 20x20s require you to manage, even the simplest ones are extremely tasking.

The rest is pretty obvious, more 20x20 as rating increases, not increasing 5x5s as the map pool is increasing which in turn lowers the proportion of 5x5 games without removing them. An increase in 10x10s since as rating increases you want more 10x10s.

This breakdown also makes the interaction between various pools extremely intuitive. You will have the pools operate in a nesting model with everything essentially being a cut of the 20 map pool of the 1800+ segment. That means if you are 500 and play a 200, you will still be playing the maps you were assigned, it will just not have a 20x20 nor one of your 5x5s as one of the potential maps. This should mean that everyone is generally operating on a similar knowledge basis and there is no extreme discontinuity interfering with the integrity of trueskill nor frustrating players.

This section is more informal and me just addressing some of the pertinent survey comments.
C: Map voting before the game starts!

A: I’m not a fan of giving players massive ability to influence their maps due to the bias it induces in trueskill. I’d only be willing to accept such a system if it increases the odds of a map showing up while still considering other maps. This requires a lot of coding work both server-side and client-side and is not easy.


C: One thing you didn't mention in your post that you should consider is 'build order whoring,' which is certain players knowing or having more efficient build orders on a map than another player. This occurs when a player is very familiar with some or all of the maps in a rotation. As you can imagine, having very few maps means this happens more frequently.

One of the worst offenders from the past (it has since been fixed) was Eye of the Storm, where a single tiny post near the spawn contained 300 mass. This allowed players that knew about it to gain almost an entire extra factory before their opponent in the early game. While all the worthwhile players knew about it, unfortunately many of the newer ones didn't. Keep in mind, this was a popular map and an extreme example, so imagine how annoying this can be across all maps.

One of the most fun ladder matchmakers we had before was a system that had a huge pool (40+ maps) and selection process to ban maps you didn't want to play, and favor maps you did like.

A: The old map pool had huge problems that made it not work out well at all. You would often play the same person on the same map due to how it favored maps you both wanted to play, making players sick of these maps. It also was extremely discouraging as a new player who had no idea what maps were good. You would be dumped on a screen of tons of maps you had zero idea about. It also had a really bad solution to situations where no maps between two players were favored (picking popular maps which inevitably were maps <800 players enjoyed which higher rated players hated). I do have a concept of a “better” matchmaker system however we have no dev power to implement it currently.

Your other comment, I feel is one that kind of misconstrues things on FAF and it’s one I see a lot. When people refer to “build orders” in the way you do, particularly when the example you cited, what they actually mean is “map with a trick.” If someone plays EotS and wants to improve, they will quickly notice an opponent sucking up this strange mass that massively boosts their total mass. Likewise, on Syrtis you can quickly see the benefit of edge building a factory. On Loki you reclaim the tree groups near your base to cut out 2 pgens from your build. These are tricks that are map specific but are not very difficult to grasp after a loss or two.

Given you as a player understand the fundamentals of a successful build order and can go 5 minutes without stalling, you can quickly see and utilize these tricks as ways to enhance the fundamental build orders. You should then be on the same page as other players. If you are not able to do so, then you likely have a failure that stems with the fundamentals rather than the map specific aspect itself.


C: I think having a few core maps (5 or so of about 20) that are always in rotation (theta passage for example) would allow for a set of maps that all ranges are familiarized with but allows for diversity and requires the player to diversify play styles and skill.

This would seemingly strike a happy medium between both sides of the argument and allow for easier learning from new people since they can familiarize themselves with those basic maps while allowing more advanced players a larger pool of maps to play on.

Additionally if need be limit the rotation of maps to 10 for players below 800 ladder and have all 20 maps for 800+.

A: There are no core maps people can agree are universally good. Making ¼ of the pool be the same maps, even the most loved maps, is going to result in a lot more hate towards those maps, which demands their removal, which defeats the purpose of the suggestion. I already get hate towards badlands and theta being in every 2nd and 3rd pool. I also get hate from people that say a map has been in tons of pools when really it’s been in 2 of 32 pools, but that’s negativity bias for you.

I also feel that this doesn't help learn the core maps much. I certainly think it would help the dudes playing ladder for 50 games a month, however, the average person is playing 3-5 hours of ladder a month. This means you will maybe get those core maps a few times a year and you will likely forget anything you were supposed to learn from your preceding games on the map by the next time you play it.


C: It’s important to create a consistency for newer players and pro players in order to create better gameplay . If the ladder pool is changed to often that hurts the game.

A: Totally agree and it’s why I decided on a uniform pool length.


C: Would there be away to place a limit on rank before seeing 20X20 maps? I'm close to 900 and almost everyone I play agaisnt (me included) just lacks the skill, apm and frankly attention to handle a 20X20 map but watching higher rated players on these maps is very enjoyable and clearly they have the skill to do this. Therefore could the ladder not be programmed to ensure anyone under say 1200 does not get placed on a 20X20 map? Idealy if this were implemented there should be a user overide option just for those rare players amoung us!
One final point, I see continued asertions in AEOLUS that people have cheated yet watching the reply and comparing to banned exploits it is clear no cheating happeed. The most recent was someone simply macroing a tec 3 bomber to a stop which is clearly not on the banned list. Why not included a link on the main page to the rules or have a seperate page such as that seen by the uniot list which explains the rules?

A: And that’s what we did! Basically. On the rules page, I think it’s a good idea. I think my analysis of the survey logistics shows that people, especially new people, hardly look at aeolus. We can see what we can do with the upcoming team matchmaker UI and if it could be included.


C: Announce pools in advance, consider running blitz tournaments on new maps or something to ensure they are playtested properly for 1v1. Lots of mapper egos getting priority over "is this map good gameplay" in pool decisions.

A: Unfortunately (for you?) this runs counter to my philosophy on priorities in FAF. I see ladder (and matchmaker) as the way to improve yourself. I see tournaments as the true area of competition for those that have improved themselves. I use new maps in the ladder to assist both arenas. In ladder, it helps improve players by forcing them to adjust to new scenarios and develop a game sense that isn’t totally reliant on meta play. As you progress more and more into the game, this skill becomes more and more important in separating the players and it’s why as you increase in rating you would begin to see more and more new or uncommon maps rather than the typical classic ones.


C: allow players to choose 50% of the maps of a list that shows most played maps at their ratting range.

A: Not a fan of choice systems in general. I would much rather have a veto system based on a predefined pool and then the ability to “buff” a map so that it has a higher likelihood of appearing assuming the player you matched up against didn’t veto it.


C: It should be more obvious when the pool is going to change. For example from-to dates in the 1vs1 overview would be nice.
And maybe a link to a short explanation of general gameplay of the maps (obviously not for new maps). A small guide with screenshots that tells a noob about the map, to make it easier to get into laddering.
And more specific for the pool: a short explanation of the current composition of the pool. Like: 8x 10km maps, 3x 20km, 3 navy maps, 1 new map.

A: Good suggestions all around. Will try to see how I can include all of this in the upcoming matchmaker UI. This was actually the comment that particularly pushed me into going for 7 maps for the <300 pool as I talked it over with the ladder team and we plan on including general tutorials for the maps that would be going into the <300 pool so that people could have some knowledge of what to do.
For the short explanation, I don’t think that will lead to as much information as you think it will. Every pool consists of the exact same ratio of map sizes and has nearly the same ratio of new/experimental/common/classic maps. The only information you would gain is the sort of gameplay bias of the pools, but I don’t think that provides much more input that just looking at the maps will. Luckily, the new matchmaker UI will show you the map previews while you’re on the tab waiting for a game.


C: Would also like to see publicly how the maps are selected (e.g who specifically asked for a certain map to be included, and some reasoning behind it. A few sentences from each member of the ladder council would do).

A: The maps are selected utilizing an aggregated poll result from the ladder team, which consists of several veteran mappers and 2k+ ladder players. The maps are rated on gameplay/variance/aesthetics in that order of importance. A map must hit 2.5/5 to meet the minimum requirement to get into ladder. It will then be put into the “new” map category until it gets played in 1-2 pools and then transitioned into either the experimental/common/classic category depending on ladder team decision.

Everyone had similar preferences for the length of the pool regardless of rating. Therefore, it will stick at a uniform 1 month interval.

For preferences, everyone wanted a pool that had mostly 10x10s with some 5x5s and some 20x20s, however, the definition of “some” changes depending on your rating range.

For size, I used the data to assist me in constructing a division that would hopefully be intuitive for all while being the most conducive to improving the general FAF player’s capability. While it doesn’t follow the survey itself, I hope people can see why I used the results as input rather than dictation.

Brackets:
<300
7 Maps (3:4:0 breakdown)

300-800
9 Maps (4:4:1 breakdown)

800-1300
12 Maps (4:6:2 breakdown)

1300-1800
15 Maps (4:8:3 breakdown)

1800+
20 Maps (4:10:6 breakdown)