The Problem With "Multiple Choice Polling"

[Evan_]

I guess this is a good place to ask the question: What happens if there's a huge conflict between ladder team and playerbase opinion? Like say the ladder team think a map is really good but the majority of voters hate it? does it kinda cancel out and the map shows up an average amount of times?

Also, what happens if a new version of a bad map is uploaded, but before the new version, it was panned so heavily it's set to almost never appears? Is the map given a second chance in another vote or pool?

[Medicraze]

Okay, as a program planner with experience in this domain, I'd like to address a couple things.

A small list of player barriers related to surveys:
1: complexity: This being the balance between holistic data collection with survey engagement. So while finding a good balance provides data, it may be incomplete. This is something we will just have to deal with, as a complete survey is more complicated, which reduces sample size.
2: Accessibility: No one wants to make an account for a survey, but most have a gmail. Using google forms for data sets may be a good idea, although, I'm sure this has been considered before.
3: Perception of Influence: Players need to know they are being listened to. One of the most effective is videos (people are lazy and like passive information). Integration of a video acknowledging feedback in the client would be helpful in persuasion. It would also provide for increased perception of "our opinion maters".

One work around is intermittent topic based surveys: for example -> Survey player opinion on simple topics such as maps with inclusion of qualitative data. Simple questions such as "rate these 5 maps from least favorite to most" With a picture of the map (no one wants to open the map vault and remind themselves of names, easy = more involvement). Bite sized weekly surveys over time will increase involvement. You can have a pop-up on client launch that says "help us improve the game! Spend 2 min answering these questions. We listen. Tell us how we can help! This week's topic, matchmaking.". Something like that. Consider the 1 vote per user...idk how you'd implement...I'm gonna stop here, this is likely falling on deaf ears.

I do want to note: Although you may reserve your opinion; Overwatch is balanced based on top tier player performance. I think balancing based on this principle is a good idea due to the condition of mastery. Although lower rated players who haven't completely refined their skills may find changes to certain units "OP/nerfed" secondary to their perception/knowledge of viable strategies. In time, they would come to find effective counters, and this perception can be minimized by acknowledgement of the changes, through (you guessed it) and informational video that is short, and to the point. The simple fact is, a unit on paper is detached from the user's skill handling them.... I just realized I'm ranting about balance and not about surveys. I'm still going to post this because now I'm laughing lol


Anyway, if you want an actual breakdown of how to do a proper survey, let me know. I am trained in it. I do have a degree in public health... that's all we did. Bare in mind, my availability is spotty, but I'm here to help.

[biass]

I guess this is a good place to ask the question: What happens if there's a huge conflict between ladder team and playerbase opinion? Like say the ladder team think a map is really good but the majority of voters hate it? does it kinda cancel out and the map shows up an average amount of times?

Vote tally doesn't directly influence the score edit: "yet" as far as we are concerned, a conflict will however force a reevaluation of said score amongst the team (who are just as happy to argue about maps as anyone else) we haven't had a major issue yet, but i would assume team members would attempt to find out why by asking around if there was one and we didn't understand the problem.

Also, what happens if a new version of a bad map is uploaded, but before the new version, it was panned so heavily it's set to almost never appears? Is the map given a second chance in another vote or pool?

Never thought about that, and that should be "my area" Yikes.
I will know if a new version is uploaded, however if a mapper has decided there is a major change they should let us know directly through any channel. Map scores can be reevaluated at any time and are constantly adjusted. If I were to adjust a score mere seconds before someone generated a pool, the changes would apply.

To be fair, I only know of two maps rated too poorly to be considered. Let's just say that we try to avoid cluttering the spreadsheet if the time investment was less on the map then it was on said sheet. You get me?

overwatch

Not only is only balancing around the pro scene just not true, why is this even here?

Are you trying to confer a point about considering low rated opinions on maps? We do already attempt that. having FtX on the team makes it easy kappa

[Morax]

It's not about whether only two maps are too poorly rated to be considered, but whether they get reviewed after fixes.

Those of us that like to improve require feedback, so if there's a "one chance" policy on map discussion, it's quite frustrating for us folk.

To Medicraze point, I think you guys definitely should improve the polling in FaF. It comes off as quite low-effort, but yields big results.

I'm glad Evan and he agree there are flaws here.

[Medicraze]

@Biass regarding the overwatch comment: This information is based on A_Seagull (former E-sports pro player for Overwatch). Obviously, take this claim with a grain of salt, but he states this information came from blizzard employees. He stated the balance was determined based on corresponding pro player affinity for a character's win percentage. Reportedly, the team would review what made some tactics so successful, and some of the abilities were adjusted based on what they gathered. I'm too lazy to look through his hundreds of videos, but I do remember this being said because I found it an interesting approach.

@morax Yes, there are flaws, but I thing it's mostly in the execution. Bare in mind, there are many barriers needing to be addressed, because we are dealing with a small sample size, and of that size, what motivates a player to respond, will that skew the data compared to those who dont, etc. There's many variables. That's why I would do that "youtube Polling" bullcrap ad that pops up when you launch a video. It's mildly inconvenient, but doesn't really keep you from doing what you want. But, being a game people want to play and improve on, I'm sure this would really help.

[Morax]

Medicraze, it's not just the execution, but there are errors in the polling itself. Their "polls and statistics" are questionable and this only gives me further reason to believe they're not very helpful:

[Turinturambar]

that quote is out of context. the actual results are (afaik) used correctly. the thing that gorton was refering to there was not the actual rating which is used to pick the pool.

[Morax]

that quote is out of context. the actual results are (afaik) used correctly. the thing that gorton was refering to there was not the actual rating which is used to pick the pool.

As far as you know, but it doesn't seem like it would be far from the truth since the poll was never advertised.

How can you explain that you guys are okay to use the poll info when not even that many people are made aware? It seems 1 poll was successful in turnout, then the rest are just "whatever." Can you give some more insight as to what critical number of votes will be needed to allow it to have an effect? E.G. if one poll has 300+ responses verse one with 40? Why should that count as much? It seems like a misuse of information.

[biass]

Why does literally every discord snip on this god forsaken server intentionally cut out the context that is mere PIXELS about the snip?



The comment is about the poll, ftx mentions that the other poll was correct, he also mentions that column that had an error was related to people voting.there is no plausible way you somehow correlate that comment with being the actual map score unless you were intentionally misleading people into having lower faith in the ladder teams level of competency.

I have one answer to the poll turn out.
P R O M O T I O N C O U N C I L O R

[Morax]

"We can't ask a poll to show up on the 'What's New' page of FaF unless there is a promotions councilor"

- The FaF ladder team