Temporary disruption on services

Announces concerning FA Forever project.

Temporary disruption on services

Postby Brutus5000 » 20 Mar 2020, 15:35

Dear commanders,

tonight we found a security vulnerability in our API. After some evaluation today we decided that we immediately need to hotfix this issue regardless of the impact it will have on our other services.

As a result of that a lot of errors will occur around FAF will break, especially in the are of statistics., The website is the most prominent example right now :(

We are in contact with the developers of the library where the problem occurs and are searching for a solution. However the current situation might last for a week or longer.

We apologize for the inconvenience, but the protection of your personal data has the highest priority.
The 9th Doctor wrote:You think it'll last forever, the people and cars and concrete. But it won't. One day it's all gone, even the sky.
Brutus5000
Councillor - DevOps
 
Posts: 324
Joined: 05 Aug 2016, 23:32
Has liked: 29 times
Been liked: 271 times
FAF User Name: brutus5000

Re: Temporary disruption on services

Postby RedX » 20 Mar 2020, 18:26

Thanks for your hard work, and I completely understand the need to shut down while it's fixed. Can we expect a post-mortem at some point detailing what was potentially at risk and what the vulnerability was?
RedX
Avatar-of-War
 
Posts: 107
Joined: 09 Mar 2014, 20:20
Has liked: 6 times
Been liked: 10 times
FAF User Name: D3matt

Re: Temporary disruption on services

Postby PhilipJFry » 29 Mar 2020, 15:25

The issues are supposed to be resolved by now.

Going to unpin/lock this topic.
cats>dogs
post logs
User avatar
PhilipJFry
Supreme Commander
 
Posts: 2631
Joined: 23 Mar 2016, 21:16
Location: Austria
Has liked: 232 times
Been liked: 348 times
FAF User Name: PhilipJFry

Re: Temporary disruption on services

Postby Brutus5000 » 09 Apr 2020, 22:52

Insights on the incident

For a few years now FAF is running an open api, offering player and game data for free use to build cool features for which the core FAF developers don’t have time. As part of that it has always been possible to query the list of players and filter on them.

Example: the API call https://api.faforever.com/data/player?f ... tus5000%22 returns my user.

If you look closely you will see that there are only very basic fields to see (in the “attributes” block): create date (of the account), update date (which is the equivalent to “last seen”) and the username.

In the database this is querying data from this table here http://faforever.github.io/db/tables/login.html and as you can see there are other very sensitive information like your email or your password hash.

This fields were also available in the api, but only after login and only for a limited set of people. Basically moderators are allowed to see all the sensitive fields (for support cases with lost email, steam id etc. and for detecting of fake accounts, duplicate accounts etc.) and every player was able to see it’s own fields. A whole bunch of tests in our application ensured the correct hiding of these fields for unauthorized people.

And so we thought we were safe. Until silenceluke launched his new service https://fafscore.nl . For some users fafscore asks for their username. Other users were magically detected. So people asked us how that was possible. When we investigated the website, we noticed that fafscore was calling the player list as above, but instead of filtering on the user name it first tried to filter on the users ip-address and checked for a match with the last login ip address.

But the ip address was supposed to be hidden away from non-authorized access. Alarm bells started ringing. As it turns out it was possible to filter on every field even if it was hidden from viewing. Even with wildcards!! And that is a huge vector of attacks:

Now imagine you want to know my last ip address. You could just use a filter query that you know to return a result and then start appending wildcard queries. If there is a match, your partial guess is correct, if there is no match, your guess is wrong.
So you could take any player query and append &ipAddress==”1*”. If it gives a match, you check for the next character: &ipAddress==”11*”. If there is no match you test the next allowed value for the character: &ipAddress==”12*”, &ipAddress==”13*”, &ipAddress==”14*”.

You do this until you found the whole value. Using this attack it was possible to resolve an ip address with a maximum of 120 http calls. Or an email with roundabout 1000 calls.

When we are able to confirm the criticality of the isse, we contact the authors of the library responsible for this (some very nice people at Yahoo) who after some explanation gave us a workaround and fixed the issue within the next week.
The 9th Doctor wrote:You think it'll last forever, the people and cars and concrete. But it won't. One day it's all gone, even the sky.
Brutus5000
Councillor - DevOps
 
Posts: 324
Joined: 05 Aug 2016, 23:32
Has liked: 29 times
Been liked: 271 times
FAF User Name: brutus5000


Return to Announcements

Who is online

Users browsing this forum: No registered users and 1 guest