By PJlucyloo Jul 8, 2013
I'm not sure where to write next. I have contacted Blizzard a couple of times and nothing is being done. I understand they might be busy to deal with these things but I am at a loss. I play on the server Kil'jaeden where there has been a huge amount of cheating and exploitation recently by many people and some guilds. One of those guilds is purporting they are #1 according to Guildox on server and the link to Guildox is how they are going about recruiting for their guild. Problem is that they got #1 through cheating/exploiting and anybody who plays this game at all pvp or pve can tell by looking at the stats reported. I'm not sure if anything can be done about this but we are having a terrible time recruiting people to server because of the activities of cheaters/exploiters who are so obvious to all it is hard to ignore. I know you get your stats from Blizzard but they seem to be reluctant or do not want to deal with what is going on not sure which is the case and they make reporting these people so difficult. I hate that your site is being used to boast cheating/exploiting toons and guilds. If you look at the Guildox stats below the guild you see as #1 on server has 8 toons on the other link (kj ladders) who used cheating/exploitation to get their rating. Please just look at the links below that is all I ask. Polar said this was a better forum to address this in than an email.

Kil'jaeden RBG ratings ladder
http://us.battle.net/wow/en/pvp/battlegrounds/?spec=&characterName=&battleGroup=&realm=9&faction=&minRating=&maxRating=&specType=all

Guildox RBG rating stats for guild and server
http://www.guildox.com/go/g.aspx?a=14&r=Kil%27jaeden-US

Thank you,
PJ
By Polar (site admin) Jul 8, 2013
Thanks for posting this up here - we get a few people write to us about cheating players.

Let me start by checking - this is just for rated battlegrounds, correct?

My main question is - how can we detect if a player is cheating/exploiting? If you can let me know then we can look to see if we can put some filters in place.
By PJlucyloo Jul 8, 2013
Yes this is for RBG's. A few things are glaring that I think can be filtered or maybe just flagged until checked out.

First thing and most obvious is number of games played and rating. If you look at KJ ladders for example look at Conradical and Rooftrollen. I am not in their guild and have no alts in their guild. They are known in game to be legit players who have had great success and they are on the KJ ladder I linked above. Look at their amount of games played close to 700 each to get to their rating of close to 2500 which is pretty much max legit rating on KJ. Now look at just a few of the top players on the ladder I linked and some have as few as 22 games played to 50 and have ratings of 3k and above.

Another thing you can look at is the achievements related to them. Most all of them were gotten in one or two days. A couple went from 1100 to 3300 in a matter of hours on the same day or over a couple day period.

Finally, the way the mmr system works is a player gets to a point where because of the teams he is que'ing into they are only getting a couple points per game up or down so you can imagine how many games you would have to play in order to go from 1200 to 3300 and how long that should take and what it would look like. The majority of toons on the first page of KJ ladders have fewer than 50 to 100 games played and ratings exceeding 2500 over a couple of days of play. It is just not possible to do that legitimately.

What this site is being used for is to bolster some of the guilds who have the most wintrader/cheaters in game in order to build recruitment. One such guild the #1 according to Guildox site on KJ has skyrocketed in players with their recruiting posts which use this site as proof of their high rating. They knocked down a legitimate guild out of first with the bogus stats.

For filters maybe some sort of a flagging system I'm not a pro at computering so cannot say what it would look like. But I would say if somebody achieved a massive rating increase in brief period of time filter should catch that and they should be flagged to check out. Then further if they not only had a huge increase but had it within a couple of days time another red flag and their achievements tell you that. There is also of course the games played compared to rating compared maybe to average games played and rating of players on server. If somebody only has 22 games played and has over 3k rating it is ridiculous. That of course would take some fine tuning to strike balance but some things are just impossible to achieve.

There are more things like the mmr system workings etc., and how it does not work to achieve this sort of rating legitimately but my ability to explain it is probably not the strongest.

Thank you for responding,
PJ
By rawrjay Jul 8, 2013
Another way to find out if someone got boosted or carried or whatever you guys call it these days is by looking under statistics for PvP Battlegrounds of their armory.

By looking at their Win/Loss ratio you can see they're almost always exactly the same, these though might only be alts, I'm sure their mains would have higher loss after getting tired of losing and buying their rating.

Take this guy as example:
http://us.battle.net/wow/en/character/kiljaeden/Johnhandlock/statistic#21:153

You'll notice under the Rated battlegrounds played section the number played is 58 and of those RBGs played, he won 57.. That is so obvious that he cheated the system, no one is that lucky or good.

You'll notice that its the same way for all of his toons.
By chilblane Jul 8, 2013
I think that being able to report players directly might be useful. Once a player has been flagged enough they can be looked at and evaluated for cheating.

I expect that this system might be abused, so there could be a high threshold. However, asking people to flag someone on GuildOx is significantly easier than the hurdles Blizzard puts you through. Plus it can work alongside the existing star rating system.

The point is that this wouldn't even be a GuildOx concern if it weren't for the fact that guilds are flaunting their illegitimate GuildOx ratings on server, and they're doing it by gaming GuildOx. No other service I know of organizes rbg ratings by guild, so fixing this issue, if Blizzard can't step up, relies on you.
By PJlucyloo Jul 8, 2013
Yes Chilblane explained that quite well. Another thing that comes up. In order for these players to have achieved these ratings they would have had to que into 3k + teams and over and over and the mmr system is not set up like that. Most of these guys were in the 1100 area how are they que'ing into 3k + teams? I am not saying it never happens but what are the odds that in the example above from Chilblane that player would que into 3300+ rated team 57x?


If you look at the #1 player accordiing to Guildox site. He went from under 1100 to 2200 all on June 6th. He then went from 2200 to 2400 all on June 12th. He didn't even get his "In Service of the Horde" till June 17th 5 days later which is an achievement for 25 RBG's won. He now rests at 3471 rating.

The player in 2nd on Guildox went from 1100 to 2400 all on same day of May 20th. He then got his "In Service of the Horde" on May 25th for 25 RBG's won he now rests at 3378 rating.

This same pattern continues almost 2/3 of the way down the first page. The guild in 1st for server has 8 listed in the ladders with the same types of bogus stats above.
By Polar (site admin) Jul 8, 2013
Keep the ideas coming but let me throw a couple of constraints in:

1. The filtering system needs to be fully automated - as a free service, we don't have bandwidth to be doing manual checks on flagged characters

2. The filtering system needs to be as accurate as possible - we can't be filtering out legitimate players

By Nayadri Jul 8, 2013
On Bg9 atm it is literally impossible to be above 2500 in Rbgs atm.

You look at games played and wins / losses, as well as dates. Take any well-known hardcore RBG player and compare them to someone on Kil'jaeden at 3300 rating right now. Papaxsmash for example has over 600 RBGs played, multiple HotH and he's got a higher win ratio. His dates aren't suspicious and he's only been to what, 2600 from Cata?

Honestly, anyone above 2500 alone is a huge red flag for wintrading. It's also suspicious to be so high in rbgs and yet 0 rating in arenas. Also, anyone in the 2.2k-2.4k bracket with less than 100 games played / close dates is a HUGE red flag.

I think a way to fix this whole ranking thing is to count every player who has atleast 100/200ish Rbgs played right off that bat to start. I'm not sure how exactly you could screen dates of achievements, but anyone who gets 0-2400 in a day should also never be accounted for.
By PJlucyloo Jul 8, 2013
Well in the two examples I posted one got all his achieves on the same date from grunt to high warlord on same day. Is there a way that these achievements can be tracked and if all in short period of time be flagged? Even better some of these guys went from grunt to high warlord and still did not have their "In service of the Horde" achieve which tracks 25 rbg wins.

If a guild jumps from 12th place on server to 1st in a short period of time can't that be caught and then investigated?
By PJlucyloo Jul 9, 2013
Maybe just do not count people over 2600 rated with less than 100 games played. That will not catch everybody but it will be a serious dent in the activities and those guilds and toons who legitimately get over that rating you know earned it.
By PJlucyloo Jul 9, 2013
Well maybe you wouldn't "know" the players over the 2600 with over 100 games played earned their rating for sure but it would be better chance than not but those who want to work their way around that will probably find a way in time. Cheaters will constantly try to get one step ahead of you and you will be constantly playing catchup. But at this time having some boundaries/filters would help for time being and bring more legitimacy to this site which gets used a lot.
By Polar (site admin) Jul 9, 2013
A (2600 rating and 100 games) filter would be fairly easy to implement.

I welcome more comments/suggestions to see if there are better ideas out there - please encourage others to share their opinions in this thread.
By Keylogthis Jul 9, 2013
What about filtering out the players that went from 1100-2400 in one day? Or even, a player that went from 1800-2400 in one day.

Is there a way to implement an automated system that checks achievement dates?
By sicarius Jul 9, 2013
Instead of 100 games how about 150 to help weed it out more?
By PJlucyloo Jul 9, 2013
Some pretty good suggestions. I think keeping interference or filtering to a minimum in beginning would lower the "hassle" factor. Like I said earlier though another way around it will be found until Blizzard can get a handle on it. I don't expect that filters implemented will get rid of them completely just let the one's who are legitimate shine thru for now. Probably have to be addressed again in the future once a way around it is figured out by the cheats.
By Nayadri Jul 9, 2013
150 games sound fair. Also if there is any way to filter out a 0-2.2k+ rating jump over the span of 2-3 days that will also help out tremendously. a

Again, the wintraders on KJ have gone from 0-2400+ (3k) in ONE day....that's virtually impossible. If you could filter that out in some way, a lot of us would be truly grateful.
By PJlucyloo Jul 9, 2013
Seems like these filters wouldn't be too bad. Not sure whether a system of filters or flags would be better but seems like filters would be less trouble as flags you would have to go into each flagged toon's stats and confirm. Either way those suggestions above are more than generous stats to filter. Most PvP'rs know getting to those ratings takes much more than these minimal filtering stats but to be fair and eliminate any possibility of a legit player being filtered these numbers are more than fair and will make this site so much more legit. I use this site frequently as many of us do it will be nice to know that every effort was made to try and stop the cheaters from using the site to boast their illegal activities. Thanks Polar for opening dialogue with us regardless of outcome.
By Polar (site admin) Jul 10, 2013
We will see how it goes with 150 games/2600 rating. We will code it up in the next few days and then it will take about a week to refresh the data. Let's aim for around the 20th to have it fully up and running.

We will review it then and see if it needs any further refining around achievement dates - although, i expect it will be looking pretty good without it.

In the meantime, feel free to post up any more suggestions.
By Keylogthis Jul 10, 2013
Thanks, Polar. Appreciate it.
By PJlucyloo Jul 10, 2013
This is wonderful news and will be greeted by many with enthusiasm and relief. Thanks so much for listening to our concerns and not allowing your site and the hard work you put into it to be used to promote cheating.
By chilblane Jul 10, 2013
Thanks!!!!
By Nayadri Jul 10, 2013
Awesome!
By Polar (site admin) Jul 10, 2013
No worries.

I should add that if you know of individuals or guilds that are actively cheating then be sure to go and rate them 1 star using our rating system. All votes make a difference.
By chilblane Jul 10, 2013
Haha I remembered the star rating system last night, and forgot to mention it here. Going to do that right now!
By PJlucyloo Jul 10, 2013
I will start using it.

Also just fyi Polar the guild on KJ that was using this site to support their recruitment by advertising they were #1 on server thru their cheating activities removed this site's stats from their recruitment post after they found out that filters were being put into place knowing they were about ready to tank. One down many to go...with these new filters in place and dialogue with the pvp community open even if Blizz cannot do anything to stop people from getting their ratings illegally at least they will not be able to benefit from their activities using your site which for me makes this a more valuable site for statistics than Blizzard atm. Thanks so much!
By Nayadri Jul 10, 2013
3
By PJlucyloo Jul 17, 2013
LOL I can hardly wait to see what this does to KJ ranking :)
By Nayadri Jul 17, 2013
Also...this change:

Is everyone under 150 wins excluded OR everyone 2600+ will be excluded if they do not have 150 wins?

I think being under 150 wins to not being ranked is a little bleh, cuz we do want 2k players to be included and it doesn't take many wins to hit 2k legitly. However if it only applies to the bunch above 2600+ then it is TOTALLY neccesary

Thanks!
By chilblane Jul 18, 2013
I'm pretty sure it's for 2600+ only.

It looks like, with the except of one player on our realm that I can see, the upcoming change will completely decimate the influence of wintrading on our realm's rankings. And since I started following the US race, it will be nice to see that go back to normal.
By Polar (site admin) Jul 18, 2013
It is for 2600+ only
By PJlucyloo Jul 19, 2013
This looks incredible so far Polar! It did not catch all but this is awesome start and definitely more closely reflects the reality in game. Thank you so much! May have to go at some point to achieves all gotten same day or something but this is such a huge improvement!
By Polar (site admin) Jul 19, 2013
It is starting to take shape nicely. It will need a few more days to crawl more people and guilds though.

Ultimately, I suspect we will need to think up some fancier rule sets for the future - it probably won't take long before people find a way around this filter.
By chilblane Jul 20, 2013
Well, as I expected, guilds are starting to stack wintraders that are just below 2600. Example:

http://www.guildox.com/go/g.aspx?a=13&n=Division+VII&r=Kil%27jaeden-US
By Polar (site admin) Jul 20, 2013
I am not sure that guilds are purposely stacking sub 2600 players - it is just that the filter is removing the 2600+ players.

In any case, what do you all recommend is the next step?
By KJplayer Jul 20, 2013
Thanks for this Polar! After reading this, it does make a lot of sense to apply filters. Few things I would like to point out though. It seems as if your new algorithm is basically a form of removing outliers, in this case, for obvious reasons. This is a quite common statistical practice as you may be aware. It would make more sense however, to apply more rigorous methods to the calculations since it does seem that the ultimate goal is to reflect some “accurate” rating score for a guild's RBGs. It would seem logical to operate under some assumptions that validate any methods used. For instance,
A.) The top 10 players do not necessarily always play on the same team within a guild. Therefore it is not necessarily a direct reflection of the performance by a particular guild's team.
B.) Players seeing this score as a reflection of a guild’s RBG activity (which is also a performance measure) may be misled by the fact that a guild has a rating of 2K. This is the case since the guild could be populated with 50 players one half of which are ranked below 500 and a guild which contains 1 lead to run the RBGs. There are several cases of this on Kil’Jaeden.
C.) The score should also have some reflection of "activity" since this is also an important factor in determining if a guild is "good" in RBGs. Guild’s having high activity in RBGs will be represented by a more stable distribution of good scores over a larger population of players.
D.) Lastly, since we are looking for an accurate “score”. When looking over larger population, methods such as Median may be more relevant. This is the case since while some guilds may have 100 RBGers, 75% of players having a rating of 1600+ and 25% at 192, we wouldn’t want to make them look like a poor RBG guild by taking the Average. Instead the Median would be reflective of the norm around the majority of the players.
Overall, these assumptions are ones that can be used to generate a more accurate model of a Guild Score for RBG rating. In the sense that the score is meant to reflect the guild’s performance, it takes into account variables that are more representative of the Guild rather than a few members at the top that may not necessarily be representative of the “whole”. This is in similar vein to what you are already doing with the “filter”
Therefore, as a proposal for further optimization to your algorithm for “scoring” that can be easily coded, I would suggest a computational method, as follows:
1.) Take the players with the top 150 scores. Trim (“filter”) players with 100 games and score 2600) – what you are doing already.
2.) Reduce player set to top 100 from above trimming.
3.) Remove players with a RBG score of Zero, since they technically have never contributed to Guild RBG rating.
4.) Take the MEDIAN value of what remains of the 100 players.
5.) OPTIONAL: when posting the score, I notice you post if “10” players where used to calculate the score or not. Therefore, I am assuming it might also be worthwhile to still post a number here but this time have it be the number of players that were used for to calculate the MEDIAN. This would also be a helpful value in the sense that it gives you an idea of the rating over a particular number of players. Some may want to see this to get an idea of the guild size.
6.) NOTE: I do also want to note that your current filter does not take into account some very real scenarios. There may be the case where a player plays 100 games then decides to wintrade. They therefore will not meet both criterias and thus remain in the pool of players used to calculate the score. Also, there are instances where players may have friends with very high ratings and get “carried” to a high rating very fast. I do not believe this is against WoW rules and therefore their score would actually be a legitimate value to consider.
I think the above algorithm, as you described, would be an evolution of the one you currently use and certainly have potential for further optimization. For instance, there are also ways to represent the distribution of values via a particular statistical score and as I mentioned above. There may also be ways to implement a more accurate “filter” than the one currently used.

I hope this is helpful and thanks for taking the community input on how to make the ranking system more fair. It's definately something that helps make the game more pleasant to play and rewards those that stick to the rules!



By KJplayer Jul 20, 2013
Also as a side note Polar...larger sets of data are more resilient to "abnormal values" that may come from cheating. Much easier to do it if only 10 data points are used.
By PJlucyloo Jul 20, 2013
I'm not sure if there is a way to track the achievement dates. Almost all of these wintraders got their achieve from 1100 to 2400 and titles on same day or over a couple of days.

Also to KJplayer thanks for a concise and well thought out post. I am not as good with computer programming and language. I am much better at rallying support to fix things that I think are unfair.
By Nayadri Jul 21, 2013
Achievement dates shouldn't be relative to filters because sadly it would screw over high rated player's alts. Good high rated players can get 0-2200ish on their alt easily because they carry the same weight on the same team just playing a diff class. For example a high rated hpally decides to run with his team on his new monk thats finally got decent gear. He's still healing a high rated team because he's the same player but on a different character.

Ever thought of implementing arena rating into here? Say you're a 2200 rbg player but you have 2400 in arenas. Maybe because you have that rating it boosts your GuildOx rating score by like 100 or so. Anyone like this idea?
By Nayadri Jul 21, 2013
or screw it that'd complicated to keep track of. Maybe arenas just have their own category but track the top 20 rated arenas
By PJlucyloo Jul 22, 2013
Yeah Nayadri that makes sense and would not work. Don't know about the arena thing either. I do not do a ton of RBG's but have managed 1800 ish but I do little to no arena's at all. Just do not find them enjoyable. But if by some miracle I got back into RBG's and pushed higer rating I would hate any of that to be affected by my arena activities which is little to none. Not sure what answer is here except to get Blizz to fix the problem. This adjustment definitely helped a lot but of course you can still see many in top 10 who have wintraders that just did not make it to 2600 so they are still counted. These ranks though definitely look better than they used to and although they do not reflect the real rankings they are a lot closer than they were. Just not sure what step could be taken next without messing up legitimate players as well.
By Polar (site admin) Jul 22, 2013
@KJPlayer - thanks for the detail and work you have put in here. I have a good grasp of statistics so I will take a look over it.

In the meantime, let's run with the simple filter we have for a bit longer and see how it all works out. We might consider dropping the threshold from 2600 to 2500 at some stage.
By KJplayer Sep 29, 2013
Hi Polar, now that we've started the new PVP season, I wanted to rehash this topic. You can see some very interesting things now represented by the progress in ranking for the various guilds. Early on, people have not had the incentive to win trade yet so you get to see guilds that are really active in RBGs as well as running core teams easily. You also see guilds that have the one or two odd players that have "somehow" managed to secure 2200+ very fast and they stick out like a sore thumb from the rankings of the rest of the guild. You also see guilds with only a few players leading the ranks and pushing the average of the guild up. So here again, I would like to bring up the topic of modifying your algorithm again. It seems like a much better representation of the guilds activity, ratings and team playing would be reflected by a greater pool of players that you use to generate your guild ranks. As I mentioned before, these are guild ranks and I think they should be more representative of a larger pool of players that are contributing to the "score". This will also be helpful to people wishing to join and participate in these guilds if they know a larger population of players are actually playing and representative of the guild score. It will also be a good way of diluting out those few individuals that find questionable means of achieving high rating in a very short period of time. My suggestion would be to start by using the top say 50 players and see how that works. Having taken a good look at the compositions in scores in various guilds, I do believe this would be a good starting point. Thanks and as always appreciate your efforts to make WoW a better playing environment. ~Kjplayer
By Polar (site admin) Oct 1, 2013
Locked - do you mind bringing this up in a new topic since I don't want us to have to wade through 3 pages of the old issue
Post Reply