Dev. of a Squidboards Weapon Tier List, Part 2: Tier Creation

[Vitezen]

For part 2 of the creation of this tier list, we'll be discussing how the tiers themselves will be set up. This includes how many tiers exist, and the qualifications for each tier. For part 1, calculating the rankings of weapons, see the first thread: http://squidboards.com/threads/dev-of-a-squidboards-weapon-tier-list-part-1-calculations.10686/

For this segment I'm going to be a little more hands-off, because I don't have a set idea of my own for how tiers should be organized yet. However, I can offer some data that I've gathered recently, to help you all understand how the weapons might be ranked. This is useful because you can more easily formulate reasons for what the cutoff should be for each tier, and how many tiers there should be

Below is a calculation of weapon rankings I made from recording 68 matches from the recent Messtival 2 tournament, both official matches and scrimmaging. Keep in mind that I couldn't see every single match, but I did see the vast majority of them. Also, to avoid people being biased by their own favorite and least favorite weapons, I've chosen to not use their actual names. In addition, as decided in the previous thread, I ranked scoped and unscoped variants of chargers as separate weapons. Finally, for simplicity, I've chosen not to include the modes or maps used for this experiment. But for reference, Messtival 2 used a variety of maps, and all modes except turf wars.

Spoiler: Messtival 2 Data

There are a few interesting facts I gathered from this data.

Out of the 60 weapons that existed in the game on that day, only 41 of them saw any play.
Out of those 41 weapons that saw any play, only 27 of them were used to win a game.
I chose to omit the 19 weapons that saw no play from the visual.

I can conclude from this, if it is an accurate microcosm of the metagame, that in competitive play, more than half of all weapons will see very little or no success, and about one third of them will see little to no play at all. Because of this, I believe it will be necessary to have a large “unranked” tier, for weapons that have a ranking of zero due to no games won with them. Other than that, I currently have nothing else to offer on how best to develop the tiers themselves in this tier list.

As before, I plan to keep discussion in this thread open for about a week, after which we'll be moving on to the third and final part of the process in developing this tier list. This thread pertains only to the creation of the tiers, not the rankings of the weapons. For discussing weapon rankings, please make your posts in the first thread. Remember to include your own suggestions and critique the methods of others.

 

[Dolphoshi]

I feel that a class system could work better as any weapon can be used to win a game (I do well with an L3-D as well as an eliter 3k with no scope) certain weapon do have an advantage over a variety of situations giving people the illusion of better weapons.

 

[NeoSeth]

I think a tier list should be divided something like this:

S Tier: Weapons that are extremely powerful in a wide variety of situations. These weapons have few, if any, weaknesses and these weaknesses can be mitigated by a strong player. These weapons are versatile and good for a wide variety of modes/maps. These weapons are always strong choices.
A Tier: Weapons that are very strong, but not as versatile as S Tier. Weapons in this tier have weaknesses that make them less desirable in certain situations, but are overall still very effective and can serve as a solid part of a team. They're never weak choices.
B Tier: Weapons that are strong in some situations, but weak in others. These weapons are often map dependent, and while they may be powerful on some maps or modes they can be inferior choices on others. Overall very good, but far less versatile than A-tier weapons.
C Tier: Weapons that are only good in a few situations, but very weak in a variety of other situations. Perhaps in a specific map/mode combination one of these weapons could be actually quite powerful, but most of the time these weapons will be outclassed or possibly even dead weight on a team.
D Tier: Weapons in this tier have almost no purpose on a high-level team. These weapons require specific situations to be useful, and even then are often still outclassed.

I think these distinctions are vague enough to allow for good debate, but also specific enough to show a division between how good weapons are.

 

[Vitezen]

@NeoSeth The purpose of this tier list isn't for subjective debate about weapon quality though, as I said in the previous thread. This tier list is purely guided through numerical rankings based on how frequently a weapon is used and how often it appears on the winning team. Instead of debating the viability of the weapon and opening the door for bias to come through, I've chosen to instead use purely numerical data gathered objectively.

For example, you've suggested 5 tiers. What would be the cutoffs for each of them? That's more of the decision we're looking for.

@Dolphoshi We've considered that idea before. The biggest problem we could foresee was how much data we have available, and whether or not it's useful when looking at the meta. You have to assume that at high level play, when someone is using a weapon, they're filling their role properly, which leaves the only factor as the weapon itself. The purpose of the tier list is to anticipate how often you'll see a weapon and how well you can expect it to do on average. If a certain weapon is very good in a very low number of situations, it still needs to be respected, but you can't really say it's a big influence on the metagame when most of the time it won't see much play. That's accounted for in the calculations themselves, since you first take how often the weapon appears, THEN multiply it by its rate of success. Even if a rarely used weapon is successful when used, it will still be ranked fairly low.

 

[NeoSeth]

@NeoSeth The purpose of this tier list isn't for subjective debate about weapon quality though, as I said in the previous thread. This tier list is purely guided through numerical rankings based on how frequently a weapon is used and how often it appears on the winning team. Instead of debating the viability of the weapon and opening the door for bias to come through, I've chosen to instead use purely numerical data gathered objectively.

For example, you've suggested 5 tiers. What would be the cutoffs for each of them? That's more of the decision we're looking for.

@Dolphoshi We've considered that idea before. The biggest problem we could foresee was how much data we have available, and whether or not it's useful when looking at the meta. You have to assume that at high level play, when someone is using a weapon, they're filling their role properly, which leaves the only factor as the weapon itself. The purpose of the tier list is to anticipate how often you'll see a weapon and how well you can expect it to do on average. If a certain weapon is very good in a very low number of situations, it still needs to be respected, but you can't really say it's a big influence on the metagame when most of the time it won't see much play. That's accounted for in the calculations themselves, since you first take how often the weapon appears, THEN multiply it by its rate of success. Even if a rarely used weapon is successful when used, it will still be ranked fairly low.

In that case, I suppose this tier list will be similar in structure to Smogon's tier list process? For reference, they have Pokemon divided into Overused, Underused, RarelyUsed, Neverused and PU (Which is just a pun, as everything in that tier stinks).

I maintain that there should be only a few tiers, as having many tiers can overcomplicate things (even if this means having a lot of weapons in a few tiers). I suppose I would like to see the usage data for weapons to see how often weapons are used so I can make an educated comment on where the usage cut-off should be. I don't really understand the data in your OP, but using those numbers:

A Tier: 20 or more.
B Tier: 10-20
C Tier: 5-9
D Tier: Less than 5, but more than 0
0 Tier: 0%. Weapons here are never seen.

 

[Vitezen]

Alright my bad. I explained how the ranks were calculated in the previous thread. The two numbers in parentheses are the frequency a weapon appears on a team, and the frequency that the weapon appears on a winning team. I divide the number of appearances by total games x2, then multiply the result by the number of wins divided by the number of appearances.

 

[NeoSeth]

Gotcha; I understood it a bit, but wasn't 100% clear. I think that the numbers I gave, based on your data, line up with clear divisions in weapon useage. The weapons valued at 25 and up seem to be CLEARLY above the rest, deserving of their own useage tier. The other numbers were just based on where I saw divisions lie in your data.

I've spent too much of my life arguing in Smogon and Smashboards, so this type of discussion always excites me. ^_^