Our own tiering system: Empirical Tiers

[DCM]

This is a post taken from a mod forum discussion we were having a while back. I'd like to make it the official Bulbagarden stance on tiering to go with our PO server plans.

I. Background
So I was doing some thinking when I saw this post in the thread about Bulbagarden tiers. I was also thinking about the primary (in my eyes) criticism of the Salamence move: it was done on a vote with no real, concrete, objective empirical evidence behind it. And so I was pondering how to remedy this.

The obvious answer is to have empirical tiers.

II. Explanation
How would they work? It actually isn't all that complex.

* The top (approximately) 2% of Pokemon, by base stats, are automatically+ banned.
* A Pokemon that is used on more than 2 in every 5 teams (>40%) is banned forever.
* A Pokemon that is used on more than 1 in every 3 teams (>33%) in a month is banned for 6 months.
* A Pokemon that is used on more than 1 in ever 4 teams (>25%) in 2 months over a 3-month period is banned for 6 months.
*A Pokemon that is used on more than 1 in every 5 teams (>20%) in 4 months over a 6-month period is banned for 2 months.

The + on automatically--this rule would exempt Pokemon whose ability obviously lowers their real stat value. In real terms, this excludes Slaking and Regigigas, so far, from the uber list. If any other conditions come in which mimic Slow Start or Truant, they will also be added to the exemption list.

III. Goal
The goal of this tier is to play with a very "open" metagame--that is, trainers have lots of options when they want to put a team together by the time the metagame develops. While the metagame is developing (expected to take 4-7 months) there will be some serious fluctuations in what's allowed, what's banned, etc. Once it's established, however, I expect it will be superior to Smogon's.

Why? Because Smogon is solely concerned with removing what makes for a playable metagame. This will be concerned with removing what makes a highly concentrated metagame. The net effect of the Bulbagarden Tier system will be a more spread-out, "open" metagame driving more innovation and more fun.

IV. In practice
So how would this work in practice?

Well, here's a hard banlist by generation:
Gen I: Mewtwo
Gen II: Mewtwo, Ho-oh, Lugia
Gen III: Mewtwo, Ho-oh, Lugia, Rayquaza, Groudon, Kyogre (awkwardly, in Gen III the 2% rule spills a bit out into the 600-level Pokes, and we don't want a blanket ban of them.)
Gen IV: Mewtwo, Ho-oh, Lugia, Rayquaza, Groudon, Kyogre, Arceus, Giratina, Palkia, Dialga.
Gen V: Mewtwo, Ho-oh, Lugia, Rayquaza, Groudon, Kyogre, Arceus, Giratina, Palkia, Dialga, Reshiram, Zekrom.

So, excepting Gen III, we've been perfect so far in banning the (agreed-upon) top-tier of Ubers.

V. Closing Comments

First, I'd love criticism and comments a lot. Go to town on this idea.

But the big theoretical concern I see that I want to address preemptively is, broadly: why not ban things like Darkrai outright?

Simple. We don't know how Gen V is gonna play. So we let it just...play out. If everyone (say, 55%) is using Darkrai in the first month, we ban him. Easy peasy. If he suddenly becomes a regular part of the metagame for whatever reason (say, 15%), we let the flow of the metagame determine that, too.

Some caveats: This will result in a very broken first month. I don't see that as a huge flaw, though. This system is entirely empirical. Nobody will be able to dispute our relegation decisions, etc. We won't have anything like the Great Salamence Schism of 2010. It's cut-and-dry, easy, and doesn't take a lot of effort on our part to maintain.

The main thing that requires additional rigor is the cutoff levels between tiers, which translates into creating a UU/NU tier if we so desire.

I've decided to move this to development due to the fact that the majority of the users seem to like the idea. However, I do see that people have problems with the suggested system. I'd love it to see other pitches for a tier system which we could later vote on.

 

[Zekurom]

So, my proposed system again, from the previous thread:

The basic gist of the system is to rank Pokémon by usage, not just by battling ability - just like with technical analysis in a stock market, the two are usually correlated, but one is more readily measurable than the other, although admittedly less useful.

In the same way that a higher price usually means a better stock to have bought, higher usage usually means that the strategy being used is better.

However, this system does not aim to ban any Pokémon, per se - just to rank them into tiers. Then, each individual tournament can decide for themselves which Pokémon are allowed or not.

And the basic mechanic of this system is that Pokémon that are used too much within a tier are promoted to the next tier, and that Pokémon that used to little within a tier are moved to a lower tier, to keep the tiers ranked by usage.

Within a tier, each Pokémon has what is called a "relative representation", which is a measure of how dominant a Pokémon is within a tier, how how commonly it is used compared to other Pokémon. This ratio is equal to the proportion of that Pokémon, divided by the proportion of any other Pokémon.

So, if, say, Garchomp accounts for 8.4% of all Tier 8 (OU1) Pokémon, its relative representation is calculated as 0.084 / 0.916 = about 0.0917, or 9.17%.

What is this means is that for every non-Garchomp Pokémon in Tier 8, there are 9.17% Garchomp - or, conversely, for every Garchomp being used, there are 1/0.0917 = 10.91 other Tier 8 Pokémon being used.

Now, each Pokémon within a tier also has a measure of how dominant it should be within a tier - its "fair share". This is simply the actual relative representation when each Pokémon is represented equally. Let's say that there are 15 Pokémon in Tier 8 - then Garchomp's fair share of relative representation is (1/15)/(14/15) = 7.14%.

Now, what I propose as the promotion/demotion system is quite simple. It is as thus:

• If a Pokémon's actual relative representation exceeds double its fair share, then it should be promoted to the next higher tier.
• If a Pokémon's actual relative representation falls short of one-half its fair share, then it should be demoted to a lower tier.

These cutoffs are just arbitrary for the time being - if Pokémon start to gravitate toward one end of the tiers, we can adjust them accordingly.

To go along with this proposal, I would also like to expand the tier counts. Currently, Smogon uses a five-tier system, which can start becoming a little unwieldy with 649 Pokémon in the metagame, even if half of them are removed due to not being fully evolved (and hence being placed in the NFE group). I propose expanding it into 9 tiers, with the names Tier 1, Tier 2, etc.

They could also have these alternate names:

Tier 9 - Ubers (Uber)
Tier 8 - Overused Level 3 / Extremely Overused (OU3/EOU)
Tier 7 - Overused Level 2 / Overused (OU2/OU)
Tier 6 - Overused Level 1 / Heavy Usage (OU1/HU)
Tier 5 - Border Line / Medium Usage (BL/MU)
Tier 4 - Underused Level 1 / Light Usage (UU1/LU)
Tier 3 - Underused Level 2 / Underused (UU2/UU)
Tier 2 - Underused Level 3 / Extremely Underused (UU3/EUU)
Tier 1 - Never Used (NU)

As you can tell, I'm really bad at making these names. But the tiers still apply.

 

[Sarcastically Insane]

I agree almost completely with Giaru. However, I don't think we need 9 tiers- certainly 2 or 3 between Uber and BL, but I don't think that splitting what we currently know as UU into 3 tiers would be as useful. 2, maybe, since some UU Pokemon (Hitmontop, Swellow) are very dominant, but not 3- or at least, not yet. If there shows a need for another tier after a while, then by all means. HOWEVER, I think that we absolutely SHOULD NOT use Smogon's tier names (except maybe Uber and NU, since they will probably correlate 90% of what's in them). This avoids confusion, since some people may want to continue use of Smogon. I think most of the names Giaru suggested would work (especially since some of them seem to be made from my original suggestions).

 

[Phoenicks]

If people want a tier system, they can go to smogon. I think that one of the benefits of the proposed system is that it doesn't segregrate the metagame by tiers. You have one set of Pokemon you can pick from, with no concerns about mixing the races or failing to keep tiers equal.

 

[Zekurom]

However, I don't think we need 9 tiers- certainly 2 or 3 between Uber and BL, but I don't think that splitting what we currently know as UU into 3 tiers would be as useful. 2, maybe, since some UU Pokemon (Hitmontop, Swellow) are very dominant, but not 3- or at least, not yet. If there shows a need for another tier after a while, then by all means.

I'm not "splitting" UU into three tiers at all.

The nine tiers are not meant to be equivalent to Smogon's tiers by name. I just used their names because it was convenient. Officially, they're referred to as Tier 9, Tier 8, etc.

Personally, I think the NU tier could use some splitting.

If people want a tier system, they can go to smogon. I think that one of the benefits of the proposed system is that it doesn't segregrate the metagame by tiers. You have one set of Pokemon you can pick from, with no concerns about mixing the races or failing to keep tiers equal.

Smogon's tiers are anything but equal. (Case in point, their NU tier for D/P takes up more than half of all the fully evolved Pokémon, where as BL is only 11 Pokémon and Ubers is 25.) Also, what do you mean by "mixing the races"?

The reason we disliked evkl's proposed system was because it banned all the overused ones, whereas the smogon!OU metagame is where most official Pokémon video game tournaments are going to be at. Eventually, as the Pokémon pool got less and less, more and more Pokémon would be banned due to simple mathematical laws. Either that, or as some people said in the idea thread, we'd all be restricted to NU Pokémon. Those are pretty evenly spread out, in my opinion, and would make for a more "diverse" game.

Not to mention - evkl's system is pretty much a two-tier system anyway - a "banned" tier and an "unbanned" tier. Not much room for any actual ranking.

Fourthly, I don't see how the game gets "segregated" at all. When somebody talks about an "OU" game, they're referring to a game that allows OU Pokémon and under. So if anybody has a UU strategy they want to use in that game, that's perfectly fine.

 

[Our Wasted Years]

I really think you guys might want to first come up with tier names. If you use Smogons terminology people are going to be confused, plus doesn't look original.

 

[Ino-Chan]

As for tier names, why not just have it Tier 1, Tier 2, etc?

It's not original, but it works, is easy to understand, and isn't Smogon.

 

[Zekurom]

As for tier names, why not just have it Tier 1, Tier 2, etc?

It's not original, but it works, is easy to understand, and isn't Smogon.

But I already suggested that ;_;

Anyway, I'm running through the mathematics of my algorithm with a buddy. We'll have it refined, don't you worry.

 

[The Outrage]

I really think you guys might want to first come up with tier names. If you use Smogons terminology people are going to be confused, plus doesn't look original.

I second this.

High
Medium
Low

Maybe categorize them like that, though I want to know how tiers are being segregated, by usage or by power?

 

[Sanic Hegehog]

though I want to know how tiers are being segregated, by usage or by power?

A tiering system should mix power with usage, both of which are achieved from experience and knowledge though testing in the metagame.

 

[Zekurom]

Maybe categorize them like that, though I want to know how tiers are being segregated, by usage or by power?

By usage. There's no objective way to determine power, except by perhaps base stats, and that's very non-strategic.

A tiering system should mix power with usage, both of which are achieved from experience and knowledge though testing in the metagame.

Usage implies power, most of the time. Strategic power, at least. We're hoping that the experience and knowledge of our competitive battlers will test this new system thoroughly enough that it will evolve into its own thing.

 

[Sanic Hegehog]

Well yes... maybe I should of worded that a little better, but at least you get the idea. Anyway, I'll contribute to this in anyway I can, hopefully with not too much bullshit.

Also regarding the tier names, I personally want to see a Greek Alphabet theme:

Tier 9 - Alpha (Uber)
Tier 8 - Beta (Extremely Overused)
Tier 7 - Gamma (Overused)
Tier 6 - Delta (Heavy Usage)
Tier 5 - Epsilon (Border Line)
Tier 4 - Zeta (Light Usage)
Tier 3 - Eta (Underused)
Tier 2 - Theta (Extremely Underused)
Tier 1 - Iota (Never Used)

 

[Zekurom]

Well yes... maybe I should of worded that a little better, but at least you get the idea. Anyway, I'll contribute to this in anyway I can, hopefully with not too much bullshit.

Don't worry about spewing BS - we'll just see it as an honest mistake.

Also regarding the tier names, I personally want to see a Greek Alphabet theme:

Tier 9 - Alpha (Uber)
Tier 8 - Beta (Extremely Overused)
Tier 7 - Gamma (Overused)
Tier 6 - Delta (Heavy Usage)
Tier 5 - Epsilon (Border Line)
Tier 4 - Zeta (Light Usage)
Tier 3 - Eta (Underused)
Tier 2 - Theta (Extremely Underused)
Tier 1 - Lota (Never Used)

That last one is "Iota", correct?

That might work. I'd have to try and come up with better, though XD

 

[Tolan]

Well yes... maybe I should of worded that a little better, but at least you get the idea. Anyway, I'll contribute to this in anyway I can, hopefully with not too much bullshit.

Also regarding the tier names, I personally want to see a Greek Alphabet theme:

Tier 9 - Alpha (Uber)
Tier 8 - Beta (Extremely Overused)
Tier 7 - Gamma (Overused)
Tier 6 - Delta (Heavy Usage)
Tier 5 - Epsilon (Border Line)
Tier 4 - Zeta (Light Usage)
Tier 3 - Eta (Underused)
Tier 2 - Theta (Extremely Underused)
Tier 1 - Lota (Never Used)

Even though I do like the Greek names, I don't think that we don't need 9 tiers. That might be kind of confusing. I think that:

Alpha (Uber)
Beta (OU)
Gamma (BL
Zeta (UU)
Theta (NU)
Iota (LC)

is a much better way.



Also, for the "Ban List", would those be Uber, or we couldn't use those, even in Ubers?

 

[Zekurom]

Even though I do like the Greek names, I don't think that we don't need 9 tiers. That might be kind of confusing. I think that:

Alpha (Uber)
Beta (OU)
Gamma (BL)
Zeta (UT)
Theta (NU)
Iota (LC)

is a much better way.

Excuse my ignorance, but what are UT and LC?

 

[Sanic Hegehog]

Excuse my ignorance, but what are UT and LC?

UT is UU, but I'm not sure what it means.

LC, however, refers to the Little Cup tier.

 

[DCM]

Let's not use Greek.

 

[The Outrage]

By usage. There's no objective way to determine power, except by perhaps base stats, and that's very non-strategic.

Smogon's is by usage too so I don't really see the difference, unless the Bulbagarden's PO server can also take statistics on what's used and what's not and we're tailoring our tier system based on that.

And since it is usage, just go with the usage terms since its most intuitive, but if using it outside of Bulbagarden, just specify its the Bulbagarden Tier.

Or just go with a variation what I suggested:

High Usage
Medium Usage
Low Usage

And then create a ban tier for each one based on whether its broken in that usage tier rather than bumping it up a tier since that's pretty misleading.

HU Ban
High Usage
MU Ban
Medium Usage
LU Ban
Low Usage

And basically, anything in a higher usage tier is intuitively banned from lower usage tiers.

Wondering if we could do tiers for other battle styles though, like Miracle Shooter and Double and Triple battle, and also how we could keep track of usage there (I guess people who battle could always report on the Pokemon they see)

EDIT: And based on statistics, probably the top 30% on the high usage, 31%-69% medium and the bottom 30% low. Things that are obviously overpowered (like the 680 legendaries) would be in the highest ban tier thus allowing every Pokemon to compete there.

 

[Sanic Hegehog]

I think Doubles, Triples, Rotational, and Miracle shooter should have sub-tiers sort of like Little Cup.

 

[Zekurom]

I think Doubles, Triples, Rotational, and Miracle shooter should have sub-tiers sort of like Little Cup.

Or we have separate tiers for double, triple, and rotational battles, but that has the potential to get really complicated.