How Riot Can Continue to Improve Their Design Philosophy by Grimlockt

Our latest Mobalytics Partners spotlight author is Grimlockt! He’s a ten-year gaming industry veteran who has been a part of nearly 150 different game releases as game designer, consultant, and producer. In this guest post, he discusses the evolution of Riot’s design philosophy and explores ways they can continue to improve. 

Let’s give him a warm welcome! Be sure to follow Grimlockt on Youtube, Twitter, and Discord. You can catch him live on Twitch from Friday through Tuesday from 7 PM to midnight CST.

Part I: The Article I Almost Wrote

I was initially going to write an article titled “Why Riot Needs to Change Their Design Philosophy.” I wanted to explore the topic of criticizing Riot’s design philosophy as a game designer myself, but I wanted to be respectful and honest with my criticisms.

It was going to be several kind suggestions, supported by statistics I found with champion releases to suggest, maybe it was time to make some changes.

And then, Riot did the unthinkable in the corporate world…they changed their design philosophy (read about it in their /dev post!).

It’s one thing to talk though and another thing entirely to back that up with action.

Enter Lillia and Yone – and now, the Riot superfan in me is so happy to report that I’m delighted with the Yone and Lillia releases. Lillia is still my go-to ban, but that’s more personal preference and less out of necessity.

Today, I had to play Darius into Yone and Yasuo in the top lane during a game with friends (they did not have a Jungler), and I ended up going 16/10/8 by the end. Yone feels great!

He has a lot of damage, but I also feel like there’s a lot of counterplay in his kit, and that is just a massive sigh of relief for me. It felt intuitive to play against him, and I did not feel cheated when he got away.

And that left me in a pickle because I was fully ready with a ton of statistics, history of champion releases, and research to back up an already solid hypothesis that Riot really needed to evaluate their way of releasing champions.

After all, it was toxic to the player base…and how dare they take full responsibility for their mistakes and take steps toward areas of opportunity when I was fully ready to go to war! (I am joking around, and if it isn’t apparent yet, I hope it is now).

No, but seriously. I am incredibly happy with Riot’s most recent champion releases. They said they were going to take steps to do the right thing for their player base, and they delivered. That is awesome! Keep up the excellent work, Riot! *High five* You earned it!

So this article will now focus on what Riot can do in the future to further improve champion releases. Things I do not see them doing at the moment, but in my professional experience, are a good idea and can benefit them in the future.

I have broken this up into three parts.

The first section (which you have already read), the second section where I will try to shed some light on why we should be more understanding of Riot during new champion releases, and finally, the third section, where I offer my suggestions on how to approach making the game even better, based on my professional experience.

But without further adieu, I will present Part II, in defense of Riot, on why champion releases will never go as planned.

Part II: Why Champion Releases Will Never Go Perfectly

A little background about me, I worked on the production side of the game industry for a big label for about six years as a specialist, consultant, and other odd jobs.

As a result, I participated in 150+ game launches? I’ve seen Murphy’s Law at work in real-time.

I learned that no plan ever survives the battlefield, and it is better to have a general outline of what to do, rather than a set list of requirements that need to be met.

And additionally, it does not matter how many Quality Assurance testers you have, players will find a way to break your game, and they’ll put it on Youtube.

In 2009, I was fortunate enough to hear Rob Pardo, the Chief Creative Officer at Blizzard speak at GDC (Game Developers Conference).

This quote from him has stuck with me, ever since:

“You could have 200 of the best QA people working on your game round the clock. The moment you release your game to a million+ players, those players will find more bugs in one day than your team of 200 QA testers found over the entire life cycle of development.”

That sounds a bit outlandish, but it’s not so out there if you think about it in terms of man-hours worked.

I want to drill into that a bit here and deep dive it for this article because I believe it can provide a perspective that we as players badly need from the developer’s point of view.

We know that Riot spends between one to two months testing champions out before they release them from an interview that August Browning did with PC Gamer a while back.

Browning stated: “We usually start balancing a champion up to a month or two before release. To be fair, we balance all the way through the process, because it’s hard to playtest a character when they’re doing 5,000 true damage on every basic attack. We really start drilling down at about the one to two month mark before they come out.”

So, let’s do a bit of math here, shall we? If we can assume 200 QA testers (with a company of Riot’s size, 200 is reasonable in my experience). They may have more QA testers.

Still, I’d bet Darius’s ultimate that they don’t have less, and assuming their QA testers are not being abused to work more than 40 hour work weeks, 40 hours a week, for roughly 4 weeks per month.

That means approximately 64,000 man-hours of testing go into testing a new champion.

40 (hour work week) x 4 (weeks) x 2(months) x 200(people) = 64,000.

Well, that’s a lot of hours! But, let’s compare that to the amount that the players of League of Legends will play said champion in a single day. Not months, not weeks, in a single day.

On release, Lillia was picked in roughly 6.33 percent of games.

Currently (as of writing this article), according to PlayerCounter, there are 2,367,864 active players. This number obviously fluctuates up and down, and if you click on the link, the amount will be different for you than it was for me, but for the purposes of the math I’m about to do, there are precisely 2,367,864 players active at any given time.

This means we can assume there will always be 2,367,864 players playing League of Legends, no matter what time it is.

2,367,864/10 (ten players per game) means 236,786 active games at one time.

If Lillia is being played in precisely 6.33% of these 236,786 games actively being played, Lillia would be being played at any one time in 14,988 games.

According to LeagueOfGraphs, excluding Master, Grandmaster, and Challenger, which account for a small portion of the League of Legends community, the average game time across all ranks is around 28:46 (or 28.77 minutes).

That means at the moment of Lillia’s release, she was played in the first games ever of Lillia for 35 minutes into her release, for a cumulative total of 7,187 hours.

It would take 200 people 35 hours to do the same amount of work. But it gets better.

If we assume time-wise, that there are also 3 minutes for champion select, two minutes to load into the game, five minutes for after the game (loading out of the game, plus chatting in the post-match screen) and an additional 5 minutes between games for bathroom breaks, etc.

This comes out to, on average, about 45 minutes per game, which means that the average player base could get 32 games per online capita over a day.

32 games x 7,187 hours of playtime gives us 229,984 hours played in-game on Lillia on the first day alone of her release, which is three times what a group of 200 QA professionals could manage over two months.

So maybe, we should cut Riot some slack, huh? They are doing a pretty darn great job, considering they have such a hungry user base for their product.

Part III: What Can Riot Improve On?

Having gone out of my way to defend them, I think I’ve earned a bit of time to speak on how I think they can work toward positive change going forward in the future.

One, it does seem like they have a good handle on champion releases and not releasing champions in overpowered states. But, I think it bears repeating.

Win rates don’t equate to champion quality. I believe that Riot needs to have more faith in their players and their product in some ways.

Coming from a creative standpoint, I know how hard it can be to release an innovative product and wonder if it succeeds, but you just gotta have faith sometimes.

However, there are safe ways to ensure champion releases go better, and that is by utilizing their most valuable resource more, their players. Here are things I think they should do:

1. Extend PBE testing phases

PBE testing phases for games tend to be almost an afterthought. It is my opinion that those phases should be twice as long as they currently are. Leverage the player base more and reward players who actively participate with incentives that players like, such as skins, chromas, and currency.

2. Increase in-house player testing

I already know that Riot does in house player testing, but I encourage them to do bigger and more sessions, before releasing to the PBE servers. The more players get involved in the process, the better it will be.

3. Send out more surveys to the players that are actively participating and gather end-user thoughts

This collects an incredibly valuable metric for analytics that can’t be gleaned from server statistics called “Overall User Satisfaction.” What if during the Yummi release Yummi had a 30% win rate, but an overwhelming 80% approval rate? That would have a massive impact on how Riot was to approach balancing that champion. Because the core philosophy was on target, but her numbers need work. Whereas, what if a champion had a 54% win rate, but a 20% user satisfaction? That also would directly shape approach, because it would be easy to tell that the champion missed the mark.

4. Have more faith in the champions they make

I think one thing that really strikes home for me while reading the article was the underlying apology between the lines. There was something there that hit home for me, and that was an underlying feeling that it was not the developers’ actual intent to release OP champions, but that they were made to do that. And though, they did not outright say that they were made to do the thing, that can only happen if big brother, a publisher steps in and makes them by exerting their muscle. When that happens, it is up to management to use one of the most powerful words in their arsenal, no.

Thanks for reading!

I will end the article on that note because I feel like League is heading in the right direction. There is a bright spot on the horizon for us. And I hope that my thoughts can have a lasting and positive impact in one way or another. As always, stay classy, summoners! DEMACIA!!!!!

Be sure to follow Grimlockt on Twitch, Twitter, and Discord. You can catch him on his Twitch from Friday through Tuesday from 7 PM to midnight CST.