GDC 2024: Embedding Player Data into Game Development

Date Published: 11/03/2024

As part of a series of talks at this year's Game Developers Conference (GDC), Sebastian Long, Managing Director at Player Research, a Keywords studio, is hosting three roundtables, all on the topic of embedding player data into game development at various stages.

What can attendees expect to learn in these sessions? We spoke to Seb about the inspiration behind the topic, and its potential takeaways for GDC-goers.

What does it mean to ‘make player data work’?

Player data is a double-edged sword: incredible at unearthing design issues, misunderstandings, and inspiring new creative ideas; an essential component in developing fun and accessible games. Yet it’s also possible for player feedback — if it’s poorly collected or managed — to undermine confidence, sap team morale, and be completely misunderstood. It’s all too possible to make a game much, much worse through player feedback, if it’s misused. These roundtables will discuss how to make player data work better for game teams: maximising positive outcomes, and staving off the negatives.

Can feedback really make game design worse?

Unfortunately yes! There’s lots of ways that can happen, but perhaps the most common are caused by the timing of player feedback, and the accuracy of the feedback. Getting either one of these wrong can lead to bad outcomes for game design.

When I say ‘the timing of feedback’ I’m referring to the gap between the critical time a game design choice is made, and the first time that that choice is really tested with a member of the public. In the best of cases that gap is really short — ideally a few days, a week, a fortnight — so teams can quickly ask: “is this button, this tutorial prompt, this signal, being played as we intend it to? Is this content fun for players?”, and so forth. For lots of reasons it’s feasible that some design choices sit unexamined for long stretches of time before they’re tested in the wild, and that can lead to unwelcome surprises, especially if there’s an overlooked issue with core controls or the core path. If there’s not enough time left to react to that feedback fully — to make deep-cutting changes needed, or to validate last-minute changes with another round of feedback-gathering — it can leave the game in worse shape.

Having inaccurate player feedback can also make the game much worse. There’s an adage on game development that you should pay close attention to players saying what’s wrong with their game experience, but ignore what they suggest to fix it. And that’s mainly true. It alludes to the reality that players’ verbatim feedback can’t tell the whole story, that it needs somehow translating and cross-checking to be the whole truth.

If you’re being really technical about this you’d say that data needs ‘triangulation’: trying to take lots of different sources of information and see where they align. That means gathering feedback from a diverse group of players, and in a controlled setting first of all. Certainly not interrupting playtesters playing too much, or hovering behind them making them nervous. It also means relying on other sources of data than just what players are able to tell you directly. In player research our richest source of data is watching players play: getting really deep into the actions players perform when playing, and what about the game’s design influenced them to do that. This often reveals far more about the game design’s flaws than asking players individually or in a group “what did you think?”.

Observation of gameplay is a science unto itself, and it can radically improve the accuracy of the player data we rely on to help dev teams make effective improvements to their games. Relying more on observation rather than surveys is just one way to improve the accuracy of the player data teams capture to improve design.

You’re hosting three roundtables on this subject. Can you tell us about what each will cover?

That’s right, I’ve chosen to split up the topic into three distinct game development phases, since the way player data is applied is radically different in each one.

The first roundtable is about getting player data into the pre-production phase of development. Feedback at this stage has the unique trait of trying to inspire radically new ideas, and gather a really foundational view of who our future players are: what makes them tick. That’s a completely different ‘voice’ of the data to later development phases, which tend to be more specific and focused on the iteration of game content.

The roundtable content will be led by the attendees, but I can expect to cover questions like: “how does player data support teams designing in a genre that’s new to them?” and “how can player feedback help define monetisation strategy early?”

My second and third roundtables deal with pre-release production and post-release ‘live operations’. Supporting ongoing development through production is typified by rapid iterative feedback sessions, reducing design risks like poor usability, accessibility, approachability and onboarding. Game development moves quickly, so scrappy feedback sessions held in playtest labs are common. Post-launch games find themselves awash with player data in the form of game analytics, and teams can find it challenging to make sense of all the real player data that’s flowing into their studio, through social media, customer support, analytics and so on.

For the production and post-prod roundtables I imagine questions such as: “how do teams keep playtests with real players secret?” and “how do we add the ‘why’/‘why not’ insight to analytics data?”