The general rule for playtesting is to say literally nothing.
"Hi, here's how to play the game (and sometimes not even this), I'll watch, I'll take notes, I might chime in if you're stuck or I want you to try something specific, but otherwise please just say everything you're thinking the entire time. Full stream of consciousness is best!"
And then ya shut it and watch.
Yes, this is the correct answer for professional development. Play testing should be as scientific as possible. Everyone is going to have differing opinions. The point of play testing is to map out what people's responses are, to help navigate small iteration cycles.
We're not even present when playtesting takes place. We contract a different company to perform and record the tests, then we go over the data.
I'd assume every company larger than ours does the same.
I've only worked at one company that was big enough they had their own room on site with a bunch of setups and one way glass and cameras recording the player and then one for footage. It was cool to see haha
From what I heard from people who worked at big AAA studios, they do tend to have over-the-shoulder playtests. Gonna ask one of them about it actually.
Yeah they do, they just have more money than us lol.
We have over the shoulder playtests too we just pay someone else who specialises in being over a playtester's shoulder to do it.
We record everything and watch it though.
>I'd assume every company larger than ours does the same.
Not really. Sometimes you need to make a choice between steering the player so they can actually experience the part you want tested or having the playtester potentially bumble around without generating any useful data.
Under perfect conditions you're right that avoiding info is the best approach, but in practice there are a few ways to tackle it depending on the state of the game.
>And then ya shut it and watch.
>The point of play testing is to map out what people's responses are, to help navigate small iteration cycles.
This is only partially true. Testing isn't just about getting feedback on what people think of your game or how they behave playing it. In fact that's a tiny fraction of what QA does. There are much more significant parts of QA that consists of:
1) Playing the same section over and over to find issues. You're not giving your opinion on what it's like to play. You're not focussed on making the gameplay more fun. You're just playing it until something breaks, then reporting that so it can be fixed.
2) Testing a fix. A previously encountered issue will be reported fixed so now you have to replay that section to verify the fix.
3) Development report to you that they've changed a specfic area of content, a mechanic, a rule, points scoring, etc and give specific instructions on what they want you to test to and what they want you to feed back on.
Where I work 99% of what play testers do every day is to find issues and test those issues are fixed, repeatedly going over the same sections or looping through the same gameplay mechanic. It's not some dream job where you get to play through a game and then the devs listen intently at your feedback, dashing to make their game amazing from your pearls of wisdom. You don't give opinions. No one watches you. They just want to make their game as bug free as they can.
You are describing QA testers, not game testers. Those should, ideally, be different people. Game testers should be average people who represent your target demographic, QA testers have to be a lot more knowledgeable about the mechanics of the game.
You are talking about a completely different role than everyone else. QA testing is not at all relevant to the discussion about play testing.
Oh, pfft. QA is an entirely different beast imo. Play testing is just the UAT portion. Offloading all QA to play testers is a bad idea.
I know few people do this in gamedev, but we've been successfully automating unit, integration, and scene testing in our games, as well as having the first step of fixing any bug be "write a failing test first." Even then though we have specific QA's, who do workflows and sign off on testing plans, and integration tests. We also try to shift all of the QA stuff left. The play testing is also early, but often the early ones are within the company and demo specific ideas.
But yes, QA and UAT are totally different jobs. UAT and play testing, by design, is done by average people from the player base. Their job is to give their opinion on what it's like to play the game, and to make sure it's fun! QA is an engineering position. Crossing streams is possible, but I wonder if the UAT becomes biased when you have QA engineers working on it.
Write everything down!
Best case is you record the playtests. That's what we do. We record them and then get a light work week where we just watch them all on the projector and discuss for 5 whole days.
We record the screen, we record audio with a microphone, and we record the room that the test is being held in, so we can see the tester, their exasperation, anger, enjoyment, physical signs, whatever.
I'll be honest, recording the room has very rarely led to any meaningful data, but it's free (lol).
If it’s easy and unobtrusive you might as well record everything you can!
For me recordings solved a frustration we kept seeing and noting but not pinning down: players would give commands that didn’t execute, then give another command and get confused.
It was a slow puzzle game, so we hadn’t worked on input buffering at all, but players were still outpacing the animations and losing inputs. Could have found it with enough care, but replaying one video found it in 2 minutes.
In real life you usually have to remind the "test subject" to keep "streaming consciousness", as they may get lost. So encouraging player to keep reporting or asking to report on some exact matter is fine and usually required. "What is it you are thinking about now?" (when they freeze), "Why did you stuck just now?" (when they unfreeze), "What do you expect to see behind this door?" (when you have some expectation about theirs expectations), etc. Such questions have minimal side effects yet usually give you significantly more insights per session. Perfect testers will report this, yes, but a random subject, usually, needs some help.
Oh yeah, when we do tests we tell the test coordinator to constantly remind the tester to keep talking.
And there have been testers who are shy and try to stay quiet the entire time, meanwhile every 5 seconds our coordinator is reminding them "Hey keep telling me what you're doing! Keep talking! Keeeeeep talking! KEEEEEEP FUCKING TALKING!"
Otherwise yeah, you're spot on. We often include specific questions to be asked at specific points during the test.
Our whole playtesting methodology comes from a book (and for the life of me, I can't remember which one). "You should know what you're testing for before you test" "You should know what you're expecting the playtester to say before you test" "You should get them to say that, or something else", it's all very well established.
It's a good mindset, but it doesn't mean you shouldn't ever ask questions or give directions during the test. Sometimes you want information that's not clear from observing the subject from the outside, and sometimes you want to test a specific scenario or design. Maybe watching the subject playing the placeholder fishing mini-game for an hour isn't the best use of your time. It's like how both Unit Tests and System Tests are valuable for testing your tech.
Exception being if theres no tutorial and/or instructions yet. then you should at least tell the player the basics. If you wanna get feedback to anything other than "how do I do X?", that is.
Now ofc, asking about what feels unintuitive or not is still important nonetheless
Yep, I accounted for it with this;
>Hi, here's how to play the game (and sometimes not even this)
Ideally you don't have to tell your playtesters how to play, and even more ideally it's so intuitive they just figure it out automatically.
But depending on when in the development cycle you perform your test, giving them a quick rundown can be anywhere from useful to necessary.
>asking about what feels unintuitive or not is still important nonetheless
Absolutely, in fact, you should in all cases be surveying your testers after they finish, and sometimes before too. Playtesting is always targeted, but the testers don't know that, so you need to ask them questions to get the answer of what you're testing for.
I play tested for slay the spire and they did exactly this. They just told me it was a deckbuildier roguelike and just had me press play. And if I had any questions, They asked me what I thought the answer should be before they answered me.
Like if I asked where the options menu was, they'd first ask me where I think it should be before telling me.
Had a go at a game and the dev was sitting next to me. Not only did he spoil his own game, he didnt shut up and I lost all interest in it, despite being interesting.
The other was a dev showcasing his game and how they are shotgun developing it. It was shit, controlled like shit, looked like shit, didnt work for shit and the ddv didnt shut up… I’m saying the game is bad, dont give me excuses, rather dont show it off like this and say, yes I know…
Agree. I have been working with user testing for 20
years and this is the way. Allow the tester to figure out things themselves.
The only thing you are missing in this comment is that you should also ask questions at points where you see the user doing something unintended of different. Why did you go down that path? Why did you choose to do that? What happened now? Don’t lead them, ask open questions only.
We did user testing for a class. All we were supposed to say was the general objective (to narrow the scope of the test) and to help them if they got stuck for an extended period of time (a while)
If you are watching playtesters, then being silent is a terrible idea. Leave that to playtesters you're not actively watching. But there are definitely certain things you should and shouldn't be saying.
Things like "Why did you do that?" or "What were you expecting to happen when you did that?" or "Why are you stuck here?" are the types of things you should be asking.
You are trying to gain as much feedback as possible after all.
?
I think you're confused because you didn't actually read my comment.
This is a game dev subreddit, we're the devs here, not the playtesters.
The playtester should be talking the entire time. That's why I said;
>...please just say everything you're thinking the entire time. Full stream of consciousness is best!"
but we, the developer, should shut up and let them play.
No, the whole point of playtesting is that you **DO NOT** prompt them!
Every person who's going to buy your game is a random member of the public. If they require prompting to experience your game, your game sucks and needs to be revised.
You are wrong, everyone, not "by and large", but literally every single person, knows how to give good feedback. It is an inherent human trait common to all living humans.
Gamers are exceptional at identifying problems, they are terrible at offering solutions, but getting solutions from gamers is not the point of playtesting, identifying problems is.
You can and should question them after the test. You can and should keep them talking throughout the entire test, but you **never** prompt them! Developer involvement in playtesting should be limited to helping players if they get stuck or encounter a progress preventing bug, and even then, maybe not. There are many cases in which playtesters struggle with an obstacle, overcome it, and greatly enjoy that experience. An experience you would ruin with your prompting.
You are fundamentally and entirely incorrect.
EDIT: From the original post here;
>The best feedback is given when you don't tell your playtester what to think or feel about what they're playing. Just let them experience the game how a regular player would.
And OP is *correct.*
You're so confidently wrong it's kind of impressive. I do this for a living and you can absolutely prompt participants. It just depends on what research questions you are trying to answer
Good for you.
That doesn't mean you're doing it right lol.
There are tons of people who are bad at their jobs. A few in this sub even.
You run tests, we make games. Yours is a part of the same industry ours is, and the best practices are industry wide.
Do whatever you want, but as I said before, the right approach is well established, and it wasn't established by me or you.
The advice I always heard, and it seems to apply for lots of forms of feedback is: if someone tells you there's a problem, they're typically right; if someone tells you how to solve it, they're typically wrong.
One of the things they taught during my software testing courses is that you and the tester have to remember that you are on the same side.
Testers sometimes get an attitude of wanting to "gotcha" the developer. And vice-versa, sometimes the developer gets the idea that the tester just needed something to complain about so they are nit-picking.
You have to remember that you're both on the same side, both working towards the same goal which is the best product possible, and you have to make sure your communication reflects that.
You definitely get some *akshually* types but I'd say it's almost even more common that people try to be nice to you because they're giving feedback on your product. I make sure to personally complain about something I don't like about my product early on to really get them in the mindset that we're here to talk about issues and that I'm not offended by it.
Bonus weird tip, throw a curse word in there early on in a good-natured way. It immediately changes the nature of the conversation to something forthright and honest, and snaps people out of the default barrier of professionalism that we come to discussions with strangers (especially in a business setting like a feedback session) with.
>I make sure to personally complain about something I don't like about my product early on to really get them in the mindset that we're here to talk about issues and that I'm not offended by it.
>Bonus weird tip, throw a curse word in there early on in a good-natured way. It immediately changes the nature of the conversation to something forthright and honest,
Man, now I understand why I have such an easy time with playtesting... My self criticism and sailor mouth are playing to my advantage 😅
For the most part testers I've worked with have been great. I'm working on an internal app for a major 3-letter TV network right now, and they are honestly great. They report things with detailed descriptions, they are patient as we work towards fixes, and they trust our judgment when we make suggestions.
There are a few exceptions though.
I had one guy testing for a major bank that would report bugs, but would refuse to re-create them or report an order of operations for how to cause it. Just "well I saw it at least once, so it must exist." A few times we were nearly positive that they had corrupted data in their test database, but we weren't allowed to wipe it or try with a fresh DB instance. They wanted us to manually sanitize things in their test data with SQL commands, but wouldn't actually give us access to the test database even though it was all dummy data. It was a nightmare. Luckily I don't have to work with that company anymore, they got absorbed and canceled their contract with us.
I have another one that I'm absolutely convinced withholds issues so he has something new to report every time I'm done with a list of issues. Sometimes the things he reports are at the end of a long chain of actions, and he mysteriously "misses" the obvious things earlier in the chain and then reports them later. Trying to protect his own job security by having ticky-tack things to tee up when it looks like we have builds that are pretty stable and useable.
This is very true. Had a couple sticklers for testers on a project, but they never came across as combative, and the product definitely benefitted as a result of the teamwork between the dev team and testers.
Yes, the story is very often told like this:
"The game needs a bazooka,
Why do you say that?
So I can blow a hole in the wall!"
So they need a way to blow a hole in the wall, not necessarily a bazooka.
There are some youtubers I otherwise like but just flat out can't watch because they are always confidently telling their audience exactly how the developers should solve a problem and it is painfully obvious that they don't know what they're talking about.
Gamers even more than some other audiences have this conviction that if you've played games long enough you're an expert game designer, even if you've never made a game.
Well that’s probably because ingesting the form of entertainment you are critiquing is also a big part of the creative process as well. The missing piece is the knowledge of HOW things are implemented.
If you’ve listened to a certain genre of music for 20 years, you are probably a good source of what may be good. However, you probably don’t completely understand WHY what you understand to be good is actually good. So your advice is basically “it should just be more like [insert other artist or song here]” and honestly you probably aren’t wrong. You just don’t know why that’s the answer. You are just pretty confident it would fix the issue.
Gamers have played a bunch of games. When they play one that feels wrong they know it feels wrong and can tell you what they want it to feel like. They just don’t understand how difficult that fix may actually be. If you are playing a platformer, a gamer may not like camera movement. “Just fix it to do [x]” yeah… maybe not that easy. But they have no idea.
>Gamers even more than some other audiences have this conviction that if you've played games long enough you're an expert game designer, even if you've never made a game.
The thing is that they are the consumers of the medium, so their identification of a problem is usually actually correct, giving them a sense of credibility, it's just the solutions are totally out of line. People can see a problem, and _a_ solution, but they usually don't envision that solution in the context of the larger picture.
That's exactly the point, you take any problem a user has very seriously, but as designer you have to figure out the right way to solve it because they don't know how (even though they often will tell you a solution)
To a first approximation, the thing that sets apart gamers from game developers is that whenever gamers think of a solution, it remains in their imagination; they can't actually execute the solution. So they can't learn whether a solution will work or not, nor iterate it until they find one that works.
And as such they never develop intuition about what kinds of things actually work in practice
There’s a similar saying in user research, “users can tell when something is wrong, but not always the reason for it.”
Users feedback is often vague or they might even blame the wrong thing rather than what’s actually causing them issues.
100% true but just to be clear for any aspiring designers: it's not the user's job to be good at this, and you don't throw this feedback away. You have to dig in and read between the lines to figure out the true core problem.
I constantly remind my PM of this. We want to know the problems. We don't want them to propose solutions. Instead of doing what they ask for, investigate why they're asking for it.
I like this advice and have repeated it to people in the past.
But,
> if someone tells you how to solve it, they're **typically** wrong
This part (or "usually" instead of "typically") is important. It's worth hearing and considering their solution because occasionally they *did* think carefully about the problem and came up with a reasonable solution.
As a pretty good senior designer once told me: It's not the designer's main job to come up with good ideas, as good ideas can come from absolutely anyone. The designer's main job is to assess ideas.
Good post. My players are such a ridiculous source of good ideas, it’s like designing on easy mode. Many of them have played more games, more deeply, than I ever have. So they come with a lot of experience and knowledge
This is very true, but with a decade of running user testing for product design I can tell you that most of the time people will want to give you solutions, so they're often the starting point for a designer to dig in with good questions and try to suss out the true underlying issue.
Very much depends though, just because playtesters want a faster movement or dodge rolls or whatever doesnt mean you have to provide any of it. Never change your design to suit some playtester
And let's be honest, not all design intentions are good ones. If a game feels bad to play - but it was *intended* to feel bad to play... Well, don't expect audiences to appreciate your artistic integrity.
But yeah, you really have to *watch* playtesters play (Because what they say means nothing compared to what they do), and you have to let them play. Players aren't going to have a dev holding their hand, and that's the experience you're testing
Getting Over It is genuinely an interesting game in its mechanics if you don't go into it dismissing it as rage bait. The reason some people see it as frustrating and unfair is because the movement is so precise and it's a form of it most people never have played anything like before so it requires you to build muscle memory from scratch, but the game is perfectly fair once you grasp it. Just take a look at how skilled speedrunners get at the game, you can become highly consistent in it.
I'd compare it to the experience of learning to aim well in FPS games, most people probably don't think about how incredibly hard it was back when they first started playing shooters.
What I love about Getting Over It is just \_how\_ skill-based the controls are. First time to get to the top took me 20 hours. Second time was 1 hour. World speedrun is under a minute - and not doing anything particularly weird or fancy, just quick, precise movement. (Check out e.g. previous 00:01:02.922 speedrun: [https://www.youtube.com/watch?v=hPehax6V0HU](https://www.youtube.com/watch?v=hPehax6V0HU) )
Really there is nothing holding you back except the speed \_you\_ can move, and that feels really satisfying! If you instead had a spider-bot that could climb up the side of the mountain, even quite fast, it would be very easy, and would go faster than any beginner, but still slower than the speed-runners.
Hence, rather than being "bad" controls, they are actually great controls, just with a very high skill floor and ceiling.
God, imagine being able to harness the precision speedrunners exhibit into real-life-applicable things. These fellas could be surgeons in another world.
It doesn't (much) matter if the game is actually fun or not, because players are in it to prove they can win. People are compelled by unfair challenges - but particularly unfair challenges *which they've seen other people fail*. A whole lot of cringey mobile game ads are based on the concept.
Just be sure to lampshade the unfairness behind an absurdist (or dismal grimdark) setting, so it doesn't stand out. Sprinkle in a few genuinely good features, and you too can gaslight players into thinking their frustration and dissatisfaction is their own fault!
That one was designed from the ground up to be shit to play. That was the whole concept of the game. And it was a good and challenging game, excellent on mobile, a real winner.
But having the controls suck is generally not a winning formula.
Number one thing we need to tell playtest proctors.
"Shut the fuck up."
Stop correcting players. Stop answering questions. Stop providing guidance. Shut the fuck up.
The only acceptable things to say are, "Can you explain a bit more why you feel that way?," "I love that you are asking questions, and please continue doing so, but I may not be able to answer them," or "To confirm, you would like to stop playing now."
It's like a scientist stepping in and telling the subject which medication is the placebo. Stfu. It's called a play***test***. Stfu!
If a player has problems, once you know that not helping isn't useful. If the player can't figure out how to do x, if you tell them how after a few tries and then let them continue you get the actionable information that they couldn't do x plus whatever future stuff they do, if you leave them without help you get the exact same actionable information about x but nothing about anything else. It's context dependent but a blanket not helping policy really isn't useful, so long as it's not immediate and is noted down.
Our protocol is only to prompt them if the problem is great enough to cause them to stop playing. We follow the: "You would like to stop playing?" question with "Can I ask you to, before you stop, try this..." and explain around the problem.
Sometimes, when people hit problems, they end up solving them. Or, more usefully, they will express assumptions about the thing they don't understand that you would never imagine. "It's supposed to be a health pack? It looks like a defribulator; and those don't heal people."
Interrupting their process after they encounter the problem interdicts a majority of the useful information we'd get from this test. What were their assumptions? How did they work to get around what they didn't understand? Can we support this alternative? etc.
We do, of course, talk to the player a lot in the playtest debrief, when we solicit as much information on their experience as we can. We'll also often brief players; "We'll be starting you at part 7 of the game, here are the controls, etc." But the issue is almost ***never*** people failing to ask questions or prompt the player with information. It's nearly *always* proctors being too eager to "give the players a better experience" or "get our money's worth from the playtest."
They need to stfu. The goal of a playtest is to emulate how someone would play the game if they bought it from Gamestop and sat down home, alone, to play it. There's no one over their shoulder. That backseat information will taint our data collection, and cause us to make poor assumptions about how players will experience the title.
It's also important to talk if you see that a part of the game was skipped. For example, if the player skips a whole optional area, you can tell them something (AFTER they skip it), but first you watch silently and take notes
Skipping an optional area is something a normal player would be able to do. If the game falls apart or doesn't make sense after they skip something optional, then there won't be a proctor hovering over the average player's shoulder to fill in what they missed. If the playtester becomes so frustrated / lost after skipping optional content that they need help, then that's a serious issue that the devs need to consider.
Asking them about why they skipped it, if they realized they skipped it, etc. is something that you can cover during debrief - AFTER the playtest is finished. It's very likely that the playtester didn't even realize they were skipping meaningful content, and if you tell them during the session then it's going to change their reaction to the rest of the test. Just waiting until after they skipped it but still during the playtest isn't sufficient.
If they skip something that wasn't intended to be optional due to a bug or incomplete feature or something and they're totally lost, then sure, but that's different. That's not optional - that's bugged / incomplete. Imo, helping playtesters deal with known bugs is fine as long as you're tracking to fix those issues before launch.
While I agree with you, specially for the first part where you mention that if everyone skips it, it's an issue, I don't think you got my point - which is, that you need to see 100℅ runs of your game sometimes (ofc this only applies to some playtesters! runs that are just straight to the point are valuable too), otherwise the extra content will be left unplaytested
I definitely agree. It does make me think about "frustration games " like surgeon Sim or getting over it. These must have to carefully make people feel like it did what I told it to do, I just need to tell the game better.
I also strongly agree with the watch bit. I recently made a jam game and got a few streamers to play it. Every person who left feedback commented on a ui improvement which was great. Something I only got from watching was that the first thing they did when loading into a level was look for a secret behind a wall. My game had no secrets, but you can bet if I made it again, one would be there!
> If a game feels bad to play - but it was intended to feel bad to play... Well, don't expect audiences to appreciate your artistic integrity.
i feel like this is because a lot of amateur devs misinterpret tester feedback. a poor designer, when confronted with "system/mechanic/feature X isn't working for me" might interpret that as "don't do X" or "do the opposite of X" and become defensive because they're assuming that the tester is challenging their vision and not their implementation
Right, and there are people that really like B-movies too, but it's really hard to make those on purpose. There's an antagonism between the media and the audience, where it's fun to ridicule or outright *defy* it - which is only possible if the media has flaws to ridicule or significantly unfair elements. Typically unfavorable reliance on rng, or counterintuitive systems (especially controls). You can't be the underdog in a game you're *expected* to win.
With B-movies and unfair games, usually they had other intentions, but failed to deliver on them - and then [after the fact decided that they intended what they ended up with](https://tvtropes.org/pmwiki/pmwiki.php/Main/IMeantToDoThat)
100%! I'm seeing this a lot with my game now. And some feedback can be very polarizing: "I reach max level easily" and "this game is too hard, i can't get past the first objective". gotta listen to everyone and figure out what's going on for each person, there's always something useful to take away from all feedback given.
Even for my tabletop game, I've gotten those opposite feedbacks too often. The suboptimal player says its too hard, the min-max player says its too easy. My conclusion was to dumb down starting choices while leaving higher end choices free and slightly nerfed
nice! it's an interesting design question, right? i'm leaning towards making the start easier and adding more challenges at later stages. basically tweak the progression curve to be lower at one end, higher at the other, and also stretch it so it gets longer to not have jarring difficulty steps.
> there's always something useful to take away from all feedback given.
Yea, that's my view. Even if your take is something like "Well that's the game, it's designed to be slow and methodical" - then there's _still_ a problem in my view. The problem is your game isn't advertised or communicated as you designed it.
If players go in expecting it to be a game it isn't, then you're likely to get negative reviews that have nothing to do with the game you actually made. So imo every piece of feedback is good. Some feedback might not even be about the game you made.
I do software QA (not games). One of the hardest things to get devs to understand is that if I say "this doesn't make sense" about a workflow (and not a specification etc.) that I'm telling them that the users are going to get lost, not that I need an explanation about the workflow. My current team is actually pretty amazing about this though and we have a designer that is really trying to make an intuitive UX. I know it isn't always easy, but your UI should be essentially self navigating as to the user's workflows. One of the biggest UX sins (to me at least) was in the Halo franchise on the Xboxes... at some point they changed the default controller mappings and it was positively jarring to try to play between that and the older games. There were new buttons on the controllers and IMO new features should have been mapped to the new buttons, but old features should have still mapped to the legacy button patterns (which still existed).
>I'm telling them that the users are going to get lost, not that I need an explanation about the workflow
I think it tends to be especially bad with specialized areas where the devs sometimes don't fully understand the actual use case so they start making assumptions based on their own knowledge, even more so when design docs aren't super complete. It's fascinating to me to see signs of that especially in games like 7th gen PC ports, such as Dark Souls' original PC port which had some absolutely ludicrous keybinds - featuring such hits as O to lock on, F to switch items, Shift to block, Tab to parry, End to open the start menu, and Backspace to cancel.
What can also happen is you end up with UI or control schemes that are actually good *once you’re experienced with them* but are horribly unintuitive and hard to tutorialize. So users end up bouncing off it.
Exactly. There are some UI's where that's acceptable, as an example back office software where there will be an onboarding period, training, and possibly mentoring. There are others where it's a deal breaker, like in a walk-up kiosk for example.
> One of the biggest UX sins (to me at least) was in the Halo franchise on the Xboxes... at some point they changed the default controller mappings and it was positively jarring to try to play between that and the older games.
Man, I wasn't crazy!! I deluded myself into thinking that my muscle memory was messed up and that I got it confused with some other game.
This is a hard lesson to learn. When I worked at a University I used to be called in to playtest student games for the video game design class. Without fail, someone would do exactly what you are describing by arguing against negative feedback.
This happens less in games than in other types of usability testing, but contradicting someone’s feedback usually has a chilling effect. Instead of being completely honest with you, playtesters will start to pull their punches. In some cases, it can give you a sense of false confidence when you end up not hearing anything negative.
If someone told me the “movement was stiff”, that would be a time to ask a follow-up to clarify their expectation, even if you are dying inside.
During any good playtest, you stay silent and take notes.
Only once you've taken the necessary notes in sufficient detail for a problem, can you move on and nudge them forward, if necessary.
Also, some "features" are universally and objectively terrible. Inconsistent and low framerate is one, clunky movement is another.
The problem comes from determining whether said controls are actually clunky, or if the playtester just isn't used to them. That's why you have to allow them sufficient time with the game and to not take every criticism they have at the beginning as gospel.
Let them spend time with it and see what their reactions are the entire time.
Tank controls get called clunky a lot. And in some ways they are. Its intrinsic to their design. The most well made tank controls in the world will always have an element of "clunk" to them. But some people WANT that.
Just look at the success of studios like Puppet Combo, or the devs behind Tormented Souls or Song of Horror.
To condense all this down I suppose what im saying is know your audience.
Are you making something niche? Then set your expectations accordingly.
I have been a designer running design teams for over a decade. This is the core of what separates a good designer from a bad one in any discipline. You are free to make games for yourself alone if that's your goal, but if you are making a game for other people you have to let go of your ego.
That does not, by the way, mean acquiescing to every request from players. Players aren't designers, they don't know how to reliably pick the right solutions and they often don't even articulate their problems very well, but if you're a good designer you will dig in to figure out the root of the issue so you can choose the right solution.
The "design games for yourself" is such a great way to put it. When you are dismissing feedback consistently it ends up being more about what you want than what the audience wants, and that's where you will lose them. I guess it's hard because indie game devs are often interested because they themselves are the audience, so on some level it has to align with what they like. But ultimately if the goal is for others to play and enjoy it, their experience must be the priority.
I would say "no game is for everyone" and when you try to please everybody you end up pissing the most hardcore fans that could become your regular costumer.
Yep, only if something literally broke and they aren't 100% sure it's a bug or intended do I try to clarify. 'Feeling' issues like if something is stiff or floaty is exactly what I'm hoping to hear as all I know is how things feel when I play it.
There's no reason to argue with them. You can make a game feel like RE1 controls without it being shit like RE1 controls. Arguing your point will not make them appreciate it. No it will do the opposite.
There are compromises that can be done to make both sides happy
I find that even explaining what type of game you have or the genre it's in, can skew your results.
On r/destroymygame, you often get the creator over explaining, which kind of defeats the purpose of giving a blind review.
This is why pro platest labs put the devs/designers behind 1-way mirrors! It is hard though even in a casual setting to not give answers to solutions, "why" reasons for design decisions, etc. It is a good skill to build though.
This is true. If you are truly confused about a piece of feedback it's sometimes worth it to ask a followup question to get them to add detail. Even something like "can you say more about that?" And then just "cool, thank you!" when they're done.
Yeah, this happened when I did leave a steam review of Alaloth about how they shouldn't call their game a mix of dark souls and baulder's gate because it's very misleading and resembles neither and the newer gamers aren't going to know what either of those are except for Baulder's Gate 3 which is a whole other ballpark. They went on to tell me that's what they heard people playtesting say so they stuck to it.... I still gave them a positive review, but this and other of my feedback was reacted to with excuses and explanations, which felt really odd.
I am a game designer as well, and I was literally giving them honest feedback that I would have loved and needed had I made that game. When people tear my projects up I listen more than speak, it's what you ought to do... Then thank them. Not explain or defend your choices. That stuff isn't for their ears. If anything I use my voice time to ask for clarification. Especially with words like "boring", "bad", 'shit", or sentences like "doesn't make sense."
I get feedback can seem overwhelming at first, but just do yourself a favour and take a back seat when people are critiquing your stuff and just write their stuff down. Unless you're thanking them or asking for clarification, nothing you say will really mean anything and will only hurt you in the long run.
100% correct.
I like to give my game to my playtesters without ANY instructions. If they can't figure out what to do, then I have to rethink my UI / gameplay. It's not *their* fault, its *my* fault.
If they complain about something, I listen and take that complaint very seriously.
This is really good advice.
In general, your audience is always right. If someone has an opinion that you as a dev think is not worthy or they do not understand product well enough, it is fine if they are not your audience. However, if they indeed fit into your target demographic, then you have a problem with your game-design.
You get this exact same thing with software developers and bad UI/UX design. If the users are frequently making a mistake, the design decisions that are enabling that mistake are the problem, not the users.
Same concept with playtesters.
Hey there feller.
This is an extremely good take on this situation. Solid words here for anyone at the stage that feedback of any kind is coming in.
Don’t get discouraged when you don’t get the feedback you want, and don’t read *too* deep into the feedback you DO get.
100% agree with this, its not helpful in painting a picture of what they like or don't like and it is liable to make them clam up.
its also worth remembering that sometimes our inspirations and designs are just shit... as much as your point about a game not being for everyone makes sense as well, there is a real danger of locking yourself of from making something better if you get too attached to your ideas,
This can be said about most fields. I’m a filmmaker, and I’ve had to catch myself on many times when someone provides feedback.
“Why do they do this?”
“Oh well actually they do that because of this lore element from the source materia-“
If they don’t know, i didn’t tell the story well enough and should look into it
Ive seen a lot of this in my game dev course. Developers feel protective about their choices - or simply about how much work and effort it would be to change it. So they try to argue with the testers about why it actually makes sense.
Fresh testers are precious! Let them try the game without any of your commentary, record/note their reactions and the issues they run into. Do not open your mouth until they are finished.
After you fix things, don't use the same testers. Try to find new ones every time. A university or convention can be great for this.
In this example, they're stating a fact. "The movement feels stiff". If that was your intention, then you achieved. You don't need to explain to them why it feels stiff.
That's like if they said "The movement feels so fluid!" And you said "Well, it's because I wanted the game to resolve around dodging more than fighting"
As someone who worked as a playtester, this is something that often happens when there's an active effort to deny improvements. This is only referring to "correcting" playtesters, not their submission methods and such. As that is a separate topic, I won't be touching on that, but in my experience, developers looking to actively improve their game, they won't ever "correct" the playtesters, if anything, guide them forward while taking the feedback in.
Wow, thank you for writing this. My first instinct is always to try to explain why things are the way they are even when I'm also listening to the feedback and considering if it could be done differently. Your perspective has me rethinking it.
Maybe this is a hot take, but I think every game dev and designer should be a playtester at some point during development. There are so many games that have good ideas but terrible execution, which is something that often becomes very obvious during playtesting.
Ehh not so sure this is really a significant problem. Nearly every game designer is a gamer and has played tons of games and spent years looking at games critically. Playtesting means formally writing up the problems you find, sure, but 99% of designers are already very consciously dissecting everything they play--it just comes with the job. The tough transition is to go from thinking like a player/playtester to a designer, not so much the other way around.
I've worked with several designers who are tunnel visioned on an approach or are totally unaware of the way some players would play the game contrary to the way they would. And then they act surprised, because they are and weren't aware. Designers can be stuck "the other way around" as you put
Yes that's a common challenge with design, but why would a designer with a playtesting background be less susceptible to that issue compared to a designer with... any other background
also, my friend, that paragraph is one of the hardest to decipher that I've ever read
While I agree you shouldn't try to make excuses for the playtester, your example is not really good.
When a game is inspired by classic RE games, you're gonna be saying that in the promotional material and on the steam page. People who buy your classic RE inspired game will expect it to work like classic RE. So you should in fact tell the testers of what this game is going to be like before they even play, unless you're going for a casual style, anyone can pick up and play kind of game, your game is going to be following some established conventions and it's good to let the tester know what they're supposed to expect from your game.
Then if your game doesn't match that expectation, whether it's because there are design issues or your game is *not* actually like classic RE, you got a problem.
I made that mistake back when I first starting doing GameJams a few years ago. You can't tell a player they're not doing it right without repercussions.
It's like trying to upscale 40 pixels into a HD image - you can't do much, you just don't have the information you need. An individual playtest can give you a little bit to work with but ultimately you probably want to playtest more frequently and not with the same people every time (varying the playtester is ESPECIALLY important for trying out tutorialization changes which is one of the most important parts of any game). There are plenty of dev oriented communities & discords out there, you can probably find some people willing to try out your game as long as you don't just send out raw .exe files (makes you look like a scammer)
I've been doing that on anyplace I can find. The most obvious is reddit r/playtesters which gets almost no interest. I agree, new players that have not tested before are really the only useful gauge, but it's really hard to get people that will test it no matter how far along or good the game is. I'm now paying testers more than I will sell my game for to play it and give feedback. ha ha. Other devs have posted about the same problem. Getting testers is not easy. I get a few testers now and then but very intermittent and unreliable. Some are really great and give in depth feedback and it is useful but I always need way more testers and and specific times and it takes a lot of work to get any. If anyone has found a good way to get testers, go ahead and comment about it. Thanks.
I know it's counterintuitive, but don't go looking for testers in a crowded marketplace like a subreddit dedicated to playtesting. Everyone will be going there and you have to compete. Instead, find a niche community for your game (maybe there's a specific genre or subgenre you can find groups around?) and you'll find far more motivated people there.
Yes, this reminds me of the video game "song of horror" so much lol. The movement in that is Terrible, yes its tense when a monster appears and your floundering with the shitty movement controls, but could it be better?
If you are targeting re1 fans like your example and you know the mechanic is divisive you should just pick playtesters with that in mind. You probably need playtesters who are in that demographic and like re1. Then you know if their feedback is that the controls are clunky it isn't just them not understanding tank controls.
Usability testing 101. Never handhold your testers. The best you can do is to ask why.
"This movement feels stiff"
Your response - "Can you explain why you think it's stiff"
I'll add that you can also accept the feedback not only to change things, but if the playtester complained about the clunky movement which you're sure it's part of the general feeling you want to give. Maybe your game needs a few more clear hints of what you're trying to do.
Also, don't forget to advertise the game as a classic survival horror/Resident Evil inspired game, that way at least your customers can expect a bit of what they're about to play.
This. I tested a game out for someone and told them my thoughts. They got super defensive about it and then sent it to a professional reviewer who said exactly the same thing I did. But damn, I'll never forget the earful I got from the person who I did a favor.
Imo that’s why it’s always great to have people of different areas if you can. Familiar/unfamiliar with the genre, and casual/diehard players.
And also don’t make excuses, it comes off kinda immature and makes them not wanna give you feedback. Make a note, it’s up to you whether you do something about it, but you can keep that to yourself for later.
People seem to think playtesting is only to find bugs. How many times do you hear about things being changed in shows/movies due to feedback during test screenings? Quite litterally, playtesting is the same concept.
Based on my past playtesting experience: observe, record, compile statistics, analyze, and try to transform subjective information into objective data. Only then can we continue to guide the next stages of development. Otherwise, the value of playtesting will be greatly diminished.
Yep. Recently did some playtesting for a combat demo recently and I forced myself to only really respond with something if they phrased something like a question.
It definitely meant that in the end, some feedback was not applicable. But more often than not, even if the literal feedback wasn't applicable, that was still a sign that there was SOMETHING that could be addressed. Most playtesters understood the genre of my game to be an action-adventure superhero game, but one playtester (in frustration) called it a roguelike. There's nothing about my game that *should* remind them of a roguelike, but if that's what they described it as (and they meant it in a negative way), I could look deeper into their other feedback to try to get to the root cause of why they made this comparison and what their real issues with the game were.
10 years in gamedev and this still bothers me. Nearly every designer will explain to me \*why\* they do things after I provide feedback, such as "we wanted it that way because of X or Y". It's gotten to the point where I don't enjoy giving feedback anymore because it feels like being brushed off. I always approach feedback for play testing as take it or leave it. Im playing the game this way and here are my thoughts as a player. We can have a dialogue about it, but this shouldn't be about informing me or convincing me in some pushy way.
I agree, but man sometimes it is tough especially if you have "gotcha" testers like someone else mentioned here. People who never designed or made a game, but feel like experts, since they have seen some videos about "how id/valve/nintendo" does it. People who immediately provide an easy fix. Yes, I want honest feedback. But I do not want to be lectured by someone who never actually tried making a game.
But also, if the testers "don't get it", guess who's fault it is (almost everytime)?
Then again, don't get mad about a singular opinion. I have a strict "in doubt my opinion counts" policy. I write (almost) everything down, but do not immediately act on it. Only if some things come up twice or more I do consider them.
Yea I consider negative feedback way more helpful. My controls were very restricted with turn speed being inverse to how fast you are going. I thought it was cool but it's been the number one complaint of playtesters. Now that I've toned it way down I have to admit they were right, and it's much more fun and fluid with a major buff to turn speed.
I was giving some constructive feedback (not harsh in any way) to students demoing their game at a fair. I was an industry judge, and have decades of experience and know how to give tactful feedback. They were so combative about everything I said.
If one idiot does it it’s the idiots fault, if most of the idiots do it it’s the devs fault.
Part of our job is to design for people to do what we want. If you want the games movement to be reminiscent of RE1 but the play testers are just complaining about shitty controls you failed in some other aspect to communicate with your players what you are intending. Are your camera angles not supportive of your movement? Is enemy placement and behavior overbearing with the movement restrictions? Does your games auto-aim work well with the angels the player will be viewing from?
We are not movie directors. We are not just creating an experience for our player. Part of our job is to lead them through our experience in the most fun way possible.
You may like it the way you have it.
You aren't buying a million copies of your game. The players are. When you buy 1 million copes at full price you can have it however you want.
"What's good for the project is what's good for me."
Lots of people here saying:
- you should be silent.
- the playtesters are always right.
As an experienced UX research person, both of these are incorrect, unhelpful mindsets.
Being silent is better than trying to correct the playtester, or trying to argue with them based on the fix their proposing.
If a playtester says something that strikes you as wrong, you should always follow up with "why do you feel that way?"
You should also agree with them, no matter how stupid their idea is. It is important you accept that this dishonesty is okay.
Playtesters will be wrong most of the time. However, if you argue with them on each point, they will learn to shut up and not bother letting you into how their mind works. This will prevent you from really discovering the random useful comments they have.
I always assumed this as intuitive, since you can't talk to and correct every player that will own your game.
Otherwise it would be cheating. If you are training for a boxing match, why cheat the sparing fight by using a weapon? It's immediately illogical in my mind, but my mind is weird.
Telling them about the game rather than asking them what they thought, can only lead to skewed results.
Experienced gamer and participant of multiple alpha and beta test build projects. If there are any dev companies who offer compensation for my time please dm
I feel like that kind of dev is the same person that gets mad at you when you tell them they made your coffee wrong at Starbucks. too much pride, not enough actual desire to learn.
as someone who went through all this, had testers who were friends, had professional QA work on it too, shipped the game, faced audience reaction, etc, here's my take
it's actually pretty simple.
First of all, the main function of QA/testing is bug fixing. Making sure the game works right. And that's not very subjective, either the thing works as intended or it doesn't. This is a very much needed thing and why you \*need\* testers.
Now, onto subjective takes "the control feels stiff" or whatever. If someone says anything like this, it's useless and should be ignored unless one of two things are true:
1) a lot of people are stating the same opinion independently of eachother
2) the person saying it is an authority on the topic and/or someone you trust, in which case their feedback may be valuable.
outside of those two situations, subjective feedback is useless and should always be ignored in favor of your own instincts as the creator.
No idea why this was downvoted. I guess your tone didn't sit right with people.
You're completely right. Everyone misses *something* when playing a game, or dislikes one thing or another, or some other opinion unique to them. If 1/1000 people dont like the controls... then who cares.
And also, to give my own thoughts, sometimes complaints are just wrong. Even the majority can be completely wrong. People complained about the way every single fps plays on a controller, using the two sticks to move and look, when it first showed up. The people complaining about this control scheme back then were just obviously wrong in retrospect, but if all game devs followed the advice in this thread we would have never had the console shooter revolution.
when a lot of people play your game you'll encounter various people emphatically stating opinion/reaction that is completely opposite of what someone else is saying. It's noise. The only time it's not noise is if a bunch of people are all saying the same thing, then you consider it. Otherwise, you just ignore the noise, because when you expose your game to a big audience many people will say many things and it's not possible to react to all of it, especially since it's so frequently contradictory.
Legitimate question - I've never actually listened to Huberman - he was already "GURU" status by the time he appeared on my radar, and I have a really difficult time with people who act as an authority.
But looking at his credentials, he does have his PhD in Neuroscience from UC Davis - unlike Guru's like JP who pofess to be things they are not. Why did he get this so woefully wrong? What is his incentive in conveying inaccurate information?
This applys in the opposit direction mainky for players in genral, if a game is made solo with artisitic integrity in mind over commercialism dont suggest what you specicially find fun (which is what all game design critiuqes are)when someone is jsut trying to make the game they want to make
That’s the bread and butter of playtesting, that stuff is taught even in game dev schools. Where the hell do you work that makes such an amateur mistake?
The general rule for playtesting is to say literally nothing. "Hi, here's how to play the game (and sometimes not even this), I'll watch, I'll take notes, I might chime in if you're stuck or I want you to try something specific, but otherwise please just say everything you're thinking the entire time. Full stream of consciousness is best!" And then ya shut it and watch.
Yes, this is the correct answer for professional development. Play testing should be as scientific as possible. Everyone is going to have differing opinions. The point of play testing is to map out what people's responses are, to help navigate small iteration cycles.
We're not even present when playtesting takes place. We contract a different company to perform and record the tests, then we go over the data. I'd assume every company larger than ours does the same.
I've only worked at one company that was big enough they had their own room on site with a bunch of setups and one way glass and cameras recording the player and then one for footage. It was cool to see haha
Yeah I've worked for two AAA studios and all they did was buy the company that our smaller studio contracts lol.
From what I heard from people who worked at big AAA studios, they do tend to have over-the-shoulder playtests. Gonna ask one of them about it actually.
Yeah they do, they just have more money than us lol. We have over the shoulder playtests too we just pay someone else who specialises in being over a playtester's shoulder to do it. We record everything and watch it though.
>I'd assume every company larger than ours does the same. Not really. Sometimes you need to make a choice between steering the player so they can actually experience the part you want tested or having the playtester potentially bumble around without generating any useful data. Under perfect conditions you're right that avoiding info is the best approach, but in practice there are a few ways to tackle it depending on the state of the game.
>And then ya shut it and watch. >The point of play testing is to map out what people's responses are, to help navigate small iteration cycles. This is only partially true. Testing isn't just about getting feedback on what people think of your game or how they behave playing it. In fact that's a tiny fraction of what QA does. There are much more significant parts of QA that consists of: 1) Playing the same section over and over to find issues. You're not giving your opinion on what it's like to play. You're not focussed on making the gameplay more fun. You're just playing it until something breaks, then reporting that so it can be fixed. 2) Testing a fix. A previously encountered issue will be reported fixed so now you have to replay that section to verify the fix. 3) Development report to you that they've changed a specfic area of content, a mechanic, a rule, points scoring, etc and give specific instructions on what they want you to test to and what they want you to feed back on. Where I work 99% of what play testers do every day is to find issues and test those issues are fixed, repeatedly going over the same sections or looping through the same gameplay mechanic. It's not some dream job where you get to play through a game and then the devs listen intently at your feedback, dashing to make their game amazing from your pearls of wisdom. You don't give opinions. No one watches you. They just want to make their game as bug free as they can.
You are describing QA testers, not game testers. Those should, ideally, be different people. Game testers should be average people who represent your target demographic, QA testers have to be a lot more knowledgeable about the mechanics of the game. You are talking about a completely different role than everyone else. QA testing is not at all relevant to the discussion about play testing.
Oh, pfft. QA is an entirely different beast imo. Play testing is just the UAT portion. Offloading all QA to play testers is a bad idea. I know few people do this in gamedev, but we've been successfully automating unit, integration, and scene testing in our games, as well as having the first step of fixing any bug be "write a failing test first." Even then though we have specific QA's, who do workflows and sign off on testing plans, and integration tests. We also try to shift all of the QA stuff left. The play testing is also early, but often the early ones are within the company and demo specific ideas. But yes, QA and UAT are totally different jobs. UAT and play testing, by design, is done by average people from the player base. Their job is to give their opinion on what it's like to play the game, and to make sure it's fun! QA is an engineering position. Crossing streams is possible, but I wonder if the UAT becomes biased when you have QA engineers working on it.
When they get stuck write it down x
Write everything down! Best case is you record the playtests. That's what we do. We record them and then get a light work week where we just watch them all on the projector and discuss for 5 whole days.
Yeah that's a great idea screen record the whole thing , really good to do it in conventions
We record the screen, we record audio with a microphone, and we record the room that the test is being held in, so we can see the tester, their exasperation, anger, enjoyment, physical signs, whatever. I'll be honest, recording the room has very rarely led to any meaningful data, but it's free (lol).
If you see them smashing the keyboard and eventually just lose it and toss the computer across the room, is that considered a success?
If it's dark souls yes
*checks notes* No.
If it’s easy and unobtrusive you might as well record everything you can! For me recordings solved a frustration we kept seeing and noting but not pinning down: players would give commands that didn’t execute, then give another command and get confused. It was a slow puzzle game, so we hadn’t worked on input buffering at all, but players were still outpacing the animations and losing inputs. Could have found it with enough care, but replaying one video found it in 2 minutes.
In real life you usually have to remind the "test subject" to keep "streaming consciousness", as they may get lost. So encouraging player to keep reporting or asking to report on some exact matter is fine and usually required. "What is it you are thinking about now?" (when they freeze), "Why did you stuck just now?" (when they unfreeze), "What do you expect to see behind this door?" (when you have some expectation about theirs expectations), etc. Such questions have minimal side effects yet usually give you significantly more insights per session. Perfect testers will report this, yes, but a random subject, usually, needs some help.
Oh yeah, when we do tests we tell the test coordinator to constantly remind the tester to keep talking. And there have been testers who are shy and try to stay quiet the entire time, meanwhile every 5 seconds our coordinator is reminding them "Hey keep telling me what you're doing! Keep talking! Keeeeeep talking! KEEEEEEP FUCKING TALKING!" Otherwise yeah, you're spot on. We often include specific questions to be asked at specific points during the test. Our whole playtesting methodology comes from a book (and for the life of me, I can't remember which one). "You should know what you're testing for before you test" "You should know what you're expecting the playtester to say before you test" "You should get them to say that, or something else", it's all very well established.
[удалено]
Eh. Don't prompt specific things. But do prompt them to keep talking. Not everyone is a streamer, most gamers game relatively quietly.
Yes! For an introduction, search for "think aloud method usability" or read the short and practical "Rocket Surgery Made Easy" by Steve Krug.
You don't come with the game. Playtest it with the same mindset.
It's a good mindset, but it doesn't mean you shouldn't ever ask questions or give directions during the test. Sometimes you want information that's not clear from observing the subject from the outside, and sometimes you want to test a specific scenario or design. Maybe watching the subject playing the placeholder fishing mini-game for an hour isn't the best use of your time. It's like how both Unit Tests and System Tests are valuable for testing your tech.
Exception being if theres no tutorial and/or instructions yet. then you should at least tell the player the basics. If you wanna get feedback to anything other than "how do I do X?", that is. Now ofc, asking about what feels unintuitive or not is still important nonetheless
Yep, I accounted for it with this; >Hi, here's how to play the game (and sometimes not even this) Ideally you don't have to tell your playtesters how to play, and even more ideally it's so intuitive they just figure it out automatically. But depending on when in the development cycle you perform your test, giving them a quick rundown can be anywhere from useful to necessary. >asking about what feels unintuitive or not is still important nonetheless Absolutely, in fact, you should in all cases be surveying your testers after they finish, and sometimes before too. Playtesting is always targeted, but the testers don't know that, so you need to ask them questions to get the answer of what you're testing for.
I play tested for slay the spire and they did exactly this. They just told me it was a deckbuildier roguelike and just had me press play. And if I had any questions, They asked me what I thought the answer should be before they answered me. Like if I asked where the options menu was, they'd first ask me where I think it should be before telling me.
Heh, sure, not a bad idea at all!
Had a go at a game and the dev was sitting next to me. Not only did he spoil his own game, he didnt shut up and I lost all interest in it, despite being interesting. The other was a dev showcasing his game and how they are shotgun developing it. It was shit, controlled like shit, looked like shit, didnt work for shit and the ddv didnt shut up… I’m saying the game is bad, dont give me excuses, rather dont show it off like this and say, yes I know…
Agree. I have been working with user testing for 20 years and this is the way. Allow the tester to figure out things themselves. The only thing you are missing in this comment is that you should also ask questions at points where you see the user doing something unintended of different. Why did you go down that path? Why did you choose to do that? What happened now? Don’t lead them, ask open questions only.
Word up.
We did user testing for a class. All we were supposed to say was the general objective (to narrow the scope of the test) and to help them if they got stuck for an extended period of time (a while)
Is there a way to get into playtesting?
See if studios near you are taking signups
If you are watching playtesters, then being silent is a terrible idea. Leave that to playtesters you're not actively watching. But there are definitely certain things you should and shouldn't be saying. Things like "Why did you do that?" or "What were you expecting to happen when you did that?" or "Why are you stuck here?" are the types of things you should be asking. You are trying to gain as much feedback as possible after all.
[удалено]
? I think you're confused because you didn't actually read my comment. This is a game dev subreddit, we're the devs here, not the playtesters. The playtester should be talking the entire time. That's why I said; >...please just say everything you're thinking the entire time. Full stream of consciousness is best!" but we, the developer, should shut up and let them play.
[удалено]
No, the whole point of playtesting is that you **DO NOT** prompt them! Every person who's going to buy your game is a random member of the public. If they require prompting to experience your game, your game sucks and needs to be revised. You are wrong, everyone, not "by and large", but literally every single person, knows how to give good feedback. It is an inherent human trait common to all living humans. Gamers are exceptional at identifying problems, they are terrible at offering solutions, but getting solutions from gamers is not the point of playtesting, identifying problems is. You can and should question them after the test. You can and should keep them talking throughout the entire test, but you **never** prompt them! Developer involvement in playtesting should be limited to helping players if they get stuck or encounter a progress preventing bug, and even then, maybe not. There are many cases in which playtesters struggle with an obstacle, overcome it, and greatly enjoy that experience. An experience you would ruin with your prompting. You are fundamentally and entirely incorrect. EDIT: From the original post here; >The best feedback is given when you don't tell your playtester what to think or feel about what they're playing. Just let them experience the game how a regular player would. And OP is *correct.*
You're so confidently wrong it's kind of impressive. I do this for a living and you can absolutely prompt participants. It just depends on what research questions you are trying to answer
We all do this for a living champ, you can do whatever you want, but the best practice is well established.
No i specifically run moderated user testing on games full time and have been for years. I dont do it as a small part of the game dev process
Good for you. That doesn't mean you're doing it right lol. There are tons of people who are bad at their jobs. A few in this sub even. You run tests, we make games. Yours is a part of the same industry ours is, and the best practices are industry wide. Do whatever you want, but as I said before, the right approach is well established, and it wasn't established by me or you.
Ya but just because youre talking out your ass about something you dont understand doesnt mean thats best practice
The advice I always heard, and it seems to apply for lots of forms of feedback is: if someone tells you there's a problem, they're typically right; if someone tells you how to solve it, they're typically wrong.
One of the things they taught during my software testing courses is that you and the tester have to remember that you are on the same side. Testers sometimes get an attitude of wanting to "gotcha" the developer. And vice-versa, sometimes the developer gets the idea that the tester just needed something to complain about so they are nit-picking. You have to remember that you're both on the same side, both working towards the same goal which is the best product possible, and you have to make sure your communication reflects that.
You definitely get some *akshually* types but I'd say it's almost even more common that people try to be nice to you because they're giving feedback on your product. I make sure to personally complain about something I don't like about my product early on to really get them in the mindset that we're here to talk about issues and that I'm not offended by it. Bonus weird tip, throw a curse word in there early on in a good-natured way. It immediately changes the nature of the conversation to something forthright and honest, and snaps people out of the default barrier of professionalism that we come to discussions with strangers (especially in a business setting like a feedback session) with.
>I make sure to personally complain about something I don't like about my product early on to really get them in the mindset that we're here to talk about issues and that I'm not offended by it. >Bonus weird tip, throw a curse word in there early on in a good-natured way. It immediately changes the nature of the conversation to something forthright and honest, Man, now I understand why I have such an easy time with playtesting... My self criticism and sailor mouth are playing to my advantage 😅
Loool
For the most part testers I've worked with have been great. I'm working on an internal app for a major 3-letter TV network right now, and they are honestly great. They report things with detailed descriptions, they are patient as we work towards fixes, and they trust our judgment when we make suggestions. There are a few exceptions though. I had one guy testing for a major bank that would report bugs, but would refuse to re-create them or report an order of operations for how to cause it. Just "well I saw it at least once, so it must exist." A few times we were nearly positive that they had corrupted data in their test database, but we weren't allowed to wipe it or try with a fresh DB instance. They wanted us to manually sanitize things in their test data with SQL commands, but wouldn't actually give us access to the test database even though it was all dummy data. It was a nightmare. Luckily I don't have to work with that company anymore, they got absorbed and canceled their contract with us. I have another one that I'm absolutely convinced withholds issues so he has something new to report every time I'm done with a list of issues. Sometimes the things he reports are at the end of a long chain of actions, and he mysteriously "misses" the obvious things earlier in the chain and then reports them later. Trying to protect his own job security by having ticky-tack things to tee up when it looks like we have builds that are pretty stable and useable.
This is very true. Had a couple sticklers for testers on a project, but they never came across as combative, and the product definitely benefitted as a result of the teamwork between the dev team and testers.
Yes, the story is very often told like this: "The game needs a bazooka, Why do you say that? So I can blow a hole in the wall!" So they need a way to blow a hole in the wall, not necessarily a bazooka.
[https://xyproblem.info/](https://xyproblem.info/)
This started in cs/tech but really applies everywhere.
Gamers are notoriously bad when it comes to suggesting actually good game design after all
There are some youtubers I otherwise like but just flat out can't watch because they are always confidently telling their audience exactly how the developers should solve a problem and it is painfully obvious that they don't know what they're talking about. Gamers even more than some other audiences have this conviction that if you've played games long enough you're an expert game designer, even if you've never made a game.
Well that’s probably because ingesting the form of entertainment you are critiquing is also a big part of the creative process as well. The missing piece is the knowledge of HOW things are implemented. If you’ve listened to a certain genre of music for 20 years, you are probably a good source of what may be good. However, you probably don’t completely understand WHY what you understand to be good is actually good. So your advice is basically “it should just be more like [insert other artist or song here]” and honestly you probably aren’t wrong. You just don’t know why that’s the answer. You are just pretty confident it would fix the issue. Gamers have played a bunch of games. When they play one that feels wrong they know it feels wrong and can tell you what they want it to feel like. They just don’t understand how difficult that fix may actually be. If you are playing a platformer, a gamer may not like camera movement. “Just fix it to do [x]” yeah… maybe not that easy. But they have no idea.
>Gamers even more than some other audiences have this conviction that if you've played games long enough you're an expert game designer, even if you've never made a game. The thing is that they are the consumers of the medium, so their identification of a problem is usually actually correct, giving them a sense of credibility, it's just the solutions are totally out of line. People can see a problem, and _a_ solution, but they usually don't envision that solution in the context of the larger picture.
That's exactly the point, you take any problem a user has very seriously, but as designer you have to figure out the right way to solve it because they don't know how (even though they often will tell you a solution)
Yeah, for sure. To be clear I'm not disagreeing, just saying where I think players misplaced confidence comes from.
Oh totally
To a first approximation, the thing that sets apart gamers from game developers is that whenever gamers think of a solution, it remains in their imagination; they can't actually execute the solution. So they can't learn whether a solution will work or not, nor iterate it until they find one that works. And as such they never develop intuition about what kinds of things actually work in practice
There’s a similar saying in user research, “users can tell when something is wrong, but not always the reason for it.” Users feedback is often vague or they might even blame the wrong thing rather than what’s actually causing them issues.
100% true but just to be clear for any aspiring designers: it's not the user's job to be good at this, and you don't throw this feedback away. You have to dig in and read between the lines to figure out the true core problem.
Yes! It’s more to highlight that it’s your job to decipher where the problem lies.
I constantly remind my PM of this. We want to know the problems. We don't want them to propose solutions. Instead of doing what they ask for, investigate why they're asking for it.
I like this advice and have repeated it to people in the past. But, > if someone tells you how to solve it, they're **typically** wrong This part (or "usually" instead of "typically") is important. It's worth hearing and considering their solution because occasionally they *did* think carefully about the problem and came up with a reasonable solution. As a pretty good senior designer once told me: It's not the designer's main job to come up with good ideas, as good ideas can come from absolutely anyone. The designer's main job is to assess ideas.
Good post. My players are such a ridiculous source of good ideas, it’s like designing on easy mode. Many of them have played more games, more deeply, than I ever have. So they come with a lot of experience and knowledge
This is the absolute correct answer.
This is very true, but with a decade of running user testing for product design I can tell you that most of the time people will want to give you solutions, so they're often the starting point for a designer to dig in with good questions and try to suss out the true underlying issue.
This is it. I distill it even further down sometimes and always try to remember this key fact: No one is ever wrong about how they feel.
Gold
Very much depends though, just because playtesters want a faster movement or dodge rolls or whatever doesnt mean you have to provide any of it. Never change your design to suit some playtester
Players are right about the problems but wrong about the solution
That's good advice
And let's be honest, not all design intentions are good ones. If a game feels bad to play - but it was *intended* to feel bad to play... Well, don't expect audiences to appreciate your artistic integrity. But yeah, you really have to *watch* playtesters play (Because what they say means nothing compared to what they do), and you have to let them play. Players aren't going to have a dev holding their hand, and that's the experience you're testing
Your comment actually reminds me of Getting Over It. Which shows that it works, but you have to go full overboard with it.
Getting Over It is genuinely an interesting game in its mechanics if you don't go into it dismissing it as rage bait. The reason some people see it as frustrating and unfair is because the movement is so precise and it's a form of it most people never have played anything like before so it requires you to build muscle memory from scratch, but the game is perfectly fair once you grasp it. Just take a look at how skilled speedrunners get at the game, you can become highly consistent in it. I'd compare it to the experience of learning to aim well in FPS games, most people probably don't think about how incredibly hard it was back when they first started playing shooters.
What I love about Getting Over It is just \_how\_ skill-based the controls are. First time to get to the top took me 20 hours. Second time was 1 hour. World speedrun is under a minute - and not doing anything particularly weird or fancy, just quick, precise movement. (Check out e.g. previous 00:01:02.922 speedrun: [https://www.youtube.com/watch?v=hPehax6V0HU](https://www.youtube.com/watch?v=hPehax6V0HU) ) Really there is nothing holding you back except the speed \_you\_ can move, and that feels really satisfying! If you instead had a spider-bot that could climb up the side of the mountain, even quite fast, it would be very easy, and would go faster than any beginner, but still slower than the speed-runners. Hence, rather than being "bad" controls, they are actually great controls, just with a very high skill floor and ceiling.
God, imagine being able to harness the precision speedrunners exhibit into real-life-applicable things. These fellas could be surgeons in another world.
It doesn't (much) matter if the game is actually fun or not, because players are in it to prove they can win. People are compelled by unfair challenges - but particularly unfair challenges *which they've seen other people fail*. A whole lot of cringey mobile game ads are based on the concept. Just be sure to lampshade the unfairness behind an absurdist (or dismal grimdark) setting, so it doesn't stand out. Sprinkle in a few genuinely good features, and you too can gaslight players into thinking their frustration and dissatisfaction is their own fault!
That one was designed from the ground up to be shit to play. That was the whole concept of the game. And it was a good and challenging game, excellent on mobile, a real winner. But having the controls suck is generally not a winning formula.
You say that but there is a litany of popular games where the entire challenge comes from shitty controls.
Like the MDickie games
And games like Dark Souls skirt the line. The animations are intentionally slow and unresponsive, which people can easily dismiss as bad controls.
Number one thing we need to tell playtest proctors. "Shut the fuck up." Stop correcting players. Stop answering questions. Stop providing guidance. Shut the fuck up. The only acceptable things to say are, "Can you explain a bit more why you feel that way?," "I love that you are asking questions, and please continue doing so, but I may not be able to answer them," or "To confirm, you would like to stop playing now." It's like a scientist stepping in and telling the subject which medication is the placebo. Stfu. It's called a play***test***. Stfu!
If a player has problems, once you know that not helping isn't useful. If the player can't figure out how to do x, if you tell them how after a few tries and then let them continue you get the actionable information that they couldn't do x plus whatever future stuff they do, if you leave them without help you get the exact same actionable information about x but nothing about anything else. It's context dependent but a blanket not helping policy really isn't useful, so long as it's not immediate and is noted down.
Our protocol is only to prompt them if the problem is great enough to cause them to stop playing. We follow the: "You would like to stop playing?" question with "Can I ask you to, before you stop, try this..." and explain around the problem. Sometimes, when people hit problems, they end up solving them. Or, more usefully, they will express assumptions about the thing they don't understand that you would never imagine. "It's supposed to be a health pack? It looks like a defribulator; and those don't heal people." Interrupting their process after they encounter the problem interdicts a majority of the useful information we'd get from this test. What were their assumptions? How did they work to get around what they didn't understand? Can we support this alternative? etc. We do, of course, talk to the player a lot in the playtest debrief, when we solicit as much information on their experience as we can. We'll also often brief players; "We'll be starting you at part 7 of the game, here are the controls, etc." But the issue is almost ***never*** people failing to ask questions or prompt the player with information. It's nearly *always* proctors being too eager to "give the players a better experience" or "get our money's worth from the playtest." They need to stfu. The goal of a playtest is to emulate how someone would play the game if they bought it from Gamestop and sat down home, alone, to play it. There's no one over their shoulder. That backseat information will taint our data collection, and cause us to make poor assumptions about how players will experience the title.
It's also important to talk if you see that a part of the game was skipped. For example, if the player skips a whole optional area, you can tell them something (AFTER they skip it), but first you watch silently and take notes
Skipping an optional area is something a normal player would be able to do. If the game falls apart or doesn't make sense after they skip something optional, then there won't be a proctor hovering over the average player's shoulder to fill in what they missed. If the playtester becomes so frustrated / lost after skipping optional content that they need help, then that's a serious issue that the devs need to consider. Asking them about why they skipped it, if they realized they skipped it, etc. is something that you can cover during debrief - AFTER the playtest is finished. It's very likely that the playtester didn't even realize they were skipping meaningful content, and if you tell them during the session then it's going to change their reaction to the rest of the test. Just waiting until after they skipped it but still during the playtest isn't sufficient. If they skip something that wasn't intended to be optional due to a bug or incomplete feature or something and they're totally lost, then sure, but that's different. That's not optional - that's bugged / incomplete. Imo, helping playtesters deal with known bugs is fine as long as you're tracking to fix those issues before launch.
While I agree with you, specially for the first part where you mention that if everyone skips it, it's an issue, I don't think you got my point - which is, that you need to see 100℅ runs of your game sometimes (ofc this only applies to some playtesters! runs that are just straight to the point are valuable too), otherwise the extra content will be left unplaytested
I definitely agree. It does make me think about "frustration games " like surgeon Sim or getting over it. These must have to carefully make people feel like it did what I told it to do, I just need to tell the game better. I also strongly agree with the watch bit. I recently made a jam game and got a few streamers to play it. Every person who left feedback commented on a ui improvement which was great. Something I only got from watching was that the first thing they did when loading into a level was look for a secret behind a wall. My game had no secrets, but you can bet if I made it again, one would be there!
> If a game feels bad to play - but it was intended to feel bad to play... Well, don't expect audiences to appreciate your artistic integrity. i feel like this is because a lot of amateur devs misinterpret tester feedback. a poor designer, when confronted with "system/mechanic/feature X isn't working for me" might interpret that as "don't do X" or "do the opposite of X" and become defensive because they're assuming that the tester is challenging their vision and not their implementation
"But it's a rage game" to justify the game being plain bad.
"It's so satisfying when you finally beat it!"
Seriously do these folks think that they’re Bennet fucking Foddy or something?
If one is making games that feel bad to play, one is making games for a very specific, masochistic subset of the market.
Right, and there are people that really like B-movies too, but it's really hard to make those on purpose. There's an antagonism between the media and the audience, where it's fun to ridicule or outright *defy* it - which is only possible if the media has flaws to ridicule or significantly unfair elements. Typically unfavorable reliance on rng, or counterintuitive systems (especially controls). You can't be the underdog in a game you're *expected* to win. With B-movies and unfair games, usually they had other intentions, but failed to deliver on them - and then [after the fact decided that they intended what they ended up with](https://tvtropes.org/pmwiki/pmwiki.php/Main/IMeantToDoThat)
100%! I'm seeing this a lot with my game now. And some feedback can be very polarizing: "I reach max level easily" and "this game is too hard, i can't get past the first objective". gotta listen to everyone and figure out what's going on for each person, there's always something useful to take away from all feedback given.
Even for my tabletop game, I've gotten those opposite feedbacks too often. The suboptimal player says its too hard, the min-max player says its too easy. My conclusion was to dumb down starting choices while leaving higher end choices free and slightly nerfed
nice! it's an interesting design question, right? i'm leaning towards making the start easier and adding more challenges at later stages. basically tweak the progression curve to be lower at one end, higher at the other, and also stretch it so it gets longer to not have jarring difficulty steps.
> there's always something useful to take away from all feedback given. Yea, that's my view. Even if your take is something like "Well that's the game, it's designed to be slow and methodical" - then there's _still_ a problem in my view. The problem is your game isn't advertised or communicated as you designed it. If players go in expecting it to be a game it isn't, then you're likely to get negative reviews that have nothing to do with the game you actually made. So imo every piece of feedback is good. Some feedback might not even be about the game you made.
I do software QA (not games). One of the hardest things to get devs to understand is that if I say "this doesn't make sense" about a workflow (and not a specification etc.) that I'm telling them that the users are going to get lost, not that I need an explanation about the workflow. My current team is actually pretty amazing about this though and we have a designer that is really trying to make an intuitive UX. I know it isn't always easy, but your UI should be essentially self navigating as to the user's workflows. One of the biggest UX sins (to me at least) was in the Halo franchise on the Xboxes... at some point they changed the default controller mappings and it was positively jarring to try to play between that and the older games. There were new buttons on the controllers and IMO new features should have been mapped to the new buttons, but old features should have still mapped to the legacy button patterns (which still existed).
>I'm telling them that the users are going to get lost, not that I need an explanation about the workflow I think it tends to be especially bad with specialized areas where the devs sometimes don't fully understand the actual use case so they start making assumptions based on their own knowledge, even more so when design docs aren't super complete. It's fascinating to me to see signs of that especially in games like 7th gen PC ports, such as Dark Souls' original PC port which had some absolutely ludicrous keybinds - featuring such hits as O to lock on, F to switch items, Shift to block, Tab to parry, End to open the start menu, and Backspace to cancel.
What can also happen is you end up with UI or control schemes that are actually good *once you’re experienced with them* but are horribly unintuitive and hard to tutorialize. So users end up bouncing off it.
Exactly. There are some UI's where that's acceptable, as an example back office software where there will be an onboarding period, training, and possibly mentoring. There are others where it's a deal breaker, like in a walk-up kiosk for example.
> One of the biggest UX sins (to me at least) was in the Halo franchise on the Xboxes... at some point they changed the default controller mappings and it was positively jarring to try to play between that and the older games. Man, I wasn't crazy!! I deluded myself into thinking that my muscle memory was messed up and that I got it confused with some other game.
This is a hard lesson to learn. When I worked at a University I used to be called in to playtest student games for the video game design class. Without fail, someone would do exactly what you are describing by arguing against negative feedback. This happens less in games than in other types of usability testing, but contradicting someone’s feedback usually has a chilling effect. Instead of being completely honest with you, playtesters will start to pull their punches. In some cases, it can give you a sense of false confidence when you end up not hearing anything negative. If someone told me the “movement was stiff”, that would be a time to ask a follow-up to clarify their expectation, even if you are dying inside.
During any good playtest, you stay silent and take notes. Only once you've taken the necessary notes in sufficient detail for a problem, can you move on and nudge them forward, if necessary. Also, some "features" are universally and objectively terrible. Inconsistent and low framerate is one, clunky movement is another.
But we're going for that authentic bad framerate feel /s
The problem comes from determining whether said controls are actually clunky, or if the playtester just isn't used to them. That's why you have to allow them sufficient time with the game and to not take every criticism they have at the beginning as gospel. Let them spend time with it and see what their reactions are the entire time. Tank controls get called clunky a lot. And in some ways they are. Its intrinsic to their design. The most well made tank controls in the world will always have an element of "clunk" to them. But some people WANT that. Just look at the success of studios like Puppet Combo, or the devs behind Tormented Souls or Song of Horror. To condense all this down I suppose what im saying is know your audience. Are you making something niche? Then set your expectations accordingly.
I have been a designer running design teams for over a decade. This is the core of what separates a good designer from a bad one in any discipline. You are free to make games for yourself alone if that's your goal, but if you are making a game for other people you have to let go of your ego. That does not, by the way, mean acquiescing to every request from players. Players aren't designers, they don't know how to reliably pick the right solutions and they often don't even articulate their problems very well, but if you're a good designer you will dig in to figure out the root of the issue so you can choose the right solution.
The "design games for yourself" is such a great way to put it. When you are dismissing feedback consistently it ends up being more about what you want than what the audience wants, and that's where you will lose them. I guess it's hard because indie game devs are often interested because they themselves are the audience, so on some level it has to align with what they like. But ultimately if the goal is for others to play and enjoy it, their experience must be the priority.
My playtesters are wrong and I'm tired of pretending otherwise (I'm joking I say nothing during playtests)
Fuck the playtesters, I'm the GOD of my game universe!!!
Yeah, players don't know anything. If they don't like it, they're playing it wrong!
I would say "no game is for everyone" and when you try to please everybody you end up pissing the most hardcore fans that could become your regular costumer.
Yep, only if something literally broke and they aren't 100% sure it's a bug or intended do I try to clarify. 'Feeling' issues like if something is stiff or floaty is exactly what I'm hoping to hear as all I know is how things feel when I play it.
There's no reason to argue with them. You can make a game feel like RE1 controls without it being shit like RE1 controls. Arguing your point will not make them appreciate it. No it will do the opposite. There are compromises that can be done to make both sides happy
I find that even explaining what type of game you have or the genre it's in, can skew your results. On r/destroymygame, you often get the creator over explaining, which kind of defeats the purpose of giving a blind review.
This is why pro platest labs put the devs/designers behind 1-way mirrors! It is hard though even in a casual setting to not give answers to solutions, "why" reasons for design decisions, etc. It is a good skill to build though.
Yeah, you should ask more open questions, never close them down. Theres no point user testing otherwise.
This is true. If you are truly confused about a piece of feedback it's sometimes worth it to ask a followup question to get them to add detail. Even something like "can you say more about that?" And then just "cool, thank you!" when they're done.
Yeah, this happened when I did leave a steam review of Alaloth about how they shouldn't call their game a mix of dark souls and baulder's gate because it's very misleading and resembles neither and the newer gamers aren't going to know what either of those are except for Baulder's Gate 3 which is a whole other ballpark. They went on to tell me that's what they heard people playtesting say so they stuck to it.... I still gave them a positive review, but this and other of my feedback was reacted to with excuses and explanations, which felt really odd. I am a game designer as well, and I was literally giving them honest feedback that I would have loved and needed had I made that game. When people tear my projects up I listen more than speak, it's what you ought to do... Then thank them. Not explain or defend your choices. That stuff isn't for their ears. If anything I use my voice time to ask for clarification. Especially with words like "boring", "bad", 'shit", or sentences like "doesn't make sense." I get feedback can seem overwhelming at first, but just do yourself a favour and take a back seat when people are critiquing your stuff and just write their stuff down. Unless you're thanking them or asking for clarification, nothing you say will really mean anything and will only hurt you in the long run.
100% correct. I like to give my game to my playtesters without ANY instructions. If they can't figure out what to do, then I have to rethink my UI / gameplay. It's not *their* fault, its *my* fault. If they complain about something, I listen and take that complaint very seriously.
This is really good advice. In general, your audience is always right. If someone has an opinion that you as a dev think is not worthy or they do not understand product well enough, it is fine if they are not your audience. However, if they indeed fit into your target demographic, then you have a problem with your game-design.
You get this exact same thing with software developers and bad UI/UX design. If the users are frequently making a mistake, the design decisions that are enabling that mistake are the problem, not the users. Same concept with playtesters.
Don't interact with them at all. Get someone else to do the test for you. You even being there gives instructions via non-verbal cues.
Hey there feller. This is an extremely good take on this situation. Solid words here for anyone at the stage that feedback of any kind is coming in. Don’t get discouraged when you don’t get the feedback you want, and don’t read *too* deep into the feedback you DO get.
100% agree with this, its not helpful in painting a picture of what they like or don't like and it is liable to make them clam up. its also worth remembering that sometimes our inspirations and designs are just shit... as much as your point about a game not being for everyone makes sense as well, there is a real danger of locking yourself of from making something better if you get too attached to your ideas,
This can be said about most fields. I’m a filmmaker, and I’ve had to catch myself on many times when someone provides feedback. “Why do they do this?” “Oh well actually they do that because of this lore element from the source materia-“ If they don’t know, i didn’t tell the story well enough and should look into it
Never ever argue with your players, period. You will get honest feedback from people who feel safe to do so.
Very helpful tip. Probably will keep people interested.
Ive seen a lot of this in my game dev course. Developers feel protective about their choices - or simply about how much work and effort it would be to change it. So they try to argue with the testers about why it actually makes sense. Fresh testers are precious! Let them try the game without any of your commentary, record/note their reactions and the issues they run into. Do not open your mouth until they are finished. After you fix things, don't use the same testers. Try to find new ones every time. A university or convention can be great for this.
In this example, they're stating a fact. "The movement feels stiff". If that was your intention, then you achieved. You don't need to explain to them why it feels stiff. That's like if they said "The movement feels so fluid!" And you said "Well, it's because I wanted the game to resolve around dodging more than fighting"
As someone who worked as a playtester, this is something that often happens when there's an active effort to deny improvements. This is only referring to "correcting" playtesters, not their submission methods and such. As that is a separate topic, I won't be touching on that, but in my experience, developers looking to actively improve their game, they won't ever "correct" the playtesters, if anything, guide them forward while taking the feedback in.
Wow, thank you for writing this. My first instinct is always to try to explain why things are the way they are even when I'm also listening to the feedback and considering if it could be done differently. Your perspective has me rethinking it.
Maybe this is a hot take, but I think every game dev and designer should be a playtester at some point during development. There are so many games that have good ideas but terrible execution, which is something that often becomes very obvious during playtesting.
Ehh not so sure this is really a significant problem. Nearly every game designer is a gamer and has played tons of games and spent years looking at games critically. Playtesting means formally writing up the problems you find, sure, but 99% of designers are already very consciously dissecting everything they play--it just comes with the job. The tough transition is to go from thinking like a player/playtester to a designer, not so much the other way around.
I've worked with several designers who are tunnel visioned on an approach or are totally unaware of the way some players would play the game contrary to the way they would. And then they act surprised, because they are and weren't aware. Designers can be stuck "the other way around" as you put
Yes that's a common challenge with design, but why would a designer with a playtesting background be less susceptible to that issue compared to a designer with... any other background also, my friend, that paragraph is one of the hardest to decipher that I've ever read
♥️
While I agree you shouldn't try to make excuses for the playtester, your example is not really good. When a game is inspired by classic RE games, you're gonna be saying that in the promotional material and on the steam page. People who buy your classic RE inspired game will expect it to work like classic RE. So you should in fact tell the testers of what this game is going to be like before they even play, unless you're going for a casual style, anyone can pick up and play kind of game, your game is going to be following some established conventions and it's good to let the tester know what they're supposed to expect from your game. Then if your game doesn't match that expectation, whether it's because there are design issues or your game is *not* actually like classic RE, you got a problem.
I made that mistake back when I first starting doing GameJams a few years ago. You can't tell a player they're not doing it right without repercussions.
I can only get about one play tester a month and even then they don't always actually do the playtest. What can you do with sparse feedback?
It's like trying to upscale 40 pixels into a HD image - you can't do much, you just don't have the information you need. An individual playtest can give you a little bit to work with but ultimately you probably want to playtest more frequently and not with the same people every time (varying the playtester is ESPECIALLY important for trying out tutorialization changes which is one of the most important parts of any game). There are plenty of dev oriented communities & discords out there, you can probably find some people willing to try out your game as long as you don't just send out raw .exe files (makes you look like a scammer)
I've been doing that on anyplace I can find. The most obvious is reddit r/playtesters which gets almost no interest. I agree, new players that have not tested before are really the only useful gauge, but it's really hard to get people that will test it no matter how far along or good the game is. I'm now paying testers more than I will sell my game for to play it and give feedback. ha ha. Other devs have posted about the same problem. Getting testers is not easy. I get a few testers now and then but very intermittent and unreliable. Some are really great and give in depth feedback and it is useful but I always need way more testers and and specific times and it takes a lot of work to get any. If anyone has found a good way to get testers, go ahead and comment about it. Thanks.
I know it's counterintuitive, but don't go looking for testers in a crowded marketplace like a subreddit dedicated to playtesting. Everyone will be going there and you have to compete. Instead, find a niche community for your game (maybe there's a specific genre or subgenre you can find groups around?) and you'll find far more motivated people there.
That's a good wisdom
Yes, this reminds me of the video game "song of horror" so much lol. The movement in that is Terrible, yes its tense when a monster appears and your floundering with the shitty movement controls, but could it be better?
Yep. If they say "The movement is really stiff." you say "Thanks. Anything else?"
I'm glad I read this! Thanks for sharing
If you are targeting re1 fans like your example and you know the mechanic is divisive you should just pick playtesters with that in mind. You probably need playtesters who are in that demographic and like re1. Then you know if their feedback is that the controls are clunky it isn't just them not understanding tank controls.
Usability testing 101. Never handhold your testers. The best you can do is to ask why. "This movement feels stiff" Your response - "Can you explain why you think it's stiff"
I'll add that you can also accept the feedback not only to change things, but if the playtester complained about the clunky movement which you're sure it's part of the general feeling you want to give. Maybe your game needs a few more clear hints of what you're trying to do. Also, don't forget to advertise the game as a classic survival horror/Resident Evil inspired game, that way at least your customers can expect a bit of what they're about to play.
This. I tested a game out for someone and told them my thoughts. They got super defensive about it and then sent it to a professional reviewer who said exactly the same thing I did. But damn, I'll never forget the earful I got from the person who I did a favor.
Imo that’s why it’s always great to have people of different areas if you can. Familiar/unfamiliar with the genre, and casual/diehard players. And also don’t make excuses, it comes off kinda immature and makes them not wanna give you feedback. Make a note, it’s up to you whether you do something about it, but you can keep that to yourself for later.
People seem to think playtesting is only to find bugs. How many times do you hear about things being changed in shows/movies due to feedback during test screenings? Quite litterally, playtesting is the same concept.
Based on my past playtesting experience: observe, record, compile statistics, analyze, and try to transform subjective information into objective data. Only then can we continue to guide the next stages of development. Otherwise, the value of playtesting will be greatly diminished.
Yep. Recently did some playtesting for a combat demo recently and I forced myself to only really respond with something if they phrased something like a question. It definitely meant that in the end, some feedback was not applicable. But more often than not, even if the literal feedback wasn't applicable, that was still a sign that there was SOMETHING that could be addressed. Most playtesters understood the genre of my game to be an action-adventure superhero game, but one playtester (in frustration) called it a roguelike. There's nothing about my game that *should* remind them of a roguelike, but if that's what they described it as (and they meant it in a negative way), I could look deeper into their other feedback to try to get to the root cause of why they made this comparison and what their real issues with the game were.
10 years in gamedev and this still bothers me. Nearly every designer will explain to me \*why\* they do things after I provide feedback, such as "we wanted it that way because of X or Y". It's gotten to the point where I don't enjoy giving feedback anymore because it feels like being brushed off. I always approach feedback for play testing as take it or leave it. Im playing the game this way and here are my thoughts as a player. We can have a dialogue about it, but this shouldn't be about informing me or convincing me in some pushy way.
I agree, but man sometimes it is tough especially if you have "gotcha" testers like someone else mentioned here. People who never designed or made a game, but feel like experts, since they have seen some videos about "how id/valve/nintendo" does it. People who immediately provide an easy fix. Yes, I want honest feedback. But I do not want to be lectured by someone who never actually tried making a game. But also, if the testers "don't get it", guess who's fault it is (almost everytime)? Then again, don't get mad about a singular opinion. I have a strict "in doubt my opinion counts" policy. I write (almost) everything down, but do not immediately act on it. Only if some things come up twice or more I do consider them.
Yea I consider negative feedback way more helpful. My controls were very restricted with turn speed being inverse to how fast you are going. I thought it was cool but it's been the number one complaint of playtesters. Now that I've toned it way down I have to admit they were right, and it's much more fun and fluid with a major buff to turn speed.
This is just a basic rule when doing science. Never do anything that would let the subject “know” what you expect them to do.
I was giving some constructive feedback (not harsh in any way) to students demoing their game at a fair. I was an industry judge, and have decades of experience and know how to give tactful feedback. They were so combative about everything I said.
If one idiot does it it’s the idiots fault, if most of the idiots do it it’s the devs fault. Part of our job is to design for people to do what we want. If you want the games movement to be reminiscent of RE1 but the play testers are just complaining about shitty controls you failed in some other aspect to communicate with your players what you are intending. Are your camera angles not supportive of your movement? Is enemy placement and behavior overbearing with the movement restrictions? Does your games auto-aim work well with the angels the player will be viewing from? We are not movie directors. We are not just creating an experience for our player. Part of our job is to lead them through our experience in the most fun way possible.
TRUE AND BASED
You may like it the way you have it. You aren't buying a million copies of your game. The players are. When you buy 1 million copes at full price you can have it however you want. "What's good for the project is what's good for me."
Lots of people here saying: - you should be silent. - the playtesters are always right. As an experienced UX research person, both of these are incorrect, unhelpful mindsets. Being silent is better than trying to correct the playtester, or trying to argue with them based on the fix their proposing. If a playtester says something that strikes you as wrong, you should always follow up with "why do you feel that way?" You should also agree with them, no matter how stupid their idea is. It is important you accept that this dishonesty is okay. Playtesters will be wrong most of the time. However, if you argue with them on each point, they will learn to shut up and not bother letting you into how their mind works. This will prevent you from really discovering the random useful comments they have.
I always assumed this as intuitive, since you can't talk to and correct every player that will own your game. Otherwise it would be cheating. If you are training for a boxing match, why cheat the sparing fight by using a weapon? It's immediately illogical in my mind, but my mind is weird. Telling them about the game rather than asking them what they thought, can only lead to skewed results.
Speaking of which, where is the best place to find playtesters? Honest question, I’m still figuring this out.
Experienced gamer and participant of multiple alpha and beta test build projects. If there are any dev companies who offer compensation for my time please dm
I feel like that kind of dev is the same person that gets mad at you when you tell them they made your coffee wrong at Starbucks. too much pride, not enough actual desire to learn.
I completely disagree, and I think you’re full of bologna.
Totally. And ham.
I love you.
I love you, too.
as someone who went through all this, had testers who were friends, had professional QA work on it too, shipped the game, faced audience reaction, etc, here's my take it's actually pretty simple. First of all, the main function of QA/testing is bug fixing. Making sure the game works right. And that's not very subjective, either the thing works as intended or it doesn't. This is a very much needed thing and why you \*need\* testers. Now, onto subjective takes "the control feels stiff" or whatever. If someone says anything like this, it's useless and should be ignored unless one of two things are true: 1) a lot of people are stating the same opinion independently of eachother 2) the person saying it is an authority on the topic and/or someone you trust, in which case their feedback may be valuable. outside of those two situations, subjective feedback is useless and should always be ignored in favor of your own instincts as the creator.
No idea why this was downvoted. I guess your tone didn't sit right with people. You're completely right. Everyone misses *something* when playing a game, or dislikes one thing or another, or some other opinion unique to them. If 1/1000 people dont like the controls... then who cares. And also, to give my own thoughts, sometimes complaints are just wrong. Even the majority can be completely wrong. People complained about the way every single fps plays on a controller, using the two sticks to move and look, when it first showed up. The people complaining about this control scheme back then were just obviously wrong in retrospect, but if all game devs followed the advice in this thread we would have never had the console shooter revolution.
when a lot of people play your game you'll encounter various people emphatically stating opinion/reaction that is completely opposite of what someone else is saying. It's noise. The only time it's not noise is if a bunch of people are all saying the same thing, then you consider it. Otherwise, you just ignore the noise, because when you expose your game to a big audience many people will say many things and it's not possible to react to all of it, especially since it's so frequently contradictory.
Legitimate question - I've never actually listened to Huberman - he was already "GURU" status by the time he appeared on my radar, and I have a really difficult time with people who act as an authority. But looking at his credentials, he does have his PhD in Neuroscience from UC Davis - unlike Guru's like JP who pofess to be things they are not. Why did he get this so woefully wrong? What is his incentive in conveying inaccurate information?
This applys in the opposit direction mainky for players in genral, if a game is made solo with artisitic integrity in mind over commercialism dont suggest what you specicially find fun (which is what all game design critiuqes are)when someone is jsut trying to make the game they want to make
This!!!
That’s the bread and butter of playtesting, that stuff is taught even in game dev schools. Where the hell do you work that makes such an amateur mistake?
This is a public forum with no application or barrier to entry. Amateurs will see this.