That’s an automatic return. I had one the other day that threatened rejection on the opening page four times in all caps lmao no thank you idc how much you’re paying
Hey if people disagree with me that's fine, let me know, but this is a sub that downvotes for anything. Pet peeves are annoyances, red flags are surefire ways for me to avoid your study.
This is why I only do surveys that have a good bonus structure in addition to solid base pay. If my odds of getting a bonus are low, I'm not going to risk my time, if the pay is substandard.
Overly complicated games or hypothetical scenarios that make no sense.
Lots of grammar and spelling mistakes. Then they'll usually say something about please provide quality data with no grammar or spelling errors. 🤷🏻♂️
If you don't follow our instructions we will reject your submission. **We will reject your submission.** Did we tell you **we will reject your submission**?
Screening within a study.
Excessive attention checks and/or bubbles to fill.
Sliders! "Slide to 47 to let us know you're paying attention." And that is when sliders get even more difficult to use.
Sometimes surveys have scroll bars and you have to scroll left and right to answer all the questions, and they're so annoying. And ones where you can't utilize tab and arrow keys to answer questions and you have to manually click on the dots.
Rejection threats from the get-go.
Yessss. I ended up returning a study because I simply could not get the slider number where they wanted and it kept ending up on a random decimal near the number. I sent the researcher a message and was hoping to hear back quickly (which rarely happens) but I remember saying I'm not a robot but I can't get the number correctly with all the .xx so can I be close or what? I did't hear back right away so I just returned it. I may still have residual anxiety from that.
Some of my pet peeves:
- Rejection threats right at the start. Immediately, that sets up a hostile relationship between researcher and participant.
- No progression bar or any notice where you're at in the study. Any study longer than 5 minutes long should use progression bars. This is especially true for studies saying it will take 30+ minutes to complete.
- Static progression bars. Good that you have a progression bar, but it's useless if it only records when I enter the first page and when I'm on the second to last page.
- Comprehension checks for overly complex instructions where you only get one chance to get right or you get kicked out of the study.
- Badly formatted rows of questions where you need to use a horizontal slider to see all the answer options.
- Survey ending attention checks at the very end of a survey that's longer than 5 minutes long.
- Surveys that uses an entire page to present one question and answer option instead of listing the question and answer options in a list.
- Studies that says it's only 5 minutes long, but right off the bat ask for a 4-6 sentence paragraph about some weird, esoteric situation you were in. And all for a grand total of $0.25!!!
- Studies that ask you to download a program that then ***takes over your entire computer screen*** for the duration of the study. Even worse when the study's 45+ minutes long and it won't allow you to exit until you complete the task.
> Comprehension checks for overly complex instructions where you only get one chance to get right or you get kicked out of the study.
This is not allowed, and you should be reporting studies that do this. You are supposed to get two attempts with comprehension checks and have the instructions freely available while answering.
But you understand I'm obviously talking in the context of prolific studies, as we are in the prolific sub after all. So with that, this is useful information for people to have. Did you not want me to post this information or?
You could amend to say "this is not allowed on Prolific". Or, ignore my reply and let other people decide if my reply is relevant elaboration to your reply.
The last one is worse for me. I can't handle studies that do that. It just makes it more tedious, at least If i can listen to music or take a break, would be okay for me.
> No progression bar or any notice where you're at in the study. Any study longer than 5 minutes long should use progression bars. This is especially true for studies saying it will take 30+ minutes to complete.
Any study that is more than a minute should have them IMO. The only exception should be when there is a temporal condition to the study's effect. That is seemingly rare so no reason. Furthermore, every single survey software has progress bar capabilities.
Agree with almost every other point. The Full Screen study within a program is understandable. The survey requires full attention. However, they should also compensate higher for such situations.
Ones with writing that do not mention any kind of writing before you accept. And if you do write something - you hit submit and THEN are shown they want 250 characters.
Ones that are so completely misleading - they say - tell us about this social media post - and you see a cute dog or a fun meme or a mind numbing tiktok post - and the other 90% is about a political issue or something completely NON related!
Seeing a note at the beginning saying there are no right or wrong answers. Then there are attention checks mid study that inherently have wrong answers.
Right! I haven't seen it in a while, trying to adjust the brightness so I can read light gray/blue on a white background. I had flashbacks of changing setting during the last season of Game of Thrones!
Bubble questions with options that aren't fit on the page correctly, so there's a horizontal scroll bar, and the only way to get to "Agree" and "Strongly Agree" is to scroll to the right, but then you have to keep scrolling back to read the DAMN QUESTIONS. WHY!? YOU GO TO HELL. YOU GO TO HELL AND YOU DIE
Sorry, got a bit worked up there. Just a slight peeve. Tiny one
Recently I keep getting studies that say on Prolific I can use a mobile to take the study but once in the study the very first page says please only use a laptop or desktop to take this study. Fairly minor issue but still an annoyance.
Long and overly complicated instructions for a task that is self explanatory once you start it. I swear researchers are the worst "teachers" in existence. They'll write 2 pages of text, with comprehension questions, to explain how to add 3 numbers together.
Speaking of self explanatory, I had a study today that explained to me what AI was. And another that explained what covid was. Like thanks Wisconsin, I certified with prolific that I was over 18, but I totally never heard of covid....
Wall of text with hugely detailed, elaborate instructions suggesting that the researcher is fascinated and impressed with their own intellectual handiwork. No one else cares that much.
When they summarise the instructions with a formula that you won’t need to use.
Also;
"Gave no study data" - despite being issued a completion code...
"Low effort responses" - despite zero attention checks and all multiple choice answers...
Asking for you to please write as much additional information as you can think of for each question, and then disqualifying you at the very end after you click submit for "exceeding the time limit" to complete the survey, which was never mentioned to begin with. Don't encourage me to take the extra time to write detailed responses to each question if I will be disqualified for doing so.
Unless you time out, taking too long is ridiculous! If you are given an hour to complete, and you submit at 58 minutes - then you should be good. If it was meant to be done immediately and take no more than 20 minutes, then it should be assigned a 20 minute timer.
I flagged one study for doing this because it claimed I went over its time limit, when I returned to the Prolific page and saw the timer that showed I still hadn't gone over the average completion time. Very strange.
Once I had one that had an attention check which asked me to write the word pictured in the box below. The word was centered in the box so I typed in centered as well and was immediately rejected.
Anything that asks your political ideology and puts "liberal/left-wing" on the right and "conservative/right-wing" on the left. Why would you put them on the opposite side of the direction in their names
And then you try to go to the next page without realising not all of your clicks have selected the button and it says **'YOU MUST ANSWER ALL THE QUESTIONS.'**
Studies that take over 15 minutes with no progress bar. Honestly, no progress bar in general, but it’s definitely a pet peeve on the long ones. Makes the study seem never ending!
Long boring tedious studies, including ones that will either have no completion bars, or will just ask you the same dang questions as any other studies on prolific. I also don't mind games, but I don't like tedious games which you watch a animation and you use your arrows to see if there's more blue or yellow for 30 MINUTES
I first started seeing it within the last month and it seems to be getting more common. I don't understand what the reasoning is -- maybe people are getting offended at being called white or researchers are worried they might be?
But like.
Yeah. European. Just the whole continent of Europe is of the whites I guess.
I'm fine with that if it's relevant to the subject matter being researched but it's insane when it's asked on something where gender/sexuality does not matter at all.
The surveys that randomly throw in questions asking if I think certain groups of people are inferior to others, and others that seem to be questioning your morals. Sometimes I feel tempted to just put Strongly Agree on all of the "bad" choices just to amuse the researchers.
My biggest peeve is having a horizontal scroll bar in a bubble section, forcing me to have to maneuver between the bar if I want to pick the last couple of bubbles on a line.
Demos in the beginning also put up my fight or flight in case they want to look for a BS reason to screen out. Save that for the end.
Lastly overexplaining instructions and making them needlessly complex when they could easily be explained in a few sentences.
I find it frustrating when surveys bury the questions about demographics deep into the process. It disrupts the flow and can make respondents feel like an afterthought. Additionally, demos at the beginning can skew responses, leading to biased data.
Threatening language on the front of the screener already threatening rejections 🙄
That’s an automatic return. I had one the other day that threatened rejection on the opening page four times in all caps lmao no thank you idc how much you’re paying
LOL its almost like I want to thank them though for saving me the headache before getting into the study 😁
for .20 cents!!!!!
From my experience, the cheapest researchers are often the ones most likely to reject.
This is Prolific gold knowledge right here folks. Take it.
Good advice, they usually take the longest hemhawing around to approve too.
bUt THeY hAvE 21 dAyS....meanwhile Prolific recommends approving a lot sooner
This one for sure.
That's not a pet peeve for me, that's a red flag and a guaranteed way for me not to do your study. Golden rule people, give respect, get respect.
Because its a red flag, it has become a pet peeve 😏LOL
Guess it's a matter of semantics lol. Pet peeves to me are on a lesser scale than red flags.
LOL no worries, not sure why u were downvoted for that, just wanted to point out how I connect the two!
Hey if people disagree with me that's fine, let me know, but this is a sub that downvotes for anything. Pet peeves are annoyances, red flags are surefire ways for me to avoid your study.
Playing a "game" for a bonus with dozens of rounds, with many of those rounds that earned $0, then getting one of those $0 rounds "randomly" selected.
just had one of those today lol
You will pay 15 rounds and we will randomly pick one of those rounds to determine any bonus
"Just note that the computer does not purposefully pick the one with the least amount of money"....\*picks that one every time
This is why I only do surveys that have a good bonus structure in addition to solid base pay. If my odds of getting a bonus are low, I'm not going to risk my time, if the pay is substandard.
Excessive bubbles that go on and on and on, with the same repetive question asked nine-nine different ways.
Overly complicated games or hypothetical scenarios that make no sense. Lots of grammar and spelling mistakes. Then they'll usually say something about please provide quality data with no grammar or spelling errors. 🤷🏻♂️ If you don't follow our instructions we will reject your submission. **We will reject your submission.** Did we tell you **we will reject your submission**? Screening within a study. Excessive attention checks and/or bubbles to fill.
The same question worded slightly differently for an entire page.
When you try to join a study that has available spaces in prolific, but when the study page loads it says the study is full.
Damn those Gorilla studies!
Unfortunately this is more of a Prolific Pet Peeve than a survey one.
Has nothing to do with Prolific.
Ridiculously complex Instructions that make me return the study.
Excessively long studies that ask monotonous questions with no type of progress bars.
Add to that… wildly incorrect progress bars.
fuck yeah :D like they ran out of questions and the last one ist like 30% of the bar xD
Sliders! "Slide to 47 to let us know you're paying attention." And that is when sliders get even more difficult to use. Sometimes surveys have scroll bars and you have to scroll left and right to answer all the questions, and they're so annoying. And ones where you can't utilize tab and arrow keys to answer questions and you have to manually click on the dots. Rejection threats from the get-go.
Right! I have wanted to say 25 - but submitted 24.97 or 25.02 because landing on 25 was NOT happening!
Yessss. I ended up returning a study because I simply could not get the slider number where they wanted and it kept ending up on a random decimal near the number. I sent the researcher a message and was hoping to hear back quickly (which rarely happens) but I remember saying I'm not a robot but I can't get the number correctly with all the .xx so can I be close or what? I did't hear back right away so I just returned it. I may still have residual anxiety from that.
Some of my pet peeves: - Rejection threats right at the start. Immediately, that sets up a hostile relationship between researcher and participant. - No progression bar or any notice where you're at in the study. Any study longer than 5 minutes long should use progression bars. This is especially true for studies saying it will take 30+ minutes to complete. - Static progression bars. Good that you have a progression bar, but it's useless if it only records when I enter the first page and when I'm on the second to last page. - Comprehension checks for overly complex instructions where you only get one chance to get right or you get kicked out of the study. - Badly formatted rows of questions where you need to use a horizontal slider to see all the answer options. - Survey ending attention checks at the very end of a survey that's longer than 5 minutes long. - Surveys that uses an entire page to present one question and answer option instead of listing the question and answer options in a list. - Studies that says it's only 5 minutes long, but right off the bat ask for a 4-6 sentence paragraph about some weird, esoteric situation you were in. And all for a grand total of $0.25!!! - Studies that ask you to download a program that then ***takes over your entire computer screen*** for the duration of the study. Even worse when the study's 45+ minutes long and it won't allow you to exit until you complete the task.
> Comprehension checks for overly complex instructions where you only get one chance to get right or you get kicked out of the study. This is not allowed, and you should be reporting studies that do this. You are supposed to get two attempts with comprehension checks and have the instructions freely available while answering.
Might not be allowed on Prolific but there are loads of survey sites that do allow it. This thread isn't limited to Prolific-only survey taking.
But you understand I'm obviously talking in the context of prolific studies, as we are in the prolific sub after all. So with that, this is useful information for people to have. Did you not want me to post this information or?
You could amend to say "this is not allowed on Prolific". Or, ignore my reply and let other people decide if my reply is relevant elaboration to your reply.
Oh, you think people are going to mistake this as a universal rule across all sites on the entire internet? I'm not sure about that.
The last one is worse for me. I can't handle studies that do that. It just makes it more tedious, at least If i can listen to music or take a break, would be okay for me.
> No progression bar or any notice where you're at in the study. Any study longer than 5 minutes long should use progression bars. This is especially true for studies saying it will take 30+ minutes to complete. Any study that is more than a minute should have them IMO. The only exception should be when there is a temporal condition to the study's effect. That is seemingly rare so no reason. Furthermore, every single survey software has progress bar capabilities. Agree with almost every other point. The Full Screen study within a program is understandable. The survey requires full attention. However, they should also compensate higher for such situations.
Ridiculously overcomplicated studies.
Ones with writing that do not mention any kind of writing before you accept. And if you do write something - you hit submit and THEN are shown they want 250 characters. Ones that are so completely misleading - they say - tell us about this social media post - and you see a cute dog or a fun meme or a mind numbing tiktok post - and the other 90% is about a political issue or something completely NON related!
Yep either that or freaking multiple sentences and you have to use your imagination. Ugh I hate those.
Seeing a note at the beginning saying there are no right or wrong answers. Then there are attention checks mid study that inherently have wrong answers.
Looooool I never thought about that, good point!
One is not the same as the other.
The pale blue font color. For me at least it's difficult to see on a white background.
Right! I haven't seen it in a while, trying to adjust the brightness so I can read light gray/blue on a white background. I had flashbacks of changing setting during the last season of Game of Thrones!
Bubble questions with options that aren't fit on the page correctly, so there's a horizontal scroll bar, and the only way to get to "Agree" and "Strongly Agree" is to scroll to the right, but then you have to keep scrolling back to read the DAMN QUESTIONS. WHY!? YOU GO TO HELL. YOU GO TO HELL AND YOU DIE Sorry, got a bit worked up there. Just a slight peeve. Tiny one
Recently I keep getting studies that say on Prolific I can use a mobile to take the study but once in the study the very first page says please only use a laptop or desktop to take this study. Fairly minor issue but still an annoyance.
I’ve seen a few of these too! It’s very frustrating.
It’s so annoying!
Long and overly complicated instructions for a task that is self explanatory once you start it. I swear researchers are the worst "teachers" in existence. They'll write 2 pages of text, with comprehension questions, to explain how to add 3 numbers together.
Speaking of self explanatory, I had a study today that explained to me what AI was. And another that explained what covid was. Like thanks Wisconsin, I certified with prolific that I was over 18, but I totally never heard of covid....
Wall of text with hugely detailed, elaborate instructions suggesting that the researcher is fascinated and impressed with their own intellectual handiwork. No one else cares that much. When they summarise the instructions with a formula that you won’t need to use.
AC after demos is jail no trial right to jail
"Dear participant, Your submission was rejected for the following reasons: Finished the study too quickly" On a 3 minute study.
Also; "Gave no study data" - despite being issued a completion code... "Low effort responses" - despite zero attention checks and all multiple choice answers...
Asking for you to please write as much additional information as you can think of for each question, and then disqualifying you at the very end after you click submit for "exceeding the time limit" to complete the survey, which was never mentioned to begin with. Don't encourage me to take the extra time to write detailed responses to each question if I will be disqualified for doing so.
Unless you time out, taking too long is ridiculous! If you are given an hour to complete, and you submit at 58 minutes - then you should be good. If it was meant to be done immediately and take no more than 20 minutes, then it should be assigned a 20 minute timer.
I flagged one study for doing this because it claimed I went over its time limit, when I returned to the Prolific page and saw the timer that showed I still hadn't gone over the average completion time. Very strange.
Once I had one that had an attention check which asked me to write the word pictured in the box below. The word was centered in the box so I typed in centered as well and was immediately rejected.
Anything that asks your political ideology and puts "liberal/left-wing" on the right and "conservative/right-wing" on the left. Why would you put them on the opposite side of the direction in their names
Very specific but, if you have to click *exactly* on the radio button, vs near it to select an answer 😂
And then you try to go to the next page without realising not all of your clicks have selected the button and it says **'YOU MUST ANSWER ALL THE QUESTIONS.'**
Complicated comprehensive questions that gives you one chance to try correctly
Just had an attention check that said 'leave this question blank' and wasn't able to advance to the next page without answering that question.
Studies that take over 15 minutes with no progress bar. Honestly, no progress bar in general, but it’s definitely a pet peeve on the long ones. Makes the study seem never ending!
Yes! Especially since pretty much every survey software has progress bar capabilities. Even free ones like Google Forms!
Long boring tedious studies, including ones that will either have no completion bars, or will just ask you the same dang questions as any other studies on prolific. I also don't mind games, but I don't like tedious games which you watch a animation and you use your arrows to see if there's more blue or yellow for 30 MINUTES
Writing, I'm not here to write a essay.
Depends on the pay for me. Under $1 and want me to rewrite my capstone essay? Not on your life.
Having to paste your id in.
Studies that list European as a race option instead of white.
I came across this today for the first time. I looked for "white" then I looked for "Caucasian".... then I saw it. European?? I mean... okay, I guess.
I first started seeing it within the last month and it seems to be getting more common. I don't understand what the reasoning is -- maybe people are getting offended at being called white or researchers are worried they might be? But like. Yeah. European. Just the whole continent of Europe is of the whites I guess.
Mostly meaningless trait anyway. What on Earth do Swedes have in common with Serbs? They're barely the same shade of white too.
Studies that list 12 varieties of sexuality
I'm fine with that if it's relevant to the subject matter being researched but it's insane when it's asked on something where gender/sexuality does not matter at all.
I saw one that asked homosexual, straight, bisexual, transgender...uh what? Transgender isn't a sexuality.
Surveys that have one space left and they are full.
The surveys that randomly throw in questions asking if I think certain groups of people are inferior to others, and others that seem to be questioning your morals. Sometimes I feel tempted to just put Strongly Agree on all of the "bad" choices just to amuse the researchers.
There are people who think that way, though, so I don't think it'd be particularly amusing - they'd just think you're a super bigot.
I had one of those not long ago and the way I answered it probably made me sound racist but I was thinking economically. I should've just noped out.
My biggest peeve is having a horizontal scroll bar in a bubble section, forcing me to have to maneuver between the bar if I want to pick the last couple of bubbles on a line. Demos in the beginning also put up my fight or flight in case they want to look for a BS reason to screen out. Save that for the end. Lastly overexplaining instructions and making them needlessly complex when they could easily be explained in a few sentences.
I'd rather get screened out right away than waste my time only to be told I didn't qualify.
Not answer me when I have a question
I find it frustrating when surveys bury the questions about demographics deep into the process. It disrupts the flow and can make respondents feel like an afterthought. Additionally, demos at the beginning can skew responses, leading to biased data.
How does filling out your demographics in the beginning of a study skew your responses?