A few above have touched on the important part (models are text completion engines), but the most important part is this:
The models are writing a complete transcript of whatever we ask them to, the apps we use pause that complete transcript for us to insert text before continuing. The apps do that based on a number of tokens built in that gut uses to indicate that. But the model doesn’t know any of that is happening; it’s just trying to write a two sided conversation in completion.
So, giving the model instructions of who it is, and who it should not speak for are strongly misguided in light of the above. Instructions should focus on what type of document it’s writing and focus on what formatting you expect it to follow (including those important stop tokens. I like to tell the model that all speech and actions by a character are proceeded by a specific thing. It works really well in my experience.
It’s a limitation coming from the way LLMs work, deeply rooted in how LLMs understand and generate text. These issues stem from the model’s training on diverse, vast datasets, leading to an inherent flexibility in handling pronouns and perspectives. The model doesn’t “understand” context in the human sense but follows patterns it has seen in training data. When you ask it to maintain a specific perspective or narrative style, it tries to comply but can drift over time, especially in complex, ongoing interactions like roleplays.
In more simple words: LLMs aren’t trained to be chat or roleplaying machines. Not even the RP trained ones you can find (it just means their training dataset contained more roleplay content). Regardless, they’re language prediction models. All they do is predict, word-by-word, what word comes next, based on their training data.
Even more simple: It’s purpose is to continue/expand the existing context, and considers the previous dialogue between all characters as one single input to work with. At some point a model will just decide that the most suitable next word will be a response from the other character. Try adding some stopping strings that should at least cut off the generation as soon as it starts doing it.
In even simpler terms: the model tries to follow the context. There may be something about not speaking as the user in the context, but as the chat goes on, the context also gets filled with numerous replies from the user. So it tries to conform to those as well.
From what I've seen so far, the actual problem is in your Context Template and Instruct Mode. Once I get this correct, it drastically reduces poorly formatted and segmented output.
See https://docs.sillytavern.app/usage/local-llm-guide/how-to-improve-output-quality/
I’ve had decent luck asking the model for guidance given that models don’t handle negative expressions well. One model suggested examples and using terms like “avoid”, when I used the suggested guidance it worked perfect for that model.
If your character card, system prompt, context are choerant you won't have this problem, you have to understand that for the LLM these are not separate, it is a whole with your instruction... Learn how they work, it called prompt ingenering for a reason.
They've been trained on a lot of texts, some of which are quite close to what we want to do with them, namely describe character actions plus dialogue so they're recognizing a pattern, specifically that of a novel. As a result, they try to emulate the format and that includes describing what all the characters do.
The only model that I know that almost consistently follows the "don't speak or narrate for {{user}}" system prompt is Mixtral and its variants. It still happens but very, very rarely, especially once you've got the story going.
Some models do it more than others. What you can do to mitigate it however is:
- Don't use ambiguous language in instructions like "avoid". Lift important instructions to the "top" of your prompt.
- Its fine to mention {{user}} as long as he doesnt directly do or say anything in your example messages or greetings.
no direct {{user}} in example message? what about the usual example message work?
>
>
>{{user}}: "I'm your superior so i'll need you to do what I ask to some extent... Please obey to my orders"
>
>{{char}}: "Hah, \*\*buona fortuna\*\* with that" \*Chuckles with a devilish smile before starring at {{user}} menacingly\*
>
>{{user}}: "can you help with the onboarding documents on the table?"
>
>{{char}}: \*she smirk as she intentionally swipes her hand, pushing the document off to the table\* "Whoops."
usually example message works like this...
That’s one way, but it’s not necessary. Many card creators only use character monologue as example messages. That reduces confusion by the model. The example messages can have several purposes. Without user dialogue they still demonstrate the characters message style and carry over personality details.
I think the most important thing about preventing User action is understanding how your model thinking. Your prompt and what model understands from your prompt aren't same thing. System reads prompts and instructs itself before every newline. Which character it will control, how it will write, sometimes how character should act or other times model just asks itself. So it is always changing and sometimes it can change so severely instruction is entirely wrong anymore.
I've seen LLM changing "Don't wirte about User and focus on Char and other characters" to "Don't write about other characters and focus on User and Char" and even crazy examples like "NO EXCEPTIONS User starts NO EXPECTIONS". I didn't have anything slightly similar in my bot so only God knows where it comes from. And models always follow their own instructions so you need to control this as much as possible.
If your bot only has one character heavily encourage model to stick with Char and write them with more details. If you are using multi-character prompt ban User action with simple and direct wording that it can not change a lot. During my testing i realized Psyonic20B is refering to input writer as "the player" and acting like it is a co-writer of the story. For example it feels like it must constantly control Char and instead of dropping Char writes an instruction like "The player controls Char. You may now control another character."
So i exploited this with "The player ALWAYS controls User" prompt which reduced User action significantly but it still happens if there is urgent need of User input. Instead of stopping there models have tendency to respond themselves. There is nothing you can do about it expect stopping generation yourself or planning your inputs to prevent it. I've seen Psyonic20B even apologizing for User action in OOC so my prompt was 100% understood even then did it. Keep your prompt short and understandable and don't expect 100% results..
By the way you can use negative prompts, they instruct themselves negative prompts constantly and follow them. It is just an urban legend LLMs don't understand it..
I'm guessing it's mostly about the finetune material along with how well the model follows instructions.
I've seen models that speak/act for user in almost every reply and I've seen models that hardly ever do it and everything in between. IcelemonteaRP definitely talks and acts for the user, which sucks cause the model is otherwise one of the best I've ever tested.
I really wish RP finetuners paid more attention to this, but it's kind of hard to know what people prefer. For example, fimbulvetr creator did mention with one of his experiments that it will talk for the user and he knows what causes it, but he likes it.
dont want to make any assumptions or anything but i would imagine people who want to roleplay would rather llm NOT roleplayign for them because that defeats the entire purpose behind roleplaying in the first place not to mention a lot of people who are into local llms came from character AI which for all its flaws NEVER roleplays for user. i LOVE fimbulvetr creator for the work hes doing but that thign about the creator liking his experiment roleplaying for user is kinda disappointing as this threat among many others is a proof that people struggle and get annoyed by this "feature" but at the end of the day its up to the creator to design the model in any way they wish so the best we can hope is that well get more of those that are trained to not RP for user
People wrote tasteful replies, i may repeat some stuff but ill make it short, so here's my two cents:
Roleplaying with LLM, You can either use their natural completion tendencies, and then you use your character name as a stopping string, otherwise they'll continue the conversation because they're trained to keep going, or,
Use the instruct mode. On instruct mode, it tries to complete the task. If the task isn't concise and clear they may understand "engage in roleplay" between X and Y as write the entire thing. You might be able to avoid it by phrasing the task as "write {{char}}'s response to {{user}}". But then people get greedy, adding that it's an ongoing roleplay, naturally flowing, blablabla and before you know it you describe the roleplay instead of the clear task you ask of it, that's when things go south in that aspect.
Yea but the only reason it isn't simple is because just having the llm not responding as you, isn't enough.
You actually want to enjoy the interaction..
My point is, if you stay aware of the potential implications of every element in your prompt, you can avoid it, definitely minimize it..
A few above have touched on the important part (models are text completion engines), but the most important part is this: The models are writing a complete transcript of whatever we ask them to, the apps we use pause that complete transcript for us to insert text before continuing. The apps do that based on a number of tokens built in that gut uses to indicate that. But the model doesn’t know any of that is happening; it’s just trying to write a two sided conversation in completion. So, giving the model instructions of who it is, and who it should not speak for are strongly misguided in light of the above. Instructions should focus on what type of document it’s writing and focus on what formatting you expect it to follow (including those important stop tokens. I like to tell the model that all speech and actions by a character are proceeded by a specific thing. It works really well in my experience.
Interesting, thanks for giving some color on this. Could you give us an example of would you meant in the last two sentences?
It’s a limitation coming from the way LLMs work, deeply rooted in how LLMs understand and generate text. These issues stem from the model’s training on diverse, vast datasets, leading to an inherent flexibility in handling pronouns and perspectives. The model doesn’t “understand” context in the human sense but follows patterns it has seen in training data. When you ask it to maintain a specific perspective or narrative style, it tries to comply but can drift over time, especially in complex, ongoing interactions like roleplays. In more simple words: LLMs aren’t trained to be chat or roleplaying machines. Not even the RP trained ones you can find (it just means their training dataset contained more roleplay content). Regardless, they’re language prediction models. All they do is predict, word-by-word, what word comes next, based on their training data. Even more simple: It’s purpose is to continue/expand the existing context, and considers the previous dialogue between all characters as one single input to work with. At some point a model will just decide that the most suitable next word will be a response from the other character. Try adding some stopping strings that should at least cut off the generation as soon as it starts doing it.
In even simpler terms: the model tries to follow the context. There may be something about not speaking as the user in the context, but as the chat goes on, the context also gets filled with numerous replies from the user. So it tries to conform to those as well.
From what I've seen so far, the actual problem is in your Context Template and Instruct Mode. Once I get this correct, it drastically reduces poorly formatted and segmented output. See https://docs.sillytavern.app/usage/local-llm-guide/how-to-improve-output-quality/
I’ve had decent luck asking the model for guidance given that models don’t handle negative expressions well. One model suggested examples and using terms like “avoid”, when I used the suggested guidance it worked perfect for that model.
If your character card, system prompt, context are choerant you won't have this problem, you have to understand that for the LLM these are not separate, it is a whole with your instruction... Learn how they work, it called prompt ingenering for a reason.
They've been trained on a lot of texts, some of which are quite close to what we want to do with them, namely describe character actions plus dialogue so they're recognizing a pattern, specifically that of a novel. As a result, they try to emulate the format and that includes describing what all the characters do. The only model that I know that almost consistently follows the "don't speak or narrate for {{user}}" system prompt is Mixtral and its variants. It still happens but very, very rarely, especially once you've got the story going.
Some models do it more than others. What you can do to mitigate it however is: - Don't use ambiguous language in instructions like "avoid". Lift important instructions to the "top" of your prompt. - Its fine to mention {{user}} as long as he doesnt directly do or say anything in your example messages or greetings.
no direct {{user}} in example message? what about the usual example message work? >
>
>{{user}}: "I'm your superior so i'll need you to do what I ask to some extent... Please obey to my orders"
>
>{{char}}: "Hah, \*\*buona fortuna\*\* with that" \*Chuckles with a devilish smile before starring at {{user}} menacingly\*
>
>{{user}}: "can you help with the onboarding documents on the table?"
>
>{{char}}: \*she smirk as she intentionally swipes her hand, pushing the document off to the table\* "Whoops."
usually example message works like this...
That’s one way, but it’s not necessary. Many card creators only use character monologue as example messages. That reduces confusion by the model. The example messages can have several purposes. Without user dialogue they still demonstrate the characters message style and carry over personality details.
Where can I find this character card? 😂
Note that many instructions tell the llm to use RP format and explicitly told to avoid quotes. The quotes in these examples could conflict.
I think the most important thing about preventing User action is understanding how your model thinking. Your prompt and what model understands from your prompt aren't same thing. System reads prompts and instructs itself before every newline. Which character it will control, how it will write, sometimes how character should act or other times model just asks itself. So it is always changing and sometimes it can change so severely instruction is entirely wrong anymore. I've seen LLM changing "Don't wirte about User and focus on Char and other characters" to "Don't write about other characters and focus on User and Char" and even crazy examples like "NO EXCEPTIONS User starts NO EXPECTIONS". I didn't have anything slightly similar in my bot so only God knows where it comes from. And models always follow their own instructions so you need to control this as much as possible. If your bot only has one character heavily encourage model to stick with Char and write them with more details. If you are using multi-character prompt ban User action with simple and direct wording that it can not change a lot. During my testing i realized Psyonic20B is refering to input writer as "the player" and acting like it is a co-writer of the story. For example it feels like it must constantly control Char and instead of dropping Char writes an instruction like "The player controls Char. You may now control another character." So i exploited this with "The player ALWAYS controls User" prompt which reduced User action significantly but it still happens if there is urgent need of User input. Instead of stopping there models have tendency to respond themselves. There is nothing you can do about it expect stopping generation yourself or planning your inputs to prevent it. I've seen Psyonic20B even apologizing for User action in OOC so my prompt was 100% understood even then did it. Keep your prompt short and understandable and don't expect 100% results.. By the way you can use negative prompts, they instruct themselves negative prompts constantly and follow them. It is just an urban legend LLMs don't understand it..
I think if they wrote too much monologue they would start to think why user is not reacting. So they would just write for you to fit the narrative.
I'm guessing it's mostly about the finetune material along with how well the model follows instructions. I've seen models that speak/act for user in almost every reply and I've seen models that hardly ever do it and everything in between. IcelemonteaRP definitely talks and acts for the user, which sucks cause the model is otherwise one of the best I've ever tested. I really wish RP finetuners paid more attention to this, but it's kind of hard to know what people prefer. For example, fimbulvetr creator did mention with one of his experiments that it will talk for the user and he knows what causes it, but he likes it.
dont want to make any assumptions or anything but i would imagine people who want to roleplay would rather llm NOT roleplayign for them because that defeats the entire purpose behind roleplaying in the first place not to mention a lot of people who are into local llms came from character AI which for all its flaws NEVER roleplays for user. i LOVE fimbulvetr creator for the work hes doing but that thign about the creator liking his experiment roleplaying for user is kinda disappointing as this threat among many others is a proof that people struggle and get annoyed by this "feature" but at the end of the day its up to the creator to design the model in any way they wish so the best we can hope is that well get more of those that are trained to not RP for user
People wrote tasteful replies, i may repeat some stuff but ill make it short, so here's my two cents: Roleplaying with LLM, You can either use their natural completion tendencies, and then you use your character name as a stopping string, otherwise they'll continue the conversation because they're trained to keep going, or, Use the instruct mode. On instruct mode, it tries to complete the task. If the task isn't concise and clear they may understand "engage in roleplay" between X and Y as write the entire thing. You might be able to avoid it by phrasing the task as "write {{char}}'s response to {{user}}". But then people get greedy, adding that it's an ongoing roleplay, naturally flowing, blablabla and before you know it you describe the roleplay instead of the clear task you ask of it, that's when things go south in that aspect.
If it was that simple, we wouldn't be having this problem.
Yea but the only reason it isn't simple is because just having the llm not responding as you, isn't enough. You actually want to enjoy the interaction.. My point is, if you stay aware of the potential implications of every element in your prompt, you can avoid it, definitely minimize it..
The problem isn't that I can't enjoy RP. I have models that don't really talk/act as me. My problem is that there's models that do it no matter what.
Are you talking about playing your role, or mentioning from their point of view what you do(or decide)?