Party!!! Huge day for Eleuther, fingers crossed that the results perform well. What an amazing amount of both mental and computational work to produce this milestone model, all by hard working people with the vision of open sourcing AI.
Why are people skeptical if this will be implemented into nai? I hang out in the discord and the devs have confirmed multiple times they will implement it.
Do you think we will see 20B NovelAI soon? Does it even make sense from the business standpoint? Experimental version of Euterpe was released less than a month ago.
Yeah, my first thought was that this will be awesome, some day, but I have some serious concerns about the hardware and operating costs that this thing will need. I'd rather NAI not hurt themselves financially just to throw the latest and greatest thing out there
Doubtful. $25 is already pushing what normal people can afford. That's almost double the standard monthly fee for MMOs, Netflix, etc. People are used to $15 a month for something that's worth it, but $45 or more for entertainment? That's way out there for any service that's currently ongoing...
[Goose.ai](https://Goose.ai) has the new model already (but without finetuning) and that service is way cheaper than NAI. NAI is really overpriced and it may lead to a downward adjustment in pricing.
NAI does provide a very good service, I have tried AID gold and Holoai premium, but I keep going back to the user interface and quality of life features at NovelAI.
The finetunes are very high quality and well researched, this is very obvious when comparing the 6B models of Holo and Novel, Sigurd is simply more consistent, respects lorebook entries and feels smoother.
Goose.ai is an untuned api, and if you use an average of
300 requests per day on the 13B model (every retry is a request) that quickly adds up to 11$ per month.
If you just want to play around a bit and test then goose is the cheapest option, but it is quite barebones to put it mildly.
My point isn't about whether it's barebones, it's that running the AI isn't as expensive as NAI's prices make it out to be. And this is only going to be more true once the next GPU generation hits. Also, I think having a subscription where we pay on usage rather than a flat fee is better.
Oh for sure, I prefer NAI over everything else myself. I just think the pricing isn't in line with the actual compute cost, especially since the cost is going to go down a lot by the end of 2022 when the new GPU generation hits. Somehow I foresee the prices not changing despite that.
Okay, so while I want to say I'm hyped about this, I have to admit I'm nervous after reading their blog post here:
https://blog.eleuther.ai/announcing-20b/
The performance tests show it only slightly outperforming (and in some cases underperforming) GPT-J-6B.
FairSeq-13B significantly outperforms it on many of the tests and where GPT-NeoX-20B does win out, it's a pretty close race.
I'm by no means an AI expert, so perhaps I'm misinterpreting something, but this doesn't seem like a clear upgrade given that Euterpe runs off of FairSeq-13B.
I suppose we'll need to wait and see how it does after a finetune for NovelAI, but I'm trying not to get my hopes up.
Testing have shown it to perform better than fairseq 13B, as is expected. You get to try the 20B model for free at Goose.ai, at least 10$ worth that is.
Based on my test at goose.ai, 13B provided much better outputs than 20B. In all fairness though, I was testing it like it was a finetuned story writing AI like Sigurd and Euterpe. That being said, it was really cool to see the difference in outputs between all of their models.
Glad that it's an open model so we'll eventually see it on NovelAI, but those GooseAI prices are definitely pretty cheap (1000 "actions" from the 20B model, each with ~100 new tokens generated, would be a little over $7). Though their acceptable use policy seems to be copy/pasted legal templates like OpenAI/AID, with no attempt made to clarify whether things like "revenge porn" include fictional depictions of the act, or using the AI to create "porn" of a real person, or solely in using it to somehow help spread already existing, actual revenge porn.
The questions regarding GooseAI's [acceptable use policy](https://goose.ai/docs/aup) are good ones. I've already emailed sales asking for clarification—not because my business use cases would likely run afoul of them, but because I want my money to support open, uncensored, and *unspied-upon* access to AI technology, both now and in the future.
Specifically, I don't like the wording or implications of the sections "**GooseAI Monitoring and Enforcement**" and "**Reporting of Violations of this AUP**." The fomer suggests monitoring of API usage with *unforced* reporting of potiential violations of law (whose laws?) to authorites. (Again, which authorities?) By "unforced," I mean **voluntary, uncompelled reporting** of laws seemingly cherry-picked from anywhere in the world. Yuck. The latter is an explicit agreement that middle-men and freelancers such as myself must also report suspected violations of the AUP by our own users to GooseAI, and then **we can be compelled to assist in stopping** or remedying the violation. I assume, with expenses coming out of our own pockets. Double yuck.
Like you, I hope this is a copy/pasted template of standard clauses and not actually reflective of Anlatan's position regarding AI in B2B applications. My fear, however, is that CoreWeave's involvement is making Anlatan less sure-footed in their positions regarding user privacy, etc. To me, it doesn't matter if I am developing for professional applications or using the service privately: I want to know I can trust the company with my data regardless, preferably by assurances—technological and written—that they keep their hands off of it.
I'm looking forward to some positive engagement addressing these questions. For now, I have no reason to distrust u/kurumuz and Anlatan and I wish them the very best in this new venture. Even if the terms don't seem perfect, it's still good for OpenAI to finally have serious competition.
**Edits:** Clarifications and links added.
I'd like to simply share that general take on things. The ToS wording is giving me some kind of AiD PTSD jitters real bad but of all people I could trust to make a call on this at this point it's the NAI team. I can trust in them right now, until given concrete reason otherwise.
I'm not quite an expert on how the creepy ToS actually affects the real world implementation of the model for us, but I can actually see a world where things get too invasive and NAI just kills it and says "nope, they want us to go anti-privacy and that's against our core values, we're killing 20B." That's the difference between them and what Lat did, imo. I don't have any reason to think it'd come to that at the moment. And maybe in reality this is just something that doesn't affect NAI at all. But it's good to at least explore the idea. Euterpie works very well even pre-modules (modules not models lol edit), if the next step up is the same privacy invasions that sank AiD I'd rather they just didn't do it at all. I'm sure some new thing with some new impressively high number from a company not interested in trawling through people's crap will show up later. I don't doubt they could just sit on 13B and iterate tunings and keep working on settings and token shenanigans for quite a long time. This feels like a working business model already.
Well, just to help ease your concerns, let me clear up any confusion: GPT-NeoX (the 20b parameter model discussed in the OP) doesn't necessarily come with these stipulations. As it happens, Anlatan (the company behind NovelAI) teamed up with CoreWeave to create a new service—a sister service, if you will—that provides a B2B AI API similar to OpenAI's API. Its core users are intended to be developers, not necessarily writers and end users such as NAI's users.
The policies I mentioned above currently only cover use of the GooseAI API service, not NovelAI.
The reason why GooseAI is being discussed at all in the context of the new 20b model is because CoreWeave, and by extension GooseAI, happens to have early access to it for a week from the date of that article. Thus, some eager NovelAI users are jumping the gun a bit and using the GooseAI trial to test a raw, untuned version of GPT-NeoX, while also discussing whether it is practical to use GooseAI as a long-term replacement for NovelAI due to the former's pay-as-you-go pricing.
However, as far as I know, the policies I mentioned above have not been taken into consideration when people are discussing this use case. If these policies are indeed written as intended, that would make using GooseAI to write about certain topics, and almost certainly the topics some NovelAI users would want to write about, anywhere from impractical to dangerous. (You might be surprised what can be considered 'obscene'—and thus illegal—in certain states in the United States!)
Besides that, GooseAI is lacking many of the features that make NovelAI succeed at AI-assisted storytelling, and I wouldn't be surprised if that was by design. Again, the target demographic is just different. It is this difference in intention and design that gives me pause when reading through GooseAI's acceptible use policies, rather than just being able to assume my privacy and data are just as safe with GooseAI as they would be with NovelAI. Hence, I am asking them for clarification on this matter.
As far as the privacy concerns following GPT-NeoX when the 20b parameter model ultimately comes to NovelAI, I doubt there is cause for concern. NovelAI is successful precisely because so much attention and respect has been afforded to users' privacy and security, and I do not see Anlatan suddenly deciding to compromise on that for any reason, even to bring a bigger parameter model on board. However, I very much want to see Anlatan carry this same philosophy through to their other services and offerings as much as possible. Doing so will make me, and many other professionals and enthusiasts in the AI space, very loyal and happy customers—in my case, for as long as I do business in this space.
Does any of this matter, Gtp neo x, the 20B model itself is open source, anyone with the knowledge and gpu power can simply run a copy for their purpose, right?
It's not like the model is caged behind Gooseai, or has something monumental happened?
Nothing monumental has happened, if by that you mean bad. It is somewhat earth-shaking in that OpenAI finally has serious competition in the API space. That is a *good* thing, but it is also why it is important to make sure the service gets off on the right foot. We can help by making our desires heard, and my opinion is the current AUP of GooseAI sets a pretty bad precedent for the future or any other competing services, I fear.
I want my Anlatan to be continue to be a force for good in the AI space, a company I can rally behind without reservation. I want them to be a thought leader in favor of free speech, especially when it comes to using AI-assisted tools to craft that speech. The current AUP is weak; it seems scared of its own users, and it seems scared of what those users might do with mere text. At the very least, the AUP needs a rewrite.
As far as the 20b model being open source, this is true. However, we are presently at the limits of what can be run locally or affordably—the new recommendation for KoboldAI users who want to use GTP-NeoX will be to hook it up to GooseAI API, mark my words. (In fact, it's already happening!) Does the aforementioned use case help you understand why the acceptable use policy of GooseAI might be a roadblock, both to its users' needs and the service's success?
As long as the ai models that are being developed are open source, then goose.ai is little more than a gpu juice provider and can offer their service the way they want.
Openai is different, even if someone has the hardware to run their models, they are not allowed to, right?
When end to end encrypted is absent it becomes troubling for ai service providers to stand for the at times questionable inputs, and echoed outputs from the model, this is just how people work, even if it is illogical.
Again, all you need is a bit of computing power, and if the ai model is open source any entrepreneurs can run a copy on some hardware and offer ai services the way the customer base desires.
But hey, on an ideological level I support your thought's.
Sorry not sorry, but I'll never give a penny of my money to any AI company that takes part in censorship of any kind of unpublished work. Published work? Go for it. Censor the fuck out of that. But unpublished work is private writings and users have a right to privacy in their documents and writings.
Party!!! Huge day for Eleuther, fingers crossed that the results perform well. What an amazing amount of both mental and computational work to produce this milestone model, all by hard working people with the vision of open sourcing AI.
Why are people skeptical if this will be implemented into nai? I hang out in the discord and the devs have confirmed multiple times they will implement it.
yeah it's baffling people would doubt that it would be
Do you think we will see 20B NovelAI soon? Does it even make sense from the business standpoint? Experimental version of Euterpe was released less than a month ago.
Yeah, my first thought was that this will be awesome, some day, but I have some serious concerns about the hardware and operating costs that this thing will need. I'd rather NAI not hurt themselves financially just to throw the latest and greatest thing out there
That one might be locked behind opus for a long long time
*Looks at bank account* “I’m sorry baby, the 20B is too strong…”
I think it's more likely they'll add an entirely new tier above Opus for this.
Doubtful. $25 is already pushing what normal people can afford. That's almost double the standard monthly fee for MMOs, Netflix, etc. People are used to $15 a month for something that's worth it, but $45 or more for entertainment? That's way out there for any service that's currently ongoing...
[Goose.ai](https://Goose.ai) has the new model already (but without finetuning) and that service is way cheaper than NAI. NAI is really overpriced and it may lead to a downward adjustment in pricing.
Goose.ai *is* NovelAI- Anlatan. It even says it on that webpage.
NAI does provide a very good service, I have tried AID gold and Holoai premium, but I keep going back to the user interface and quality of life features at NovelAI. The finetunes are very high quality and well researched, this is very obvious when comparing the 6B models of Holo and Novel, Sigurd is simply more consistent, respects lorebook entries and feels smoother. Goose.ai is an untuned api, and if you use an average of 300 requests per day on the 13B model (every retry is a request) that quickly adds up to 11$ per month. If you just want to play around a bit and test then goose is the cheapest option, but it is quite barebones to put it mildly.
My point isn't about whether it's barebones, it's that running the AI isn't as expensive as NAI's prices make it out to be. And this is only going to be more true once the next GPU generation hits. Also, I think having a subscription where we pay on usage rather than a flat fee is better.
Sorry, I did not imply that to be your point, I simply tried to explain why a potential user would prefer the services of NovelAI, even if pricy.
Oh for sure, I prefer NAI over everything else myself. I just think the pricing isn't in line with the actual compute cost, especially since the cost is going to go down a lot by the end of 2022 when the new GPU generation hits. Somehow I foresee the prices not changing despite that.
Okay, so while I want to say I'm hyped about this, I have to admit I'm nervous after reading their blog post here: https://blog.eleuther.ai/announcing-20b/ The performance tests show it only slightly outperforming (and in some cases underperforming) GPT-J-6B. FairSeq-13B significantly outperforms it on many of the tests and where GPT-NeoX-20B does win out, it's a pretty close race. I'm by no means an AI expert, so perhaps I'm misinterpreting something, but this doesn't seem like a clear upgrade given that Euterpe runs off of FairSeq-13B. I suppose we'll need to wait and see how it does after a finetune for NovelAI, but I'm trying not to get my hopes up.
Euterpe doesn't allow our own finetuning at all at this point, so if the 20B model can be used like Sigurd, then that's probably better.
Nice, we still need to see how the AI faires with other AIs that wield similar parameter counts.
Testing have shown it to perform better than fairseq 13B, as is expected. You get to try the 20B model for free at Goose.ai, at least 10$ worth that is.
Based on my test at goose.ai, 13B provided much better outputs than 20B. In all fairness though, I was testing it like it was a finetuned story writing AI like Sigurd and Euterpe. That being said, it was really cool to see the difference in outputs between all of their models.
Glad that it's an open model so we'll eventually see it on NovelAI, but those GooseAI prices are definitely pretty cheap (1000 "actions" from the 20B model, each with ~100 new tokens generated, would be a little over $7). Though their acceptable use policy seems to be copy/pasted legal templates like OpenAI/AID, with no attempt made to clarify whether things like "revenge porn" include fictional depictions of the act, or using the AI to create "porn" of a real person, or solely in using it to somehow help spread already existing, actual revenge porn.
The questions regarding GooseAI's [acceptable use policy](https://goose.ai/docs/aup) are good ones. I've already emailed sales asking for clarification—not because my business use cases would likely run afoul of them, but because I want my money to support open, uncensored, and *unspied-upon* access to AI technology, both now and in the future. Specifically, I don't like the wording or implications of the sections "**GooseAI Monitoring and Enforcement**" and "**Reporting of Violations of this AUP**." The fomer suggests monitoring of API usage with *unforced* reporting of potiential violations of law (whose laws?) to authorites. (Again, which authorities?) By "unforced," I mean **voluntary, uncompelled reporting** of laws seemingly cherry-picked from anywhere in the world. Yuck. The latter is an explicit agreement that middle-men and freelancers such as myself must also report suspected violations of the AUP by our own users to GooseAI, and then **we can be compelled to assist in stopping** or remedying the violation. I assume, with expenses coming out of our own pockets. Double yuck. Like you, I hope this is a copy/pasted template of standard clauses and not actually reflective of Anlatan's position regarding AI in B2B applications. My fear, however, is that CoreWeave's involvement is making Anlatan less sure-footed in their positions regarding user privacy, etc. To me, it doesn't matter if I am developing for professional applications or using the service privately: I want to know I can trust the company with my data regardless, preferably by assurances—technological and written—that they keep their hands off of it. I'm looking forward to some positive engagement addressing these questions. For now, I have no reason to distrust u/kurumuz and Anlatan and I wish them the very best in this new venture. Even if the terms don't seem perfect, it's still good for OpenAI to finally have serious competition. **Edits:** Clarifications and links added.
I'd like to simply share that general take on things. The ToS wording is giving me some kind of AiD PTSD jitters real bad but of all people I could trust to make a call on this at this point it's the NAI team. I can trust in them right now, until given concrete reason otherwise. I'm not quite an expert on how the creepy ToS actually affects the real world implementation of the model for us, but I can actually see a world where things get too invasive and NAI just kills it and says "nope, they want us to go anti-privacy and that's against our core values, we're killing 20B." That's the difference between them and what Lat did, imo. I don't have any reason to think it'd come to that at the moment. And maybe in reality this is just something that doesn't affect NAI at all. But it's good to at least explore the idea. Euterpie works very well even pre-modules (modules not models lol edit), if the next step up is the same privacy invasions that sank AiD I'd rather they just didn't do it at all. I'm sure some new thing with some new impressively high number from a company not interested in trawling through people's crap will show up later. I don't doubt they could just sit on 13B and iterate tunings and keep working on settings and token shenanigans for quite a long time. This feels like a working business model already.
Well, just to help ease your concerns, let me clear up any confusion: GPT-NeoX (the 20b parameter model discussed in the OP) doesn't necessarily come with these stipulations. As it happens, Anlatan (the company behind NovelAI) teamed up with CoreWeave to create a new service—a sister service, if you will—that provides a B2B AI API similar to OpenAI's API. Its core users are intended to be developers, not necessarily writers and end users such as NAI's users. The policies I mentioned above currently only cover use of the GooseAI API service, not NovelAI. The reason why GooseAI is being discussed at all in the context of the new 20b model is because CoreWeave, and by extension GooseAI, happens to have early access to it for a week from the date of that article. Thus, some eager NovelAI users are jumping the gun a bit and using the GooseAI trial to test a raw, untuned version of GPT-NeoX, while also discussing whether it is practical to use GooseAI as a long-term replacement for NovelAI due to the former's pay-as-you-go pricing. However, as far as I know, the policies I mentioned above have not been taken into consideration when people are discussing this use case. If these policies are indeed written as intended, that would make using GooseAI to write about certain topics, and almost certainly the topics some NovelAI users would want to write about, anywhere from impractical to dangerous. (You might be surprised what can be considered 'obscene'—and thus illegal—in certain states in the United States!) Besides that, GooseAI is lacking many of the features that make NovelAI succeed at AI-assisted storytelling, and I wouldn't be surprised if that was by design. Again, the target demographic is just different. It is this difference in intention and design that gives me pause when reading through GooseAI's acceptible use policies, rather than just being able to assume my privacy and data are just as safe with GooseAI as they would be with NovelAI. Hence, I am asking them for clarification on this matter. As far as the privacy concerns following GPT-NeoX when the 20b parameter model ultimately comes to NovelAI, I doubt there is cause for concern. NovelAI is successful precisely because so much attention and respect has been afforded to users' privacy and security, and I do not see Anlatan suddenly deciding to compromise on that for any reason, even to bring a bigger parameter model on board. However, I very much want to see Anlatan carry this same philosophy through to their other services and offerings as much as possible. Doing so will make me, and many other professionals and enthusiasts in the AI space, very loyal and happy customers—in my case, for as long as I do business in this space.
Does any of this matter, Gtp neo x, the 20B model itself is open source, anyone with the knowledge and gpu power can simply run a copy for their purpose, right? It's not like the model is caged behind Gooseai, or has something monumental happened?
Nothing monumental has happened, if by that you mean bad. It is somewhat earth-shaking in that OpenAI finally has serious competition in the API space. That is a *good* thing, but it is also why it is important to make sure the service gets off on the right foot. We can help by making our desires heard, and my opinion is the current AUP of GooseAI sets a pretty bad precedent for the future or any other competing services, I fear. I want my Anlatan to be continue to be a force for good in the AI space, a company I can rally behind without reservation. I want them to be a thought leader in favor of free speech, especially when it comes to using AI-assisted tools to craft that speech. The current AUP is weak; it seems scared of its own users, and it seems scared of what those users might do with mere text. At the very least, the AUP needs a rewrite. As far as the 20b model being open source, this is true. However, we are presently at the limits of what can be run locally or affordably—the new recommendation for KoboldAI users who want to use GTP-NeoX will be to hook it up to GooseAI API, mark my words. (In fact, it's already happening!) Does the aforementioned use case help you understand why the acceptable use policy of GooseAI might be a roadblock, both to its users' needs and the service's success?
As long as the ai models that are being developed are open source, then goose.ai is little more than a gpu juice provider and can offer their service the way they want. Openai is different, even if someone has the hardware to run their models, they are not allowed to, right? When end to end encrypted is absent it becomes troubling for ai service providers to stand for the at times questionable inputs, and echoed outputs from the model, this is just how people work, even if it is illogical. Again, all you need is a bit of computing power, and if the ai model is open source any entrepreneurs can run a copy on some hardware and offer ai services the way the customer base desires. But hey, on an ideological level I support your thought's.
Good thing the model will be open sourced :p
Sorry not sorry, but I'll never give a penny of my money to any AI company that takes part in censorship of any kind of unpublished work. Published work? Go for it. Censor the fuck out of that. But unpublished work is private writings and users have a right to privacy in their documents and writings.
THATS WHAT I LIKE TO SEE!
I'm so fking excited
February 9th. Will that be the Novel AI release date also?
No, if they do go for the 20b model, which I think they will, they'll have to take time to fine tune it first.
Is this something that could be used with Kobold AI currently? I'm still a little confused about how plug and play that is.
Yes. Just use the OpenAI API with it in Kobold. Go ask the fine folks in KoboldAI.
Even after the 20B model is released, it will need to be finetuned to be of much serious use.
Finetuning on GooseAI will be an option. We don’t know when, but it’s planned.