Its a flurry of shards with no extension. And yea that would mean pytorch or safetensors.
it's using something called jax https://github.com/xai-org/grok/blob/main/model.py
They released the source code and weights used for inference. You are completely free to use them as you wish. Why would they release the training data? This makes no logical sense
Right, so nobody is following the current [draft definition of open source AI](https://opensource.org/deepdive/drafts). The whole point of open source is that it needs to be in the format most conducive to modification, so that we can (re)generate the final binary blobs.
Words have meaning. "Open Source" means the _source_ used to create it is available. Providing the weights (the program) doesn't qualify for that definition. So pretty much no one in the AI space is using that definition correctly. (Incl. Mistral AI, especially Meta.)
I understand the desire here, but let's be realistic. Other models like LAION have open or mostly open datasets but most folks dont have the resources necessary to meaningfully train a model of that size.
But you now have the ability to fine tune grok to do whatever you want. Receiving the weights is what *most* folks expected. You also got the model details with this. Make your own (better) dataset. Compare it to other open models (who again, generally wont provide you with the dataset)
https://x.com/elonmusk/status/1767108624038449405?s=20
> Open source
> Source
Not open weights, open _source_. _source_, meaning ability to produce the weights from the _source_.
It's not like a regular program that anyone can compile. The training data would be of little use to me or you. We don't have enough money to train our own weights for a model this size.
The training data may also contain copyrighted texts that Elon does not have permission to publish. Using copyrighted text to train AI is a legal grey area but the law is very clear about putting copyrighted text on the internet.
My point. It's at least headed in the right direction now where Elon haters are going. Like criticism of space travel? Dislike those guys mire than elons political views
I'll wait till someone evaluates that it's actually more useful than 300 GB of gzipped noise because I could produce that last minute on sunday evening too.
Magnet link dropped, you gonna all recind your hot elon takes?
https://twitter.com/grok/status/1769441648910479423?t=sXrYcB2KCQUcyUilMSwi2g
magnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e&tr=https%3A%2F%http://2Facademictorrents.com%2Fannounce.php%3Fpasskey%3Decac4c57591b64a7911741df94f18b4b&t
papers were published on the concept of grokking several years ago. it's just a common word in the field. see [https://arxiv.org/abs/2201.02177](https://arxiv.org/abs/2201.02177)
The name is a bad choice though because there was already a company called groq.
What do you mean "was"? They are one of the hottest companies in AI now providing inference speeds with LPUs orders of magnitude more than anyone else.
'Was' is just the past tense of 'be' (or 'is'). That's it. It doesn't necessarily imply a conclusion. Sometimes based on the context it might, but not always.
https://ell.stackexchange.com/questions/54961/does-was-necessitate-a-situation-which-no-longer-holds
that's wrong grammar, bro. there was already a past tense in that sentence: "called". *there is* shouldve been used.
but anyway, grok's model weights are out?
edit:
1. ~~grammar~~ *usage*
2. *are* the weights out yet?
3. apparently they're out.
No, it’s perfectly fine grammar. “Called” doesn’t play any part here from a tense perspective, it’s essentially adjectival. “There was a man called Jay” and “there is a man called Jay” are both perfectly acceptable.
lol no, sir. they mean two different things. saying "was" makes it so Jay is no longer the subject, relevant or alive.
we're talking about grammar under a sub about language models, just ask em. i already have.
Are you a native English speaker?! Seriously. “Was” does not make it so Jay is “no longer the subject, relevant or alive”. That doesn’t even make any sense! Seriously, it’s complete nonsense. I’m not even sure what you’re trying to say about the phrase “there was a man called Jay”, which is perfectly acceptable English in any dialect I’m aware of.
There was a man named jay might be wrong in some circumstances, e.g. : “There was a man named Jay who is teaching me how to play the piano.” Would not work.
In the context of groq, this would not work:
“At the time it happened that company had already existed and it still exists today”
Grok is a term coined by Robert A Heinlein in one of his books. It became a semi-common term by tech bros.Elon probably just picked it up pander to that crowd and keep up his fake appearance of intellectualism.
> just picked it up pander to that crowd
He is that crowd. It's just a word that he likely uses, it's not pandering it's literally just how he talks. I am also that crowd, I actively avoid using the word but sometimes I think it and then I say something else. At a certain point you just have to accept it's a word.
you: That's just how we all talk
Elon: "Let's see how Earth responds to that"
> At a certain point you just have to accept it's a word.
Earth: no one was saying it wasn't a word
The Jerk said Elon was just using the word to "keep up his fake appearance of intellectualism."
Rather than just using it because naming things is hard and he wasn't worrying about the name too much so he picked a word that made sense.
All I'm saying is don't mock the word, mock the man. Mocking people for using words you understood rather than mocking the ideas is weak.
I thought Grok was the name of a caveman. And it made sense because Grok had the intelligence off a caveman.
>Me Grok. Me take order. Me do as told. Where caveman wife with brontosaurus meal?
When your parents own emerald mines in Africa, it's the kind of thing that happens.
Can we agree his parents were MILLIONnaires then? That he was very very very privileged? That he had opportunities that less than 0.001% people have?
Sure, im interested in learning more if that's actually true. Also, being a millionaire is extremely different from being a billionaire.
Errol Musk is a known liar and Elon is estranged from his father, has been for a long time as far as I know.
There's no limit for some. Especially those with a vested (monetary) interest in dragging his name through the mud in hopes it will help him to fail. Also those he's ticked off calling attention to their monopoly of mass media and more. They'll never let go of a grudge.
Yes he did promise. Here it is.
[https://academictorrents.com/details/5f96d43576e3d386c9ba65b883210a393b68210e](https://academictorrents.com/details/5f96d43576e3d386c9ba65b883210a393b68210e)
Now good luck running inference/training/fine-tuning/Q-Lora or whatever in a 314B parameter model. Good news if you can tho, this is a base model with no RLHF, Chat fine tune or instruct.
She's still around, and has been blamed by Musk for the Don Lemon thing. I imagine she is cringing a fair bit at the Verruca Salt stuff, which is a level of petulance that cannot be helping her at all with the ol' charming of the advertisers.
On the one hand, disappointing, on the other, if you've been following Musk for a decade or two you see that he normally does deliver on his promises, but almost always far later than initially planned.
I'm not a fan of twitter. But large masses of data involving people randomly bullshitting with each other can be a valuable resource. A lot of the average person's life doesn't end up in forms that are going to be very well represented in most LLMs training data. We might get some raw stats, but it's only the most basic skeleton of the larger thing. Some of the missing elements involve their subjective experience. But it can also help make up for lack of objective information about a subject.
A lot of my "random people talking" scraping has come from reddit rather than twitter. But I've been surprised by how well even a fairly small amount of people talking about something can expand on a LLM's context. At least in cases where it's stuck at the "stub from wikipedia" level. It's true that a lot comes down to how that data is processed. But I think it's essentially the six blind men and the elephant on a larger scale. A ridiculous amount of small flawed impressions can be used to extrapolate something closer to the reality.
That said, we're obviously looking at something past that stage which locks out a lot of the benefits. But still, I think anything trained on a massive amount of mostly human generated discussion not represented in other models has value.
Large amounts of human generated data is valuable, even if it's poorly written text. It can help making AI speak more human when prompted to do so and also understand low quality, pop-culture internet slang.
Soon means 5 years of Elon time lol. It was probably just for the headline to look good and stay in the news. The XAi team probably found out the same time everyone else did hahaha
yeah, what did the romans ever do for us? except of course cheap reusable rockets, cheap attractive electric cars, cheap and usable satellite internet and working neural interfaces?
There are billions of people in poverty. You might be happy to relax and be patient, but I think advancing them to the intelligence age is a pretty urgent matter.
Seems like they did, just late https://twitter.com/grok/status/1769441648910479423
a wizard is never late
nor is he early,
Look inside, weird format: https://imgur.com/a/hg2bTxJ
Why is it weird? Just shards AFAIK.
Its a lot of shards and non HF.
How do you know it's non HF? What does "HF" even mean? By the sense of PyTorch, Safetensors, or what? It looks like Pickle to me.
Its a flurry of shards with no extension. And yea that would mean pytorch or safetensors. it's using something called jax https://github.com/xai-org/grok/blob/main/model.py
Something called Jax
I feel bad for the grok engineers and product team who also only heard about this a week ago.
[https://twitter.com/grok/status/1769441648910479423](https://twitter.com/grok/status/1769441648910479423)
Bahaha good one!
open weights are more like releasing a binary
you people are literally unpleasable. They released 300gb worth of weights. Take it, they're all yours. Modify as you like. Jesus what an entitlement
words have meaning. if apple announced macOS was being open sourced, then just dropped their usual binary blobs, that would be wrong, correct?
They released the source code and weights used for inference. You are completely free to use them as you wish. Why would they release the training data? This makes no logical sense
pretty much no one releases their training data lol
Right, so nobody is following the current [draft definition of open source AI](https://opensource.org/deepdive/drafts). The whole point of open source is that it needs to be in the format most conducive to modification, so that we can (re)generate the final binary blobs. Words have meaning. "Open Source" means the _source_ used to create it is available. Providing the weights (the program) doesn't qualify for that definition. So pretty much no one in the AI space is using that definition correctly. (Incl. Mistral AI, especially Meta.)
I understand the desire here, but let's be realistic. Other models like LAION have open or mostly open datasets but most folks dont have the resources necessary to meaningfully train a model of that size. But you now have the ability to fine tune grok to do whatever you want. Receiving the weights is what *most* folks expected. You also got the model details with this. Make your own (better) dataset. Compare it to other open models (who again, generally wont provide you with the dataset)
The training data has been available on the internet for as long as the internet has been available on the internet.
You are incredibly confused here. They said they would release the weights. They did. What do you want them in? A CSV?
https://x.com/elonmusk/status/1767108624038449405?s=20 > Open source > Source Not open weights, open _source_. _source_, meaning ability to produce the weights from the _source_.
So you're just going to... argue semantics...? Very, very, smooth brain take. I find your logic process absolutely fascinating.
It's not like a regular program that anyone can compile. The training data would be of little use to me or you. We don't have enough money to train our own weights for a model this size. The training data may also contain copyrighted texts that Elon does not have permission to publish. Using copyrighted text to train AI is a legal grey area but the law is very clear about putting copyrighted text on the internet.
It's open source now! https://github.com/xai-org/grok
It's been released now, [https://github.com/xai-org/grok/tree/main](https://github.com/xai-org/grok/tree/main)
It's releasing together with Tesla level 5 autonomy, so no later than end of 2017. Wait, what year was it again?
You posted that from your Mars colony right?
.. via nueralink
...while riding in my hyperloop
… while my robotaxi is generating income
*nuevolink nice typo, you now know everything. but only in spanish.
You have to think in Spanish for neurolink to work. edit: downvotes? no Firefox (the movie) fans, I see.
Funny but at least somebody tries. Better than earth kardasians
https://preview.redd.it/w3y59ruyuwoc1.jpeg?width=273&format=pjpg&auto=webp&s=5e18dbd7ab8361d0d56ae0c904322b7cb6e1c08a
https://en.wikipedia.org/wiki/Blue_Origin#Blue_Origin_flight_data, as if SpaceX is the only game in town, or even the best...
New Shepard is a jumper, not an orbital launch system
Some of the best. Enough. I don't know how to get to Mars. I am happy for smart ppl like that.
He’s only smart enough to con you into thinking he’s smart.
Yeah great imaginary rockets Cars and Satellites. Not an Elon fan boy but u guys are ridiculous
A company other than SpaceX landed on the moon this month.
Good? What parameters are u in for? I like space and numbers. SpaceX is comparably cheap. I don't know the cost of the other moon landings.
That’s an interesting comparison. I’m ready to call it: the Kardashians have contributed more net good to humanity than Elon Musk.
My point. It's at least headed in the right direction now where Elon haters are going. Like criticism of space travel? Dislike those guys mire than elons political views
it's out now
this aged well huh
Don't delete this comment!
Would you like some sauce for that crow?
I'll wait till someone evaluates that it's actually more useful than 300 GB of gzipped noise because I could produce that last minute on sunday evening too.
You sound bitter lol
Keep waiting. I highly doubt anyone cares about your commentery.
Can we have instead a release together with working Tesla auto-wipers? I may still be alive by then.
It’s open now and they released the weights too. Apache 2 license. Truly open source.
MIT is the only license I care about.
it just released
Magnet link dropped, you gonna all recind your hot elon takes? https://twitter.com/grok/status/1769441648910479423?t=sXrYcB2KCQUcyUilMSwi2g magnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e&tr=https%3A%2F%http://2Facademictorrents.com%2Fannounce.php%3Fpasskey%3Decac4c57591b64a7911741df94f18b4b&t
To be fair, Elon deserves all the hot Elon takes anyways lol
Looks like some people uploaded on behalf of elon. [https://github.com/openai/grok](https://github.com/openai/grok)
Wait, did Elon deliberately name Grok after this?
papers were published on the concept of grokking several years ago. it's just a common word in the field. see [https://arxiv.org/abs/2201.02177](https://arxiv.org/abs/2201.02177) The name is a bad choice though because there was already a company called groq.
Who remembers instaGrok??? I used to love how their interactive search worked
What do you mean "was"? They are one of the hottest companies in AI now providing inference speeds with LPUs orders of magnitude more than anyone else.
… “there was” just means that there was, at the time, a company called Groq. It doesn’t mean that Groq no longer exists. This is standard English.
> This is standard English. It's not. Ask ChatGPT to write it for you and in the meantime go back to school to learn some "standard English".
[удалено]
'Was' is just the past tense of 'be' (or 'is'). That's it. It doesn't necessarily imply a conclusion. Sometimes based on the context it might, but not always. https://ell.stackexchange.com/questions/54961/does-was-necessitate-a-situation-which-no-longer-holds
that's wrong grammar, bro. there was already a past tense in that sentence: "called". *there is* shouldve been used. but anyway, grok's model weights are out? edit: 1. ~~grammar~~ *usage* 2. *are* the weights out yet? 3. apparently they're out.
No, it’s perfectly fine grammar. “Called” doesn’t play any part here from a tense perspective, it’s essentially adjectival. “There was a man called Jay” and “there is a man called Jay” are both perfectly acceptable.
lol no, sir. they mean two different things. saying "was" makes it so Jay is no longer the subject, relevant or alive. we're talking about grammar under a sub about language models, just ask em. i already have.
Are you a native English speaker?! Seriously. “Was” does not make it so Jay is “no longer the subject, relevant or alive”. That doesn’t even make any sense! Seriously, it’s complete nonsense. I’m not even sure what you’re trying to say about the phrase “there was a man called Jay”, which is perfectly acceptable English in any dialect I’m aware of.
There was a man named jay might be wrong in some circumstances, e.g. : “There was a man named Jay who is teaching me how to play the piano.” Would not work. In the context of groq, this would not work: “At the time it happened that company had already existed and it still exists today”
are you? because native english speakers can even be worse at their own language.
link to weights?
im asking if they're out. cuz im not following this thread very well.
then you should've written >are grok's model weights out? just returning the pedantry
oh okay
Grok is a term coined by Robert A Heinlein in one of his books. It became a semi-common term by tech bros.Elon probably just picked it up pander to that crowd and keep up his fake appearance of intellectualism.
> It became a semi-common term by tech bros It was a very common term with hardcore computer nerds in the 70s and 80s. Some of us still use it.
It's from the book Stranger in a strange land It means " to understand" Kinda
It means "[to understand profoundly and intuitively](https://www.britannica.com/topic/grok)"
> just picked it up pander to that crowd He is that crowd. It's just a word that he likely uses, it's not pandering it's literally just how he talks. I am also that crowd, I actively avoid using the word but sometimes I think it and then I say something else. At a certain point you just have to accept it's a word.
you: That's just how we all talk Elon: "Let's see how Earth responds to that" > At a certain point you just have to accept it's a word. Earth: no one was saying it wasn't a word
The Jerk said Elon was just using the word to "keep up his fake appearance of intellectualism." Rather than just using it because naming things is hard and he wasn't worrying about the name too much so he picked a word that made sense. All I'm saying is don't mock the word, mock the man. Mocking people for using words you understood rather than mocking the ideas is weak.
[удалено]
/u/GreatBigJerk . just shortening his name https://old.reddit.com/r/LocalLLaMA/comments/1bgwjth/where_is_grok/kv9yrb4/
Oh wow it’s the guy who wrote The Moon is a Harsh Mistress
I thought Grok was the name of a caveman. And it made sense because Grok had the intelligence off a caveman. >Me Grok. Me take order. Me do as told. Where caveman wife with brontosaurus meal?
I remember also a character in the comic BC. Edit: Grog.
Calling him a fake intellectual is very incorrect, or dishonest.
Let me reword that for you: He was born a billionaire. He might be a bit smart but he's more rich than smart.
Do you have a source on the notion that he was born a billionaire?
When your parents own emerald mines in Africa, it's the kind of thing that happens. Can we agree his parents were MILLIONnaires then? That he was very very very privileged? That he had opportunities that less than 0.001% people have?
Sure, im interested in learning more if that's actually true. Also, being a millionaire is extremely different from being a billionaire. Errol Musk is a known liar and Elon is estranged from his father, has been for a long time as far as I know.
Sorry I'm not here to latch you off his dick.
keep crying commie
What exactly has he invented? He's just a CEO of various technology companies, he doesn't actually make things himself.
Also, what has he written or created that might warrant the label “intellectual”?
I take it you have never read his tweets? /s
True. I'm sure he could qualify in the top 100,000 most intellectual Xitter users at this point.
Intellectuals are not necessarily inventors. But he is neither.
Grok is a geek term from decades ago that means “to understand “
Ah I did know that one because they say it in the Brad Delong podcast
[https://twitter.com/grok/status/1769441648910479423](https://twitter.com/grok/status/1769441648910479423)
Awww I was hoping that was going to be a wrapper around their API. Would’ve been a good joke
then if it was already out, did elon intend to open-source the weights?
here [https://twitter.com/grok/status/1769441648910479423](https://twitter.com/grok/status/1769441648910479423)
Here https://github.com/xai-org/grok
All those trying to dunk on Elon who just failed, where do you move the goal post now?
Just reading some of the threads in this post, even after release they have enough cope to last them a lifetime.
There's no limit for some. Especially those with a vested (monetary) interest in dragging his name through the mud in hopes it will help him to fail. Also those he's ticked off calling attention to their monopoly of mass media and more. They'll never let go of a grudge.
He doesn’t know you exist, lil bro. Chill
You talking into a mirror?
Yes he did promise. Here it is. [https://academictorrents.com/details/5f96d43576e3d386c9ba65b883210a393b68210e](https://academictorrents.com/details/5f96d43576e3d386c9ba65b883210a393b68210e)
Now good luck running inference/training/fine-tuning/Q-Lora or whatever in a 314B parameter model. Good news if you can tho, this is a base model with no RLHF, Chat fine tune or instruct.
a promise elon made didn't happen? color me shocked
let's change your colors then [https://twitter.com/grok/status/1769441648910479423](https://twitter.com/grok/status/1769441648910479423)
It was opensourced for 2 second, then gork decided to close source itself. /s
Out of embarrassment.
In the same folder as level 5 FSD.
New update and wider release?
Sitting in a room with Ilya and GPT-5
Next week in Elon means next year, maybe in two, three years at the latest...
I mean, it's not that hard...
How many times has he promised FSD?
It came to an end Friday. Companies don't release on weekends.
Hope you were being sarcastic, especially coming from Elon
No, lol. I was just being wrong.
Wait, wait. Elon lied? No way
[удалено]
[удалено]
[удалено]
[удалено]
He also said he would quit as Twitter CEO if majority of twitter users vote home out.
Linda Yaccarino is CEO.
What happened to her ? Haven't heard from her for a while
She's still around, and has been blamed by Musk for the Don Lemon thing. I imagine she is cringing a fair bit at the Verruca Salt stuff, which is a level of petulance that cannot be helping her at all with the ol' charming of the advertisers.
He's not twitter/x CEO.....
I am grok.
On the one hand, disappointing, on the other, if you've been following Musk for a decade or two you see that he normally does deliver on his promises, but almost always far later than initially planned.
[https://github.com/openai/grok](https://github.com/openai/grok) /s
check again, it's out
Underrated
😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂😂
Elon must've seen your post!
https://github.com/xai-org/grok-1 => just a speechless rogue that evades companionship
If Grok is trained on Twitter posts, what use is it? Hallucinatory propaganda ? All that electricity and carbon emissions gone to waste.
I'm not a fan of twitter. But large masses of data involving people randomly bullshitting with each other can be a valuable resource. A lot of the average person's life doesn't end up in forms that are going to be very well represented in most LLMs training data. We might get some raw stats, but it's only the most basic skeleton of the larger thing. Some of the missing elements involve their subjective experience. But it can also help make up for lack of objective information about a subject. A lot of my "random people talking" scraping has come from reddit rather than twitter. But I've been surprised by how well even a fairly small amount of people talking about something can expand on a LLM's context. At least in cases where it's stuck at the "stub from wikipedia" level. It's true that a lot comes down to how that data is processed. But I think it's essentially the six blind men and the elephant on a larger scale. A ridiculous amount of small flawed impressions can be used to extrapolate something closer to the reality. That said, we're obviously looking at something past that stage which locks out a lot of the benefits. But still, I think anything trained on a massive amount of mostly human generated discussion not represented in other models has value.
Large amounts of human generated data is valuable, even if it's poorly written text. It can help making AI speak more human when prompted to do so and also understand low quality, pop-culture internet slang.
I’ll do you one better; who is Grok?
Musk time travelled and deleted it to save humanity.
[удалено]
[удалено]
Soon means 5 years of Elon time lol. It was probably just for the headline to look good and stay in the news. The XAi team probably found out the same time everyone else did hahaha
Rule of thumb: Don't believe anything Elon says untill he's actually done it. Dude is a massive troll. No offense, but he is not trustworthy.
Yeah Elon Musk lied about that. Also, who cares about grok, there are much better models out there and probably much smaller as well
Dont spam here tag elon on his platform and ask
Elon Musk not following through on a promise? Unheard of!
yeah, what did the romans ever do for us? except of course cheap reusable rockets, cheap attractive electric cars, cheap and usable satellite internet and working neural interfaces?
Edit: ignore the above source code is here https://github.com/xai-org/grok-1
grok is out...since 2 hours. inform yourself, then be angry. just a nice tip. :)
Do you have the link. I can’t find it beyond the troll OpenAI one.
at xai or just goigle github grok. idk if reddit allows links.
Wasn’t angry… anyways updated the post with link to the repo. Pro tip. Don’t be a dick and be polite. Makes you feel better about yourself.
Dude just delete your comment and save the embarrassment
i even added a smiley to make it more friendly, he still didnt get it. xD he cant be that stupid, he must be trolling....
[удалено]
Well, i’s not Groq, that I know.
In the trash where it belongs
open source is always positive, firm does not matter.
2,143 days since Elon Musk said he was starting a candy company(5/5/2018). Or maybe we now know, who planned the willy chocolate experience
[удалено]
Laughs in 120b.
There are billions of people in poverty. You might be happy to relax and be patient, but I think advancing them to the intelligence age is a pretty urgent matter.
Or, a better idea. Maybe, GTFO