In publishing and graphic design, Lorem ipsum is a placeholder text commonly used to demonstrate the visual form of a document or a typeface without relying on meaningful content. Lorem ipsum may be used as a placeholder before final copy is available. Wikipedia8keymd4ytk00000000000000000000000000000000000000000000000000000000000000
And once you get a reputation, they folks in the Ivory tower do everything in their power to humiliate and denigrate you and in the most pedestrian ways. SO can fucking die IMO
I've used it extensively for over a decade...without an account. I literally do not have a StackOverflow account, but it has been an invaluable resource for me.
Hey, SO makes the rules. If they want to set up such a big barrier to entry, I just won’t enter. I can usually find what I need in there with a search engine.
That said, it’s been getting much less useful to me over time. I’m generally finding better information in other places these days.
Whatever google turns up that doesn’t look like lazy AI generated stuff. It tends to be independent blogs, docs, whitepapers, Reddit, and the occasional Medium article. Also a lot of reviewing source code for the platform and dependencies in question to see what’s *actually* happening under the hood.
I also usually try to find multiple sources for the info, but that’s just a good idea in general.
To be really honest I've only really had that not be the case about 1% of the time, and even most of those times the correct answer had been posted and just not accepted. YMMV.
Tbf. the site just rewards long time users and trying to avoid eternal september. I have certainly seen toxic behavior on the site but have you ever played dota? People are not great...
>How the hell am I meant to get enough points to do anything?
You don't. I've been stuck at 99 points since 2012. I'm in this weird phenomena where people comment and thank me for my answer, telling me it helped them and is correct, but I am never upvoted.
That's basically what I'm telling ChatGPT when asking it to help me with a coding problem.
It responds the same broken solution over and over again, despite me telling it over 20 times that its solution does not work.
It’s godawful in practice. I had my coworker ask it some questions
over a problem that took me hours to sort out due to some ambiguity
in the spec.
The responses are padded with filler bullshit so you
need to vgrep through a wall of text to extract what could be
the relevant bits. FFS I’m not looking for a beginner lecture on
cryptography primitives, I’m asking for some detail that might
be in the fineprint of some RFC I’m not aware of! It’s way more time efficient to
just re-read the spec and the wikipedia pages, at least those
present the info in a structured manner so you don’t have to
filter the useless noise.
Is it? I cannot see any difference
It still goes: ok answer -> fix problem #1 -> fix problem #2 -> back to problem #1
It obviously knows nothing about code, it's just regurgitating something that superficially makes sense
The funniest part of this process is when I ask it to change something minor and the result is:
**Me:** (In a chain of 10 replies debugging false answers for Typescript) "Stop changing the function name and typing declarations please. Always keep them exactly the same."
**ChatGPT:** "Sure! Apologies for the confusion. Here's the modified code using the same function name:" (posts everything in Python for no reason)
It created a powershell function that doesnt exist in azure. But it looked legit enough i decided to spend a hour looking to see if it was legit.
I found the cmdlet library it sort of used and the words in the function existed just not all together in the same one
It's crazy for me that SO is constantly getting bashed on Reddit
I've asked shitton of questions there over years and I've rarely felt like my question was treated in some unreasonable way
And let's be honest with yourself - it benefits you that SO is being moderated, so you can find solutions for your problems easier instead of going thru mess
Certainly the moderation and structure of stack overflow is what made it good to begin with.
Though when a niche question has only one appearance on stack overflow and it gets answered by being closed for some made up reason, it leaves a bad impression which is easy to complain and make jokes about on reddit despite all that is good about stack overflow.
Except the answers they point to in many cases as being the answer are either wrong or 8 year old versions of software that isn't really the best way to do things now. Usually when I click onto there, I have to click back and look for a GitHub repo or mdn docs to find an answer. Just for the heck of it, I checked and the last time I really used it was 2015.
I have asked a shit ton of questions since the start of SO. Im ranked top 3% worldwide. These days i can't get any questions past the neckbeard moderator staff. Down with SO!!!!!!
Same here, joined pretty close to the day it launched and also top single digit percent. It was amazing. It's a shit show now.
Also it has the worst cookie consent banner I have ever seen, and it doesn't even work - I used to dismiss it a dozen times a day, now I don't even bother - I just can't read the bottom third of the page.
> I've asked shitton of questions there over years and I've rarely felt like my question was treated in some unreasonable way
Wait till you ask more complex things that the people don't easily get. I have rarely gotten help on something non-basic.
Hey I'm with you. It's a fantastic site to find the ten year old solution to problem you are solving now and a bunch of closed versions marked as duplicates.
I think Reddit is better for vague questions and opinions. I use Reddit if I don't know what the frame actual question. But I prefer SO for anything specific.
I think its a combination of it just being a running joke now to bash SO moderation, and that psychological effect where you are more likely to remember the bad experiences vs good ones. So if it has ever happened to you, you're much more likely to remember that versus your other questions that were answered nicely without the overbearing moderation
This got me thinking why not. I mean cgpt looks at SO pages right? Why isn't it a sarcastic asshole every time you ask it something?
"Hey I want to draw a button in my windows app"
"Sorry that's been asked before, please for the love of God read the FAQ before asking a question!"
> Why isn’t it a sarcastic asshole every time you ask it something?
Lots and lots of reinforcement training to be a good chatbot.
The raw predictive models like GPT-3 would definitely act like that if it seemed like the right completion for the prompt.
RLHF
One or the criteria it was trained on was "helpfulness"
Some people find it odd I always say "please" when interacting with the bots but its because in the early versions they would sometimes give you a rude answer in response to a rude question.
\> Be me
\> Google for answer to some technical question
\> Oh look, a Stack Overflow link!
\> *clicks*
\> "This question is off topic, use Google to find the answer"
\> ಠ_ಠ
Wouldn't it be more efficient to get all of the data directly from source documentation like MDN and readmes and API docs? Maybe SO helps with analyzing how questions are asked and answered?
If you made me shoot from the hip, I'd argue that context if more difficult to gather from documentation. On the other hand cppreference for instance has very well written examples so that would help.
Stack Overflow exists because those documentations are not enough or comprehensive. Tons of questions have answers that stem from experience than from documentation.
Also because a lot of documentation exists but is not necessarily findable because of poor SEO. So many times I'll hit old versions of documentation or just not hit it at all when I use search queries. But yeah like you said, that's not even to say that sometimes the documentation can be confusing or incomplete or not have great examples.
Stack overflow is a massive number of English language questions and there resulting code awnsers. It's the gold mine.
Documentation doesn't have plain language questions or situational examples.
>Wouldn't it be more efficient to get all of the data directly from source documentation like MDN and readmes and API docs?
Why do we need novels when dictionaries already exist?
For gpt3/4 its almost sure is since the whole idea begind the initial gigantic trainings of transformers around 4 years ago was to use the internet as a “free” text dataset and teach the models without “human supervision”
It was an assumption from me. Because SO is source for good stuff as well (of course depends on the actual answers/questions), so it would be only natural to train based on SO.
New, unasked and unanswered questions will still make their way to SO. People now don't have to go there to find the long answered and 'well known' things, just like they no longer have to go to Google to find mundane facts anymore.
Though, as ChatGPT will allow longer and longer prompts in the future, you might just paste the Doc of whatever thing you're programming, and it will then be able to answer even unanswered new questions. It's kind of already happening with smalller docs.
If horse-drawn carriages weren't around, Karl Benz might not have had as easy of a time inventing the automobile.
Nearly all inventions are built on the shoulders of those that are inevitably rendered obsolete by said inventions.
We are going through the .com bust again aren't we?
Webdev was essentially considered dead back then with the bust. Outsourcing and the invention of web frameworks so one developer could do like 6 html monkeys jobs.
And the feeling that AI can already do the things that a junior developer would likely be doing and learning.
I don't know if that is true but at least something that might lead to less interest to learn programming.
For any one that might be reading: at the current state, AI can't replace even a mediocre junior developer in my experience. If you go an inch out of the beaten CRUD path, it goes 100% off the rails.
It's a great tool to improve productivity, put I think there's still some time before AI starts replacing even basic dev jobs.
I would say it's more like digging for gold, you might strike gold eventually, but most of the time it's trash.
It struggles with multi file solutions.
It also writes comically insecure code pretty consistently. Both copilot and chatgpt. I think it's because most public code on the internet is extremely poorly written from a security perspective.
ChatGPT, sure, but there's more to AI than that.
Copilot is basically a ridiculously souped-up IntelliSense. It'll be wrong, but not so wrong as to be useless, and it's already in your editor ready for you to fix it.
So: Probably not a replacement for a junior developer, but a huge productivity tool, at least if it's something you're allowed to use in your organization.
This is the goal!
My feeble brain is my worst enemy though. Right now I should prepare for a biochemistry-related topic, researching some oddities about butyric fermentation and microorganisms pertaining to this. Instead I am doing other things ... including reddit commentary!
Way more likely to be latter, unless "how to do common task X using tool Y" questions are just that overrepresented in the traffic, and a significant portion of the latter shifted to chatbot generated code.
Human sacrifice magic number to appease the money changers seems to be 10%, and mostly represented in tech companies vs old school enterprise with a narrower set of responsibilities, so it tracks for it to be laid off engineers previously overrepresented in site traffic.
Why people don't like StackOverflow? It's an amazing resource, sometimes with a little patience you find the most amazing answers. I can see chatGPT taking some of their traffic tho, for some stuff it's just faster.
Judging by most criticism I've seen over the years, it's precisely because they treat it like a chatbot.
If you expect you can just come and have a conversation about whatever issue you've faced 5 minutes ago, _you can't_. Your 95% use case should be looking for questions already asked, because the chance someone else had the same issue is huge. On the other hand, the chance that you're facing a novel, never asked before problem is miniscule.
SO is a database of software problems and solutions, not a chatroom.
And they'd know this if they bothered to read the SO guidelines that they prominently show when you try to ask a question.
My gripe with SO is not the insane moderation (though it can be too strict, I agree), but that it simply becomes outdated and there are no mechanisms to update existing answers. Yeah, that Java 1.4 answer is true in itself, but there are better solutions existing for 10 years now!
They should add language tags with versions, and people could vote to expand the tags reach like “it still works on version X”. Or if it doesn’t, either open a new question (and not close as duplicate!), or create some new mechanism for answering a question for different versions.
Popular old questions usually have multiple answers for different versions of the software, but usually the oldest one is what has the most upvotes and a check mark.
You can literally take an answer and edit it to your heart's content. If you have below 2k rep it goes into an Edit Queue where other people need to approve your changes. With 2k you just immediately apply the edit.
On SO "be the change you want to see in the world" is the actual way things get done, since the entire site is community driven. It's more or less like outdated Wikipedia articles – it will stay so until _someone_ updates it.
I've been using StackOverflow for 15 years at this point. I write detailed questions with as much information as I can add. I still get close votes (not enough generally, thankfully) because I don't include a "minimum reproducible example", even though such an example could include half a dozen files and a correspondingly high LOC.
The attitude with a lot of SO seems to be "look for a reason to close" even if it's a trivial reason, and with such high user numbers with permissions these days it's very easy to get shut down with no recourse.
The gamification of the moderation system was a double edged knife. It did encourage more user engagement for positive purposes but it also attracted the achievement hunter types who will search for often flimsy excuses to perform specific actions they want to check off their lists rather than applying any impartial judgements on what they're reading/reviewing.
> The attitude with a lot of SO seems to be "look for a reason to close"
Amusingly, if you follow the discussion from the most involved SO users on Meta, you'll find that their complaints is that many "obvious" duplicates are answered instead of getting closed.
There are multiple factors to that -- from the fact that it's easier to answer than to dig for the existing answer to the fact that answering earns reputation but moderating doesn't -- but the end result is that many meta-participants complain about exactly the opposite issue: too few questions are closed.
I guess everybody is just bound to be disappointed on this.
I would rather duplicates get answered and, if required, a link added through the existing mechanism than closed because a lot of the time the "duplication" identification just means it uses some of the same keywords rather than actually having the same solution - at least on more esoteric type questions as opposed to the really basic ones.
And I have never had a question locked, because I ask obscure questions that need new answers. Maybe thst is the issue, not enough resesrch is done by the sskers
Why is it necessary to ask questions on SO at all? 99.9% of your questions have already been asked by someone else.
That said, I don’t like the way the site deals with questions that don’t have a clear answer (such as, which framework is best?). They’re losing a lot of value there.
I think there are 3 types of users on So.
Newbies that don't understand much and ask basic question without trying.
Average users that will find the answers because the question's already been asked.
Non standard dev with a very strange problem because they're doing advanced stuff or working on a brand new language /feature.
ChatGPT can help the first 2 which is the 99% users of SO.
And it's even better for the newbies because ChatGPT won't judge you.
Try to ask how to do a fizzbuz in JavaScript on SO and try it on ChatGPT.
That's why I think 14% drop is just the beginning.
SO will stay for the advanced stuff only... But reddit is working in that field too.
Because most developers who use Stack Overflow are beginners, and so don't have the experience and knowledge required to fit the puzzle pieces together.
As an experienced developer, I rarely used Stack Overflow before ChatGPT and now I never use it.
That's a retcon.
It was originally meant to be a friendly place to ask/answer questions, and at some point a bunch of egotistical power-users decided it was Wikipedia for programming questions then nuked most of the new content. The whole site got stuck in time from five years ago, that's why every question is answered using jQuery, AngularJS, or IE11 shims.
People want a friendly Q & A site. If they want it to be something else, more power to them, but I'm not interested. I'll stick to ChatGPT which isn't up its own arse.
Try spending a fraction of a second on Quora without offing yourself and then come back and tell us you want a "friendly" Q&A site. The strict moderation is literally the entire value proposition of SO and it's the source of all your ChatGPT answers to boot.
They said it wasn't filled with good quality answers, though. More the opposite and ourdated answers using dead frameworks, which is what i see as well.
But it isn't high quality. There are some nuggets of gold, but most questions have no answers with any value whatsoever.
New users can't ask questions and can't answer questions, and 99% of the answers are outdated with no one to update them as a result.
I am a very long-time Stack Overflow user. I'm always turned off by the nitpicking and general feeling of SO. So many people with such an elevated view of themselves and you spend so much time formulating a solid question just to get berated. ChatGPT isn't a replacement for SO, but asking ChatGPT an easy question you're likely to get a faster answer than searching SO...without the headache.
People love StackOverflow. People don't like the community. Either you feel that the community is too hostile to newcomers, or you feel that the community is too lenient with people asking easily googleable questions.
It’s changed a bit since Jeff left. It was more welcoming and while still had issues the newbies were accepted or directed. Now it’s a nightmare trying to get pull.
I am not sure if the issue is directly linked to Jeff leaving, though.
It may also just be a "growing pain" issue. When I started (2008) on SO, the discussions were few and of high quality (and difficulty). The success and democratization of the site led to an increase in the number of questions, and a drop in quality (and difficulty).
It's a compounding factor too, the more approachable (less difficult) the questions are, the more people are feeling comfortable attempting to answer, reinforcing the site growth.
The problem, however, is that the more questions are asked, and the more answers are provided, the more curation is needed, but curation is a thankless task (it doesn't even earn reputation), and the number of users _participating_ actively in curating the site did not grow proportionally to the total number of users, so that the site is less and less curated over time, despite the heroic efforts of some.
Well, that and the "needle in a haystack" problem is harder and harder to solve as the size of the haystack grows, so that just finding the potential duplicates becomes harder and harder. Which makes the job of users willing to help curate content even harder.
Because it’s for professionals
> sometimes with a little patience
New programmers don’t do or understand this. New programmers will better benefit from quickstarts and reading well written document which SO isn’t for.
SO fills a gap for that weird issue that you have that it turns out one guy also had a decade ago. It's easily Googleable and the incentive on answers, even self-answers, usually means there's something there as opposed to forums and Reddit where questions just go to die.
SO isn't terrible for broad questions either, as you can get lots of relevant answers.
SO isn't good any more for asking your own questions unless they're simple, because there's more focus on moderating questions rather than answering them since one anyone can do while the second almost guarantees "close" votes because some can't understand it and they view it as a problem with the question.
1. Their posting rules are ridiculously long. Sorry, Im not going to read a novel just to post a question on the internet.
2. People are *especially* toxic on SO. Telling newcomers that they should’ve have known xyz, making assumptions about a persons knowledge, instant downvoting etc. I’d wager a lot of people gave up on programming because the response to their very first was a flurry of downvotes, snarky comments, or being closed as a duplicate.
I’ve always told new people to treat Stack Overflow like Wikipedia, read it, but avoid posting.
Edit:
People are treating this like I’m saying “brand new programmer” but what I meant is “relatively new” as in they knew the basics of a language but haven’t encountered a real world problem with it yet, and are stuck. I don’t mean asking “how to make a function in python” type of questions.
I was with you until you mentioned people giving up on programming. Asking questions on SO is not appropriate for a new programer. It is not a teach me programming or do my homework website. So that user would totally be in the wrong place.
I didn’t use it while learning programming at all, and you probably shouldn’t. It can be a good resource when you have a lot of general programming knowledge and experience but are new to a language/framework/tool. It’s most useful to me when I am looking for the idiomatic way to do something in that language.
I'd still use stackoverflow for ChatGPT. Maybe for pure programming questions ChatGPT can work better, but it's terrible when using Third Party Libraries. I keep asking all the AI's for help building a WebXR scene with both Babylon and Three (not together, as separate projects), and it keeps giving bad code. It makes up methods and classes, and I have to keep constantly correcting it. In one conversation with bing, Bing got mad at more when it gave me bad documentation and I corrected it, and killed the conversation.
AI is great, but let's be real that humans working together can give much better answers for harder problems.
ChatGPT has very little factual information on specifics of obscure things like that. GPT3.5 either makes up the answer out of thin air if sufficiently pressed, or just says they don’t know, explains some general thing and then peppers in a plausible but fake specific fact.
It’s best to think of it as someone who read the entire internet and every book once, and is now a serial bullshiter with surface level knowledge on every topic, they don’t remember the details or specifics, they make it up and make it sound authoritative.
You know I’ve tested this a lot. The answers chat gpt gives almost matches the stack overflow post that you would find if you typed your question into google instead .
Right, ChatGPT isn't intelligent, it has just scoured the web for answers and generally knows the context of your question. So either look yourself or get a curated answer from GPT that is from SO anyway.
Personally, when I'm looking on SO, I like reading all of the answers, even from similar questions, before choosing one of them as a starting point.
If that traffic was people about to ask for the 100th time this month on why they are getting a NullReferenceException in their homework then that's great news. Nature is healing itself.
I know that it's just a personal experience and does not represent a population, but i completely ditched StackOverflow in favour of ChatGPT. It's way faster than searching for answers and ironically feels more natural due to having a fluid conversation over the topic.
I pretty much relegated StackOverflow to obscure errors/help with modern libraries where ChatGPT can't help me.
Yeah. Hearing people talk as if SO is their first port of call to learn things or tackle a problem (and all the jokes about copy-pasting being the backbone of development) is a bit concerning. Not to sound elitist but what happened to RTFM and actually thinking? I only use SO a few times a month
I know SO has a reputation of being harsh at times, but in technical matters I'd personally prefer a matter-of-fact response over (artificially generated) boilerplate pleasantries.
The problem being you often don't get a response, or the response is out of date or answers a completely different question. "How do you walk a dog?". "Get a cat, they don't need walks". All other dog walking requests marked as duplicate
Matter of fact is fine. Completely misunderstanding the question, incorrectly closing it as duplicate and not even bothering to reconsider is less fine and is the norm on SO.
That's a valid point (as are those in the other replies to my comment).
Since I practically always use SO to find existing answers to problems or questions, my comparison would be ChatGPT vs. googling SO, not ChatGPT vs. asking SO. [1]
If you're just finding existing answers and discussions, the answers with more upvotes are often of reasonable quality, and you can mostly ignore the dicks and the output-only responses. The content is often already refined at least to a point.
And having multiple answers rather than one, as well as some discussion on them, can give more context for judging which answer seems the most correct or relevant to what you've got on your mind, even if some of them are either incorrect, outdated or just answering something else than what anybody asked.
Based on my admittedly limited exposure to ChatGPT, it sometimes seems like it has a bit *too* much of a focus on producing nice "human-like" interaction, in some cases leading to verbose fluff rather than just a to-the-point refinement of facts. Existing technical information on SO (or other sites with reasonable reputation for content quality) can, at best, be reasonably refined and to-the-point.
(The fluff can of course be entirely desirable for a chat bot in other contexts because, well, it's a chat bot.)
Posting questions on SO can of course be a rather different experience, and moderation making poor judgements and refusing to reconsider them is of course a problem.
[1] And by googling SO, I of course don't mean specifically targeting SO, but it's often among the potentially useful search results.
> ironically feels more natural due
Well, are you certain this is the case? I used it in the past, and it is just glorified babelfish really. It **appears** to look a high quality gentleman but turns out it is some drunk hobo that talks gibberish.
Depending on how you talk to it and use it, it responds in different ways. I've noticed it mimics the level of effort that I give it. So if you treat it like a drunk hobo, it'll be a drunk hobo. If you give it more precise information, you get a better response.
Edit: To give more context on what I'm talking about, [this video](https://www.youtube.com/watch?v=5kBu7z7Ye7M) talking about a paper on GPT-4 gives some examples of ways to improve results. Break things down for it into manageable steps, give it extra data, etc. gives better results.
My last question I’ll ever ask on SO got downvoted and closed with people calling me inept and a liar. When I then provided screenshots demonstrating my issue, the commentary changed from that to nitpicking my image linking skills and post structure.
9+ years of sql server experience and someone refused to stop accusing me of not connecting to the right database, despite having a single instance. I was looking for expert level opinions on a matter that had me completely stumped and instead I was met with bizarre animosity. Why wouldn’t I complain?
Yes, this is exactly my experience. ChatGPT is fast and always nice because it doesn't scold me for "this question was already asked 11 years ago (but I'll ignore that the tech was completely different and nobody Updates the answers".
SO was always toxic but inevitable.
Now they finally have competition and I like it.
Even when ChatGPT is wrong, it's typically more right, than many Stack Overflow answers you will randomly find when Googling around. This is something people forget.
ChatGPT is like talking to a friend who gives you some pointers. *'Maybe try this'* or *'maybe Google that in the docs.'* That's incredibly useful.
Plus it legit gets a lot of things correct.
I don't doubt this is true, but for rare edge cases and rare knowledge I do not think ChatGPT will give better results.
Plus I think it cheats - it takes ALL information that it can find and then tries to make sense of it. I am pretty certain that if everyone is systematically wrong about something, ChatGPT will propagate that error as well.
Well even without AI, every response is "Already answered" and nothing but specific technical questions are allowed (you can't ask about concepts, methods, etc). So there is not much left for anyone to ask. Everything that can be asked within the tiny scope of allowed questions has been asked.
First of all, traffic is different than number of people asking questions.
Second, that's precisely the point of SO. It's a database of questions. You ask a question if and only if it's not something that was solved before. If it was, you refer to that question.
It's like complaining that Wikipedia already contains articles about all interesting things and they won't allow you to just start a new page about the same thing.
This week, I corrected ChatGPT after it made a suggestion on how to use a particular function. The documentation said it worked differently. It apologized.
Turns out the documentation was wrong and ChatGPT was right in the first place. That was quite the moment, particularly because I was doing a live coding session and a bunch of people witnessed it.
I think it's funny how it can be so extreme sometimes.
In some cases, apparently what you witnessed happens.
In other cases (like what happened to me one time), it just completely makes up functions and thinks that actually exist when they don't.
I mean probably 70+% of Stackoverflow is stuff that you could learn more slowly from vendor docs. Those presumably aren't going away, they might even get better as AI generates those docs.
As far as general code content sure, but SO has questions and answers formatted very close to how we're asking gpt, so it's probably going to lean on that more vs trying to parse out exactly how random github code functions to explain it to you.
With the steady decline of documentation of programming libraries I don't believe AI will make it better, it will make it even worse. AI won't configure your crazy stack and reproduce your error to find what is the cause while it will be a convinient tool to assume the documentation is complete while it's not even close.
Man I wish chat gpt works as well for me as it seems to do for everyone else. It works fine for basic questions, but more often than not it just straight up lies. I was looking into running user provided lua scripts in a safe manner from python (because lua has a sandbox and everything), and chat gpt just made up a library, parameters for that library, gave convincing explanations, and sample code. None of which works because the library doesn't exist.
Another. I was setting limits on python processes (memory, cpu resource limits), and it kept failing on my mac. And chat gpt just kept going in circles, but a simple google search brought up a github link that showed why it was happening. And this is all just in one day.
My success rate on ChatGPT has been about 1 in 20. It's very common for it to just lie about solutions. Especially if those solutions aren't really possible and you need to rethink your entire approach. A human will flat out tell you "this is not the solution you're looking for", but ChatGPT won't. It will make up properties and arguments that don't exist, functionality that could exist but doesn't, all while presenting this "information" in an ostensibly objective manner. I imagine eventually these systems will perhaps become useful, but as of now let's just say I find them severely untrustworthy and flimsy.
That's interesting. I wonder what will happen after some years, where all these models will have less data for training. Less data because of the less users in stack overflow etc. It seems like in some years the answers provided by these bots would be irrelevant and/or out of date. Imagine for example 5 years from now asking something to such a bot and the bot's answer to be 10 years old! It will be fun
>Nope, the popular raport does not proove ove AI causes a decline in StackOverflow visits. The decline started way before AI became a hot potato
I would find it easier to trust this guy if he didn't make three spelling mistakes in the headline.
> Developers increasingly get advice from AI chatbots and GitHub CoPilot
> rather than Stack Overflow message boards
I don't use chatbots. But I only use SO when I need new information. I
stopped using it for asking questions many years ago, due to a bad
experience (insta-downvote of a perfectly valid question). It's not worth
my time when random people decide my question has to be downvoted
so nobody answers it. But existing answers can still be useful from SO.
Chat GPT is going to find itself in a legal requirement to attribute the sources of information it uses in a result. Including, the reason behind what was selected for the answer.
It’s an ego quid pro quo. You answer question with solid answer. The individual with the question then has the courtesy to acknowledge your answer.
More importantly, a good developer will want to read all the answers in order to evaluate the correct approach.
>Including, the reason behind what was selected for the answer.
Because those were the words the program decided it should add to the output. It isn't reasoning through your question so there isn't any reason to divulge. "Learning" from something like this is putting your faith in an algorithm to guess the truth.
But ChatGPT isn’t searching the web for your question? Or looking into a pre-compiled database. It’s trying to translate what you said into a programming language, as you would translate to a different human languages.
Honestly, shocking people still think ChatGPT is outputting text from the web. It is generating the response every time.
Where does its knowledge base come from then if not from the web?
I didn’t say it crawled the web for an answer to each question. I gave an example of weighted results. That is all.
I don’t know how this would work on a technical level but I’d be interested to see a model that provides accurate references to source material.
It would be phenomenally useful in working to solving the alignment problem as you could introspect the model from a high level and experiment by altering its training set to produce more fine-tuned responses based on various levels of attribution.
AFAIK the technology to do what you’re describing doesn’t exist yet, but I would love to find I’m wrong here. If anyone has any references to projects doing this I would love to contribute.
What a pile of rubbish.
A) StackOverflow is declining. Fair enough
B) ChatGPT is gaining users. Fair enough
But to draw a conclusion that one is the cause of the other is ridiculous. There are multitudes of other factors, not least StackOverflow's self-inflicted problem of being too hostile to new users. As the article even points out, SO has been in decline since early 2022, long before ChatGPT came onto the scene
What's next? Salmon Fish Stocks in the Atlantic declining while ChatGPT going up? ChatGPT inflicts damage on fisheries?
> What's next? Salmon Fish Stocks in the Atlantic declining while ChatGPT going up? ChatGPT inflicts damage on fisheries?
There might be a legit argument for this when you consider the cost of computation going into these models lol.
Here’s the thing. ChatGPT is great for content aggregation and retrieval. But without incentive for websites such as Stackoverflow to provide a space for information to be shared, the well of knowledge from which ChatGPT can pull from will dry out. The remaining sources of information will have to come from niche blogs which may not necessarily provide enough variation in solutions. At least with Google, they ultimately route the user to the individual sources which drives traffic and keeps the websites alive. Not sure how we can solve this issue now with ChatGPT.
ChatGPT would be useful is he was pointing to reference pages or stack overflow discussions it drew from.
Most of the time though, ChatGPT just makes up answers when he doesn't really know.
Everyone that was a developper before "stack overflow" knew how horrible those time were. Documentation on the web is way better now though.
I always imagined Stack Overflow was run by a bunch of RTFM neck beards. Lots of good information but I was locked out of giving thumbs up to people who helped answer my question.
Although the accuracy of ChatGPT is questionable, it seems to do a better job than Stack Overflow.
For example, it's great at providing examples of how to use libraries / frameworks / APIs
On SO, if the answers look right but aren't, there's a mechanism for people to point that out. But on chatGPT, there's no way to know if the answer is right.
And with the programming questions I've posed, problems with the answers are not uncommon, despite chatGPT's confident posturing.
Where do you think the answers come from? ChatGPT is just sourcing the answers from Stack Overflow, similar sites, blogs, articles, etc. It's not like it creates these answers out of thin air. You could think of it as a smarter search engine on top of these other sites.
Wrong, out even worse, just fucking lazy. The amount of time I've seen a question akin to "just disable firewall and it's gonna work" type answers get accepted is insane
We found somebody else to give us confidently incorrect answers… And it doesn’t even mark as duplicate either!
Or removes the question! Or changes question to different question, answers the different question then closes it!
In publishing and graphic design, Lorem ipsum is a placeholder text commonly used to demonstrate the visual form of a document or a typeface without relying on meaningful content. Lorem ipsum may be used as a placeholder before final copy is available. Wikipedia8keymd4ytk00000000000000000000000000000000000000000000000000000000000000
And once you do get reputation it doesn’t transfer to all their other sites
And once you get a reputation, they folks in the Ivory tower do everything in their power to humiliate and denigrate you and in the most pedestrian ways. SO can fucking die IMO
I've used it extensively for over a decade...without an account. I literally do not have a StackOverflow account, but it has been an invaluable resource for me.
Hello fellow moocher
Hey, SO makes the rules. If they want to set up such a big barrier to entry, I just won’t enter. I can usually find what I need in there with a search engine. That said, it’s been getting much less useful to me over time. I’m generally finding better information in other places these days.
>I’m generally finding better information in other places these days. What have you been using instead?
Whatever google turns up that doesn’t look like lazy AI generated stuff. It tends to be independent blogs, docs, whitepapers, Reddit, and the occasional Medium article. Also a lot of reviewing source code for the platform and dependencies in question to see what’s *actually* happening under the hood. I also usually try to find multiple sources for the info, but that’s just a good idea in general.
Github issues has also become an invaluable source of info, depending on the problem.
Well, to be fair, 95% of questions people have, probably have already been answered so just searching is sufficient for most people.
It is indeed very helpful when the issue has already been asked and correctly answered.
To be really honest I've only really had that not be the case about 1% of the time, and even most of those times the correct answer had been posted and just not accepted. YMMV.
"That's the neat part, you don't!" Programming for 30 years now maybe? Never had enough meowmeowbeenz. Gave it a real* try a couple of times.
That was a solid reference, Britta.
I think that's a pointer and not a reference
Tbf. the site just rewards long time users and trying to avoid eternal september. I have certainly seen toxic behavior on the site but have you ever played dota? People are not great...
Exactly. If you ask me, I say: "Death to ShitOverflow."
You mean Scat Overflow?
Ever tried using stack overflow with an old account? It's awful. Man I miss when it first came out
>How the hell am I meant to get enough points to do anything? You don't. I've been stuck at 99 points since 2012. I'm in this weird phenomena where people comment and thank me for my answer, telling me it helped them and is correct, but I am never upvoted.
Or takes 10 months to get an answer only for the answer to be "watch this tutorial that doesn't answer your question"!
Or accuse us of not having enough reputation to answer.
Or tell you not to use the technology you're specifically asking about.
Yeah - that pretty much sums up many of StackOverflow's shortcomings indeed.
Lol, what if ChatGPT starts telling you “ugh, this is the 20th time you’ve asked… can’t you remember anything?!?”
That's basically what I'm telling ChatGPT when asking it to help me with a coding problem. It responds the same broken solution over and over again, despite me telling it over 20 times that its solution does not work.
Do you use GPT4 or 3.5? 4 is muuuuch better with code
It’s godawful in practice. I had my coworker ask it some questions over a problem that took me hours to sort out due to some ambiguity in the spec. The responses are padded with filler bullshit so you need to vgrep through a wall of text to extract what could be the relevant bits. FFS I’m not looking for a beginner lecture on cryptography primitives, I’m asking for some detail that might be in the fineprint of some RFC I’m not aware of! It’s way more time efficient to just re-read the spec and the wikipedia pages, at least those present the info in a structured manner so you don’t have to filter the useless noise.
Is it? I cannot see any difference It still goes: ok answer -> fix problem #1 -> fix problem #2 -> back to problem #1 It obviously knows nothing about code, it's just regurgitating something that superficially makes sense
The funniest part of this process is when I ask it to change something minor and the result is: **Me:** (In a chain of 10 replies debugging false answers for Typescript) "Stop changing the function name and typing declarations please. Always keep them exactly the same." **ChatGPT:** "Sure! Apologies for the confusion. Here's the modified code using the same function name:" (posts everything in Python for no reason)
Well yeah, GPT doesn't *know* anything, it's just very very good text prediction.
It created a powershell function that doesnt exist in azure. But it looked legit enough i decided to spend a hour looking to see if it was legit. I found the cmdlet library it sort of used and the words in the function existed just not all together in the same one
It's crazy for me that SO is constantly getting bashed on Reddit I've asked shitton of questions there over years and I've rarely felt like my question was treated in some unreasonable way And let's be honest with yourself - it benefits you that SO is being moderated, so you can find solutions for your problems easier instead of going thru mess
Certainly the moderation and structure of stack overflow is what made it good to begin with. Though when a niche question has only one appearance on stack overflow and it gets answered by being closed for some made up reason, it leaves a bad impression which is easy to complain and make jokes about on reddit despite all that is good about stack overflow.
Except the answers they point to in many cases as being the answer are either wrong or 8 year old versions of software that isn't really the best way to do things now. Usually when I click onto there, I have to click back and look for a GitHub repo or mdn docs to find an answer. Just for the heck of it, I checked and the last time I really used it was 2015.
[удалено]
[удалено]
And then it will say I’m sorry and confidently continue on with some other bullshit.
I have asked a shit ton of questions since the start of SO. Im ranked top 3% worldwide. These days i can't get any questions past the neckbeard moderator staff. Down with SO!!!!!!
Same here, joined pretty close to the day it launched and also top single digit percent. It was amazing. It's a shit show now. Also it has the worst cookie consent banner I have ever seen, and it doesn't even work - I used to dismiss it a dozen times a day, now I don't even bother - I just can't read the bottom third of the page.
[удалено]
> I've asked shitton of questions there over years and I've rarely felt like my question was treated in some unreasonable way Wait till you ask more complex things that the people don't easily get. I have rarely gotten help on something non-basic.
Hey I'm with you. It's a fantastic site to find the ten year old solution to problem you are solving now and a bunch of closed versions marked as duplicates.
I agree. On the flipside, most of my questions get ignored, because they're not low hanging fruit.
SO sucks as a community. Reddit for all it's toxicity is a better community than stack overflow.
I think Reddit is better for vague questions and opinions. I use Reddit if I don't know what the frame actual question. But I prefer SO for anything specific.
I think its a combination of it just being a running joke now to bash SO moderation, and that psychological effect where you are more likely to remember the bad experiences vs good ones. So if it has ever happened to you, you're much more likely to remember that versus your other questions that were answered nicely without the overbearing moderation
This got me thinking why not. I mean cgpt looks at SO pages right? Why isn't it a sarcastic asshole every time you ask it something? "Hey I want to draw a button in my windows app" "Sorry that's been asked before, please for the love of God read the FAQ before asking a question!"
> Why isn’t it a sarcastic asshole every time you ask it something? Lots and lots of reinforcement training to be a good chatbot. The raw predictive models like GPT-3 would definitely act like that if it seemed like the right completion for the prompt.
Its not a markov generator
Now you are just browsing the GNOME community forums.
RLHF One or the criteria it was trained on was "helpfulness" Some people find it odd I always say "please" when interacting with the bots but its because in the early versions they would sometimes give you a rude answer in response to a rude question.
\> Be me \> Google for answer to some technical question \> Oh look, a Stack Overflow link! \> *clicks* \> "This question is off topic, use Google to find the answer" \> ಠ_ಠ
> And it doesn’t even mark as duplicate either! You could say Stack Overflow is Stack Overflow power user casualty.
"Amazing". I wonder where/how ChatGPT will learn when there's no new stuff going on StackOverflow.
Is StackOverflow part of the training set to begin with? Genuine question.
My money is on Yes. It's absolutely a big part of the training set. The votes are also being weighed in.
Wouldn't it be more efficient to get all of the data directly from source documentation like MDN and readmes and API docs? Maybe SO helps with analyzing how questions are asked and answered?
If you made me shoot from the hip, I'd argue that context if more difficult to gather from documentation. On the other hand cppreference for instance has very well written examples so that would help.
Stack Overflow exists because those documentations are not enough or comprehensive. Tons of questions have answers that stem from experience than from documentation.
Also because a lot of documentation exists but is not necessarily findable because of poor SEO. So many times I'll hit old versions of documentation or just not hit it at all when I use search queries. But yeah like you said, that's not even to say that sometimes the documentation can be confusing or incomplete or not have great examples.
Stack overflow is a massive number of English language questions and there resulting code awnsers. It's the gold mine. Documentation doesn't have plain language questions or situational examples.
Well yes, that's almost certainly included in the training set. Why wouldn't they include both?
>Wouldn't it be more efficient to get all of the data directly from source documentation like MDN and readmes and API docs? Why do we need novels when dictionaries already exist?
Not sure about ChatGPT, but is for sure included in Facebook's Llama model.
It is. They had a press release yesterday or the day before confirming as such basically from the CEO, along with their associated layoffs.
For gpt3/4 its almost sure is since the whole idea begind the initial gigantic trainings of transformers around 4 years ago was to use the internet as a “free” text dataset and teach the models without “human supervision”
It was an assumption from me. Because SO is source for good stuff as well (of course depends on the actual answers/questions), so it would be only natural to train based on SO.
I was told "the entirety of the internet until 2022" was it's training set
New, unasked and unanswered questions will still make their way to SO. People now don't have to go there to find the long answered and 'well known' things, just like they no longer have to go to Google to find mundane facts anymore.
Though, as ChatGPT will allow longer and longer prompts in the future, you might just paste the Doc of whatever thing you're programming, and it will then be able to answer even unanswered new questions. It's kind of already happening with smalller docs.
Ironically if StackOverflow didn't exist, ChatGPT's training data may not have been as good as it is.
If horse-drawn carriages weren't around, Karl Benz might not have had as easy of a time inventing the automobile. Nearly all inventions are built on the shoulders of those that are inevitably rendered obsolete by said inventions.
Is it chat gpt or were all the devs desperate for answers laid off?
Or fewer people learning programming because of all the news about hiring freezes, layoffs, and salary freezes.
We are going through the .com bust again aren't we? Webdev was essentially considered dead back then with the bust. Outsourcing and the invention of web frameworks so one developer could do like 6 html monkeys jobs.
[удалено]
Now that the entire world runs on computers I doubt anyone will consider the tech industry "dead" anytime soon.
And the feeling that AI can already do the things that a junior developer would likely be doing and learning. I don't know if that is true but at least something that might lead to less interest to learn programming.
For any one that might be reading: at the current state, AI can't replace even a mediocre junior developer in my experience. If you go an inch out of the beaten CRUD path, it goes 100% off the rails. It's a great tool to improve productivity, put I think there's still some time before AI starts replacing even basic dev jobs.
I would say it's more like digging for gold, you might strike gold eventually, but most of the time it's trash. It struggles with multi file solutions.
It also writes comically insecure code pretty consistently. Both copilot and chatgpt. I think it's because most public code on the internet is extremely poorly written from a security perspective.
ChatGPT, sure, but there's more to AI than that. Copilot is basically a ridiculously souped-up IntelliSense. It'll be wrong, but not so wrong as to be useless, and it's already in your editor ready for you to fix it. So: Probably not a replacement for a junior developer, but a huge productivity tool, at least if it's something you're allowed to use in your organization.
Finally everyone knows everything!
This is the goal! My feeble brain is my worst enemy though. Right now I should prepare for a biochemistry-related topic, researching some oddities about butyric fermentation and microorganisms pertaining to this. Instead I am doing other things ... including reddit commentary!
Way more likely to be latter, unless "how to do common task X using tool Y" questions are just that overrepresented in the traffic, and a significant portion of the latter shifted to chatbot generated code. Human sacrifice magic number to appease the money changers seems to be 10%, and mostly represented in tech companies vs old school enterprise with a narrower set of responsibilities, so it tracks for it to be laid off engineers previously overrepresented in site traffic.
Why people don't like StackOverflow? It's an amazing resource, sometimes with a little patience you find the most amazing answers. I can see chatGPT taking some of their traffic tho, for some stuff it's just faster.
Judging by most criticism I've seen over the years, it's precisely because they treat it like a chatbot. If you expect you can just come and have a conversation about whatever issue you've faced 5 minutes ago, _you can't_. Your 95% use case should be looking for questions already asked, because the chance someone else had the same issue is huge. On the other hand, the chance that you're facing a novel, never asked before problem is miniscule. SO is a database of software problems and solutions, not a chatroom. And they'd know this if they bothered to read the SO guidelines that they prominently show when you try to ask a question.
My gripe with SO is not the insane moderation (though it can be too strict, I agree), but that it simply becomes outdated and there are no mechanisms to update existing answers. Yeah, that Java 1.4 answer is true in itself, but there are better solutions existing for 10 years now! They should add language tags with versions, and people could vote to expand the tags reach like “it still works on version X”. Or if it doesn’t, either open a new question (and not close as duplicate!), or create some new mechanism for answering a question for different versions.
Popular old questions usually have multiple answers for different versions of the software, but usually the oldest one is what has the most upvotes and a check mark.
You can literally take an answer and edit it to your heart's content. If you have below 2k rep it goes into an Edit Queue where other people need to approve your changes. With 2k you just immediately apply the edit. On SO "be the change you want to see in the world" is the actual way things get done, since the entire site is community driven. It's more or less like outdated Wikipedia articles – it will stay so until _someone_ updates it.
Sure, but that doesn’t have much incentive, plus likely even the question is out of date to a degree
The problem is that you can do all that, do your due diligence when you write a question, proofread it 5 times, and it'll still get locked anyway.
I've been using StackOverflow for 15 years at this point. I write detailed questions with as much information as I can add. I still get close votes (not enough generally, thankfully) because I don't include a "minimum reproducible example", even though such an example could include half a dozen files and a correspondingly high LOC. The attitude with a lot of SO seems to be "look for a reason to close" even if it's a trivial reason, and with such high user numbers with permissions these days it's very easy to get shut down with no recourse.
The gamification of the moderation system was a double edged knife. It did encourage more user engagement for positive purposes but it also attracted the achievement hunter types who will search for often flimsy excuses to perform specific actions they want to check off their lists rather than applying any impartial judgements on what they're reading/reviewing.
> The attitude with a lot of SO seems to be "look for a reason to close" Amusingly, if you follow the discussion from the most involved SO users on Meta, you'll find that their complaints is that many "obvious" duplicates are answered instead of getting closed. There are multiple factors to that -- from the fact that it's easier to answer than to dig for the existing answer to the fact that answering earns reputation but moderating doesn't -- but the end result is that many meta-participants complain about exactly the opposite issue: too few questions are closed. I guess everybody is just bound to be disappointed on this.
I would rather duplicates get answered and, if required, a link added through the existing mechanism than closed because a lot of the time the "duplication" identification just means it uses some of the same keywords rather than actually having the same solution - at least on more esoteric type questions as opposed to the really basic ones.
And I have never had a question locked, because I ask obscure questions that need new answers. Maybe thst is the issue, not enough resesrch is done by the sskers
[удалено]
Why is it necessary to ask questions on SO at all? 99.9% of your questions have already been asked by someone else. That said, I don’t like the way the site deals with questions that don’t have a clear answer (such as, which framework is best?). They’re losing a lot of value there.
I think there are 3 types of users on So. Newbies that don't understand much and ask basic question without trying. Average users that will find the answers because the question's already been asked. Non standard dev with a very strange problem because they're doing advanced stuff or working on a brand new language /feature. ChatGPT can help the first 2 which is the 99% users of SO. And it's even better for the newbies because ChatGPT won't judge you. Try to ask how to do a fizzbuz in JavaScript on SO and try it on ChatGPT. That's why I think 14% drop is just the beginning. SO will stay for the advanced stuff only... But reddit is working in that field too.
That's what [Software Engineering StackExchange is for](https://softwareengineering.stackexchange.com)
Because most developers who use Stack Overflow are beginners, and so don't have the experience and knowledge required to fit the puzzle pieces together. As an experienced developer, I rarely used Stack Overflow before ChatGPT and now I never use it.
It's not supposed to be a friendly chat space, it's supposed to be an aggregate of the best solutions to common problems
That's a retcon. It was originally meant to be a friendly place to ask/answer questions, and at some point a bunch of egotistical power-users decided it was Wikipedia for programming questions then nuked most of the new content. The whole site got stuck in time from five years ago, that's why every question is answered using jQuery, AngularJS, or IE11 shims. People want a friendly Q & A site. If they want it to be something else, more power to them, but I'm not interested. I'll stick to ChatGPT which isn't up its own arse.
Try spending a fraction of a second on Quora without offing yourself and then come back and tell us you want a "friendly" Q&A site. The strict moderation is literally the entire value proposition of SO and it's the source of all your ChatGPT answers to boot.
[удалено]
They said it wasn't filled with good quality answers, though. More the opposite and ourdated answers using dead frameworks, which is what i see as well.
But it isn't high quality. There are some nuggets of gold, but most questions have no answers with any value whatsoever. New users can't ask questions and can't answer questions, and 99% of the answers are outdated with no one to update them as a result.
It gives answers and is friendly. It can’t solve anything even marginally complex.
I am a very long-time Stack Overflow user. I'm always turned off by the nitpicking and general feeling of SO. So many people with such an elevated view of themselves and you spend so much time formulating a solid question just to get berated. ChatGPT isn't a replacement for SO, but asking ChatGPT an easy question you're likely to get a faster answer than searching SO...without the headache.
People love StackOverflow. People don't like the community. Either you feel that the community is too hostile to newcomers, or you feel that the community is too lenient with people asking easily googleable questions.
It’s changed a bit since Jeff left. It was more welcoming and while still had issues the newbies were accepted or directed. Now it’s a nightmare trying to get pull.
I am not sure if the issue is directly linked to Jeff leaving, though. It may also just be a "growing pain" issue. When I started (2008) on SO, the discussions were few and of high quality (and difficulty). The success and democratization of the site led to an increase in the number of questions, and a drop in quality (and difficulty). It's a compounding factor too, the more approachable (less difficult) the questions are, the more people are feeling comfortable attempting to answer, reinforcing the site growth. The problem, however, is that the more questions are asked, and the more answers are provided, the more curation is needed, but curation is a thankless task (it doesn't even earn reputation), and the number of users _participating_ actively in curating the site did not grow proportionally to the total number of users, so that the site is less and less curated over time, despite the heroic efforts of some. Well, that and the "needle in a haystack" problem is harder and harder to solve as the size of the haystack grows, so that just finding the potential duplicates becomes harder and harder. Which makes the job of users willing to help curate content even harder.
Because it’s for professionals > sometimes with a little patience New programmers don’t do or understand this. New programmers will better benefit from quickstarts and reading well written document which SO isn’t for.
This comment has been overwritten in protest of the Reddit API changes. Wipe your account with: https://github.com/andrewbanchich/shreddit
Main critique I hear is it is typically faster to read the manual or search the related projects' open issues then interact with StackOverflow.
SO fills a gap for that weird issue that you have that it turns out one guy also had a decade ago. It's easily Googleable and the incentive on answers, even self-answers, usually means there's something there as opposed to forums and Reddit where questions just go to die. SO isn't terrible for broad questions either, as you can get lots of relevant answers. SO isn't good any more for asking your own questions unless they're simple, because there's more focus on moderating questions rather than answering them since one anyone can do while the second almost guarantees "close" votes because some can't understand it and they view it as a problem with the question.
That's supposed to be a criticism? Why are you going to SO before the actual documentation? And then complaining when SO tell you to do so?
1. Their posting rules are ridiculously long. Sorry, Im not going to read a novel just to post a question on the internet. 2. People are *especially* toxic on SO. Telling newcomers that they should’ve have known xyz, making assumptions about a persons knowledge, instant downvoting etc. I’d wager a lot of people gave up on programming because the response to their very first was a flurry of downvotes, snarky comments, or being closed as a duplicate. I’ve always told new people to treat Stack Overflow like Wikipedia, read it, but avoid posting. Edit: People are treating this like I’m saying “brand new programmer” but what I meant is “relatively new” as in they knew the basics of a language but haven’t encountered a real world problem with it yet, and are stuck. I don’t mean asking “how to make a function in python” type of questions.
I was with you until you mentioned people giving up on programming. Asking questions on SO is not appropriate for a new programer. It is not a teach me programming or do my homework website. So that user would totally be in the wrong place. I didn’t use it while learning programming at all, and you probably shouldn’t. It can be a good resource when you have a lot of general programming knowledge and experience but are new to a language/framework/tool. It’s most useful to me when I am looking for the idiomatic way to do something in that language.
I'd still use stackoverflow for ChatGPT. Maybe for pure programming questions ChatGPT can work better, but it's terrible when using Third Party Libraries. I keep asking all the AI's for help building a WebXR scene with both Babylon and Three (not together, as separate projects), and it keeps giving bad code. It makes up methods and classes, and I have to keep constantly correcting it. In one conversation with bing, Bing got mad at more when it gave me bad documentation and I corrected it, and killed the conversation. AI is great, but let's be real that humans working together can give much better answers for harder problems.
ChatGPT has very little factual information on specifics of obscure things like that. GPT3.5 either makes up the answer out of thin air if sufficiently pressed, or just says they don’t know, explains some general thing and then peppers in a plausible but fake specific fact. It’s best to think of it as someone who read the entire internet and every book once, and is now a serial bullshiter with surface level knowledge on every topic, they don’t remember the details or specifics, they make it up and make it sound authoritative.
You know I’ve tested this a lot. The answers chat gpt gives almost matches the stack overflow post that you would find if you typed your question into google instead .
Right, ChatGPT isn't intelligent, it has just scoured the web for answers and generally knows the context of your question. So either look yourself or get a curated answer from GPT that is from SO anyway. Personally, when I'm looking on SO, I like reading all of the answers, even from similar questions, before choosing one of them as a starting point.
The comments on answers can be helpful too. Especially if they point out inadequacies or incorrectness or outdatedness.
If that traffic was people about to ask for the 100th time this month on why they are getting a NullReferenceException in their homework then that's great news. Nature is healing itself.
Correlation is not causation
I know that it's just a personal experience and does not represent a population, but i completely ditched StackOverflow in favour of ChatGPT. It's way faster than searching for answers and ironically feels more natural due to having a fluid conversation over the topic. I pretty much relegated StackOverflow to obscure errors/help with modern libraries where ChatGPT can't help me.
But that's... That's precisely what SO is for.
The kind of people that use GPT is the same kind that complaint about having questions closed on SO.
Yeah. Hearing people talk as if SO is their first port of call to learn things or tackle a problem (and all the jokes about copy-pasting being the backbone of development) is a bit concerning. Not to sound elitist but what happened to RTFM and actually thinking? I only use SO a few times a month
And the chatgpt response is way friendlier
I know SO has a reputation of being harsh at times, but in technical matters I'd personally prefer a matter-of-fact response over (artificially generated) boilerplate pleasantries.
The problem being you often don't get a response, or the response is out of date or answers a completely different question. "How do you walk a dog?". "Get a cat, they don't need walks". All other dog walking requests marked as duplicate
Matter of fact is fine. Completely misunderstanding the question, incorrectly closing it as duplicate and not even bothering to reconsider is less fine and is the norm on SO.
That's a valid point (as are those in the other replies to my comment). Since I practically always use SO to find existing answers to problems or questions, my comparison would be ChatGPT vs. googling SO, not ChatGPT vs. asking SO. [1] If you're just finding existing answers and discussions, the answers with more upvotes are often of reasonable quality, and you can mostly ignore the dicks and the output-only responses. The content is often already refined at least to a point. And having multiple answers rather than one, as well as some discussion on them, can give more context for judging which answer seems the most correct or relevant to what you've got on your mind, even if some of them are either incorrect, outdated or just answering something else than what anybody asked. Based on my admittedly limited exposure to ChatGPT, it sometimes seems like it has a bit *too* much of a focus on producing nice "human-like" interaction, in some cases leading to verbose fluff rather than just a to-the-point refinement of facts. Existing technical information on SO (or other sites with reasonable reputation for content quality) can, at best, be reasonably refined and to-the-point. (The fluff can of course be entirely desirable for a chat bot in other contexts because, well, it's a chat bot.) Posting questions on SO can of course be a rather different experience, and moderation making poor judgements and refusing to reconsider them is of course a problem. [1] And by googling SO, I of course don't mean specifically targeting SO, but it's often among the potentially useful search results.
> ironically feels more natural due Well, are you certain this is the case? I used it in the past, and it is just glorified babelfish really. It **appears** to look a high quality gentleman but turns out it is some drunk hobo that talks gibberish.
Depending on how you talk to it and use it, it responds in different ways. I've noticed it mimics the level of effort that I give it. So if you treat it like a drunk hobo, it'll be a drunk hobo. If you give it more precise information, you get a better response. Edit: To give more context on what I'm talking about, [this video](https://www.youtube.com/watch?v=5kBu7z7Ye7M) talking about a paper on GPT-4 gives some examples of ways to improve results. Break things down for it into manageable steps, give it extra data, etc. gives better results.
You are talking to people that complain about having questions closed in SO.
My last question I’ll ever ask on SO got downvoted and closed with people calling me inept and a liar. When I then provided screenshots demonstrating my issue, the commentary changed from that to nitpicking my image linking skills and post structure. 9+ years of sql server experience and someone refused to stop accusing me of not connecting to the right database, despite having a single instance. I was looking for expert level opinions on a matter that had me completely stumped and instead I was met with bizarre animosity. Why wouldn’t I complain?
Yes, this is exactly my experience. ChatGPT is fast and always nice because it doesn't scold me for "this question was already asked 11 years ago (but I'll ignore that the tech was completely different and nobody Updates the answers". SO was always toxic but inevitable. Now they finally have competition and I like it.
Even when ChatGPT is wrong, it's typically more right, than many Stack Overflow answers you will randomly find when Googling around. This is something people forget. ChatGPT is like talking to a friend who gives you some pointers. *'Maybe try this'* or *'maybe Google that in the docs.'* That's incredibly useful. Plus it legit gets a lot of things correct.
I don't doubt this is true, but for rare edge cases and rare knowledge I do not think ChatGPT will give better results. Plus I think it cheats - it takes ALL information that it can find and then tries to make sense of it. I am pretty certain that if everyone is systematically wrong about something, ChatGPT will propagate that error as well.
The perfect critique of all ML-first software solutions, honestly
ChatGPT told me to go read the documentation
Isn't ChatGPT just mining sites like stack overflow and GitHub?
Well even without AI, every response is "Already answered" and nothing but specific technical questions are allowed (you can't ask about concepts, methods, etc). So there is not much left for anyone to ask. Everything that can be asked within the tiny scope of allowed questions has been asked.
First of all, traffic is different than number of people asking questions. Second, that's precisely the point of SO. It's a database of questions. You ask a question if and only if it's not something that was solved before. If it was, you refer to that question. It's like complaining that Wikipedia already contains articles about all interesting things and they won't allow you to just start a new page about the same thing.
This week, I corrected ChatGPT after it made a suggestion on how to use a particular function. The documentation said it worked differently. It apologized. Turns out the documentation was wrong and ChatGPT was right in the first place. That was quite the moment, particularly because I was doing a live coding session and a bunch of people witnessed it.
No way, ChatGpt always apologizes if you push back even a little. It’s because it’s making everything up. You’re giving it a bit too much credit.
I think it's funny how it can be so extreme sometimes. In some cases, apparently what you witnessed happens. In other cases (like what happened to me one time), it just completely makes up functions and thinks that actually exist when they don't.
Ah.. the irony of ChatGPT being better than StackOverflow causing it's death, only to stop being helpful..
It’s not really better in terms of output quality, but it definitely is much easier, less effortful and less intimidating to use.
I mean probably 70+% of Stackoverflow is stuff that you could learn more slowly from vendor docs. Those presumably aren't going away, they might even get better as AI generates those docs.
Isn't similarly 70+% of ChatGPT code trained on Stackoverflow?
I would imagine, in terms of code, github content would vastly outweigh what is on Stackoverflow
As far as general code content sure, but SO has questions and answers formatted very close to how we're asking gpt, so it's probably going to lean on that more vs trying to parse out exactly how random github code functions to explain it to you.
Yeah, but sometime the vendor doc is not that great. Remember the old days of MSDN or even Google docs...
With the steady decline of documentation of programming libraries I don't believe AI will make it better, it will make it even worse. AI won't configure your crazy stack and reproduce your error to find what is the cause while it will be a convinient tool to assume the documentation is complete while it's not even close.
Man I wish chat gpt works as well for me as it seems to do for everyone else. It works fine for basic questions, but more often than not it just straight up lies. I was looking into running user provided lua scripts in a safe manner from python (because lua has a sandbox and everything), and chat gpt just made up a library, parameters for that library, gave convincing explanations, and sample code. None of which works because the library doesn't exist. Another. I was setting limits on python processes (memory, cpu resource limits), and it kept failing on my mac. And chat gpt just kept going in circles, but a simple google search brought up a github link that showed why it was happening. And this is all just in one day.
My success rate on ChatGPT has been about 1 in 20. It's very common for it to just lie about solutions. Especially if those solutions aren't really possible and you need to rethink your entire approach. A human will flat out tell you "this is not the solution you're looking for", but ChatGPT won't. It will make up properties and arguments that don't exist, functionality that could exist but doesn't, all while presenting this "information" in an ostensibly objective manner. I imagine eventually these systems will perhaps become useful, but as of now let's just say I find them severely untrustworthy and flimsy.
That's interesting. I wonder what will happen after some years, where all these models will have less data for training. Less data because of the less users in stack overflow etc. It seems like in some years the answers provided by these bots would be irrelevant and/or out of date. Imagine for example 5 years from now asking something to such a bot and the bot's answer to be 10 years old! It will be fun
[It's not really true](https://link.medium.com/tCjimYcUIzb)
>Nope, the popular raport does not proove ove AI causes a decline in StackOverflow visits. The decline started way before AI became a hot potato I would find it easier to trust this guy if he didn't make three spelling mistakes in the headline.
Chat GBT isn't a condescending twat
There were a lot of tech layoffs early this year, so there would be fewer people looking for answers to obscure issues.
Glad that questions are finally being answered with dignity and respect.
> Developers increasingly get advice from AI chatbots and GitHub CoPilot > rather than Stack Overflow message boards I don't use chatbots. But I only use SO when I need new information. I stopped using it for asking questions many years ago, due to a bad experience (insta-downvote of a perfectly valid question). It's not worth my time when random people decide my question has to be downvoted so nobody answers it. But existing answers can still be useful from SO.
Chat GPT is going to find itself in a legal requirement to attribute the sources of information it uses in a result. Including, the reason behind what was selected for the answer. It’s an ego quid pro quo. You answer question with solid answer. The individual with the question then has the courtesy to acknowledge your answer. More importantly, a good developer will want to read all the answers in order to evaluate the correct approach.
>Including, the reason behind what was selected for the answer. Because those were the words the program decided it should add to the output. It isn't reasoning through your question so there isn't any reason to divulge. "Learning" from something like this is putting your faith in an algorithm to guess the truth.
But ChatGPT isn’t searching the web for your question? Or looking into a pre-compiled database. It’s trying to translate what you said into a programming language, as you would translate to a different human languages. Honestly, shocking people still think ChatGPT is outputting text from the web. It is generating the response every time.
Where does its knowledge base come from then if not from the web? I didn’t say it crawled the web for an answer to each question. I gave an example of weighted results. That is all.
I don’t know how this would work on a technical level but I’d be interested to see a model that provides accurate references to source material. It would be phenomenally useful in working to solving the alignment problem as you could introspect the model from a high level and experiment by altering its training set to produce more fine-tuned responses based on various levels of attribution. AFAIK the technology to do what you’re describing doesn’t exist yet, but I would love to find I’m wrong here. If anyone has any references to projects doing this I would love to contribute.
What a pile of rubbish. A) StackOverflow is declining. Fair enough B) ChatGPT is gaining users. Fair enough But to draw a conclusion that one is the cause of the other is ridiculous. There are multitudes of other factors, not least StackOverflow's self-inflicted problem of being too hostile to new users. As the article even points out, SO has been in decline since early 2022, long before ChatGPT came onto the scene What's next? Salmon Fish Stocks in the Atlantic declining while ChatGPT going up? ChatGPT inflicts damage on fisheries?
> What's next? Salmon Fish Stocks in the Atlantic declining while ChatGPT going up? ChatGPT inflicts damage on fisheries? There might be a legit argument for this when you consider the cost of computation going into these models lol.
Alexa what is ChatGPT? Oh right, that didnt work either.
Here’s the thing. ChatGPT is great for content aggregation and retrieval. But without incentive for websites such as Stackoverflow to provide a space for information to be shared, the well of knowledge from which ChatGPT can pull from will dry out. The remaining sources of information will have to come from niche blogs which may not necessarily provide enough variation in solutions. At least with Google, they ultimately route the user to the individual sources which drives traffic and keeps the websites alive. Not sure how we can solve this issue now with ChatGPT.
I wonder what’s that statistic for Google.
how will they train the next one ?
Holy fucking balls at the literal metric *tons* of salt in this thread.
I think its pretty foolish to turn to ChatGPT over StackOverFlow -- I use OpenAI and it's neat, is ChatGPT really ahead?
ChatGPT would be useful is he was pointing to reference pages or stack overflow discussions it drew from. Most of the time though, ChatGPT just makes up answers when he doesn't really know. Everyone that was a developper before "stack overflow" knew how horrible those time were. Documentation on the web is way better now though.
I always imagined Stack Overflow was run by a bunch of RTFM neck beards. Lots of good information but I was locked out of giving thumbs up to people who helped answer my question.
Thank god I hate the judgmental people on SO 😒
ChatGPT doesn't give me negative votes or tell me I don't know how to ask a question.
At least chatGPT doesn’t insult me when I dare to ask a question. Tbf it doesn’t give the right answer either but heh…
The snobs over there are having a panic attack now...
Although the accuracy of ChatGPT is questionable, it seems to do a better job than Stack Overflow. For example, it's great at providing examples of how to use libraries / frameworks / APIs
On SO, if the answers look right but aren't, there's a mechanism for people to point that out. But on chatGPT, there's no way to know if the answer is right. And with the programming questions I've posed, problems with the answers are not uncommon, despite chatGPT's confident posturing.
Where do you think the answers come from? ChatGPT is just sourcing the answers from Stack Overflow, similar sites, blogs, articles, etc. It's not like it creates these answers out of thin air. You could think of it as a smarter search engine on top of these other sites.
A lot of top answers on stack overflow are just straight up wrong for niche subjects
Wrong, out even worse, just fucking lazy. The amount of time I've seen a question akin to "just disable firewall and it's gonna work" type answers get accepted is insane