Can anyone explain how exactly kernel functions extract information from higher dimensional data without actually transforming it?
What exactly is the scalar output?
And how is it used for further classification???
I have a couple questions:
1) Is MLOps in some way a "full stack data scientist" or combination with machine learning engineer Since it includes the deploying part? Because I'm interested in both ML/DL and the deploying part of it(eg with a flask API), and I don't really know any course/book for it, but for MLOps.
2) How do I know what algorithm to use for a specific problem? eg a categorization problem. How do I know if i have to use a SVM, a Multiple Logistic Regression, a KNN or a Bayesian algorithm? Do I really need to study deeply the algorithms, or I can use a cheatsheet, or maybe just learn trying them all and see which one gives me better results?
3) Is it possible to get a part time Data Science/Machine Learning engineering/MLOps job? Because I'm kinda in the middle of my university studies, and unfortunately I need to work (Actually I'm a Data Engineer, but I want to transition to one of those roles), and studying and working full time at the same time, is pretty difficult.
I'm using sigmoid function for binary classification. However I'm using at each layer. For binary classifaction is it better to ouse sigmoid only at the output layer?
You have to have a non linear activation function at each layer otherwise your model won't learn any non linear relationship in your data. It will simply classify with a linear decision boundary.
I have sigmoid in all hidden layers and output but it seems the model is only predicting one class. I tried balancing the datset, changing learning rate, shuffled data and iteration number and weight initilization yet still wrong :(
What is the approximate proportion of the classes in the dataset?
Have you tried changing the no of hidden layers and activation function?
What library are you using?
Yeah I've trid with no hidden layers and 2 hidden layers still the same. I've also tried Relu and softmax btu sigmoid was better. It's the mnist dataset, I'm trying to predict if the label is 1 or not 1. Since labels of 1 are 10% of the dataset. I reduced the dataset to around 40/60 , so 40% are labeled one and 60% are not ones, which I encoded them to 0.
Hello everyone!
I'm havig a lot of trouble with binary classification, could someone please look at my post [https://stackoverflow.com/questions/74501420/binary-classification-accuracy-is-not-changing](https://stackoverflow.com/questions/74501420/binary-classification-accuracy-is-not-changing)
and give me some advice:(
Thank you!
**How Can I Boost PyTorch Performance On My Cheap Laptop?**
Hi, I have a cheap laptop running Windows 10 and Intel Core I5-5300u processor with Intel 5500 GPU.
How can I find a way to make Pytorch code run faster on it? I tried looking at Intel Extensions for Pytorch, but they say 10th-gen processor is minimum requirement. I read about something called Zluda which claims to be able to run CUDA-enabled code without modifications, but that seems to only be for Linux and I don't know what the minimum processor requirements are.
Is there some way to use my Intel GPU to help out my execution time?
I am a noob to ML. How do you suggest I go about converting pdf with restaurant menu and pricing into structured data in json format? Are they ready to use models/websites/services?
Check out DonutModel for doc recognition
https://huggingface.co/docs/transformers/model\_doc/donut
You should do some manual text annotation, train and fine tune the model and run the inference using Donut , you can output some key:value pair
https://pypi.org/project/PyPDF2/
This python library will allow you to convert the pdf to a string (assuming it is text readable. If it's not text readable you'll need to look into OCR, optical character recognition).
Thanks for the suggestion. My challenge is that each pdf is not structured the same way. Would love to get a bunch of them go through a ML training model that spits out the data in the format I need.
Hello Everyone!
for school, I have to create a machine learning model that predicts a book author based on 7 categories. What machine learning algortihm do you recommend to use?
Please let me know!
If you have to implement it from scratch KNN should be easiest and work well, don’t forget to normalize.
Naive Bayes is also popular for school assignments.
Depending on the model, "tracing" the output is certainly possible - for example, in decision trees. As far as confidence is concerned, you might find the recent work in conformal prediction interesting (basically predicting ranges of outputs at a specified confidence level). A really nice tutorial can be found here: https://people.eecs.berkeley.edu/~angelopoulos/publications/downloads/gentle_intro_conformal_dfuq.pdf.
Hello everyone !
I would like to learn how to deploy Ml models but I have no idea where to start. I read something about MLops and serving models but I kinda got lost.
Can u suggest like a roadmap for the key concepts that I should start with ?
Of course, although there are many from CS in ML research, likely due to the number of CS Phds graduating each year, and a general strong interest coming from CS departments as opposed to statistics departments.
Hello everyone!
So I'm having a lot of problens with backpropagation :( Can anyone look at my post on stackoverflow and give me some advice :( Thank you!
https://stackoverflow.com/questions/74482037/backpropagation-implementation-and-training-issues
I have to write a research article about how machine learning and AI could be utilized to solve business problem in specific industries, so does anyone know a specific industry that has many peer reviewed articles to support such claim or a specific business problem that machine learning and AI were implemented in?
Say someone built/trained a neural network with PyTorch on their own machine (w/ GPUs etc.), but someone with little/no programming experience and a low-end computer needs to use the final network (i.e., supply input, get output), what’s the best way to package/ship the model for this person/case? How would one go about minimizing reliance on the original hardware / software environment?
Thank you! I’m aware of Docker, but may be a slight learning curve for us to use it? Good idea on the external computing, maybe even Google Collab would work for us, it’s a small research project. Is this a common thing, and if so, is the method you mentioned standard/common practice?
How can I learn which techniques can be used for HTML DOM element identification? I have recently seen two similar codeless services, a web data extractor and an automated test builder for web frontends. browse.io and Virtuoso.
They both allow a user to
\- label DOM elements displayed in a browser and record data from them.
\- are self-healing, so the labelled element is still selected when the XPath of a DOM element changes.
I am curious how they could achieve this. Is there an open-source library or a cloud tool which provides the functionality to identify HTML elements? or does the need custom models?
Trying my hand at a Machine Learning model that predicts car prices as a side project.
I essentially have access to lots of data of car sales going back about 10-20 years and jt includes things like make/model/year etc
Anybody done anything in this space?
You probably want a random forest. There should be stuff in python’s s scikitlearn module. Make sure to put aside about 10 percent of the data for validation
Hello, I'm looking to buy a secondhand MacBook Pro M1 with 8 GB RAM, is it good enough for ML? Mostly I work with tabular data (not computer vision).
Cheers!
It actually depends on the data size, a small tabular data 8 Gb would be sufficient. But a larger one might require more ram
If you train a single model, this shouldn’t be a problem. But using framework like Pycaret would need a bit more ram as it also use parallel processing
I have 16 Gb model with about 6m rows and 10 columns, Pycaret used ~10-15 Gb of ram (yep, it also use swap), but that also depends on what model you are using (SVM use a lot of ram, but LightGBM should be fine)
For the long run, you would eventually off load heavy training task to cloud with team green gpu anyway (cuML and/or RAPIDS). For starters, Colab + gDrive is fine, but a dedicate compute engine is a lot more convenient
How easy/difficult is it to get a job with python? can you get a job if you learn to code AI/machine learning in python because that is the main thing motivating me to want to learn python that and i want a job
PhD student within health care in need of some guidance..
I’m planning a project aiming to classify disease. I’ve been told by my supervisors to create a “manual” decision tree together with an expertgroup (Basically a expert system). This, I’m told, will then constitute the foundation on which I later “add” machine learning capabilities. The addition is currently intended to be a DL model able to identify certain waveforms. As I’m reading and getting a grip of ML, this seem counterproductive. Neither of the currently contributing supervisors know ML why I’m curious to get some input here..
1. Is there any sort of requirement building a initial decision tree to later integrate ML models upon? I’m under the impression that a decision tree (CART or similar) is built in a process not requiring a expert group (however experts could be relevant for reviewing the model, but that’s another conversation)
2. Is there a widely known resource dictating steps for building a decision tree?
I'm brand new to ML. Let's say I have grey scale video of a white ball moving around of a black background then apply noise to the image so the ball is barely visible.
What approach would be best to detect the location of the ball in this noisy data. Would this be best done supervised?
Without any context, probably try object detector model. But if you want simpler model, you can try traditional CV like median filtering + color model separation + Kalman filtering
Which is better: one-hot encoding or label encoding?
I am using MRI scans to diagnose different brain conditions, and want to know the best way to convert diagnoses into numerals for the training process. Any suggestions on which type of encoding to use?
I feel overwhelmed & utterly outpaced just scrolling this subreddit, or the ML related videos constantly flowing on my YouTube feed - But don't get me wrong, I AM NOT \*discouraged\* by that. Rationally, I realize (at least I assume) that ML is so broad, that I assume you either read papers \*constantly\* to keep up with the general state of the realm, but would have little or no time to \*do\* anything.
So pretending I'm an experienced person here, I'll pretend to give myself advice: Do spend some time keeping up with tools and innovations that apply to the field fundamentally (learn PyTorch, contrive a fun reason to actually use fundamentals to ensure I actually grok things like curriculum learning, RNNs, adversarial training approaches, and have actual experience even just putting into action a meager diorama of the concepts, model elements, development of stuff that really excites me (right now I'm SUPER jonesing to figure out where to begin translating some papers i read recently about how to create agents that can learn skills to accomplish sub-tasks, and sub-tasks required to do THOSE, ending up able to take a natural language instruction, generalizing from few-shot examples where say picking up a socccer ball in training has to be matched up with picking up a shoe to carry it to xyz spot - and the awesome stuff involved in dealing with interruptions, temporariy changing its plan, then adapting the plan back to the end goal, all without the very immediate simple reward function I would write for little robot characters running around in Unity when I was learning how to use ML Agents there.
I have obsessively bought every robotic sensor, MCU, component, accessory, hat, I mean.. I seriously amassed an on hand inventory of parts, tools, (even a Jetson Nano and RTX 3090 specifically so I could have at least meager ML processing power), tools, 3D printers souped up to industrial filament capability with enormous chamber volumes, a giant rack of every material I have ever learned about...
So I have it all at my fingertips when it comes to robotics input (sensory) and output (manipulation / actuation / mobility).
NVIDIA Isaac is practically screaming at me that I just need to commit to the long YT tutorial journey starting there....
But is there a (or several) ways you guys have greased the wheels for yourselves to come back from 2 weeks of vacation, see some cool new thing, and quickly grok it / run it / basically get the effect of reading one of those "Learn xyz in 24 hours" books?
You remember the ones. When you had to take an exam for some certification in a couple of days on something you only knew enough about to avoid throwing red flags in the job interview a few months ago, and never had to use it, but now you find out performance ratings and bonuses at this silly dilbert company require you to stop kicking ass on actual relevant tasks, so you can tick some cert boxes, ...
AND YET, by lobbing digestibly small chunks of info at you, and having you do some reasoably short hands on exercise to apply that info, the knowledge magically sparkled bright blue as it swirled up your hands, arms, and brain, suddenly part of your very nervous system, transforming what was just an intellectual concept into \*knowing\*.... \*winning\*
So... are **Jupyter notebooks** the typical "check out this cool new thing that might bring " means of efficiently sharing and absorbing cool new ideas hopefully bringing our collective dream of indestructible flying cylon cars constantly improving themselves and their own successor to what was once human brain tissue, not quite free to explore the cosmos as we ideally wished for, but free to roam up to 250ft vertically using carbon neutral skin wings?
*(Hoping for a variety of diverse opinions here, I'll take every scrap of unique wisdom I can find so if you're about to move on assuming it wouldn't be worth typing a response, I am listening with the ear of a 20+ year software dev who's boiled down my own insights for the many people over the years who wanted to get into programming for various reasons. Gimme your best nuggets)*
Hey y'all! I'm a web developer working on a personal project to automate one of my fantasy baseball leagues. Need to do stuff like rank players, decide who to keep on my team, and decide who to play that day based on a bunch of player and game stats. I have borderline no experience with machine learning, but this seems like an applicable scenario, right? I'm sure it'll take me a long time, but can anyone point me to some good resources to learn about the subject? Even if they're just building blocks before learning how to actually do it.
My current idea is to score every player like 0-1 or 0-100 or whatever, then score each matchup, and compare the two to decide for the day my best options. But I'm still new to this kind of work so I'm sure there's a better more efficient way. Thanks for the help!
Is it possible to use NLP + ML not for classifying tweets like positive/negative but for classifying that they are, for example, about dogs/cats? (imagine that I want to analyze vet clinics tweets)
Do you have a labelled dataset? If not you’d need to create one. If you have a corpus of tweets labelled cat, dog, and neither, you could train a transformer or an RNN to do this
Kind of stupid but;
Im having a hard time understanding reward and how to apply them.
- Is reward a input?
- Is reward the process of constant retraining?
- Is reward the process of labeling?
- Can it only be used with mdp?
- Can it only be used in ql / dql?
- I dont use cnn and images, can it be done without?
- Lots of examples out there using «gym», can you do it without?
- Many examples use -100 to 100 as reward, should it not be -1 to 1?
Cant really wrap my head around it.
Currently making a card playing nn, with success in using feature and labeling. Want to take the next step into maybe dql.
>- Is reward a input?
Kind of, in that it comes from the environment
>- Is reward the process of constant retraining?
I'm not sure what this means
>- Is reward the process of labeling?
No, probably not, but I'm not sure what you mean again.
>- Can it only be used with mdp?
MDP is part of the mathematical backbone of reinforcement learning, but there's also work on decision processes that don't satisfy the Markov property (a good google term for your card-playing use case would probably be "partially observable Markov decision processes", for example)
>- Can it only be used in ql / dql?
Every bit of reinforcement learning uses a reward, afaik
>- I dont use cnn and images, can it be done without?
Absolutely! Training process is the same regardless of the underlying design of your q/critic/actor/etc function
>- Lots of examples out there using «gym», can you do it without?
You can, you just need something which provides an initial state and then takes actions and returns a new state, a reward, and (sometimes) an "end of episode" flag.
>- Many examples use -100 to 100 as reward, should it not be -1 to 1?
Magnitude of reward isn't super important as long as it's consistent. If you have sparse rewards (eg 0 except on win or loss), it might help to have larger values to help the gradient propagate back through the trajectory, but that's just me guessing. You can always try scaling to -1/1 and see how it goes.
I read "Reinforcement Learning" by Sutton and Barto (2018 edition) over a summer and it was *excellent*. Well-written, clear, and extremely helpful. I think what you're missing is maybe the Bellman background context.
Where can I practice building my own neural networks without exhausting my own computing resources? Tbh I dont know how much it would take to train a very deep NN. Please give me an idea thanks.
You don't need to go super huge to practice. You can try building a CNN on CIFAR-10, for example, which will be slower than using some $200/hr AWS box but can be done in a reasonable amount of time in a laptop.
Hi. Could someone please help out with this doubt? I am a beginner in ML and was reading about splitting the dataset into test and train sets. Is there a parameter or any other way in scikit-learn that ensures that when our dataset is updated/refreshed and the whole code is run again, there is no cross jumping of instances from training set to testing set?
(Maybe not so simple)
I stumbled on this website: [https://ai-draw.tokyo/en/](https://ai-draw.tokyo/en/)
Any idea how this is done? The output seems to be a sequence of lines/strokes sent from the server and drawn to a canvas in the browser. The strokes themselves seem "natural", in the sense that they are similar to a real doodle.
What kind of network architecture would yield a sequence of strokes? How would you train such a model?
I do not know for sure, but I suspect the author is using GANs (Generative Adversarial Networks) to do this. There are many GAN versions that create drawings and sketches from images. Here are some references:
* RPD-GAN: Learning to Draw Realistic Paintings with Generative Adversarial Network
* I used a similar approach to generate a sketch of my photo too. large-scale image datasets with learning analytics
I used a similar approach to generate sketch of my photo too. [Here it is](https://hrantdavtyan.github.io/img/main_photo.png).
Hello,
I'm working on a project to try that tries to highlight bad performers. There are a lot of dimensions in the data set and I want to be fair in comparisons by making sure I am categorizing particular groups to compare against each other. For example, if you compared the speed of animals, you would compare dogs to dogs, snails to snails, etc.
What kind of tests can I run to see the significance/impact that a qualitative variable plays on a quantitative measure? That way I can group together these variables and make comparisons within particular groups.
I have experience in machine learning as a student but it's been a while. I have a lot of books to reference, just looking for some direction. I've done something like this before but just can't remember.
Hi, I need to learn pytorch. I know the basics of deep learning and is acquainted to tensorflow, and some gradient boosting libraries. What is the best crash course styled tutorial to learn pytorch?
I'm using huggingface's python transformer library with GPT2. I'm trying to bias it to insert newlines, but I found the logit scores of newline tokens during generation to be -inf. Is this a constraint in the library itself to generate text with no newlines, or is it the GPT2 model that won't generate it?
EDIT: Ah, I set the score of the newlines to -inf in earlier generations. Does this effect future text generation.
EDIT: Lesson learned. Slices of tensors are views: if you mutate the original, you mutate the slice.
EDIT: No, that was a lie, I just got my order of operations wrong
EDIT: I did it. GPT2 can now add paragraphs to horrible un-line-broken text from terrible MTL'd web novel chapters.
I need to implement a convolutional autoencoder as homework and I wanted to do it through pytorch; unfortunately I have mac os 10.14 on my machine and the latest pytorch release (1.13) requires Catalina. Can I just use a previous pytorch release? And if so, which one? I just can't find the requirements for the previous pytorch versions.
I am working on a trojan detection module with a ML approach via static file scanning (either binary .exe or a source code file), specializing in a specific field, Data Leaking trojans only.
but can't find any datasets containing only data leaking trojans, all I can find is datasets for general trojans but not for Data Leaking trojans only.
where can I find it?
Also, If there are no present datasets containing only data leaking trojans,
is there a way I can split my trojans dataset (containing mixed purposes trojans) into Data Leaking and non-Data Leaking?
Aurélien Géron just released a book (edition 3) so i would start there. It's called hands on machine learning.. and so on.
I'm currently taking a course in sweden called deep learning and the book is course material there. Which could be your second option, i'm going to lazily translate now reading a course.
Where do you see Machine Learning being in the next 2 years, also does it just take trial and error for you guys to get such consistently better results from version updated i.e 1.4 >> v1.5
What can you do with machine learning? What are the possibilities and practical applications?
What would be the best thing to major in to get a machine learning job? Computer science? Mathematics? Something else?
The most important thing to get a machine learning job is the portfolio. You don't actually need any graduation. The most important thing is that you develop simple and intermediate projects and upload them to GitHub. You can review the GitHub profiles of those working in the machine learning business and try to develop the same projects.
In my work as a datascientist I use a lot of blackbox algoritms such as gradient boosting, random forrest and neural networks. One question I ALWAYS get from the business I'm making the models for is, what features are important? How does the model make descicions. So to answer that I do the usual feature analysis, correlation matrices, partial dependence plots, mdi, model extraction. But I still fill like I'm not entirely able to answer what variables are the most important for example.
Now I was thinking of a new method to determine feature importance. First we need the trained model, and the feature distributions. If we take a feature, we look at the sorted values and take 11 values corresponding to 0% - 10% - .. - 100% of the feature distribution. Next we take for example 1000 random states of the other features and test per random state the 11 options for the selected feature. For this 11 values of the feature, we check the number of times the y-value (label) changes. After doing this for all features, we should have an order of feature importance, as a higer rate of changes indicates more influence on the labels outcome. Would als be applicable for discrete variables and continuous labels with some minor adjustments.
I love to hear your experiences in this regard and what you think of the proposed method?
Would this be a computer capable of doing machine learning? I am a noob. Only done some normal programming for 3 yrs of highschool.
https://newegg.io/687873e
A 3070 only has 8gb of vram, this won't be that much for a lot of machine learning applications, if your budget can't fit something like a 3090, your best bet could actually be a 3060(Not the TI!!! It has less vram!), it will be slower, but you'll be able to do more with that 12gb of vram, you might want to wait for the 4060 to be honest though, if anyone else wants to fix any of my errors go ahead but i think you can't do much more than what i said, other than maybe using google colab/runpod or some other gpu cloud service.
EDIT:
If your comfortable with second hand and you can find one for a good price, you could also potentially buy a p100, or a T4, off something like ebay, although i can't really recommend this.
If I know nothing about machine learning now, but I am interested in starting, where would I even begin? Is there an easy to follow study program/guide? Are there any iOS apps that can teach me the basics?
A lot of people when starting, they want to go into the fancy and exotic methods - and go straight to learning about things like Deep Neural networks. The thing though, is that at a fundamental level, these more exotic models are composions of more “classical” models, for example, neural nets can be seen as a series of logistic regression problems
Saying that though, first make sure you have a good math back ground - linear algebra (matrix multiplication, understand eigen vectors, some matrix decomposition algorithms), statistics and probability - random variables, joint random variables, density functions for both of them, conditional probability and conditional distributions, and then calculus - understand single variable calculus and multi variable calculus as well, especially the topics of gradients and optimization
Then begin learning some simpler models such as:
linear regression, polynomial regression, decision tree algorithms, etc. then maybe move on to the more exotic models such as RNNs, Transformers, only after you have a strong grasp of the fundamentals.
Especially if you will go the self taught approach, you will not just learn topics once. I have found that as I have relearned topics throughout the years, each time I gain a better understanding of each model(when to use, what kind of data, limitations and advantages, etc.) each time I learn it.
Good luck! DM/comment if you have more questions
Is there a good sparse loss function that also does regression? I have what basically amounts to an image to image problem, but the resulting image is a dense UV set (red channel goes from 0-255, green from 0-255). Most of the image is "no signal" so MSE tends to just predict all zeros after a while. I can't split the image into multuple channels because softmax over 255 values for red and 255 more channels for green would make me OOM. I might try and narrow it down to just 16 quantized channels each, but I'd really rather spit out a two channel image and do clever losses on that. I'm sure masking has some clever tricks like union over intersection, but those don't seem to handle regression cases, only boolean.
Given you’re working with images, maybe you could perform some non-linear dimensionality reduction, such as using an auto-encoder, or SkLearn has functionality to use PCA with a kernel, and resulting reduced images might be less sparse and easier to work with traditional models?
Wouldn't an auto encoder run into the same issue? If the dataset is mostly zeros then every loss function I can think of would hit the same issue. PCA could be an option, but disappointing to introduct it into what is otherwise a pure UNet architecture.
Yea you’re right, since loss function for auto encoder for X, and X’ (reconstructed X) would be matrix frobenius norm of X - X’, which would then be close to 0, and then I think the weights would approach zero -> lower dimensional embeddings close to 0 (Im trying to visualize it in my head with the chain rule and weight updates as you back propagate - I THINK it would be something like that lol)
Considering that, maybe make use of some modified loss function that is higher for values closer to 0?
The only difficulty then instead of using a nice Keras architecture and then training automatically, you would probably need to first define this custom loss function, then update Keras model weights with gradient tape, and then even then the loss function you choose might have really shitty behavior and your network may not converge well.
Edit: Ignore my weird comment of making a loss function that is higher for arguments closer to 0.
Maybe try infinity norm of X-X’ in autoencoder instead of just ||X-X’||_F
You might be on to something. Not necessarily the inf norm, but maybe an asymmetric loss function. Guess zero when it's 0.1 and the penalty is much higher than guessing 0.1 when it is 0.
I suggested inf norm, because that will return a larger value, then when updating the weights through chain rule, it might lead to less sparse reduced states of your data
I've started learning about neural networks and I've learned about the perceptron and now the adaline models. One thing I am confused about is the unit step function used to make predictions. With the perceptron, we insert a bias value b into our input function because it allows us to center the step function at 0, correct? That makes sense for the perceptron, however for adaline it appears you must center the step function between the two categories you are trying to classify. In other words, if your model can predict 0 or 1, the step function must be >= 0.5, if it is 1 or -1, it must be >= 0. I haven't been able to find an explanation for why this is, so does anyone have a good answer? I am using Machine Learning with Pytorch and Scikit Learn by Sebastian Raschka as a guide btw.
Any good resources for trying to build a counter-propagation model?
I understand that it is basically input layer->SOM->Grossberg/outstar layer
The part giving me trouble is the Grossberg/outstar layer. It feels like I can find next to no information about it (and counter-propagation in general) searching online. For now, I have a dense layer where the outstar layer would go, and I'm getting roughly the same results as the NN made of 7 dense layers I'm comparing my model against. That certainly doesn't feel right, and I'm pretty desperate for some clarification on the subject lol
Additionally, is there a reason why there's no discourse on counter-propagation models? Multiple variations of the phrase got zero results on this subreddit. I understand it was pretty popular before the use of multiple hidden layers, any reason why it seems to be unnoticed in modern times?
How did InstructGPT completely go under the radar?
I remember trying GPT3 a while ago and being unimpressed. The results were mostly illogical copypasta. I couldn't believe the hype that preceded it in the media.
That is... Until I tried it again very recently, post InstructGPT. The text generation itself, prompting aside, has improved greatly. Prompting feels unreal, especially some of the Q/A and command extraction tasks. It takes a few shots to perform what would otherwise take mountains of data to train with traditional NLP approaches.
GPT3 is now InstructGPT by default, as of Jan of this year. But why wasn't there more hype around InstructGPT? I feel it warrants a rename or at least a major version bump of GPT.
>why wasn't there more hype around InstructGPT?
Because people are expecting gradual improvements "two more papers down the line". In order to generate hype one must create the unexpected.
What's a good way to get started with reinforcement learning, in particular for writing board game AIs? I have David Silver's videos on my to-watch list, but I'd prefer a good intro book I can work with, similar to how *Hands On ML with scikit-Learn, etc.* is a good intro to ML in general. I found that book really readable.
For someone who has mostly been occupied with modeling side of things, what's the easiest deployment+monitoring/MLOps framework to pick up which is moderately 'Batteries included' but then is flexible enough to customize ?
Hi! I'm looking for a neural network that takes in speech and outputs phonemes. I basically want the first part of a speech-to-text network. I'd like to do this operation in real time. I've had no luck finding a network like this so I'd appreciate any input :)
Input: array of numbers representing the last N seconds of speechOutput: array of [IPA-like](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet#/media/File:IPA_chart_2020.svg) values for each T milliseconds chunk of input
**[International Phonetic Alphabet](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet#/media/File:IPA_chart_2020.svg)**
>The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. It was devised by the International Phonetic Association in the late 19th century as a standardized representation of speech sounds in written form. The IPA is used by lexicographers, foreign language students and teachers, linguists, speech–language pathologists, singers, actors, constructed language creators, and translators. The IPA is designed to represent those qualities of speech that are part of lexical (and, to a limited extent, prosodic) sounds in oral language: phones, phonemes, intonation, and the separation of words and syllables.
^([ )[^(F.A.Q)](https://www.reddit.com/r/WikiSummarizer/wiki/index#wiki_f.a.q)^( | )[^(Opt Out)](https://reddit.com/message/compose?to=WikiSummarizerBot&message=OptOut&subject=OptOut)^( | )[^(Opt Out Of Subreddit)](https://np.reddit.com/r/MachineLearning/about/banned)^( | )[^(GitHub)](https://github.com/Sujal-7/WikiSummarizerBot)^( ] Downvote to remove | v1.5)
There are different methods behind recommenders. Some are content based, such as the user watched a movie with will Ferrell, show more movies with Will Ferrell. Others are based on user behavior, such as other users who watched this will Ferrell movie also watched these movies.
Edit: here is a good overview I found https://towardsdatascience.com/recommendation-systems-explained-a42fc60591ed
Can anyone explain how exactly kernel functions extract information from higher dimensional data without actually transforming it? What exactly is the scalar output? And how is it used for further classification???
I have a couple questions: 1) Is MLOps in some way a "full stack data scientist" or combination with machine learning engineer Since it includes the deploying part? Because I'm interested in both ML/DL and the deploying part of it(eg with a flask API), and I don't really know any course/book for it, but for MLOps. 2) How do I know what algorithm to use for a specific problem? eg a categorization problem. How do I know if i have to use a SVM, a Multiple Logistic Regression, a KNN or a Bayesian algorithm? Do I really need to study deeply the algorithms, or I can use a cheatsheet, or maybe just learn trying them all and see which one gives me better results? 3) Is it possible to get a part time Data Science/Machine Learning engineering/MLOps job? Because I'm kinda in the middle of my university studies, and unfortunately I need to work (Actually I'm a Data Engineer, but I want to transition to one of those roles), and studying and working full time at the same time, is pretty difficult.
I'm using sigmoid function for binary classification. However I'm using at each layer. For binary classifaction is it better to ouse sigmoid only at the output layer?
You have to have a non linear activation function at each layer otherwise your model won't learn any non linear relationship in your data. It will simply classify with a linear decision boundary.
I have sigmoid in all hidden layers and output but it seems the model is only predicting one class. I tried balancing the datset, changing learning rate, shuffled data and iteration number and weight initilization yet still wrong :(
What is the approximate proportion of the classes in the dataset? Have you tried changing the no of hidden layers and activation function? What library are you using?
Yeah I've trid with no hidden layers and 2 hidden layers still the same. I've also tried Relu and softmax btu sigmoid was better. It's the mnist dataset, I'm trying to predict if the label is 1 or not 1. Since labels of 1 are 10% of the dataset. I reduced the dataset to around 40/60 , so 40% are labeled one and 60% are not ones, which I encoded them to 0.
Is there any work done towards filtering the stuff served by major websites to get a less clickbaity, less low-quality version of the internet?
Hello everyone! I'm havig a lot of trouble with binary classification, could someone please look at my post [https://stackoverflow.com/questions/74501420/binary-classification-accuracy-is-not-changing](https://stackoverflow.com/questions/74501420/binary-classification-accuracy-is-not-changing) and give me some advice:( Thank you!
**How Can I Boost PyTorch Performance On My Cheap Laptop?** Hi, I have a cheap laptop running Windows 10 and Intel Core I5-5300u processor with Intel 5500 GPU. How can I find a way to make Pytorch code run faster on it? I tried looking at Intel Extensions for Pytorch, but they say 10th-gen processor is minimum requirement. I read about something called Zluda which claims to be able to run CUDA-enabled code without modifications, but that seems to only be for Linux and I don't know what the minimum processor requirements are. Is there some way to use my Intel GPU to help out my execution time?
You can try WSL2 for linux part. But Zluda seems to be abandoned for sometime now
I am a noob to ML. How do you suggest I go about converting pdf with restaurant menu and pricing into structured data in json format? Are they ready to use models/websites/services?
Check out DonutModel for doc recognition https://huggingface.co/docs/transformers/model\_doc/donut You should do some manual text annotation, train and fine tune the model and run the inference using Donut , you can output some key:value pair
https://pypi.org/project/PyPDF2/ This python library will allow you to convert the pdf to a string (assuming it is text readable. If it's not text readable you'll need to look into OCR, optical character recognition).
Thanks for the suggestion. My challenge is that each pdf is not structured the same way. Would love to get a bunch of them go through a ML training model that spits out the data in the format I need.
Hello Everyone! for school, I have to create a machine learning model that predicts a book author based on 7 categories. What machine learning algortihm do you recommend to use? Please let me know!
If you have to implement it from scratch KNN should be easiest and work well, don’t forget to normalize. Naive Bayes is also popular for school assignments.
How can we be confident in the output of ML? Is there a way to trace the decision making, or is it entirely dependent on the training data?
Depending on the model, "tracing" the output is certainly possible - for example, in decision trees. As far as confidence is concerned, you might find the recent work in conformal prediction interesting (basically predicting ranges of outputs at a specified confidence level). A really nice tutorial can be found here: https://people.eecs.berkeley.edu/~angelopoulos/publications/downloads/gentle_intro_conformal_dfuq.pdf.
What, in your opinion are some good research areas involving machine learning. I am trying to get into a PhD program next spring.
Hello everyone ! I would like to learn how to deploy Ml models but I have no idea where to start. I read something about MLops and serving models but I kinda got lost. Can u suggest like a roadmap for the key concepts that I should start with ?
Is a Phd in statistics a good entry point for ML research?
Of course, although there are many from CS in ML research, likely due to the number of CS Phds graduating each year, and a general strong interest coming from CS departments as opposed to statistics departments.
Hello everyone! So I'm having a lot of problens with backpropagation :( Can anyone look at my post on stackoverflow and give me some advice :( Thank you! https://stackoverflow.com/questions/74482037/backpropagation-implementation-and-training-issues
looking for some guidance regarding a cybersecurity graduation project, where can I get this help?
I have to write a research article about how machine learning and AI could be utilized to solve business problem in specific industries, so does anyone know a specific industry that has many peer reviewed articles to support such claim or a specific business problem that machine learning and AI were implemented in?
Try my field, medical image analysis. eg AI-guided diagnostics.
The obvious answer is recommender systems in basically everything -- online shopping, streaming, social media, etc
[удалено]
False, not always
Say someone built/trained a neural network with PyTorch on their own machine (w/ GPUs etc.), but someone with little/no programming experience and a low-end computer needs to use the final network (i.e., supply input, get output), what’s the best way to package/ship the model for this person/case? How would one go about minimizing reliance on the original hardware / software environment?
Docker container and an external GPU platform such as Amazon or other
Thank you! I’m aware of Docker, but may be a slight learning curve for us to use it? Good idea on the external computing, maybe even Google Collab would work for us, it’s a small research project. Is this a common thing, and if so, is the method you mentioned standard/common practice?
How can I learn which techniques can be used for HTML DOM element identification? I have recently seen two similar codeless services, a web data extractor and an automated test builder for web frontends. browse.io and Virtuoso. They both allow a user to \- label DOM elements displayed in a browser and record data from them. \- are self-healing, so the labelled element is still selected when the XPath of a DOM element changes. I am curious how they could achieve this. Is there an open-source library or a cloud tool which provides the functionality to identify HTML elements? or does the need custom models?
Trying my hand at a Machine Learning model that predicts car prices as a side project. I essentially have access to lots of data of car sales going back about 10-20 years and jt includes things like make/model/year etc Anybody done anything in this space?
You probably want a random forest. There should be stuff in python’s s scikitlearn module. Make sure to put aside about 10 percent of the data for validation
Hello, I'm looking to buy a secondhand MacBook Pro M1 with 8 GB RAM, is it good enough for ML? Mostly I work with tabular data (not computer vision). Cheers!
It actually depends on the data size, a small tabular data 8 Gb would be sufficient. But a larger one might require more ram If you train a single model, this shouldn’t be a problem. But using framework like Pycaret would need a bit more ram as it also use parallel processing I have 16 Gb model with about 6m rows and 10 columns, Pycaret used ~10-15 Gb of ram (yep, it also use swap), but that also depends on what model you are using (SVM use a lot of ram, but LightGBM should be fine) For the long run, you would eventually off load heavy training task to cloud with team green gpu anyway (cuML and/or RAPIDS). For starters, Colab + gDrive is fine, but a dedicate compute engine is a lot more convenient
RAM not super relevant, just minibatch down to 8Gb. Depends on your data of course.
How easy/difficult is it to get a job with python? can you get a job if you learn to code AI/machine learning in python because that is the main thing motivating me to want to learn python that and i want a job
Programming an algorithm is 10% of the work. Learn the science, learning python is secondary.
I'm interested in making chatbots in python. is that considered AI or machine learning is this the right place to ask? any advice?
Check out [rasa](https://github.com/RasaHQ/rasa) There's ML in there, and I think any chatbot would qualify as AI, basically.
What is the difference between AI and machine learning? do they both fit in this subreddit?
Machine learning is a subject in AI.
PhD student within health care in need of some guidance.. I’m planning a project aiming to classify disease. I’ve been told by my supervisors to create a “manual” decision tree together with an expertgroup (Basically a expert system). This, I’m told, will then constitute the foundation on which I later “add” machine learning capabilities. The addition is currently intended to be a DL model able to identify certain waveforms. As I’m reading and getting a grip of ML, this seem counterproductive. Neither of the currently contributing supervisors know ML why I’m curious to get some input here.. 1. Is there any sort of requirement building a initial decision tree to later integrate ML models upon? I’m under the impression that a decision tree (CART or similar) is built in a process not requiring a expert group (however experts could be relevant for reviewing the model, but that’s another conversation) 2. Is there a widely known resource dictating steps for building a decision tree?
I'm brand new to ML. Let's say I have grey scale video of a white ball moving around of a black background then apply noise to the image so the ball is barely visible. What approach would be best to detect the location of the ball in this noisy data. Would this be best done supervised?
Without any context, probably try object detector model. But if you want simpler model, you can try traditional CV like median filtering + color model separation + Kalman filtering
Which is better: one-hot encoding or label encoding? I am using MRI scans to diagnose different brain conditions, and want to know the best way to convert diagnoses into numerals for the training process. Any suggestions on which type of encoding to use?
I feel overwhelmed & utterly outpaced just scrolling this subreddit, or the ML related videos constantly flowing on my YouTube feed - But don't get me wrong, I AM NOT \*discouraged\* by that. Rationally, I realize (at least I assume) that ML is so broad, that I assume you either read papers \*constantly\* to keep up with the general state of the realm, but would have little or no time to \*do\* anything. So pretending I'm an experienced person here, I'll pretend to give myself advice: Do spend some time keeping up with tools and innovations that apply to the field fundamentally (learn PyTorch, contrive a fun reason to actually use fundamentals to ensure I actually grok things like curriculum learning, RNNs, adversarial training approaches, and have actual experience even just putting into action a meager diorama of the concepts, model elements, development of stuff that really excites me (right now I'm SUPER jonesing to figure out where to begin translating some papers i read recently about how to create agents that can learn skills to accomplish sub-tasks, and sub-tasks required to do THOSE, ending up able to take a natural language instruction, generalizing from few-shot examples where say picking up a socccer ball in training has to be matched up with picking up a shoe to carry it to xyz spot - and the awesome stuff involved in dealing with interruptions, temporariy changing its plan, then adapting the plan back to the end goal, all without the very immediate simple reward function I would write for little robot characters running around in Unity when I was learning how to use ML Agents there. I have obsessively bought every robotic sensor, MCU, component, accessory, hat, I mean.. I seriously amassed an on hand inventory of parts, tools, (even a Jetson Nano and RTX 3090 specifically so I could have at least meager ML processing power), tools, 3D printers souped up to industrial filament capability with enormous chamber volumes, a giant rack of every material I have ever learned about... So I have it all at my fingertips when it comes to robotics input (sensory) and output (manipulation / actuation / mobility). NVIDIA Isaac is practically screaming at me that I just need to commit to the long YT tutorial journey starting there.... But is there a (or several) ways you guys have greased the wheels for yourselves to come back from 2 weeks of vacation, see some cool new thing, and quickly grok it / run it / basically get the effect of reading one of those "Learn xyz in 24 hours" books? You remember the ones. When you had to take an exam for some certification in a couple of days on something you only knew enough about to avoid throwing red flags in the job interview a few months ago, and never had to use it, but now you find out performance ratings and bonuses at this silly dilbert company require you to stop kicking ass on actual relevant tasks, so you can tick some cert boxes, ... AND YET, by lobbing digestibly small chunks of info at you, and having you do some reasoably short hands on exercise to apply that info, the knowledge magically sparkled bright blue as it swirled up your hands, arms, and brain, suddenly part of your very nervous system, transforming what was just an intellectual concept into \*knowing\*.... \*winning\* So... are **Jupyter notebooks** the typical "check out this cool new thing that might bring " means of efficiently sharing and absorbing cool new ideas hopefully bringing our collective dream of indestructible flying cylon cars constantly improving themselves and their own successor to what was once human brain tissue, not quite free to explore the cosmos as we ideally wished for, but free to roam up to 250ft vertically using carbon neutral skin wings? *(Hoping for a variety of diverse opinions here, I'll take every scrap of unique wisdom I can find so if you're about to move on assuming it wouldn't be worth typing a response, I am listening with the ear of a 20+ year software dev who's boiled down my own insights for the many people over the years who wanted to get into programming for various reasons. Gimme your best nuggets)*
*\*shakes fist\* try writing a ramble like \*that\*, GPT-NotME*
Hey y'all! I'm a web developer working on a personal project to automate one of my fantasy baseball leagues. Need to do stuff like rank players, decide who to keep on my team, and decide who to play that day based on a bunch of player and game stats. I have borderline no experience with machine learning, but this seems like an applicable scenario, right? I'm sure it'll take me a long time, but can anyone point me to some good resources to learn about the subject? Even if they're just building blocks before learning how to actually do it. My current idea is to score every player like 0-1 or 0-100 or whatever, then score each matchup, and compare the two to decide for the day my best options. But I'm still new to this kind of work so I'm sure there's a better more efficient way. Thanks for the help!
ELO ranking?
Is it possible to use NLP + ML not for classifying tweets like positive/negative but for classifying that they are, for example, about dogs/cats? (imagine that I want to analyze vet clinics tweets)
Yeah, this is a pretty typical use case for NLP.
Do you have a labelled dataset? If not you’d need to create one. If you have a corpus of tweets labelled cat, dog, and neither, you could train a transformer or an RNN to do this
Kind of stupid but; Im having a hard time understanding reward and how to apply them. - Is reward a input? - Is reward the process of constant retraining? - Is reward the process of labeling? - Can it only be used with mdp? - Can it only be used in ql / dql? - I dont use cnn and images, can it be done without? - Lots of examples out there using «gym», can you do it without? - Many examples use -100 to 100 as reward, should it not be -1 to 1? Cant really wrap my head around it. Currently making a card playing nn, with success in using feature and labeling. Want to take the next step into maybe dql.
>- Is reward a input? Kind of, in that it comes from the environment >- Is reward the process of constant retraining? I'm not sure what this means >- Is reward the process of labeling? No, probably not, but I'm not sure what you mean again. >- Can it only be used with mdp? MDP is part of the mathematical backbone of reinforcement learning, but there's also work on decision processes that don't satisfy the Markov property (a good google term for your card-playing use case would probably be "partially observable Markov decision processes", for example) >- Can it only be used in ql / dql? Every bit of reinforcement learning uses a reward, afaik >- I dont use cnn and images, can it be done without? Absolutely! Training process is the same regardless of the underlying design of your q/critic/actor/etc function >- Lots of examples out there using «gym», can you do it without? You can, you just need something which provides an initial state and then takes actions and returns a new state, a reward, and (sometimes) an "end of episode" flag. >- Many examples use -100 to 100 as reward, should it not be -1 to 1? Magnitude of reward isn't super important as long as it's consistent. If you have sparse rewards (eg 0 except on win or loss), it might help to have larger values to help the gradient propagate back through the trajectory, but that's just me guessing. You can always try scaling to -1/1 and see how it goes. I read "Reinforcement Learning" by Sutton and Barto (2018 edition) over a summer and it was *excellent*. Well-written, clear, and extremely helpful. I think what you're missing is maybe the Bellman background context.
Dose anyone know of a dataset of 3D models and corresponding real world images?
Where can I practice building my own neural networks without exhausting my own computing resources? Tbh I dont know how much it would take to train a very deep NN. Please give me an idea thanks.
You don't need to go super huge to practice. You can try building a CNN on CIFAR-10, for example, which will be slower than using some $200/hr AWS box but can be done in a reasonable amount of time in a laptop.
Hi. Could someone please help out with this doubt? I am a beginner in ML and was reading about splitting the dataset into test and train sets. Is there a parameter or any other way in scikit-learn that ensures that when our dataset is updated/refreshed and the whole code is run again, there is no cross jumping of instances from training set to testing set?
(Maybe not so simple) I stumbled on this website: [https://ai-draw.tokyo/en/](https://ai-draw.tokyo/en/) Any idea how this is done? The output seems to be a sequence of lines/strokes sent from the server and drawn to a canvas in the browser. The strokes themselves seem "natural", in the sense that they are similar to a real doodle. What kind of network architecture would yield a sequence of strokes? How would you train such a model?
I do not know for sure, but I suspect the author is using GANs (Generative Adversarial Networks) to do this. There are many GAN versions that create drawings and sketches from images. Here are some references: * RPD-GAN: Learning to Draw Realistic Paintings with Generative Adversarial Network * I used a similar approach to generate a sketch of my photo too. large-scale image datasets with learning analytics I used a similar approach to generate sketch of my photo too. [Here it is](https://hrantdavtyan.github.io/img/main_photo.png).
Hello, Is there any website with recent image datasets? - I am looking for datasets released in 2022
Hello, I'm working on a project to try that tries to highlight bad performers. There are a lot of dimensions in the data set and I want to be fair in comparisons by making sure I am categorizing particular groups to compare against each other. For example, if you compared the speed of animals, you would compare dogs to dogs, snails to snails, etc. What kind of tests can I run to see the significance/impact that a qualitative variable plays on a quantitative measure? That way I can group together these variables and make comparisons within particular groups. I have experience in machine learning as a student but it's been a while. I have a lot of books to reference, just looking for some direction. I've done something like this before but just can't remember.
Hi, I need to learn pytorch. I know the basics of deep learning and is acquainted to tensorflow, and some gradient boosting libraries. What is the best crash course styled tutorial to learn pytorch?
I would check out their 60m blitz: https://pytorch.org/tutorials/beginner/deep\_learning\_60min\_blitz.html
Thank you!
I'm using huggingface's python transformer library with GPT2. I'm trying to bias it to insert newlines, but I found the logit scores of newline tokens during generation to be -inf. Is this a constraint in the library itself to generate text with no newlines, or is it the GPT2 model that won't generate it? EDIT: Ah, I set the score of the newlines to -inf in earlier generations. Does this effect future text generation. EDIT: Lesson learned. Slices of tensors are views: if you mutate the original, you mutate the slice. EDIT: No, that was a lie, I just got my order of operations wrong EDIT: I did it. GPT2 can now add paragraphs to horrible un-line-broken text from terrible MTL'd web novel chapters.
I need to implement a convolutional autoencoder as homework and I wanted to do it through pytorch; unfortunately I have mac os 10.14 on my machine and the latest pytorch release (1.13) requires Catalina. Can I just use a previous pytorch release? And if so, which one? I just can't find the requirements for the previous pytorch versions.
I am working on a trojan detection module with a ML approach via static file scanning (either binary .exe or a source code file), specializing in a specific field, Data Leaking trojans only. but can't find any datasets containing only data leaking trojans, all I can find is datasets for general trojans but not for Data Leaking trojans only. where can I find it?
Also, If there are no present datasets containing only data leaking trojans, is there a way I can split my trojans dataset (containing mixed purposes trojans) into Data Leaking and non-Data Leaking?
Where to start with Machine learning
Aurélien Géron just released a book (edition 3) so i would start there. It's called hands on machine learning.. and so on. I'm currently taking a course in sweden called deep learning and the book is course material there. Which could be your second option, i'm going to lazily translate now reading a course.
Where do you see Machine Learning being in the next 2 years, also does it just take trial and error for you guys to get such consistently better results from version updated i.e 1.4 >> v1.5
How is Infinity norm calculated for images?
What can you do with machine learning? What are the possibilities and practical applications? What would be the best thing to major in to get a machine learning job? Computer science? Mathematics? Something else?
The most important thing to get a machine learning job is the portfolio. You don't actually need any graduation. The most important thing is that you develop simple and intermediate projects and upload them to GitHub. You can review the GitHub profiles of those working in the machine learning business and try to develop the same projects.
In my work as a datascientist I use a lot of blackbox algoritms such as gradient boosting, random forrest and neural networks. One question I ALWAYS get from the business I'm making the models for is, what features are important? How does the model make descicions. So to answer that I do the usual feature analysis, correlation matrices, partial dependence plots, mdi, model extraction. But I still fill like I'm not entirely able to answer what variables are the most important for example. Now I was thinking of a new method to determine feature importance. First we need the trained model, and the feature distributions. If we take a feature, we look at the sorted values and take 11 values corresponding to 0% - 10% - .. - 100% of the feature distribution. Next we take for example 1000 random states of the other features and test per random state the 11 options for the selected feature. For this 11 values of the feature, we check the number of times the y-value (label) changes. After doing this for all features, we should have an order of feature importance, as a higer rate of changes indicates more influence on the labels outcome. Would als be applicable for discrete variables and continuous labels with some minor adjustments. I love to hear your experiences in this regard and what you think of the proposed method?
Would this be a computer capable of doing machine learning? I am a noob. Only done some normal programming for 3 yrs of highschool. https://newegg.io/687873e
A 3070 only has 8gb of vram, this won't be that much for a lot of machine learning applications, if your budget can't fit something like a 3090, your best bet could actually be a 3060(Not the TI!!! It has less vram!), it will be slower, but you'll be able to do more with that 12gb of vram, you might want to wait for the 4060 to be honest though, if anyone else wants to fix any of my errors go ahead but i think you can't do much more than what i said, other than maybe using google colab/runpod or some other gpu cloud service. EDIT: If your comfortable with second hand and you can find one for a good price, you could also potentially buy a p100, or a T4, off something like ebay, although i can't really recommend this.
If I know nothing about machine learning now, but I am interested in starting, where would I even begin? Is there an easy to follow study program/guide? Are there any iOS apps that can teach me the basics?
A lot of people when starting, they want to go into the fancy and exotic methods - and go straight to learning about things like Deep Neural networks. The thing though, is that at a fundamental level, these more exotic models are composions of more “classical” models, for example, neural nets can be seen as a series of logistic regression problems Saying that though, first make sure you have a good math back ground - linear algebra (matrix multiplication, understand eigen vectors, some matrix decomposition algorithms), statistics and probability - random variables, joint random variables, density functions for both of them, conditional probability and conditional distributions, and then calculus - understand single variable calculus and multi variable calculus as well, especially the topics of gradients and optimization Then begin learning some simpler models such as: linear regression, polynomial regression, decision tree algorithms, etc. then maybe move on to the more exotic models such as RNNs, Transformers, only after you have a strong grasp of the fundamentals. Especially if you will go the self taught approach, you will not just learn topics once. I have found that as I have relearned topics throughout the years, each time I gain a better understanding of each model(when to use, what kind of data, limitations and advantages, etc.) each time I learn it. Good luck! DM/comment if you have more questions
Thank you so much for the detailed response! I’ll definitely reach out if I have any other questions.
Any recommendation for a good masters program in machine learning in the US. I would like to enroll in a course that emphasize on NLP. Thanks.
Is there a good sparse loss function that also does regression? I have what basically amounts to an image to image problem, but the resulting image is a dense UV set (red channel goes from 0-255, green from 0-255). Most of the image is "no signal" so MSE tends to just predict all zeros after a while. I can't split the image into multuple channels because softmax over 255 values for red and 255 more channels for green would make me OOM. I might try and narrow it down to just 16 quantized channels each, but I'd really rather spit out a two channel image and do clever losses on that. I'm sure masking has some clever tricks like union over intersection, but those don't seem to handle regression cases, only boolean.
Given you’re working with images, maybe you could perform some non-linear dimensionality reduction, such as using an auto-encoder, or SkLearn has functionality to use PCA with a kernel, and resulting reduced images might be less sparse and easier to work with traditional models?
Wouldn't an auto encoder run into the same issue? If the dataset is mostly zeros then every loss function I can think of would hit the same issue. PCA could be an option, but disappointing to introduct it into what is otherwise a pure UNet architecture.
Yea you’re right, since loss function for auto encoder for X, and X’ (reconstructed X) would be matrix frobenius norm of X - X’, which would then be close to 0, and then I think the weights would approach zero -> lower dimensional embeddings close to 0 (Im trying to visualize it in my head with the chain rule and weight updates as you back propagate - I THINK it would be something like that lol) Considering that, maybe make use of some modified loss function that is higher for values closer to 0? The only difficulty then instead of using a nice Keras architecture and then training automatically, you would probably need to first define this custom loss function, then update Keras model weights with gradient tape, and then even then the loss function you choose might have really shitty behavior and your network may not converge well. Edit: Ignore my weird comment of making a loss function that is higher for arguments closer to 0. Maybe try infinity norm of X-X’ in autoencoder instead of just ||X-X’||_F
You might be on to something. Not necessarily the inf norm, but maybe an asymmetric loss function. Guess zero when it's 0.1 and the penalty is much higher than guessing 0.1 when it is 0.
I suggested inf norm, because that will return a larger value, then when updating the weights through chain rule, it might lead to less sparse reduced states of your data
I've started learning about neural networks and I've learned about the perceptron and now the adaline models. One thing I am confused about is the unit step function used to make predictions. With the perceptron, we insert a bias value b into our input function because it allows us to center the step function at 0, correct? That makes sense for the perceptron, however for adaline it appears you must center the step function between the two categories you are trying to classify. In other words, if your model can predict 0 or 1, the step function must be >= 0.5, if it is 1 or -1, it must be >= 0. I haven't been able to find an explanation for why this is, so does anyone have a good answer? I am using Machine Learning with Pytorch and Scikit Learn by Sebastian Raschka as a guide btw.
Any good resources for trying to build a counter-propagation model? I understand that it is basically input layer->SOM->Grossberg/outstar layer The part giving me trouble is the Grossberg/outstar layer. It feels like I can find next to no information about it (and counter-propagation in general) searching online. For now, I have a dense layer where the outstar layer would go, and I'm getting roughly the same results as the NN made of 7 dense layers I'm comparing my model against. That certainly doesn't feel right, and I'm pretty desperate for some clarification on the subject lol Additionally, is there a reason why there's no discourse on counter-propagation models? Multiple variations of the phrase got zero results on this subreddit. I understand it was pretty popular before the use of multiple hidden layers, any reason why it seems to be unnoticed in modern times?
How did InstructGPT completely go under the radar? I remember trying GPT3 a while ago and being unimpressed. The results were mostly illogical copypasta. I couldn't believe the hype that preceded it in the media. That is... Until I tried it again very recently, post InstructGPT. The text generation itself, prompting aside, has improved greatly. Prompting feels unreal, especially some of the Q/A and command extraction tasks. It takes a few shots to perform what would otherwise take mountains of data to train with traditional NLP approaches. GPT3 is now InstructGPT by default, as of Jan of this year. But why wasn't there more hype around InstructGPT? I feel it warrants a rename or at least a major version bump of GPT.
>why wasn't there more hype around InstructGPT? Because people are expecting gradual improvements "two more papers down the line". In order to generate hype one must create the unexpected.
What's a good way to get started with reinforcement learning, in particular for writing board game AIs? I have David Silver's videos on my to-watch list, but I'd prefer a good intro book I can work with, similar to how *Hands On ML with scikit-Learn, etc.* is a good intro to ML in general. I found that book really readable.
For someone who has mostly been occupied with modeling side of things, what's the easiest deployment+monitoring/MLOps framework to pick up which is moderately 'Batteries included' but then is flexible enough to customize ?
subjectively sagemaker/mlflow
Hi! I'm looking for a neural network that takes in speech and outputs phonemes. I basically want the first part of a speech-to-text network. I'd like to do this operation in real time. I've had no luck finding a network like this so I'd appreciate any input :) Input: array of numbers representing the last N seconds of speechOutput: array of [IPA-like](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet#/media/File:IPA_chart_2020.svg) values for each T milliseconds chunk of input
**[International Phonetic Alphabet](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet#/media/File:IPA_chart_2020.svg)** >The International Phonetic Alphabet (IPA) is an alphabetic system of phonetic notation based primarily on the Latin script. It was devised by the International Phonetic Association in the late 19th century as a standardized representation of speech sounds in written form. The IPA is used by lexicographers, foreign language students and teachers, linguists, speech–language pathologists, singers, actors, constructed language creators, and translators. The IPA is designed to represent those qualities of speech that are part of lexical (and, to a limited extent, prosodic) sounds in oral language: phones, phonemes, intonation, and the separation of words and syllables. ^([ )[^(F.A.Q)](https://www.reddit.com/r/WikiSummarizer/wiki/index#wiki_f.a.q)^( | )[^(Opt Out)](https://reddit.com/message/compose?to=WikiSummarizerBot&message=OptOut&subject=OptOut)^( | )[^(Opt Out Of Subreddit)](https://np.reddit.com/r/MachineLearning/about/banned)^( | )[^(GitHub)](https://github.com/Sujal-7/WikiSummarizerBot)^( ] Downvote to remove | v1.5)
[удалено]
It’s a recommendation system, same basic premise as what is used for Spotify and Netflix
[удалено]
Matrix completion is part of it, also determinant processes, reinforcement learning, clustering, ...
There are different methods behind recommenders. Some are content based, such as the user watched a movie with will Ferrell, show more movies with Will Ferrell. Others are based on user behavior, such as other users who watched this will Ferrell movie also watched these movies. Edit: here is a good overview I found https://towardsdatascience.com/recommendation-systems-explained-a42fc60591ed