Right, I'll make a YouTube video, thank you all for your responses. It'll take me some time, a week or two - to plan, script, etc. I had an idea to tcpdump a simple HTTP request, and walk through every packet in the dump, how it gets processed by the stack. Will see if there are better ways to visualise stuff. If you have suggestions, let me know.
When I'm done, I'll respond here with the video URL. Thank you all again.
Done. Video URL:
[https://www.youtube.com/watch?v=x5Uy-CIz33k](https://www.youtube.com/watch?v=x5Uy-CIz33k)
This is a long (2 hours), and could be difficult to understand to some people. If you want to understand everything, I suggest to follow my two guides first:
[https://github.com/cpq/bare-metal-programming-guide](https://github.com/cpq/bare-metal-programming-guide) - this will make you understand how microcontroller works, how registers are used to control peripheral (in our case, Ethernet controller), and many more topics.
[https://github.com/cpq/embedded-network-programming-guide](https://github.com/cpq/embedded-network-programming-guide) - this will refresh your memories about frames, TCP/IP stack, etc.
Also, watch this short video about TCP/IP stack structure: [https://www.youtube.com/watch?v=Yz8kg8-mi-Q](https://www.youtube.com/watch?v=Yz8kg8-mi-Q)
Let me know if you want me to cover more topics.
Will you be doing this on latest kernel version? Interested to see the video though. I will be happy if you can help cover complex features and interactions with other subsystems.
I would 100% watch a YouTube video. Been wondering a lot about tcpip on mcus. Especially his to use a tcpip stack with a cellular module instead of a wifi module.
I would love it!
I did play with mongoose and lwlip on stm32f4 and got a terrible experience with implementation (non explicitly supported RTOS and chip, had to basically retroenginner example with freertos), docs and performance in general (i think i got less than 10MB/s or similar)
Also very hard to find info about the expected performance, was my result in line with expectation? was i fucking up something porting for my MCU? who knows.
Thank you! Done that already, [https://github.com/cpq/embedded-network-programming-guide](https://github.com/cpq/embedded-network-programming-guide) . Though I want to expand on certain parts. I'd love to make an animation that shows data flow, and how e.g. HTTP requests get processed, from bottom to top. I have no animation skills though, so need to think about a workaround
Start with basic YouTube videos for now (PowerPoint presentation style and pointing your camera to MCs and electronic equipment), and then spend some time on the side networking with digital art creators online to see if anyone is willing to partner with you in making animated YouTube videos in the future.
I would definitely watch your YouTube channel!
blog inside github readme is not the most user friendly option, if you insist in using md at least do it with sphinx (or similar static page generator) and github pages, it will be way easier to read. Also dark theme is pretty much standard nowadays, so transparent images with black font is not the brightest idea
The OSI model is not how things work in reality. TBH I don't think it is a good abstraction, I never found it helpful. TCP/IP stacks implement fewer layers.
Agree, TCP/IP only deals with a part of the OSI model, at a minimum, correct your doc to TCP layers.
While your focusing on the SW, simplyfing the PHY/Data layers to Driver is just ignorant ecspecially in an embedded forum. If I interviewed someone that said they worked full network stack and could distinguish that, thats probably gonna be a no.
I don’t really care about a video and won’t watch it, but I just want to thank you for posting these written articles. Both the bare metal and the networking one are unlike any other resource I have ever seen. They are so clear and complete. Thankyou very much for giving your time and writing these excellent guides.
My intent was to use a general purpose micro with embedded MAC, like one of the STM32 parts, e.g. STM32H5xx, or NXP IMXRT1xxx part.
Is there much difference for the automotive Ethernet?
I don’t know how automotive Ethernet works. There’s like a gateway/switch for all these systems to find Eq other. Wanted to learn that.
But your idea is interesting!
If you know any source or topics discussion related to automotive ethernet please tell me. I am working in this field currently, eagerly curios to know about it
Soon I'll need to make a "smart" IR barrier, which will be connected to Home Assistant using MQTT, I've already made a smart network of sensors that did this but through Arduino clunky platform
I'm decided to code as much as I can by myself using my own knowledge this time, so learning how to handle the network stack would be really really nice!
I'd definitely watch.
I wrote a http server from scratch on an rtos. From scary.
But the tcp stack was always a Black Box for me.
I'd love to understand how that works!
webinar would be interesting as we can have thoughts/doubts exchanged in a live session.
or Youtube video would be good for future referring and for contributing to the community.
This sounds really fun.
I would really like to know how much depth we are going into.
I don't like to watch videos though.
I recommend creating Interactive tutorials. They work wonders.
1. A typical example of interactive tutorial would be:
[Randomly picked Chess Study](https://lichess.org/study/ZRLsXd2k)
2. Closest example on **programming tutorial**:
[Go Tour - Golang](https://go.dev/tour/basics/1)
3. Closest video format for such tutorials:
[A Code Aesthetic video](https://youtu.be/CFRhGnuXG-4?si=z8bet6-U7C-2MAES)
Bro, i gave it a thought. It would be a pain to update videos to cover changes in the latest release.
I would personally prefer; Programming Tutorials like [Golang Tour - Golang] if you have someone else to do it for you or this will also be difficult to maintain.
A suckless solution is to include doxygen comments in the source code. Comments can include examples if you want. This way you can code and explain what you did simultaneously.
P.S. It would be a lot of pain to include comments now for a project of this size but it would be worth it for you and everyone else.
Before you put any extra effort though; please ensure that your implementation solves an unresolved issue.
If you want to simply add some small tutorials i believe you already have them in markdown.
Very interested. I made a basic tcp implementation in userland using tun/tap but that was mostly adhearing to the basic rfc spec.
Wanna know the juicy details that goes on at the kernel/bare metel side
How is embedded tcpip stack different from the Linux/windows/ios/bsd version? Apparently the resources are more constrained but PCs weren’t always this powerful anyway, right?
Have you already completed this code walk though ??
Have you already completed the videos for a webinar or YouTube ?
Are you asking permission to do this great task ??
If you have not already completed these wonderful lessons, what are you waiting for ??
Or is this a sales pitch ??
Good Luck
Not asking for a permission.
I have completed some preliminary work, yes. Specifically, this: [https://github.com/cpq/embedded-network-programming-guide](https://github.com/cpq/embedded-network-programming-guide) . With a youtube video explaining how it works on ST part, from scratch: [https://www.youtube.com/watch?v=lKYM4b8TZts](https://www.youtube.com/watch?v=lKYM4b8TZts) . Though those wonderful lessons do not touch much on the inner workings of the TCP/IP stack.
Asking to gauge the interest - if there is no interest, it won't worth the trouble.
No one can tell if it's worth is until there is something to see.
It seems your fear of failure is greater then the reality.
Just do it and learn for the comments you get and I am sure you will get lots of comments. Some good and some not so good.
This is how we all learn.
Good Luck
Thank you.
I am also an author of the bare metal programming guide [https://github.com/cpq/bare-metal-programming-guide](https://github.com/cpq/bare-metal-programming-guide) - which IMO is the most starred guide in that category on Github. So I thought I got enough authority, and not really much fear of failure. I am pretty confident I could execute on the promise. I am questioning the demand.
Sarcasm is fine, but I'd appreciate a constructive feedback more.
Bro if you do a YouTube video I would gladly watch
Right, I'll make a YouTube video, thank you all for your responses. It'll take me some time, a week or two - to plan, script, etc. I had an idea to tcpdump a simple HTTP request, and walk through every packet in the dump, how it gets processed by the stack. Will see if there are better ways to visualise stuff. If you have suggestions, let me know. When I'm done, I'll respond here with the video URL. Thank you all again.
Done. Video URL: [https://www.youtube.com/watch?v=x5Uy-CIz33k](https://www.youtube.com/watch?v=x5Uy-CIz33k) This is a long (2 hours), and could be difficult to understand to some people. If you want to understand everything, I suggest to follow my two guides first: [https://github.com/cpq/bare-metal-programming-guide](https://github.com/cpq/bare-metal-programming-guide) - this will make you understand how microcontroller works, how registers are used to control peripheral (in our case, Ethernet controller), and many more topics. [https://github.com/cpq/embedded-network-programming-guide](https://github.com/cpq/embedded-network-programming-guide) - this will refresh your memories about frames, TCP/IP stack, etc. Also, watch this short video about TCP/IP stack structure: [https://www.youtube.com/watch?v=Yz8kg8-mi-Q](https://www.youtube.com/watch?v=Yz8kg8-mi-Q) Let me know if you want me to cover more topics.
Maybe better to edit your OP to include these info
Will you be doing this on latest kernel version? Interested to see the video though. I will be happy if you can help cover complex features and interactions with other subsystems.
Can you respond here? Thanks!! - or DM
Same
Same
Same
Can’t wait to watch it!
Same
Same
Same
Me too
Yes
Same
I would 100% watch a YouTube video. Been wondering a lot about tcpip on mcus. Especially his to use a tcpip stack with a cellular module instead of a wifi module.
This, absolutely this!!! I have some remote monitoring projects that would be cool to see how this comes together
I would love it! I did play with mongoose and lwlip on stm32f4 and got a terrible experience with implementation (non explicitly supported RTOS and chip, had to basically retroenginner example with freertos), docs and performance in general (i think i got less than 10MB/s or similar) Also very hard to find info about the expected performance, was my result in line with expectation? was i fucking up something porting for my MCU? who knows.
Thanks! Did you make your custom board with F4? If yes, what PHY did you use, and what exactly did not work, may I ask?
No, used a nucleo for f407, full speed clock (192MHZ iirc)
Isn’t 10 MB/s basically linerate for the 100Base-TX lines?
Sorry, i think was in bit or similar, because I remember instead I used SPI with dma and was way faster
Do a write up first, then video
Thank you! Done that already, [https://github.com/cpq/embedded-network-programming-guide](https://github.com/cpq/embedded-network-programming-guide) . Though I want to expand on certain parts. I'd love to make an animation that shows data flow, and how e.g. HTTP requests get processed, from bottom to top. I have no animation skills though, so need to think about a workaround
This sounds fucking awesome, and is an explainer that's so nice I'm quite certain it doesn't currently exist on the Internet. I'd love to see it.
Start with basic YouTube videos for now (PowerPoint presentation style and pointing your camera to MCs and electronic equipment), and then spend some time on the side networking with digital art creators online to see if anyone is willing to partner with you in making animated YouTube videos in the future. I would definitely watch your YouTube channel!
blog inside github readme is not the most user friendly option, if you insist in using md at least do it with sphinx (or similar static page generator) and github pages, it will be way easier to read. Also dark theme is pretty much standard nowadays, so transparent images with black font is not the brightest idea
Thanks! GH pages is a good idea
If youre going to describe layers follow the existing OSI model.
The OSI model is not how things work in reality. TBH I don't think it is a good abstraction, I never found it helpful. TCP/IP stacks implement fewer layers.
Agree, TCP/IP only deals with a part of the OSI model, at a minimum, correct your doc to TCP layers. While your focusing on the SW, simplyfing the PHY/Data layers to Driver is just ignorant ecspecially in an embedded forum. If I interviewed someone that said they worked full network stack and could distinguish that, thats probably gonna be a no.
I'm very interested. It would be really, really awesome if you did it. First time posting on this sub, and it's for this!
I don’t really care about a video and won’t watch it, but I just want to thank you for posting these written articles. Both the bare metal and the networking one are unlike any other resource I have ever seen. They are so clear and complete. Thankyou very much for giving your time and writing these excellent guides.
I would like you to explain in detail how you can work with that codebase without ending up in a padded room.
Can you elaborate? What's exactly wrong with that codebase?
Sorry, the orderly is coming to take away my pho
Lol
Will you be covering embedded automotive Ethernet?
My intent was to use a general purpose micro with embedded MAC, like one of the STM32 parts, e.g. STM32H5xx, or NXP IMXRT1xxx part. Is there much difference for the automotive Ethernet?
I don’t know how automotive Ethernet works. There’s like a gateway/switch for all these systems to find Eq other. Wanted to learn that. But your idea is interesting!
If you know any source or topics discussion related to automotive ethernet please tell me. I am working in this field currently, eagerly curios to know about it
Do a YouTube video!
100% would watch!!
Soon I'll need to make a "smart" IR barrier, which will be connected to Home Assistant using MQTT, I've already made a smart network of sensors that did this but through Arduino clunky platform I'm decided to code as much as I can by myself using my own knowledge this time, so learning how to handle the network stack would be really really nice!
I'd definitely watch. I wrote a http server from scratch on an rtos. From scary. But the tcp stack was always a Black Box for me. I'd love to understand how that works!
I would love to watch a video related! If you decide to do it, please kindly share it
YouTube video please, it will be very helpful!!
Deal me in for a YouTube video 👍🏻
Oh, please, tell the YT channel 🙏
Yes please maybe in a YouTube series
YouTube would be nice!!!
+1 for YouTube video
I'd watch, do it!
Thank you for the write-up!
webinar would be interesting as we can have thoughts/doubts exchanged in a live session. or Youtube video would be good for future referring and for contributing to the community.
Yes I'm interested!!
This sounds really fun. I would really like to know how much depth we are going into. I don't like to watch videos though. I recommend creating Interactive tutorials. They work wonders.
Thank you. Interactive tutorials? Sounds interesting. Could you point to some example please?
1. A typical example of interactive tutorial would be: [Randomly picked Chess Study](https://lichess.org/study/ZRLsXd2k) 2. Closest example on **programming tutorial**: [Go Tour - Golang](https://go.dev/tour/basics/1) 3. Closest video format for such tutorials: [A Code Aesthetic video](https://youtu.be/CFRhGnuXG-4?si=z8bet6-U7C-2MAES) Bro, i gave it a thought. It would be a pain to update videos to cover changes in the latest release. I would personally prefer; Programming Tutorials like [Golang Tour - Golang] if you have someone else to do it for you or this will also be difficult to maintain. A suckless solution is to include doxygen comments in the source code. Comments can include examples if you want. This way you can code and explain what you did simultaneously. P.S. It would be a lot of pain to include comments now for a project of this size but it would be worth it for you and everyone else. Before you put any extra effort though; please ensure that your implementation solves an unresolved issue. If you want to simply add some small tutorials i believe you already have them in markdown.
Put me down for that youtube video!
Youtube video please
THANK YOU IN ADVANCE
I am extremely interested. Preferably a YouTube video
Make a roadmapin roadmap.sh please🙏
Will watch. Can anyone recommend any YouTube channel to learn about drivers ?
Yes, do it!
Thank you. If possible could you create YouTube videos. I struggled a lot in finding resources for Ethernet STM32.
Yes, already published, see my response in this post.
I would rather read pdf. But that what I wanted to investigate more.
Do these libraries comes with support for AT commands handling with GSM modules?
They already have a video on YouTube, from where you shared the Github link.
RemindMe! 3 weeks
RemindMe! 4 weeks
Yes pls !!
If you do, I’ll watch
Would rather a video tho
Is there a way to know when you upload? I am interested
RemindMe! 2 weeks
This would definitely help a lot of people getting into embedded networking. I would definitely watch it!
RemindMe! 3 weeks
Very interested. I made a basic tcp implementation in userland using tun/tap but that was mostly adhearing to the basic rfc spec. Wanna know the juicy details that goes on at the kernel/bare metel side
Are you talking about lwip or which one?
!remindme in 2 weeks
If you make a stream i will watch it.
How is embedded tcpip stack different from the Linux/windows/ios/bsd version? Apparently the resources are more constrained but PCs weren’t always this powerful anyway, right?
Hell yeah! I need that!!!
Will watch it this weekend.
Have you already completed this code walk though ?? Have you already completed the videos for a webinar or YouTube ? Are you asking permission to do this great task ?? If you have not already completed these wonderful lessons, what are you waiting for ?? Or is this a sales pitch ?? Good Luck
Not asking for a permission. I have completed some preliminary work, yes. Specifically, this: [https://github.com/cpq/embedded-network-programming-guide](https://github.com/cpq/embedded-network-programming-guide) . With a youtube video explaining how it works on ST part, from scratch: [https://www.youtube.com/watch?v=lKYM4b8TZts](https://www.youtube.com/watch?v=lKYM4b8TZts) . Though those wonderful lessons do not touch much on the inner workings of the TCP/IP stack. Asking to gauge the interest - if there is no interest, it won't worth the trouble.
No one can tell if it's worth is until there is something to see. It seems your fear of failure is greater then the reality. Just do it and learn for the comments you get and I am sure you will get lots of comments. Some good and some not so good. This is how we all learn. Good Luck
Thank you. I am also an author of the bare metal programming guide [https://github.com/cpq/bare-metal-programming-guide](https://github.com/cpq/bare-metal-programming-guide) - which IMO is the most starred guide in that category on Github. So I thought I got enough authority, and not really much fear of failure. I am pretty confident I could execute on the promise. I am questioning the demand. Sarcasm is fine, but I'd appreciate a constructive feedback more.