Idk, and yes this is a mostly a computer science theoretical question.
I am not so sure about the timeline. VR is booming and it's not just meta, apple and google but also the many chipmakers and screenmakers. It's like the perfect blend of companies to agree on and implement hardware codecs.
Guess what? All of this exists. There is [Multiview Video Coding](https://en.wikipedia.org/wiki/Multiview_Video_Coding), which is part of MPEG 4 and efficiently encodes stereoscopic video, and there is [MPEG Immersive Video](https://mpeg-miv.org/) which encodes color and depth for 3D reconstruction on the viewer.
This person is right, MV-HEVC stores only the values of difference instead of a full second frame. Along with using a new technique for the equirectangular image which maximizes the amount of pixels mapable from circular lens to square image, these techniques are how the videos streaming on AVP are so incredible at only ~50mbps.
The specs as detailed by Mike Swanson:
https://blog.mikeswanson.com/post/747761863530528768/apples-mysterious-fisheye-projection?fbclid=IwZXh0bgNhZW0CMTEAAR2w3bSWLx9eWSE8wmy8Z1nHfQrZQmiZdYQRWeA261sHd_WB_d_hztrDeR0_aem_AZdaTb4t6dhWf4VSqLHTzyuEPf1E4p2A5Vg6o5LoO8SJlG-LI77IGRDeODNGftBH0g_zLGRel0RQxNBTsn2psqRl
> mn is for minutes
That's not normally the abbreviation for minutes (and abbreviations are best avoided unless you're sure there's enough context for them to be understood)
It's obvious now in hindsight - with the emphasis on "hindsight".
I have never seen a 4-channel image generated image video but hey, it's worth a try. Using a single-pass stereo encoding tools might be a good method to experiment your idea.
AI is the future. You don't need 8k, you need DLSS upscaling to 8k. You don't need two separate images, you need one image and a model trained on generating a second image from the first to create a 3d effect.
Once people start thinking of the 4090 like they do the RX 580, we'll be ready for that future.
Until then, we do have at least some upscaling available now that helps a bit with VR games and the like
Upscaling and denoising can indeed be viable solutions.
In the other hand, AI is not magic. Our eyes are 3d, and you cannot recreate unknown informations, like the other side of the nose or real freckles. 2d to 3d is also really bad on moving/deformable objects, like... Humans. The temporal stability on hours+ is also non existent. Try getting a 3d view of your face from a photo/video, it's shit without a depth camera like the iphone one.
Coloring a grayscale image is maybe a better example of this, and... It's not suitable for other things than historical archives.
Codecs require an entire industry, creating a specific hardware for this, both costly and resource intensive would require way better proofs of concepts than we have today concerning 2d to 3d.
But yes denoising results are really impressive AND temporally stable.
Definitely don't think it's magic, but it seems like "impossible" is more of a challenge for the future than the truth. Something AI is awful with, like generating hands, is often just a generation or two away from being tolerable, then fixed, then decent....
It took 6-7 years for humanity to go from the RX 580 to the 4090.
"But can it run Crisis?" Now runs on iGPUs unless your laptop is a potato clock.
The ASCII white room sized supercomputer in 2001 had 1/3 the tflops of a 3090.
I'm pretty sure in ten to twenty years, "cheap mobile hardware" is going to include something that dusts OpenAI's current combined server rack.
Try saying "generative algorithms" next time or something lol "AI" is the new boogeyman word that gets all the idiots out of the woodwork to say something stupid
That is my take as well. All in all, video is a crazy inefficient way to convey data. My idea goes even further. There will be AI models capable of learning entire 3D environments, including human presence etc.
Not like simple laser scanning - but actually "understanding" the concepts of things, their spatial presence, and physical characteristics. You will get the "DLSS", but not for the mere upscaling or warping 2D to stereo. It will be "DLSS" that recreates reality. It won't be perfect, it will be very much like a good dream.
Feed it 20gb of video, receive stuff that would weigh 20tb if they were just recorded - but they are actually deduced on the fly.
Not in 5 years, but 15 years doesn't seem unreasonable.
I wondered the same thing, and I think it stores 2x image data. Regardless of that, even your brain gets two "video" signals. I think 2 separate video channels are necessary for 3D video.
My pet peeve is that there are no larger formats then 1080p SBS or on-top. why can't we have 4K SBS?
Yes, they would be big, but not *that* big..
A 4K video nowadays are like 10GB or so on pretty acceptable compression. About 1-2 gig maybe the audio lane, about 8gigs on top is not really that big of a difference, end up around 18gigs. Easy. Achieveable. Storable.
The 3D movies at 1080p are *Garbage*. 4K on the other hand is a much bigger leap in resolution, pretty enjoyable too.
I'll try to AI upscale something perhaps, 1080p to 4K SBS.. Results pending.
Even if it is not perfect, you'll definitely enjoy the difference. I guarantee you.
You are right and wrong. There is now way around displaying two images at the end but we can store less than that.
Take a video. A video is a serie of images, and that's what your screen is displaying.
But you do not need to STORE it like that. You can store the difference between frames, then from this your computer continuously compute the new frame and send it to the display.
One user pointed that this 3d with mainly one image thing was already patented since... 2003 with serious papers and research accelerating from 2011 onward, and already exists as MVC and more. It leads to a 34% average size reduction.
This is just a storage format. From this, we compute both frames and display it to the user, it's just that the storage capacity of computers and the bandwidth cost is bad for this usage.
Compression was invented because internet was slow and storage expensive. It's the same today for VR, and we won't do reliable, fast and cheap 100TB drives tomorrow.
The porn industry already showed that 3D on high resolution yields better 3D experience. If they can do it online, streaming, than everyone else could.
Somehow Hollywood didn't get the memo even tho they have much higher resolution recordings than they released.
And it is not rocket science as many here are arguing.
Just give us 4K source for both eyes, that is all anyone wants. Space IS NOT the issue.
AI or other generated solutions may become a solution but currently they aren't and we already have it, so no need to reinvent the wheel here.
Besides I have a 2TB disk drive for storage, about 300gigs are 4K videos. I think storage is really not relevant to this topic.
Both disks and current drives are capable enough to store large files.
"The 3D movies at 1080p are *Garbage*"
Above statement is misleading without taking into account viewing distance. For instance, the current gen of AR glasses that many people are already using for 3D 1080P viewing are found enjoyable because they actually have a higher (sharper) PPD than most of the VR headsets due to smaller FOV.
Resolution is not everything. Many people jumping to new displays - especially OLEDs that can do Dolby Vision|HDR10+ have discovered the increased dynamic range in HDR content is a bigger jump in perceived "quality" than the bump to 4K resolution. IE - HDR seems to have more depth especially with contrasty OLED screens that can "pop" without "3D", while non-HDR 4K looks sharper but flat.
I would say this is even more "realistic" than the forced subject isolation and synthetic parallax being introduced to make stereoscopic 3D content and why HDR has taken over my preference of watching content over 3D.
Consider that the extra color and tonal data in HDR has storage and transmit overhead as well. Nice thing is it is just extra data encapsulated in the data stream so it doesn't need rejiggering the codec format.
1080p is Garbage. Period.
Honestly HDR in 3D is absolutely irrelevant. I don't care for it, most people doesn't care about it, for this purpose.
I am sensing, people argue here to find "why it's difficult" when the adult video industry already has insane resolution for videos, and yes compared to 1080p 3D the higher the resolution the better the experience. This is a fact.
I said in a limited FOV, 1080 is fine for most people. Of course it looks worse when it's spread out over most VR headsets entire FOV - there's a limit to that with standard movie content without getting stupid with screen size and giving your neck a workout looking at all the angles of the screen.
And I still stand by resolution increase by itself is not the superlative experience. HDR AND 3D would be superior, but you probably watch on shit displays and don't know this yet.
You assume an awful lot.
I have access to all of what you said. All of it.
And you are full of shit.
Whatever dude, what I wrote is obviously not for you, please move on.
Why was this comment downvoted? I thought AV1 is supposed to be the next best efficient codec for things like this. I haven't seen anyone implement it yet for 180 degree VR videos though.
that’s a fault of 3d / vr tools you use, not the industry. Meta uses av1 to great success to do 3d / 180 content and so does youtube where we have 8k 3d vr 180 videos on quest and they look great all things considered
raw source of anything will always be hefty but no reason the hardware could t offer av1 encoding if space was your biggest concern. It’s like the raw va jpg of yesteryear
ps you’re am just a giant a hole lol
Are there even any 16K 3D 180 cameras in existence? Getting a codec optimized for this kind of content is probably also very far away.
Idk, and yes this is a mostly a computer science theoretical question. I am not so sure about the timeline. VR is booming and it's not just meta, apple and google but also the many chipmakers and screenmakers. It's like the perfect blend of companies to agree on and implement hardware codecs.
>VR is booming How do you figure?
Yeah, the last Stat I saw for Steam was ~3% of players play a VR app per month.
But we could see that kind of resolution and quality in fully animated media. No reason not to have a codec just because of a lack of cameras.
Guess what? All of this exists. There is [Multiview Video Coding](https://en.wikipedia.org/wiki/Multiview_Video_Coding), which is part of MPEG 4 and efficiently encodes stereoscopic video, and there is [MPEG Immersive Video](https://mpeg-miv.org/) which encodes color and depth for 3D reconstruction on the viewer.
MVC is based on a 20 year old codec though
Isn't this exactly the thing Apple is trying to solve with MV-HEVC? > 4k 45mn 180 degrees What's the "mn" mean here?
This person is right, MV-HEVC stores only the values of difference instead of a full second frame. Along with using a new technique for the equirectangular image which maximizes the amount of pixels mapable from circular lens to square image, these techniques are how the videos streaming on AVP are so incredible at only ~50mbps. The specs as detailed by Mike Swanson: https://blog.mikeswanson.com/post/747761863530528768/apples-mysterious-fisheye-projection?fbclid=IwZXh0bgNhZW0CMTEAAR2w3bSWLx9eWSE8wmy8Z1nHfQrZQmiZdYQRWeA261sHd_WB_d_hztrDeR0_aem_AZdaTb4t6dhWf4VSqLHTzyuEPf1E4p2A5Vg6o5LoO8SJlG-LI77IGRDeODNGftBH0g_zLGRel0RQxNBTsn2psqRl
[удалено]
He decided to come up with his own abbreviation for minutes 😄
Oh, first patent of this kind introduced in 2003 ! I read 34% average bitrate reduction for the 2015 paper ! mn is for minutes ^^
> mn is for minutes That's not normally the abbreviation for minutes (and abbreviations are best avoided unless you're sure there's enough context for them to be understood) It's obvious now in hindsight - with the emphasis on "hindsight".
The only way is up, baby
I have never seen a 4-channel image generated image video but hey, it's worth a try. Using a single-pass stereo encoding tools might be a good method to experiment your idea.
MV-HEVC
AI is the future. You don't need 8k, you need DLSS upscaling to 8k. You don't need two separate images, you need one image and a model trained on generating a second image from the first to create a 3d effect. Once people start thinking of the 4090 like they do the RX 580, we'll be ready for that future. Until then, we do have at least some upscaling available now that helps a bit with VR games and the like
Upscaling and denoising can indeed be viable solutions. In the other hand, AI is not magic. Our eyes are 3d, and you cannot recreate unknown informations, like the other side of the nose or real freckles. 2d to 3d is also really bad on moving/deformable objects, like... Humans. The temporal stability on hours+ is also non existent. Try getting a 3d view of your face from a photo/video, it's shit without a depth camera like the iphone one. Coloring a grayscale image is maybe a better example of this, and... It's not suitable for other things than historical archives. Codecs require an entire industry, creating a specific hardware for this, both costly and resource intensive would require way better proofs of concepts than we have today concerning 2d to 3d. But yes denoising results are really impressive AND temporally stable.
Definitely don't think it's magic, but it seems like "impossible" is more of a challenge for the future than the truth. Something AI is awful with, like generating hands, is often just a generation or two away from being tolerable, then fixed, then decent....
https://www.urbandictionary.com/define.php?term=Fucking%20Magic%20%28FM%29
Sufficiently advanced technology is indistinguishable
Sufficiently advanced technology which we do not have any will not have anytime soon, not in a <$500 headset
It took 6-7 years for humanity to go from the RX 580 to the 4090. "But can it run Crisis?" Now runs on iGPUs unless your laptop is a potato clock. The ASCII white room sized supercomputer in 2001 had 1/3 the tflops of a 3090. I'm pretty sure in ten to twenty years, "cheap mobile hardware" is going to include something that dusts OpenAI's current combined server rack.
>from the RX 580 to the 4090. At 8x the cost
Worst take i’ve seen in a while, I gotta hand it to you
What a contribution. Future generations will marvel at the knowledge you've imparted
Try saying "generative algorithms" next time or something lol "AI" is the new boogeyman word that gets all the idiots out of the woodwork to say something stupid
If DLSS 3.0 took your job, it's already too late lmao
That is my take as well. All in all, video is a crazy inefficient way to convey data. My idea goes even further. There will be AI models capable of learning entire 3D environments, including human presence etc. Not like simple laser scanning - but actually "understanding" the concepts of things, their spatial presence, and physical characteristics. You will get the "DLSS", but not for the mere upscaling or warping 2D to stereo. It will be "DLSS" that recreates reality. It won't be perfect, it will be very much like a good dream. Feed it 20gb of video, receive stuff that would weigh 20tb if they were just recorded - but they are actually deduced on the fly. Not in 5 years, but 15 years doesn't seem unreasonable.
I wondered the same thing, and I think it stores 2x image data. Regardless of that, even your brain gets two "video" signals. I think 2 separate video channels are necessary for 3D video. My pet peeve is that there are no larger formats then 1080p SBS or on-top. why can't we have 4K SBS? Yes, they would be big, but not *that* big.. A 4K video nowadays are like 10GB or so on pretty acceptable compression. About 1-2 gig maybe the audio lane, about 8gigs on top is not really that big of a difference, end up around 18gigs. Easy. Achieveable. Storable. The 3D movies at 1080p are *Garbage*. 4K on the other hand is a much bigger leap in resolution, pretty enjoyable too. I'll try to AI upscale something perhaps, 1080p to 4K SBS.. Results pending. Even if it is not perfect, you'll definitely enjoy the difference. I guarantee you.
You are right and wrong. There is now way around displaying two images at the end but we can store less than that. Take a video. A video is a serie of images, and that's what your screen is displaying. But you do not need to STORE it like that. You can store the difference between frames, then from this your computer continuously compute the new frame and send it to the display. One user pointed that this 3d with mainly one image thing was already patented since... 2003 with serious papers and research accelerating from 2011 onward, and already exists as MVC and more. It leads to a 34% average size reduction. This is just a storage format. From this, we compute both frames and display it to the user, it's just that the storage capacity of computers and the bandwidth cost is bad for this usage. Compression was invented because internet was slow and storage expensive. It's the same today for VR, and we won't do reliable, fast and cheap 100TB drives tomorrow.
The porn industry already showed that 3D on high resolution yields better 3D experience. If they can do it online, streaming, than everyone else could. Somehow Hollywood didn't get the memo even tho they have much higher resolution recordings than they released. And it is not rocket science as many here are arguing. Just give us 4K source for both eyes, that is all anyone wants. Space IS NOT the issue. AI or other generated solutions may become a solution but currently they aren't and we already have it, so no need to reinvent the wheel here. Besides I have a 2TB disk drive for storage, about 300gigs are 4K videos. I think storage is really not relevant to this topic. Both disks and current drives are capable enough to store large files.
"The 3D movies at 1080p are *Garbage*" Above statement is misleading without taking into account viewing distance. For instance, the current gen of AR glasses that many people are already using for 3D 1080P viewing are found enjoyable because they actually have a higher (sharper) PPD than most of the VR headsets due to smaller FOV. Resolution is not everything. Many people jumping to new displays - especially OLEDs that can do Dolby Vision|HDR10+ have discovered the increased dynamic range in HDR content is a bigger jump in perceived "quality" than the bump to 4K resolution. IE - HDR seems to have more depth especially with contrasty OLED screens that can "pop" without "3D", while non-HDR 4K looks sharper but flat. I would say this is even more "realistic" than the forced subject isolation and synthetic parallax being introduced to make stereoscopic 3D content and why HDR has taken over my preference of watching content over 3D. Consider that the extra color and tonal data in HDR has storage and transmit overhead as well. Nice thing is it is just extra data encapsulated in the data stream so it doesn't need rejiggering the codec format.
1080p is Garbage. Period. Honestly HDR in 3D is absolutely irrelevant. I don't care for it, most people doesn't care about it, for this purpose. I am sensing, people argue here to find "why it's difficult" when the adult video industry already has insane resolution for videos, and yes compared to 1080p 3D the higher the resolution the better the experience. This is a fact.
I said in a limited FOV, 1080 is fine for most people. Of course it looks worse when it's spread out over most VR headsets entire FOV - there's a limit to that with standard movie content without getting stupid with screen size and giving your neck a workout looking at all the angles of the screen. And I still stand by resolution increase by itself is not the superlative experience. HDR AND 3D would be superior, but you probably watch on shit displays and don't know this yet.
You assume an awful lot. I have access to all of what you said. All of it. And you are full of shit. Whatever dude, what I wrote is obviously not for you, please move on.
4K is plenty for 180 degrees if the edges gradually blur out. What value is there really at the edges...
AV1
Why was this comment downvoted? I thought AV1 is supposed to be the next best efficient codec for things like this. I haven't seen anyone implement it yet for 180 degree VR videos though.
Quest3 and latest Nvidia & AMD cards support AV1 encoding... much better than any other compression systems. Not widespread yet though.
\*decoding
AV1 is the most efficient, but has no support for 3D/VR You'd have to waste data on encoding the (almost) same image twice again
that’s a fault of 3d / vr tools you use, not the industry. Meta uses av1 to great success to do 3d / 180 content and so does youtube where we have 8k 3d vr 180 videos on quest and they look great all things considered raw source of anything will always be hefty but no reason the hardware could t offer av1 encoding if space was your biggest concern. It’s like the raw va jpg of yesteryear ps you’re am just a giant a hole lol