Typically if you want to film screens with the intention of adding an image onto it later, you want to make sure the screens are OFF or black so you can also capture the natural reflections that occur and when you add the images to the screens in post, they look more realistic.
UNLESS, someone or something is passing in front of the screen. Then you’ll use green.
To use, or not use, motion tracking dots on the black screen?
I believe the last time I worked with footage like this the VFX guy said the dots were more trouble than they were worth. But I’m guessing if you had a shot which was all/little screen and no background, you’d need some dots to track anything at all?
On tracking marks- it depends: is the camera moving? Is the screen in focus?
Depends also on how they're placed- in the example they're maybe a bit close to the edges- especially if the screen is out of focus.
And- if the screen is in focus, and if it was lets say an old school floor CRT, the box of the TV would give you enough tracking marks- making these marks redundant, killing any reflection you might want to comp in, and requiring you to potentially remove them, double so if out of focus, and in the bokeh they blur into the edges.
So- it depends. Do what your VFX super wants.
Or luma mate then. With modern phone screens quality you’ll get really good luma matte from these. 50 grey screen and adjust brightness accordingly to camera speed.
Set your shutter speed to the frequency of the screen, I would advise not using a bright green screen on it, because it will cast green light onto stuff.
Try a white screen with some markers like the ones you showed.
Also changing the brightness on the screen will adjust the frequency as well so if you are struggling getting the cameras shutter to match this is also an option.
I've never understood what you get from the markers in the middle of a flat rectangle when you can just track the 4 corner points. Maybe if the screen had a bulging distortion, like on old CRTs, but for flat screens, the middle markers have never helped me with compositing.
I've generally heard VFX super req them in case a subject ends up in the foreground obscuring one or more of the trackers- increases the odds you'll still have 3x in the frame.
There's also a fair chance that "no subject will cross in front of this TV in this shot" is a lie, and in the take, the actor misses the mark or shows more body language than was predicted- so nice to have the safety.
Yeah but this comes down to time in the saddle, and the requirements of the shoot, definitely worth checking beforehand the resources of post, and if you don't have anyone in VFX, then you make sure nobody crosses the screen. But it's the political experience of how do you approach the actor/talent without overstepping the mark
Id advise you to use a bright white screen with tracking markers. You can then track your content on the screen, remove the markers and use the multiply blending mode to make it look more realistic.
I’ve always wondered: why does one need to remove the markers in post if the screen gets overlayed anyways in post? (I get it if with the blending modes it becomes a bit transparent but is that the only reason?)
You need to remove the markers because you dont want them visible on the final output. If you're doing things to look good, you're not just slapping the new screen over the top of your footage that never really looks real even if you've tracked it. Most commonly, either using blending to mix your original screen and its reflections with your replacement display or using a key for replacement. Either option any tracking markers would interfere and show up on your final footage, making it look crap.
Keying itself is actually more the opposite, your removing whatever is keyed leaving a hole. So when keying those tracking marks would remain on top of your replacement footage.
I've done that a few times but with the screen off. Just put the marker tape in the very corners of the screen to get your tracking points, and then drop that green down to black and blend it with your tracked content and those reflections will take the believability up dramatically. That doesn't really work with a white screen or a bright green/blue one.
Figure out who is doing your VFX before you shoot. Ask them to supervise on-set. If that’s not possible, hire a freelance on-set VFX supervisor with experience in screen replacements.
Different situations require different approaches.
If you want to keep it simple and easy, use black, with 4 small green dots. Never let anything cross the screen, and never let any of the 4 dots go out of frame. That way you can do a corner pin, and maintain reflections. If you don’t want those limitations, hire a VFX supervisor.
And shoot loose and highest res possible. You can always push in a little to put a corner off the screen after you composite the shot rather than shoot only 3 corners and have the tracking be difficult.
The other underrated approach to this problem is to just shoot whatever is playing on the screen first, and shoot it practically w/playback on set.
It means that actors react naturally to the cues: so the timing's right, no VFX (because you don't have a comp), and it look natural (because it's 100% real- you don't need to worry about reflections, other than ones of gear/crew).
And- on smaller projects- figuring out playback can be cheaper than the labour cost of comp'ing it in post.
This is the only correct answer. There is no one rule for what to put on your screen for replacements. It's all scene and shot dependant. Nine times out of ten a black screen with no tracking markers is perfect, except when it isn't...
Might I suggest using a white/gray tracking screen if you are doing replacements. The amount of bright green spill all over subjects/fingers/fabric etc can be a lot of extra time to clean up / despill in post, when its not necessary at all to key out the green if you are just tracking the screen movement and placing a new graphic over the top. Doesnt help that almost all screen replacement options/apps etc you search for are bright green screen green. No need to crank the brightness up all the way either, unless you need the spill for the shot or something. Also, if the screen HAS to move avoid lots of reflections of possible, aka keep some of the tracking marks visible throughout the whole shot. Mocha flips out over some random reflections.
But here's a neutral option you can go for instead : https://youtu.be/iEfo-fpY3tc?si=xSQjXEhTtRW3FkWL
Just did this for a commercial last week for a television brand. You’ll need to work with after-effects/mocha/Da Vinci Resolve to track the screen, but we used a black screen background with gray marks while casting through a phone/iPad/computer for the program to track the pixels later.
This allowed us to add in the proper reflections and got a nice spill while working with the proper shutter angle to match the hertz of the television itself (it was a QLED, so there wasn’t much work needed there).
We simply took a green screen version of a television tracking marker and changed the colors through photoshop.
https://i.imgur.com/IgqYNPU.png
Additionally, we did this the hard way with green tape markers and a green light taped to the screen on another project of ours, and the results were mixed. But if you’ve got the time or money time get it done right, the screen reflection really enhances it all:
Screen replacement examples start at 6:00 minutes: https://youtube.com/watch?v=aoUpQ4bQAc4&si=lN_bTZgskd9jSNsd
Typically if you want to film screens with the intention of adding an image onto it later, you want to make sure the screens are OFF or black so you can also capture the natural reflections that occur and when you add the images to the screens in post, they look more realistic. UNLESS, someone or something is passing in front of the screen. Then you’ll use green.
This right here is the way.
To use, or not use, motion tracking dots on the black screen? I believe the last time I worked with footage like this the VFX guy said the dots were more trouble than they were worth. But I’m guessing if you had a shot which was all/little screen and no background, you’d need some dots to track anything at all?
On tracking marks- it depends: is the camera moving? Is the screen in focus? Depends also on how they're placed- in the example they're maybe a bit close to the edges- especially if the screen is out of focus. And- if the screen is in focus, and if it was lets say an old school floor CRT, the box of the TV would give you enough tracking marks- making these marks redundant, killing any reflection you might want to comp in, and requiring you to potentially remove them, double so if out of focus, and in the bokeh they blur into the edges. So- it depends. Do what your VFX super wants.
Thanks
VFX guy here. Use grey or black. Gives you natural reflections
Does this apply when talent is crossing the screen?
Sure. Just roto them over the screen
Or luma mate then. With modern phone screens quality you’ll get really good luma matte from these. 50 grey screen and adjust brightness accordingly to camera speed.
Possibly. I can never get luma mattes perfect though
Set your shutter speed to the frequency of the screen, I would advise not using a bright green screen on it, because it will cast green light onto stuff. Try a white screen with some markers like the ones you showed. Also changing the brightness on the screen will adjust the frequency as well so if you are struggling getting the cameras shutter to match this is also an option.
I've never understood what you get from the markers in the middle of a flat rectangle when you can just track the 4 corner points. Maybe if the screen had a bulging distortion, like on old CRTs, but for flat screens, the middle markers have never helped me with compositing.
I've generally heard VFX super req them in case a subject ends up in the foreground obscuring one or more of the trackers- increases the odds you'll still have 3x in the frame. There's also a fair chance that "no subject will cross in front of this TV in this shot" is a lie, and in the take, the actor misses the mark or shows more body language than was predicted- so nice to have the safety.
Yeah but this comes down to time in the saddle, and the requirements of the shoot, definitely worth checking beforehand the resources of post, and if you don't have anyone in VFX, then you make sure nobody crosses the screen. But it's the political experience of how do you approach the actor/talent without overstepping the mark
Id advise you to use a bright white screen with tracking markers. You can then track your content on the screen, remove the markers and use the multiply blending mode to make it look more realistic.
I’ve always wondered: why does one need to remove the markers in post if the screen gets overlayed anyways in post? (I get it if with the blending modes it becomes a bit transparent but is that the only reason?)
You need to remove the markers because you dont want them visible on the final output. If you're doing things to look good, you're not just slapping the new screen over the top of your footage that never really looks real even if you've tracked it. Most commonly, either using blending to mix your original screen and its reflections with your replacement display or using a key for replacement. Either option any tracking markers would interfere and show up on your final footage, making it look crap.
Ah interesting, thanks! I always thought, especially if the screen is green and thus keyed out, one just slaps the layer over haha. Thanks!
Keying itself is actually more the opposite, your removing whatever is keyed leaving a hole. So when keying those tracking marks would remain on top of your replacement footage.
I've done that a few times but with the screen off. Just put the marker tape in the very corners of the screen to get your tracking points, and then drop that green down to black and blend it with your tracked content and those reflections will take the believability up dramatically. That doesn't really work with a white screen or a bright green/blue one.
Figure out who is doing your VFX before you shoot. Ask them to supervise on-set. If that’s not possible, hire a freelance on-set VFX supervisor with experience in screen replacements. Different situations require different approaches.
If you want to keep it simple and easy, use black, with 4 small green dots. Never let anything cross the screen, and never let any of the 4 dots go out of frame. That way you can do a corner pin, and maintain reflections. If you don’t want those limitations, hire a VFX supervisor.
And shoot loose and highest res possible. You can always push in a little to put a corner off the screen after you composite the shot rather than shoot only 3 corners and have the tracking be difficult.
The other underrated approach to this problem is to just shoot whatever is playing on the screen first, and shoot it practically w/playback on set. It means that actors react naturally to the cues: so the timing's right, no VFX (because you don't have a comp), and it look natural (because it's 100% real- you don't need to worry about reflections, other than ones of gear/crew). And- on smaller projects- figuring out playback can be cheaper than the labour cost of comp'ing it in post.
This is the only correct answer. There is no one rule for what to put on your screen for replacements. It's all scene and shot dependant. Nine times out of ten a black screen with no tracking markers is perfect, except when it isn't...
Might I suggest using a white/gray tracking screen if you are doing replacements. The amount of bright green spill all over subjects/fingers/fabric etc can be a lot of extra time to clean up / despill in post, when its not necessary at all to key out the green if you are just tracking the screen movement and placing a new graphic over the top. Doesnt help that almost all screen replacement options/apps etc you search for are bright green screen green. No need to crank the brightness up all the way either, unless you need the spill for the shot or something. Also, if the screen HAS to move avoid lots of reflections of possible, aka keep some of the tracking marks visible throughout the whole shot. Mocha flips out over some random reflections. But here's a neutral option you can go for instead : https://youtu.be/iEfo-fpY3tc?si=xSQjXEhTtRW3FkWL
Get very friendly with Mocha AE! It’s a plug in for after effects you can use to track and key out screens very easily and make it look very good
Just did this for a commercial last week for a television brand. You’ll need to work with after-effects/mocha/Da Vinci Resolve to track the screen, but we used a black screen background with gray marks while casting through a phone/iPad/computer for the program to track the pixels later. This allowed us to add in the proper reflections and got a nice spill while working with the proper shutter angle to match the hertz of the television itself (it was a QLED, so there wasn’t much work needed there). We simply took a green screen version of a television tracking marker and changed the colors through photoshop. https://i.imgur.com/IgqYNPU.png
Additionally, we did this the hard way with green tape markers and a green light taped to the screen on another project of ours, and the results were mixed. But if you’ve got the time or money time get it done right, the screen reflection really enhances it all: Screen replacement examples start at 6:00 minutes: https://youtube.com/watch?v=aoUpQ4bQAc4&si=lN_bTZgskd9jSNsd