xscreensaver analogtv #1
Comments
|
Awesome. Please use the ffmpeg_ntsc version of the code. The ffmpeg_to_composite code fakes it in the YUV colorspace which is not quite right, while ffmpeg_ntsc converts it to the NTSC YIQ colorspace first before processing and converting back to RGB. |
|
Right, thx! |
|
One other request: when you port the code could you please mention this project (and the github URL) in the comments? |
|
Yes! I do it always, you'll be credited fully with links and etc. My task is to port only. Generally, I'm making only video part for still images. So your tool is more generic and more usable. Do you plan to release executables? I had some problems with compiling on Ubuntu (exactly versions of ffmpeg libraries). And I couldn't compile it under Cygwin (windows) at all. One more thing: during porting I've found several places to optimization (eg. calculating the same sin/cos for every line in video_chroma_phase_noise part) and one bug (uv rotation formula is wrong). I can prepare pull request if you wish. |
|
Works for me. Let me know when to pull. |
|
First PR ready. Question: do you have idea how to incorporate noise like this: https://www.youtube.com/watch?v=6zoJavBcU5o? Or kind of broken tape / head errors? |
|
I do know of one form of head error (that I sort of emulate at this time) that has to do with the tape heads as they pass over the tape. If you look at the drum head of a VCR, there are two notches 180 degrees apart from each other. 4-head and HiFi VCRs have 4 of these notches, the other two are used to read/write the hifi audio into the video portion on tape, but that's beyond the scope at the moment. The VCR's loading mechanism wraps the videotape around the head so that one of the two heads is in contact with the tape at any time. The way a VCR normally works is that it rotates the drum head at the frame rate of the video standard it's designed for (30 for NTSC, 25 for PAL). One complete rotation of the head in the time of one frame means that the first head passes over the tape to read out the first field, then the second head passes over the tape for the second field. This is why, when one of the heads gets dirty and stops reading the tape, playing a tape gives you a picture that's stable but flickers between picture and static. To make this work, the VCR has to decide at some point to switch from one head to another. This is aided by tracking pulses and the VCR's tracking system. But it's not perfect. The heads are not exactly 180 degrees away from each other due to manufacturing tolerances, which is why when the VCR makes that switch, the video at the switching point is bent or offset slightly. To hide this, most VCRs when they record set up the tracking pulses and switching point so that the switchover happens about 4-8 scanlines before vertical retrace so that the distortion is hidden by your TV set's overscan area. That's the odd distortion you see from your TV capture card at the bottom when you play a VHS tape through it. The discoloration you see at that switching point is because of the way NTSC color works. NTSC color is amplitude modulated atop the luminance (black and white Y signal) picture as two AM signals 90 degrees apart that carry (I and Q). In order for your TV set to decode the color correctly, it needs to read the reference "colorburst" in the horizontal sync pulse and synchronize the color subcarrier clock. But if you cut NTSC video in a way that delays the picture like that, you throw the phase of the color subcarrier off and that's why video at the head switching point often has the wrong colors. Another form of distortion by VHS is that the VCR does not directly record the video signal to tape. Instead, when you record, the VCR separates the color and luma signals from the composite signal and then handles them differently (the "color under" modulation you've probably heard of). Color is modulated at a lower frequency on tape than luminance. When you play it back, your VCR reads the color and luma signals from tape and then recombines them back into a composite video signal. One thing noted in the source code, found during research, is that the color signal played back from VHS isn't exactly what you recorded. VHS has fairly poor horizontal color resolution, so they hide that to some extent by using an analog delay circuit to mix the color signal with the color signal of the previous scanline (or, I think it has to do with how noisy the color is). The effect is like software outputting a scanline that is the average of the previous and current, vertically smearing the color a bit. |
|
Part of the emulation in this program is how non-hifi VHS records audio. In the original VHS format, video is recorded by the drum head across the tape and audio at the edge of the tape. Personal experience says that, as you'd expect, you can quietly hear the sync pulses among the hiss and noise when the audio track is quiet. But the audio bandwidth is also fairly poor especially at EP speeds. Since tape can't record DC signals, only analog signals that change, a lot of what goes onto videotape is modulated as either AM or FM to some fixed frequency. So on VHS, you have the color subcarrier modulated at one frequency and the luminance to another. On hi-fi VHS, you have a third frequency that carries frequency modulated FM audio (left and right channels separated) stored between chroma and luma, which is why hi-fi VHS sounds so much better than the analog linear tracks. To help make hi-fi sound even better, there's an analog 2:1 companding system to keep the audio signal above any noise and hiss that still gets into the signal. At the same time, because you're now modulating FM audio into the video head like that, hi-fi decks now have tighter tolerances with timing and switchover, else you get that "buzzing" sound in hi-fi audio playback. |
|
One of the effects I know I can add later, is from experience with VHS and EP recording speeds. Dropouts in EP don't become static unless they're large, they tend to sort of "cross-blend" the previous scanline. I think it has to do with luminance FM modulation and how EP recording speeds put the video tracks so close together on the tape, the "FM capture effect" I think it's called. Minor tape damage of this kind from my experience tends to look more like a minor rippling effect in the picture that moves downwards as the slightly damaged portion passes over the head. If it has hi-fi audio and you're playing it on a hi-fi VCR, you'll also hear some minor buzzing because it affects the hi-fi audio too. If the dropouts are severe enough or the buzzing gets loud enough most VCRs I've used will just shut off hi-fi audio and play the linear audio tracks until the damaged part passes. The SP speeds put the tracks farther apart, which is why the same effect doesn't happen much or at all with tape damage. |
|
I'm not sure yet how to recreate the tracking noise and sparkles, but I think it has something to do with luminance and FM modulation on the tape, since it is a bit like how an FM radio given a fading signal transitions from the radio station to loud static, rather than AM radio where a fading signal becomes quiet. Tape dropouts probably have the same effect even if momentary during the video signal. I do know that some VHS tapes have that constant static bar at the bottom because VCRs as they age will drift in timing and end up putting tracking pulses on tape that don't actually line up with where it's supposed to switch heads, so what you're seeing is the VCR switching heads too early when the head it switched to isn't touching the tape yet, thus, the static bar. You can move the static bar around with the tracking controls, but it's always somewhere in the signal. I have a few tapes recorded like that. I know it's the age of the VCR because some tapes recorded 2 years earlier on the same VCR play just fine with no tracking error. |
|
I hope this technical research isn't too jarring to people who thought VHS was just "record video signal to tape", there's some analog hackery involved in making VHS work. Don't mean to shatter any rose-tinted nostalgia glasses :) |
|
Thank you for deep explanation. I'm old enough to have VCRs as part of my life. It's kind of nostalgia to try to recreate it digitally. Unfortunately I'm not too much confident with signal processing and can't say that expected effect is a result of FM or AM modulation or the other thing. However I precisely understand what are you talking about as well as I understand what is going in the code. |
|
More technical information if interested https://www.av-iq.com/AVCAT/images/documents/pdfs/TT189%20-%204611.pdf Luminance is FM modulated, as I thought. So logically, dropouts register in the video signal the same way FM signal dropouts turn to noise. That's why when the tape is blank your VCR just shows black & white noise all over the screen. Something to consider in further development. |
|
Also, about Hi-Fi audio: |
|
I'm testing FM right now. I placed it just after first chroma_from_luma and before video_chroma_noise part. Result is like that: https://www.dropbox.com/s/745b76f5vbocg78/vicfm.jpg?dl=0 Visible noise created during demodulation of luma (chroma untouched). Let me know if it's expected. I made it in Processing now. This article about Hi-Fi audio is great, I didn't know about it. Recording audio and video on the same place of the tape is tricky concept :) |
|
VCRs as I understand it do lowpass the video signal to the bandwidth given. Can you bandwidth limit the video signal before FM modulation? I would also expect that VCRs retrieve each part of the signal then lowpass each part before processing. The noise does look like VHS signal loss but my experience is that the dropouts are either brief (the momentary sparkles) or only span a few scanlines at the bottom of the picture (common tracking error). Can you emulate that? |
|
My approach here is quite straightforward. I take data from fY, normalize it (to values between -1 and 1), do modulation and first step of demodulation (differentiation). On some condition based on noise and sin I set result of demodulation to maximum (value: 1), than filter out carrier and map back to fY values. Regarding last issue, I think yes, it's a matter of creating a function which triggers demodulation error. |
|
Can you modulate to a higher carrier frequency? I believe the reason chroma and luma are modulated as far apart as they are is so that when the VCR heterodynes them down to baseband it becomes easier to filter out frequencies above or below the desired center frequency. Since the documented bandwidth for luma is 1MHz, try something above 1MHz. http://hyperphysics.phy-astr.gsu.edu/hbase/audio/sumdif.html |
|
When I try to get higher carried freq I'm loosing precision... Maybe I'm doing something wrong or demodulation method (differentation + envelope detection) is poor. Maybe you can play with it. I can pull request with fm part if you wish. |
|
Does VS2017 have Perl in it?? |
|
No. But there are third party Perl installations you can use in Windows. |
|
OK, This time it'll work, but what project type do I have to make this in VS2017? |
|
Despite it's C++, but I don't know what type of program it is, GUI or plain console (quite unsure)? |
|
Plain console. Also, you'll need to compile it against FFMPEG 3.0 or higher in it's current state. It was written for Linux, not Windows, so there's that. |
|
And how do I perform it? |
|
I'm not even sure you can compile FFMPEG 3.0 in VS2017. You may have to use MinGW at the command line. All of the tools take options at the command line and process the video as instructed. The "scanimate" code requires you to modify a function to define how (by time t) to distort the picture. |
|
I will use the source code of FFMPEG 3.4.2, right. |
actually it's not an issue but do you know xscreensaver mode called xanalogtv? https://www.youtube.com/watch?v=VmM1KkFsry0
Code simulates NTSC TV.
https://www.jwz.org/xscreensaver/download.html (analogtv.c).
I've ported this to Processing/Java - https://github.com/tsulej/GenerateMe/tree/master/ntsc_analogtv
I thought it might be helpful.
(I plan to port your code to Processing/Java - to filter images, VHS simulation is holy grail in glitch community)
The text was updated successfully, but these errors were encountered: