Jump to content
Mr.RC-Cam

Analog HD FPV Video. Why Not?

Recommended Posts

Quote

Seems like main problem with using AHD camera is scaler in video googles. I will continue research.

Yes, one of the challenges is to find a low cost AHD1.0 to HDMI scaler with minimal added latency and friendly behavior with weak or missing FPV signals (no "blue" screen, shows snow, fast video re-sync recovery).

Share this post


Link to post
Share on other sites

I just have to comment on this thread.

I'm (too) currently investigating a possibility to have a digital link / HD image on FPV systems.

What most of the people overlook, is the channel bandwidth. On 5.8GHz FPV vTX-link, we're looking at about 20 MHz. - What was the link width of a WiFi system again? Yes, the same 20 MHz (some might have MIMO these days, which have 40 or 80 MHz). So the default 20 MHz channel width is more than plenty, since I know, for a fact, that there are millions of Netflix users out there, who have 20 MHz WiFi at home. And it works. So... digital HD image fits well in that 20 MHz band (even several streams at once).

Now, once we're over the bandwidth thingy, let's look at the signals. FPV vTX link is "FM" (frquency modulation, e.g. BPSK), but WiFi links are "AM" (amplitude modulated, e.g. QAM64). If, however, we look at the colorburst section of an analog video, it's actually kind of amplitude modulated. So all the vTX transmitters, even cheap ones, are capable of sending at least "kind of" AM. The downside of colorburst in CVBS signal is that it's very sensitive -> most often one looses the color info before your whole picture goes blank.

However what this all sounded like to me as a developer, is that one could send something else than raw CVBS data through the vTX link.

So I went on and bought a cheap Sony Starvis (IMX291) CCTV camera and a DVR board. The camera hardware is capable of creating a H.265 stream (and DVR capable of decoding it). The chip on the cam is Hi3516C V300, and these very common CCTV chips (along with other HiSilicon chips of the same family) have been hacked and utilised before. – Does someone else recall the Mirai botnet? :D

Now, here comes the tricky part.. The H.265 signal must be "wrapped" in an "analog CVBS shell" (along with checksums, etc.), and sent via ordinary, cheap vTX transmitter (because, why not ;), after all, expensive things are always expensive). On the receiving end, this "non-picture picture" must be decoded by some wizardry. I've got ADV7181C chips for this job, which rip out RGB-data (ITU-BT.656) from CVBS data, and the "real" bitstream can be obtained by interpreting the the decoded RGB-values.

After we have this real data (reconstructed H.265 stream from RGB-data), it must be fed back to the system somehow. For this, I have the DVR. It's capable of receiving and decoding H.265 stream (Hi3798M chip). From the DVR I get out both CVBS and HDMI.

There are some advantages to analog signal over purely digital WiFi. One obvious thing to note is, that CVBS signal is "running" all the time. There are 625 visible horizontal lines (PAL), which can be utilised individually. Traditionally an ethernet package is something like 1500 bytes, but there are a gazillion other protocols too (e.g. MODBUS, CAN, ..), which have a totally different package length (even variable lengths). So, why not pick a convenient timing and package length of one CVBS line?

The CVBS signal lines give 18750 (625 x 30) timed packages a sec. One data package can be e.g. 704 bytes, if each pixel on the line represents one byte, which means there must be 256 separable shades of gray. This might be unrealistic, but it would offer a whopping 13.2 Mbps of digital bandwidth, by just using the luma values of CVBS data! Recommendation for 1280x720 video is something between 1-2 Mbps, so in an ideal case there would be plenty of digital bandwidth. In addition to the b/w luma channel, there's the chroma, or colorburst. Colorburst is a bit more challenging to utilise, because it's a lot more sensitive, and it's resolution is only half of the luma (i.e. 352x312, or something like that..). This would, however, give some additional bandwidth e.g. for checksums, replay packages, and things like that.

I have only begun testing this setup, and I'm still waiting for some parts to arrive, but I'm already able to generate CVBS luma data from an Arduino (using TVout module), sending it via vTX link, and translating pixel data to real bytestream on receiving end. Next step is producing color signal with Arduino Due, and getting more data. Third step is to hack the Hi3516C completely to get the H.265 data out and process that into a CVBS (with e.g. an FPGA or Cortex-M7 or something like that).

Using a digital video stream there's always the problem of frames. So far each and every practical video codec I've come across, wants to study a full frame at a time. Where as analog signal is "just transmitted". In analog world, it's just: read capacitor values from CCD / CMOS, pipe the voltages to a vTX transmitter, receive those voltages on the receiving end (vRX), convert these values to digital, and pipe them via LVDS to your display.

Using a fast CMOS (e.g. Sony Starvis IMX291), even the speeds of 120 fps are achievable (8ms latency), but might not fit in the channel (with checksums 'n' stuff). 60 fps might be doable, which would give a frame latency of 17 ms + encode, wrap, transmit, unwrap, decode and LVDS output. A fast CMOS and a fast encoder might shave off some miliseconds, like newer Qualcomm Snapdragons with modern mobile phone oriented CMOS cameras (120-240 fps is common nowadays). This will, however, increase the cost. Ideally the whole setup would be <100 EUR/USD.

Share this post


Link to post
Share on other sites

Great idea and iteresting research!  Wow, cool things usually simple :)

 I'm familiar in Hisilicon SoC and I can write own firmware for IP cameras / dvrs for example, based on HI3516 / 18 / 20 , I can help in this research.

So your main proposal is to transmit digital video stream, encoded in analog levels? I think, encoding in CVBS not really needed, because radio must be transparent (need doublechek). Radio modulation doesn't matter at this point, so all what we need is modulate our digital signal using AM (as in your proposal with CVBS), but in this case we have no limits and do not must follow CVBS protocol. Its more flexible way. Also we may use audio channel for same purpose, extending bandwidth.

I plan to do some tests related to this post. I have all needed hardware, but don't have fast cameras, only standart 30fps.

Plan is to create simple encoder / decoder for various bitstream like in your proposal, but without CVBS.

 

Edited by HardRock

Share this post


Link to post
Share on other sites

I got myself a CCTV from AliExpress, it was like 30 USD:

https://www.aliexpress.com/item/H-265-H-264-3-0MP-2048-1536-IP-StarLight-Camera-Module-Board-3516C-IMX291-FishEye/32822600792.html

That camera does not utilize the full possible frame-rate of IMX291, but it's "easy" to start with. I've got the HiSilicon SDK for 3516C and the likes, and another SDK set for 3798M, which is the chip in the DVR board (also about 30 USD):

https://www.aliexpress.com/item/8CH-CCTV-H-265-NVR-Board-4K-5MP-4MP-HI3798M-Security-NVR-Module-4CH-5MP-8CH/32810674862.html

 

HiSilicon has even more powerful chips, but I cannot even find the Hi3516C from Mouser and the likes, so it's kind of hard to create hardware-prototypes. But perhaps those can be found from somewhere, AliExpress, Taobao, Dark web? :D Other thing is MIPI-CSI2 standard. I found some draft from the web, but it would cost like a gazillion dollars to be a member in MIPI organisation, and have full access to CSI-2 specs (the bus used in most cameras).

Then there's the H.265 codec. A free(?) alternative is VP9, but since HiSilicon already has the H.265 built-in, why not use it, at least for the tests.

I was thinking of utilising the CVBS as a "carrier", to re-use as much of existing hardware as possible. I don't have a HAM license (at least, not yet), so I cannot just blast "anything" from and to the skies. There would also be an advantage to using existing Foxeer/RunCam/Caddx/what-ever FPV cameras with this tech, since packing the data saves bandwidth. Even if one has a link of semi-poor quality, perfectly clear picture could be transmitted back to the goggles / ground station.

Analog signals are inherently more vibrant than digital, but can also be contaminated more easily with noise. Also, there is no error-checking and RAID-like recovery options for an analog signal. Digital signal can also be compressed, saving that precious bandwidth.

From those FPV-video over WiFi -projects I've studied (e.g. from Github), I started thinking, how the heck does WiFi actually work. I found some university lectures and a lot of other material by googling, also with references and explanations to what actually are the BPSK, QPSK, QAM, QAM4, QAM16, QAM64, and the likes. The foundational thing is, that you've got an antenna. What you do with it, is up to you. The WiFi specs are just the ones people are used to (99.999% of users never even knowing it). Still, nothing keeps from utilising the existing hardware differently.

I think – I haven't tested – that those cheap vTX transmitters don't really know what the passing data is. I might be wrong, in which case it wouldn't be an option to choose, whether or not to wrap data in CVBS stream. But, if those devices are dumb enough, they can be utilised far more effectively than an ordinary 100mW WiFi link. In which case AM and some convenient modulation is also completely possible.

Edited by androidi

Share this post


Link to post
Share on other sites

I have try to pass variable data over radio and this works. I put video input wire of vTX on oscilloscope test signal output pin and see same signal on video output of vRX. It has noise and level shift, but it the same signal as input. vTX is TS832 and vRX is RC805. So vTX and vRX is just dummy modulator / demodulator.

Where are you from? Normally you don't need any license when using general purpose frequencies like 5.8Ghz. Nobody care what signal flies in air at given frequency.

Edited by HardRock

Share this post


Link to post
Share on other sites

From Finland. I guess it won't matter. But I'm not sure if it's legal to actually create radios of any kind in here without the license.

So those devices really are dummies. That's beyond great news! No more Arduino tinkering, yesss... I actually have one custom SAMD21/RTOS-project right now, but in a couple of days I will return to this project and start poking around the CCTV board with the HiSilicon SDK.

It's great to have more people interested in doing this "the other way around". :)

Share this post


Link to post
Share on other sites

Laws may be different depending on country, but i sure it's completly legal. Limitations applied only on transmission power for selected frequency, but not for protocol. For example, 2.4Ghz RC transmitters use own protocols, not Wi-Fi, and its completly legal.

Share this post


Link to post
Share on other sites

I'm interested in working on this subject as well. I've been looking at the commercial solutions for so long and found that following the 802.11 spec is where latency lies. 

For example in the recent drone Parrot Anafi, they use a combination of 802.11 and rtsp streams. Parrot has made the effort to modify rtsp -library and g-streamer heavily to minimize latency ( down to 100ms, which is still too much ) , this integrated non-SOC implementation can be replicated with the following project https://github.com/bortek/EZ-WifiBroadcast

I think I still need more inspection hardware to get involved more in a required lower level, talking about oscilloscopes and evaluation boards for both encoding and decoding. I've been convinced by Androidi theory and boards produced by auvIDEA. (https://auvidea.com/e10/)

Fpvmodels "teaser" about building something that can fit quads reveals that they are using a wifi -module from pins in conjuction with a auvIDEA's E10 encoder. seen here https://www.facebook.com/pg/fpvmodel/photos/?tab=album&album_id=154256488115370  https://www.facebook.com/fpvmodel/photos/a.154256488115370/963966883810989/?type=3&theater Their tests for the latency are inaccurate, since they are tied to the devices they are using, but it reveals they are on the path.

Really hoping we can gather a good posse to work with, does rc-cam have a slack / discord I can hop into?

Share this post


Link to post
Share on other sites

Ok, so I pondered this thing a bit. After digging some practical information, I found the relevant links.

Looking at PAL, which has a higher frequency than NTSC, Wikipedia tells about as much as is needed for determining the data speed. – Both PAL and NTSC are irrelevant after this, except considering the modulation capabilities of el cheapo vtxs..
https://en.wikipedia.org/wiki/PAL

So we are looking at 8 MHz max frequency, which is achievable quite easily. First hit points to a TI set (from 1994!):
http://www.ti.com/lit/an/slwa022/slwa022.pdf

Instead of just sending FM, QAM should be a no-brainer:
https://www.radio-electronics.com/info/rf-technology-design/quadrature-amplitude-modulation-qam/what-is-qam-tutorial.php
https://en.wikipedia.org/wiki/QAM_(television)

64QAM would give a beefy 30Mbps digital bandwidth, which is more than plenty. (Even for several HD cameras :D)

I think I can ditch all that ADV-HDTV-chip sheizze, and just focus on pure signals, as HardRock pointed out. CVBS wrapper would have been a nice twist, but is totally unneeded, and is only limiting the creativity. So, the only thing left is to have H.265 output data from a HiSilicon chip, pass it through vtx-vrx pair, and decode it with another chip on the other end. Easy peasy.

The problems start to emerge, when signal quality starts to suffer. By downgrading from 64QAM to QPSK/BPSK the transmission speed is lower, but still more than enough (like 5Mbps or something).

One other thing to include would be some kind of error-checking / fixing mechanism. RAID striping with parity comes to mind, but it's designed for hard drives. CRC does not help, it only throws invalid data to trash. Ideally the fixing mechanism could reconstruct some useful data from a low-quality signal. There must be hundreds of these around, it's just a matter of digging one up, and implementing the needed bit-fiddling.

It would be superb to be able to change the HEVC encoding bitrate and/or resolution on the fly, by e.g. utilising the RSSI from "controller" rx (that 2.4 GHz thingy), via FC board, or something (I'll have to dig the BetaFlight source some more to get a better understanding of how things work there). Resolution change on the fly could be more of a problem, but I'd imagine changing the bitrate would just be some request to encoder process (depending on the implementation, of course).

But remember! Digital link for PAL video only takes some hundreds of Kbps, which already is quite "ground-breaking" (for FPV pilots, that is. The rest of the world already did this like 40 years ago. :DDD).

Edited by androidi

Share this post


Link to post
Share on other sites

It's now Oct 2019. Time for an update.

A few months ago Fatshark pre-announced a new HD FPV Video link called Byte Frost. Although most details are unclear, it appears to use AHD (720p) type cameras.
 https://www.facebook.com/FatSharkRC/photos/a.931142817012812/2189117317882016

But the Byte Frost system is not a wireless AHD system. It uses a pair of DM5680 chips. These use uncompressed digital encoding instead of simpler analog RF.

Edited by Mr.RC-Cam
Added link to DM5680

Share this post


Link to post
Share on other sites
On 5/8/2018 at 5:23 PM, Mr.RC-Cam said:

Yes, one of the challenges is to find a low cost AHD1.0 to HDMI scaler with minimal added latency and friendly behavior with weak or missing FPV signals (no "blue" screen, shows snow, fast video re-sync recovery).

Hey, I am just reading in as i am researching some of the same challenges. Quite happy to see that you are still on the project after such a long time. One of my main goals was to be able to get CVBS on my Blackmagic Video Assist (field monitor with integrated ProRes (Broadcast Quality) Recorder, and I happen to have found a converter that might be low signal friendly: https://www.aliexpress.com/item/32897673378.html?spm=a2g0s.9042311.0.0.27424c4dYjTfU1

First test were quite good, as it seems to have some sort of signal sync generator, but I have yet to test this recorder in the field.

Just so that you know, this is what a recording on ProRes looks like with CVBS footage: https://youtu.be/ID-fd5XVzuA

It just so happend that this converter can also see AHD, so I purchased a bunch of AHD camera's, that got sidelined to another project and are currently installed in a birds nest, but I'll get some more and share on the researching part.

Share this post


Link to post
Share on other sites

OK, I'm joining in on this adventure! I posted on reddit asking about AHD video systems wondering the feasibility of them. Looks like it isn't a totally far-fetched idea!

 

My thoughts on the AHD system:

Mr.RC-Cam, I commend your progress , I saw the DVR video of the FPV using AHD system, this would be a promising avenue for higher quality on small builds. There is still some work to be done as you pointed out, we need improve error handling a LOT, and allow the signal to degrade gracefully. It looks like it is FM moduated, which has the characteristic of digital where it either works or it doesn't (compare the footage at 53:00 vs the rest of the video). Frequency Modulation also wastes bandwidth which isn't desirable, but it does help with the noise immunity...

I'm wondering if the AHD standard can be modified even more to allow 720i60, I think 60i would be better than 30p as it would have 720p resolution with still image and about half that when moving but with much smoother motion. Deinterlacing filters have gotten pretty good! You can actually extend this concept of analog video compression to Multiple sub-Nyquist sampling encoding or MUSE, which was a failed quasi analog/digital standard that pushes a 1080i videostream down an 8 MHz channel, this should be easy to do with the current analog systems on the market, as you mentioned with the 960H systems!

There are quite a few standards out there: TVI, CVI, AHD, MUSE, then of course the ones that require multiple channels; S-video, component, VGA, etc.

The biggest limitation of analog systems is the difficulty in compressing the datastream meaningfully. That is; removing redundancy. You can look into digtial compression schemes to see how it's achieved, but it adds latency by design, AND by reducing redundancy small errors result in much more severe consequences and frame drops! It's a 2-edged sword!

 

Digital FPV:

Of course nowadays we can't ignore the presence of the DJI digital FPV system. But I think there are still some advantages to analog as Joshua Bardwell pointed out in his review of them. I for one take some dislike to it for being proprietary, although it is understandable given how much R&D had to go into developing them. Digital systems like this are going to be inherently complex state-full systems. Just look at VP.9 encoding if you want a taste of just how complicated digital compression is! Complicating things further, it seems the DJI system has variable bit-rate, which requires RSSI calculation and a bi-directional link to allow the goggles to report to the transmitter how much power to push! (or maybe not; maybe DJI has ways of recovering partial data better!)

The other outstanding issue with digital FPV is not so much that it's digital, it's that it after compression, it's a lot less tolerant to data loss. And compression by design requires having a number of buffered frames as decoding any frame requires the previous frames (they are effectively deltas) and future frames. Like in the case of MPEG, if you lose a frame it corrupts every frame to follow because they build on that corrupt data.

 

RF output / transmission:

This is a very hard topic. There are chips out there that effectively abstract this so it isn't too much of a concern unless you wanted something better, but worth noting a high level overview of different technologies:

AM (or more accurately, DSB-SC) is what I believe is used for NTSC. DSB-AM is quite wasteful, literally HALF the transmitted power is just transmitting a DC bias, which is mixed up to a the RF center frequency. This was well known even in the very early days of radio, it was used anyway cause it's darn easy to demodulate, literally a single diode (envelope detector) is all that's needed! The fact that it's Double Sideband  means that if your signal has 5MHz of bandwidth, it takes 10MHz bandwidth once modulated. FM is even worse! Look at Carson's rule to understand the workings of FM. It is more immune to noise. Again, a trade-off between bandwidth, noise immunity, and quality! It's like there's some theoretical limit haha.

Then you have the more advanced methods, like QAM. QAM is still pure analog means of data transmission, it allows you to transmit/receive 2 channels concurrently! an 'I' component and a 'Q' component, which are 90 degrees out of phase with each other which means that changes to one component do not affect the other, they are 100% independent (in theory at least). This allows you to fully utilize the band, so 5MHz baseband takes up 5MHz when mixed up to RF. But now you lack the elegant redundancy of DSB-SC and have more noise susceptibility...

You can take all theses methods, and simply feed them with digital data to turn them into their equivalent digital modulation counterparts. AM becomes OOK or ASK, FM becomes FSK (or variants like GSK MSK, etc.) and QAM is still called QAM but with a number at the end telling you how many symbols are in the constellation diagram. More symbols means more digital data within the same bandwidth, but means also higher noise susceptibility and the need for a cleaner channel. Hence QAM tends to be used mostly for high bandwidth low-loss coaxial data links, like cable internet.

If you want to get REALLY complicated, just look at the voodoo that mobile phone carriers are doing with 3G, 4G, and 5G. One of the hallmark technologies is the use of ultra wide-band communication with time and frequency division multiplexing and orthogonal frequency-division multiplexing (OFDM) which massively improves on problems like multipathing, which is important for cellular service. I suspect 5G with the touted low latency, may enable FPV over 5G and IP! The latest development here are large antanna arrays where you control the phase of the signal to each one to "beam" a signal in a particular direction. You can constuct a number of virtual signals in an FPGA or ASIC unique for each device on the network and optimize how much of a signal goes to each antenna to "beam" the signal in a particular direction! And each user has their specific traffic beamed towards them by putting carefully broadcasting those RF components on each antenna at just the right amplitude and phase to allow maximum reception. It's over my head for sure!

 

Solving the issue's at hand with AHD:

I recently bought a Xylinx Arty Z7 FPGA/SoC with HDMI outputs, and I used a similar dev board back in my university days. I might attempt to build a decoder for these signals and see. Low latency will hopefully be easy to achieve, just make the VGA output each line of data synchronous to the data from the AHD signal. Hopefully I can get a software/HDL ibrary that also allows compression and writing to a file to the SD card slot.

Another far-fetched goal of mine is to integrate the recording camera and the FPV camera. Having 2 separate cameras just seems redundant. Why not have a camera that outputs a videostream that can be used as your FPV? Many camcorders offer composite or even microHDMI outputs although these are useless for FPV as they are provided only for purposes of focus, framing/composition, and replaying video on the big-screen, all cases where latency is of little concern. Hopefully I can reverse-engineer one of those little MIPI/CSI camera sensors and use my FPGA to record 4K video from them and also because it's an FPGA, I can whip up some HDL that implements a AHD video output!

Things that will take time for me to figure out is how video compression works, reverse-engineering a camera sensor, and learning more about RF stuff.

 

 

Edited by PowerMax
clairity

Share this post


Link to post
Share on other sites

Umm, decoding demodulated NTSC even in an FPGA may be a bit more of a hassle than I expected. One challenge is going to be all the DSP filtering that needs to be done to separate the luma signal from the chroma subcarrier and modulation. Decoding color information will require implementing a phase locked loop within the FPGA to lock on to the colorburst, and require analog means to decode it. GNU radio and the SDR community might give me some insight this all works. My first prototype will probably be black and white, and may even have the dot crawl all over everything lol! It looks like an uncompressed digital stream of from something like HD-SDI might actually be easier and "better" as the signal encoding is more "naive" and easy to decode, although finding cheap hackable security cameras that use digital signals over coax is proving to be more challenging than expected. Then again, with uncompressed 720p30 you effectively need more than double the bandwidth, it's just how it goes. Intra-frame compression will help some.

I've considered ATSC, too. ATSC occupies the same bandwidth as NTSC, and even uses the same AM modulation! This means you should be able to continue using current 5.8GHz transmitters, a huge plus! But ATSC does rely on video compression algorithms, which makes it a stateful system and bit errors take some time to resolve. (like if you lose too much data within a frame to fully reconstruct, that bad data must be used to calculate future frames and adjacent pixels, leading to the classic digital corruption look with several blocks of bad pixels being shifted around until a full I-frame is sent to fix accumulated errors) So it stands to be scene how flyable a system like this would be when the whole image gets corrupted for up to a couple seconds.

I suspect the best approach is to use ODFM modulation as it was engineered specifically to be resilient against multipathing and doppler shift, combined with a video compression algorithm that relies mostly on intraframe compression (to prevent 1 corrupt frame from screwing up the rest) and can tolerate errors elegantly (such that the result is simply increased noise, or reduced resolution rather than missing blocks). ODFM is outside my scope of knowledge, it would require a PhD in RF electronics to do it right! Unless there is an IC that allows ODFM communication over 5.8GHz band. If anyone knows of a digital video transmission that meets these requirements, can be transmitted over a single channel, and occupies the same bandwidth of NTSC, I'm all ears! So far the only thing that comes to mind is ATSC and EX-SDI.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...