TBS 5927

[satopsguy]

That is a poor approach: If the card with the modified driver really outputs the raw bbframe data (possible, but probably only on usb cards and even then only if the driver uses that as the default format), then most 0x47 bytes will simply not be there, except for a single one at the start of the bbframe (in the bbframe header) and some spurious 0x47 data bytes. So such an approach would be very error prone even on perfect streams

After reading more technial background - especially the DVB-S2 standard - things are much clearer to me. I might still be wrong but I understand, that USB driver filtered for MATTYPE to be equal to MPEG TS streams only by synchronizing on 0x47 sync byte. This would discard all GSE types. So they just removed the filtering of the stream type in the driver. My reference to raw block data was wrong. I have not further investigated, why the usb driver itself syncs on the MPEG TS stream. I would have suspected a video player to do so.

Your other explanations make perfectly sense to me now. Let me summarize for the other readers:

The DVB-S2 standard defines the following stream types (MATYPE):
* Single or multiple Transport Streams (TS)
* Generic Packetized
* Generic continuous

That means base band frames are the lowest format that the standard defines. Everything before is physical layer demodulation and forward error correction. Each BBFrame contains one of the three MATYPEs. That means - as you correctly pointed out - requesting stid135 to embed BBFrames into a private TS is enough and complete.
Using neumodvb, data streams are also perfectly identified as GSE or Non-TS.

As a last step, I need to investigate, what container format, 'don't look up' expects. It "works already" by letting the python script parse the recording of neumodmx since data is only containerized - so an embedded TCP frame would be correctly parsed if not packetized into several frames.

 

[vma]

 

[deeptho]

After reading more technial background - especially the DVB-S2 standard - things are much clearer to me. I might still be wrong but I understand, that USB driver filtered for MATTYPE to be equal to MPEG TS streams only by synchronizing on 0x47 sync byte. This would discard all GSE types. So they just removed the filtering of the stream type in the driver. My reference to raw block data was wrong. I have not further investigated, why the usb driver itself syncs on the MPEG TS stream. I would have suspected a video player to do so.

Your other explanations make perfectly sense to me now. Let me summarize for the other readers:

The DVB-S2 standard defines the following stream types (MATYPE):
* Single or multiple Transport Streams (TS)
* Generic Packetized
* Generic continuous

That means base band frames are the lowest format that the standard defines. Everything before is physical layer demodulation and forward error correction. Each BBFrame contains one of the three MATYPEs. That means - as you correctly pointed out - requesting stid135 to embed BBFrames into a private TS is enough and complete.
Using neumodvb, data streams are also perfectly identified as GSE or Non-TS.

As a last step, I need to investigate, what container format, 'don't look up' expects. It "works already" by letting the python script parse the recording of neumodmx since data is only containerized - so an embedded TCP frame would be correctly parsed if not packetized into several frames.

There are actually 4 possible types.
11 = Transport
00 = Generic Packetized
01 = Generic continuous
10 = GSE-HEM
A few other things can be set, resulting in 256 possible matypes, but all datastreams are based on bbframes. The complexity is about what is contained in the bbframes. For instance, siminn is a Generic packetized stream that contains a transport stream that contains a datagram stream that contains udp packets that contain a transport stream. And that one is still simple.

It is all described in standards documents, which are sometimes unclear, so there is room for interpretation.
Soms streams also contain other suprises such as all kinds of headers.

Several pieces of software are available, which attempt to parse all of this, e.g., check the github repo by newspaperman (but expect errors)

 

[Morty]

Maybe one more comment about "We modified the tuner card’s driver to record a raw transport stream at the DVB-S/S2(X) protocollayer and above instead of an MPEG-TS video/audio stream. MPEG-
TS frames are prefixed with a synchronization byte 0x47 [14, 15];when this byte is not observed, the driver assumes it has lost synchronization and discards bytes until the next available 0x47. This approach works for an MPEG-TS video/audio stream temporarily corrupted by noise, but it fails for non-MPEG-TS streams, such as raw DVB-S2(X), which do not use 0x47 for frame synchronization. To capture raw streams of DVB-S/S2(X) bytes, we modified and recompiled the tuner card’s Linux driver to disable this MPEG-TS internal filtering and forward the raw demodulated data directly to the demux0 device. Our modified driver is open source to support future satellite research [71].
"

That is a poor approach: If the card with the modified driver really outputs the raw bbframe data (possible, but probably only on usb cards and even then only if the driver uses that as the default format), then most 0x47 bytes will simply not be there, except for a single one at the start of the bbframe (in the bbframe header) and some spurious 0x47 data bytes. So such an approach would be very error prone even on perfect streams

On the other hand, on pci-express cards I think the fpga prevents sending raw data (at least on tbs6909x), so that approach then
cannot work (the OS does not matter! It is probably the FPGA that blocks it). However, all cards based on stv091x and stid135 can embed bbframes in a transport stream, and that includes a header to easily find the bbframes. And that is the format already supported by drivers for years now. So it could have been used.

Note that the paper itself points to the authors not having much experience with satellite. E.g., they seem to have rediscovered the principle of the polar mount.. :->
" We introduce a new method to self-align a motorized dish to improve signal quality." and "Prior to this work, achieving accurate alignment required an expensive dual-axis steerable dish or phased-array steerable antenna. We demonstrate that with hidden alignment data in
satellite feeds, even a hobbyist-grade single axis rotor dish can achieve good enough alignment to receive high-quality signal from
all visible satellites. " The "hidden" data turns out to be the nit tables and public websites.

Yeah we are still learning : ).