This is experimental and shouldn't be used in production. Don't transmit random stuff on ISM bands without understanding local duty cycle limits, ERP restrictions and other regulations!
In this cute post :3
- Testing RX for LoRa, FSK and MSK with a second board
- Implementing BPSK framing: preamble, syncword, length, CRC
- Adding data whitening for better signal quality
Part 4 - RX testing and improving BPSK!
Introduction
Today I waited WHOLE DAY to receive a package with a second STM32WLE5JC board, but when it finally happened, I immediately wanted to test if everything I worked on was worth it. After all, if RX part of the crate wasn't working, it'd be a big waste of time! I quickly opened up the package, soldered pin headers and programmed my two boards, one with lora-tx example, and another one with lora-rx one. Of course, everything worked properly. I was pretty happy, but not surprised as LoRa is the selling point of this board after all!
Okay, the tension was rising, now let's test the less fancy sister of the quirky STM32WLE5JC radio family - FSK. It didn't work at first, just threw a CrcInvalid error, and on the subsequent tests I didn't receive anything at all. It was a bit expected honestly, because I pulled the modulation parameters straight from nowhere. I made them up. Imagined, they were revealed to me in a dream, even. This time I decided to pick more sensible sounding options, and I went with 380 Hz freq dev (instead of a fucking 32kHz one... for 600 bps signal...) and smaller RX bandwidth. Of course I was careful to not create a MSK signal by accident, so I calculated the modulation index - it wasn't 0.5, good. I also changed the default packet type to be of PacketLengthType::Variable - that was the source of the original CrcInvalid error. After that, it worked just fine!
Omgomgomgomgomg
Alright, now we're getting into the unexplored zone. As I wrote in my previous post here, the radio on the chip doesn't actually allow MSK reception, only transmission. I also found out that MSK is a subtype of FSK, which means that under certain circumstances (only if the modulation has a continuous phase), it is possible to transmit MSK using a FSK modulator. The signal in the previous post looked like MSK, but I wasn't sure if it would work. I fired up the msk-tx and msk-rx examples, and oh wow:
0.028411 [INFO ] waiting for msk stuffs... (msk_rx src/bin/msk_rx.rs:48)
0.028442 [DEBUG] Updating payload length to 255 (stm32wl_subghz src/modulations/msk.rs:95)
0.028503 [DEBUG] Setting packet params [0, 20, 4, 40, 0, 1, ff, 2, 1] (stm32wl_subghz stm32wl-subghz/src/radio.rs:694)
0.028656 [DEBUG] Setting buffer base to tx_base 0 rx_base 0 (stm32wl_subghz stm32wl-subghz/src/radio.rs:768)
0.028747 [DEBUG] Clearing IRQs, mask 3ff (stm32wl_subghz stm32wl-subghz/src/radio.rs:874)
0.028839 [DEBUG] Setting IRQs on DIO1, mask 24a (stm32wl_subghz stm32wl-subghz/src/radio.rs:855)
0.028869 [DEBUG] Setting DIO IRQs, masks: irq 24a dio1 24a dio2 0 dio3 0 (stm32wl_subghz stm32wl-subghz/src/radio.rs:835)
0.029022 [DEBUG] Setting rx timer on preamble detect (true) or sync/header (false) to true (stm32wl_subghz stm32wl-subghz/src/radio.rs:757)
0.029144 [DEBUG] RF switch in rx mode (stm32wl_subghz stm32wl-subghz/src/radio.rs:301)
0.039398 [DEBUG] Radio in Rx mode (stm32wl_subghz stm32wl-subghz/src/radio.rs:399)
0.039428 [DEBUG] Polling IRQ, mask 242 (stm32wl_subghz stm32wl-subghz/src/radio.rs:882)
3.436157 [DEBUG] Clearing IRQs, mask 3ff (stm32wl_subghz stm32wl-subghz/src/radio.rs:874)
3.436248 [DEBUG] RF switch off (stm32wl_subghz stm32wl-subghz/src/radio.rs:308)
3.436279 [DEBUG] Radio in Standby mode (stm32wl_subghz stm32wl-subghz/src/radio.rs:892)
3.436401 [DEBUG] Got RX buffer status a6 0 (stm32wl_subghz stm32wl-subghz/src/radio.rs:924)
3.436645 [INFO ] yay :3 got 166 bytes: [68, 69, 69, 69, 69, 69, 20, 68, 65, 6c, 6c, 6f, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 74, 68, 69, 73, 20, 69, 73, 20, 61, 20, 6c, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 6e, 67, 20, 74, 65, 78, 74, 21, 20, 76, 65, 72, 79, 20, 6c, 6f, 6e, 67, 20, 3a, 3e, 20, 61, 6e, 64, 20, 63, 75, 74, 65, 21, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 3a, 33, 20, 75, 6d, 6d, 6d, 6d, 20, 75, 72, 67, 68, 68, 68, 20, 61, 77, 77, 77, 6f, 6f, 6f, 6f, 6f, 6f, 6f, 20, 77, 6f, 6f, 66, 20, 77, 6f, 6f, 6f, 6f, 6f, 66, 20, 77, 6f, 6f, 66] (msk_rx src/bin/msk_rx.rs:51)It actually worked! I'm still a bit unsure if I did everything correctly, but seems like so! I'm pretty sure it's the only Rust crate that exposes that as MSK :> (though to be completely honest you could do that with any SX1262 crate yourself if you knew which modulation params you have to set exactly)
But yeah, I'm super happy that I could verify that everything transmits correctly and the boards talk to each other cutely!!!
Improving BPSK
After looking at yesterday's waterfall, I knew something was off in my BPSK signal. It looked too non-uniform, too repeatable. The modulation and everything might be correct, but you're not decoding anything if the data isn't packed using some clever tricks (reference to Trixie from My Little Pony fr).
Why tf framing exists?
Framing is important to basically sync up the transmitting and receiving devices. Without framing, the receiver has no idea where the actual data starts, how quickly are the bytes sent, what's noise and what's useful. STM32WLE5JC only exposes the bare minimum of its BPSK modem and unlike other modulation types, doesn't give the user any hardware-backed framing. So sending raw bytes is useless and as far as I know, it's very hard/impossible to recover anything from that.
I looked at RM0461 (I have this number burned in my retinas after all that time) and searched for some info about how framing for FSK looks like. It can be found here: "Figure 9. Generic packet frames format"! Looking at the implementation, they use this structure for a generic frame type (variable length one):
[ Preamble ][ Syncword ][ Len ][ Addr ][ Payload ][ CRC ]
<----------><-- 0-8b --> <------0-255b-----><- 0-2b->
<--------- CRC ---------->
<----------- Whitening ----------->I looked everything up and tried to get as much info as possible about everything. What I found was super interesting and it gave me an idea how to proceed with software-backed implementation of BPSK packet framing.
After some planning, here's the final packet structure I ended up implementing:
#[derive(Clone, Copy, defmt::Format)]
pub enum BpskPacket {
/// No framing, just send raw data
Raw,
/// Use a configurable framing
Framing {
/// Length of the preamble (0xAA) in bytes
preamble_len: usize,
/// Synchronization word (max 32 bytes)
sync_word: [u8; 32],
/// Sync word length
sync_word_len: usize,
/// Enable/disable reporting length in the packet
include_len: bool,
/// CRC size (0, 1 or 2 bytes)
crc_type: CrcType,
/// Whitening algorithm
whitening: Whitening,
/// Whitening LFSR seed (9-bit, 0x000..0x1FF)
whitening_seed: u16,
},
}Preamble
Preamble is crucial for synchronizing clocks on both sides of the transmission, as they're never perfectly synced. The preamble (hex 0xAA = 10101010 in binary) gives the receiver an alternating pattern of symbols to lock its timing onto. One well known example of using a preamble is for example Ethernet - 10BASE-T, which does the exact same thing but over copper instead of radio frequencies. It sends the exact same pattern (0xAA, well 0x55 actually but sends LSB first so it looks like 0xAA over the wire) followed by a frame delimiter, analogous to a syncword I'll explain next.
Syncword
Once the receiver is synced, it needs to know one thing - when does the data start? What if I wanna send a fuckton of 0xAAs after a 0xAA preamble? Well, it would have no way of knowing that. That's why it's necessary to include a syncword, which is a bit pattern known by both the transmitter (obv) and the receiver. When the receiver notices it, it knows the next bit in the data will be the useful payload itself. After reading for a bit, I picked 0x1F35, which is one of the known Barker codes - the longest one found (13-bits long). Barker codes have ideal autocorrelation properties, meaning that (from what I understood) it's very hard to miss it or confuse it with something else. When it hasn't arrived yet, the receiver most probably will know it hasn't. And if it has arrived, it will match very strongly. It's used in eg. 802.11b WiFi!
Length byte
A very SHRIMPLE and elegant way of telling the receiver when the payload ends and CRC starts. Shrimple as.
CRC
It's used for error detection, which happens a lot in RF. Bits might flip, some interference might arrive - CRCs are used as a fast way of detecting such errors. The longer the CRC is, the less chance there is of a false negative. I decided to implement both 1-byte CRC (a common CRC-8 with 0x07 polynomial) and a CCITT 2-byte CRC (which uses a 0x1021 polynomial).
Whitening
This is probably the most interesting part and I learned a lot of things I didn't know about earlier. Whitening is an operation that transforms data into a pseudorandom form - which means that ideally there's a low chance of low-entropy, repeatable patterns. It's important, because without it, data patterns create similar patterns in the signal. BPSK encodes bits as phase transitions, so for example, if I were to send a lot of 0x00s one after another, there would be no phase transitions at all - the only signal I'd get would be the carrier. This creates several problems, the worst one of all is that clock recovery fails completely: the BPSK receiver needs the phase transitions to stay synced. If there are none, the clock might drift mid-packet, losing data.
The solution is to XOR the data with a pseudorandom sequence before transmitting. A common technique is using a LFSR (linear feedback shift register). LFSR works like a normal shift register, where bit positions rotate to the right (eg. bit 1 becomes bit 2), but it also looks at certain bit positions (I used CCITT whitening LFSR with polynomial x^9 + x^4 + 1 - so bits 9 and 4 are tapped), XORs them together and puts them in bit 1 position. This creates a pseudorandom pattern with desirable properties, such as low autocorrelation or roughly equal distribution of 0s and 1s. This pattern is then XORed with the payload and that produces the proper payload, ready to be sent! When both the seed and polynomial are known to the receiver, it can generate the exact same pseudorandom sequence and basically undo all this stuff to recover the original data. Neat :3
Here's the implementation in the code:
impl Whitening {
fn apply(self, seed: u16, data: &mut [u8]) {
match self {
Whitening::None => return,
Whitening::Ccitt => {}
}
// Calculate CCITT whitening using x^9 + x^4 + 1 polynomial and LFSR
let mut lfsr: u16 = seed & 0x1FF;
for byte in data.iter_mut() {
let mut mask = 0u8;
for bit in 0..8 {
let feedback = ((lfsr >> 8) ^ (lfsr >> 3)) & 1;
lfsr = ((lfsr << 1) | feedback) & 0x1FF;
mask |= (feedback as u8) << (7 - bit);
}
*byte ^= mask;
}
}
}Implementing that sequence in code was really straightforward. It was mostly about putting right stuff in the right places, and calculating some almost-black-magic formulas I found here and there. But yeah I have to say the to_bytes impl of BpskPacket looks really nice :3
fn to_bytes(self, payload: &[u8], buf: &mut [u8]) -> Result<u8, RadioError> {
match self {
BpskPacket::Raw => {
// Simple copy operation, no modifications made
buf[..payload.len()].copy_from_slice(payload);
payload.len().try_into().map_err(|_| RadioError::PayloadTooLarge)
}
BpskPacket::Framing {
preamble_len,
sync_word,
sync_word_len,
include_len,
crc_type,
whitening,
whitening_seed,
} => {
let len_field_size = if include_len { 1 } else { 0 };
let crc_size = match crc_type {
CrcType::None => 0,
CrcType::Crc8 => 1,
CrcType::Crc16 => 2,
};
// Validate packet length
let total = preamble_len + sync_word_len + len_field_size + payload.len() + crc_size;
if total > buf.len() {
return Err(RadioError::PayloadTooLarge);
}
// Keeps track of the current position in the buffer
let mut pos = 0;
// Write preamble which consists of 0xAA symbols
buf[pos..pos + preamble_len].fill(0xAA);
pos += preamble_len;
// Write sync word
buf[pos..pos + sync_word_len].copy_from_slice(&sync_word[..sync_word_len]);
pos += sync_word_len;
// Actual payload starts here
let data_start = pos;
// If enabled in the config, write length info
if include_len {
let payload_len: u8 = payload.len().try_into().map_err(|_| RadioError::PayloadTooLarge)?;
buf[pos] = payload_len;
pos += 1;
}
// Copy the original payload itself
buf[pos..pos + payload.len()].copy_from_slice(payload);
pos += payload.len();
// Compute CRC before the whitening
let (crc, crc_len) = crc_type.compute(&buf[data_start..pos]);
crc_type.write(crc, &mut buf[pos..]);
pos += crc_len;
// Apply whitening
whitening.apply(whitening_seed, &mut buf[data_start..pos]);
// Additional validation - if buffer position can't fit in u8, it's invalid
pos.try_into().map_err(|_| RadioError::PayloadTooLarge)
}
}
}Testing and conclusions
Even after learning all of that shit I don't feel ready to fight the omnious GNU (GNU'S NOT UNIX) RADIO program... So for now I was still going to inspect the SDR waterfalls manually. I did a quick test with the same text I sent yesterday, and just look at that (top signal - Raw packet mode, bottom - Framing packet mode with Whitening::Ccitt):
The regularities are mostly gone, the signal looks way cleaner and the preamble is clearly visible. Now for another test, a stream of 0x00s:
YAAAAAAY!!! It looks SOOOOO different. A shitty carrier vs something that actually looks decodable. Nice af :>
The code is available here, as usual!
I have several plans for the next part, including continuous rx mode, sending longer packets than 255 bytes ("The payload length can be extended beyond 255 bytes. Refer to Section 4.6: Sub-GHz radio data buffer for more details." OwO) or trying to decode the BPSK with GNU-fucking-Radio.
Stay fuzzy and comfy and see ya :3
References
- WHO WOULD'VE GUESSED RM0461
- Cool blog post about data whitening
- Barker code
- Probably some other stuffs I closed and forgor (I'm eepy af it's 3am.........)