Sunday, September 30, 2012

USB telemetry receiver & rant about flux residues

I'll be covering this here before I write the article of my transmitter, because this thing was actually built long before the transmitter. The first tests of my receiver were in fact done using this device acting as the transmitter.

I needed a nice way to debug all the radio stuff I was doing in the spring of 2011, and the whole thought of hassling with serial ports gave me a headache. I had known for quite a while of a bit-banging USB solution on low-end 8bit Atmel microcontrollers, called the V-USB. I had been eager to try it, but I never really had a need for it, until now.

So, the vision was a USB dongle which could be used to receive and transmit packets easily through the radio module. In addition to help with debugging, I needed it to act as a PC interface to my other projects, such as home automation. Using it as a telemetry receiver, however, didn't come up until quite some time later, but as things turned out that's what I use it for mostly these days.
Bottom side view

Top side with a rubber ducky antenna in the SMA connector
The whole thing was supposed to be very simple. I Just copied the reference design of V-USB, attached a radio module through the SPI port, wrote up some software and I thought I was done (see schematics, layout). There were a couple of problems, which made things a bit more difficult.

  • ICSP and radio share SPI
  • Missing bypass capacitors
  • Flux residues
The radio module was attached to the microcontroller through SPI and will act as a slave device, which means it will listen on MOSI and talk on MISO. The microcontroller of course acts as a master device when talking to the radio. However, the microcontroller is programmed through the SPI port and will act as an SPI slave device during this time. Meaning it will too listen on MOSI and talk on MISO. Since both devices are talking on the same line at the same time, we have a conflict and the programming fails.

Okay. It's not like I didn't think of that when I was designing the board. The thing is: the radio will only talk on the MISO line if its slave select line is pulled low. As the microcontroller is being programmed, it should release the slave select line and a pull-up resistor should deselect the radio. For some reason this does not always happen, of which I'm still unclear why. Anyway, in all of my later designs I added a jumper, or some other method, which can be used to disconnect the radio from the SPI bus. This problem didn't very much affect the usability of the dongle, because it can be programmed through USB anyway. The SPI programming was only used a couple of times at first, just to get the USB boot loader programmed.


I'm not sure if it ever caused any issues, but inspecting the board later on I noticed, that I had completely forgotten about the chip level bypass capacitors. There is only one capacitor on the output side of the 3.3V regulator. I bodged a couple of small ceramics between tracks, to make sure that insufficient bypass was not an issue.

What turned out to be a real problem was flux residue. You would assume flux used in electronics solder is non-conductive, right? Wrong! I'm pretty sure it depends on the type of flux, but whatever is in my soldering paste (I think it's just kolophonium) is a bastard. The problem is that the flux flows between the radio module PCB and the main PCB as well as between the microcontroller package and the main PCB, and causes "shorts". Of course they are not zero resistance shorts or anything, but they present a significant finite resistance, which changes rise times of especially the very weak drivers of the radio module (I think they are in the order of 1mA max.). Now, getting rid of the flux residues is a real pain in the ass. Kolophonium is not really soluble in water. It is supposed to be soluble in isopropanol, but even after keeping the boards submerged in isopropanol for a full day, the residues are still there. I tried:
  • water & soap
  • isopropanol
  • commercial flux cleaning agent (smells and looks a lot like soapy water)
  • ethanol
  • acetone
  • turpentine
  • paint remover
And I'm happy to report that paint remover works amazingly well! Anything else on that list is not really worth the effort. The stuff I was using is something called Nitromors, but I've heard that many other paint removers work well too. This one has dichloromethane listed as the active ingredient and is a gooey consistency. Next bottle I buy, I'll probably go for something that's a bit easier to wash off.

Receiver v1

In the summer of 2011 I decided I had played around sufficiently with the radio modules and finally decided to continue with my project to build the RC radios. I ordered a kit of Graupner's Elektro Rookie as my plane. It came with a brushless motor and a speed controller (at first I had plans of building my own speed controllers as well, but realizing how cheap they were, I didn't bother).

I had no idea how big the space for the receiver was inside the Elektro Rookie, so I had to wait for the plane to arrive. I was surprised of the amount of space there was, but of course it was still quite small.
Horrible picture to illustrate the space for the receiver

Hardware

I was planning on making the boards at home using the standard photoresist method, and at that time I didn't really have the possibility of making two sided boards, so I decided on having multiple one-sided boards with interconnects. This way I could make each individual board have quite a small outline, but still fit all the stuff I needed. I ended up splitting the functionality to three boards as follows:
  1. External connectors & voltage regulation (schematics, layout)
  2. Processor (schematics, layout)
  3. Radio (schematics, layout)
One idea behind doing the split this way was that I could upgrade just the radio board later to get antenna diversity. I'm still not sure how useful antenna diversity really is, but the commercial radios have it implemented, and as I can add it with almost no extra cost then why not.

The connector board ended up having 9 servo channels, a UART channel, two analog channels and an external power connector which would be used to power FPV gear. I decided on 9 channels because that was the largest amount I could still easily fit on the board.

Most of the components I used were already in my parts bin. It was mostly stuff that I had bought before at a bargain price or stuff I had salvaged out of old electronics. Very few parts I actually had to buy specifically for this project.

An almost completed assembly of the connector board and the processor board.
BTW, it's annoyingly difficult to solder pin headers on the wrong side of the board...
Fully assembled receiver under the first tests.
The receiver in its designed place. It's a snug fit.

Since the radio module I was using (RFM22B @ 868MHz) is a transceiver, I could trivially have telemetry. I wanted to have at least a GPS receiver on board as well as voltage and current monitoring.

I had bought a bunch of GPS receivers some time before at a local electronics store for 10€ a piece (still the best price for GPS receivers I've seen!), which communicate at logic-level RS232. This was very nice, because the microcontroller I chose to use (Atmel's ATmega168) has a hardware UART, which directly supports the output of the GPS. Also, if I wanted, I could use the UART for adding additional sensors.

The voltage measurement was done just by resistively dividing the battery voltage to the range 0-1.1V and then measured against the microcontrollers internal 1.1V reference. The resistor divider was not included on the board to give a bit more flexibility.

Current measurements were initially done using a 1:5000 Hall-effect current transducer, which was salvaged from an old HVAC variable-frequency drive. The output was converted as a voltage in the range 0-1.1V and fed to the ADC of the microcontroller. The current transducer was quite heavy, so I just quit measuring current later on.

If you look closely at the connectors in the connector board, you'll soon find the first problem I encountered. I didn't double check my reference for the pinout of the servo connectors, so I was bound to get it wrong. Turns out I ended up using an Airtronics pinout rather than the more common Futaba/Hitec/JR. I think part of the mix-up was that I thought it to be obvious that the ground pin should stay the same even if the connector was reversed. Oh well, I just had to rearrange the pins on my servos (the connector pins are easily removed by using tweezers to release a locking tab and pulling on the wire).

The second problem was that even though I had two separate power connectors for powering the processor and radio (and four separate ground connections), I still had a brownout during almost every flight. These brownouts were so short that you could not really notice them in flight (at least I couldn't, but then again, I had never flown before). I only discovered them after reviewing telemetry from the plane. I changed the GPS VCC to the 5V rail instead of the regulated 3.3V to reduce the loading on the regulator. I also added 100 uF of additional capacitance to each of the three boards and the problem went away.

The third problem was that I needed to take off the radio board every time I wanted to program the processor board. This was originally by design, because the radio uses the same SPI-bus through which the programming of the processor is done. Anyhow, over time this constant disassembling and re-assembling of the board stack became extremely frustrating. I solved this by soldering an additional connector for debugwire (Atmel's in-circuit programming and debugging protocol that goes through the reset pin of the microcontroller) onto the processor board. This was quite of a kludgy solution, but it did fix the problem and didn't cause new ones.

The heavy three board design turned out to be quite susceptible to damage in crashes. The solder joints easily cracked, which wouldn't be too easily noticed. This became obvious during a couple of weeks, when a bad connection together with an incorrectly handled error condition in software caused a complete control lock-out and quite bad crashes. It wasn't until one particular crash, after which the electronics still remained powered on, that I could finally debug what was happening and understand the issue.

Software

The software part of the receiver was quite simple. Unfortunately I have overwritten the original source, so I'm not able to publish it here.

At the heart was the servo update routines, where the servo pulses were driven every 20 milliseconds using two timer ISRs. The first one was used to set the signal lines high. The second one was called one millisecond later and it busylooped for 1 millisecond to set the signals low at the correct time. As servo pulses are always at least 1 millisecond long, this way of implementation allowed me to save that millisecond for other use.

About 19 milliseconds out of 20 the code was running a busyloop, which checked a couple of different things
  • Do we have received data over the radio?
  • Do we have received data from the GPS?
  • Has an analog-to-digital conversion completed?
  • Is it time to put the channels to fail safe?
If a packet was received from the radio, the software would decode its contents, i.e. store the received new servo pulse widths in memory and check if servo fail safe values should be set. It would also read the RSSI (receive signal strength indicator) of that packet. After that it would build a reply packet that consisted of the most recent GPS data, the most recent measured voltage and current, the RSSI value of the received packet, a count of servo updates, a count of received packets and a couple of flags that indicated different error conditions. A reply packet was sent for each received packet.

If a complete NMEA sentence was received from the GPS, it would be decoded and its contents stored in memory. The NMEA sentences were received a byte at a time from the GPS, with a FIFO buffer of 2 bytes, which corresponds approximately to 2 milliseconds at 9600bps. This is the most important reason in saving the additional millisecond by using two ISR routines.

If an analog-to-digital conversion was completed, the value would be stored to the appropriate memory location (either voltage or current), the ADC channel was changed and a new measurement was started.

If it had been 500 milliseconds since the last packet from the transmitter, the servo pulse widths were loaded with fail safe values. The fail safe values were at first hard-coded but later set as the first received values after each power-on.

Friday, September 28, 2012

Some background info

After visiting ModelExpo exhibition (their website is in Finnish) here in Helsinki in around the mid 2000s, I really got interested in RC airplanes. I mean, I had visited ModelExpo already in the early 90s as a kid, and for a while I tried to get my parents to buy me an RC plane, but they didn't and the whole idea soon died out. This time, however, after exploring the different possibilities I came to the conclusion that as a student I just couldn't afford a hobby so expensive. One of the main problems was the radio system. I didn't want to just buy a 4 channel "cheap" transmitter, because I knew I'd definitely want to update it sometime in the future. Still, even the 4 channel ones were quite expensive as the 2.4GHz revolution hadn't yet really taken off.

I'd already been a keen electronics hobbyist for quite many years back then, so I decided that building my own RC radios was no big deal. For a couple of years, I would play around every now and then with home made 27 MHz transmitters and super-regenerative receivers, but I never really got them to work reliably. While practically given up on my goal, I happened to stumble across a company called HopeRF, and especially their radio module RFM12, which was used in quite many small wireless microcontroller projects. After looking for a supplier for these modules, I found Octamex in Germany, which also sold the modules RFM22B and RFM23B at only a marginally more expensive price. After reading the datasheets, it was a no-brainer: the RFM22/23 were much more advanced, so I ordered a couple for operation at 868MHz without ever looking back at RFM12. (For reference: this was around Christmas time in 2010).

It turned out the modules were based on Silicon Labs Si4430 family of EZRadioPRO UHF ICs. The datasheets provided by HopeRF ended up being just badly copied versions of the original Silabs documents. Within just a week or two, I got the modules talking to microcontrollers (which I think were Atmel ATtiny2313) and talking to each other over the air. As I usually can't really keep focused on just one thing for a long time, I decided to first pursue other projects I could use the radio modules with, including home automation and remote sensing. I continued with the RC stuff the following summer.