Overview + Hardware & Software (+data)
A bit of background first. The cardiovascular system is mostly controlled by autonomic regulation through the activity of sympathetic and parasympathetic pathways of the autonomic nervous system. Analysis of HRV permits insight in this control mechanism , since heavy training is responsible for shifting the cardiac autonomic balance toward a predominance of the sympathetic over the parasympathetic drive.
A large body of studies demonstrated an association between endurance exercise training and HRV [2,3] (long-term changes in HRV), with more recent studies proposing HRV-based training programs, where daily HRV measurement were used to control training intensity  (short-term changes in HRV). However, a significant amount of cross-sectional studies showed no relation between HRV features and VO2max or other measures of fitness (e.g. heart rate recovery time after a maximal test) [5,6,7]. Even in longitudinal studies [3,8,9], relation between HRV and VO2max are often inconclusive. Some researcher found a significant increase in HRV features following an intervention , while most studies typically report changes in resting Heart Rate, but no changes in HRV [8,9].
On the other hand, short-term changes in HRV features, used to assess training load and recovery, seem to be a more reliable measure. Most research was able to show a significant relation between HRV features and training load/performance [10,11], with some studies showing that HRV-guided training can be used to increase VO2max faster .
In my research on energy expenditure I developed a methodology which uses heart rate distributions in different contexts to estimate a surrogate of cardiorespiratory fitness and then normalize heart rate, improving energy expenditure estimation accuracy . Then, I investigated time and frequency domain HRV features, to understand if the relation between HRV features in different contexts and fitness could also help in normalizing heart rate and improve energy expenditure estimation accuracy . However, in my cross-sectional analysis I did not find any feature able to go beyond what heart rate could do. These findings, together with most of what is reported in literature, point out that the link between HRV and fitness should be analyzed at the individual level, establishing a personal baseline from which deviations can be representative of changes in training load and fitness level.
HRV can easily be determined from ECG recordings, resulting in time series (RR-intervals) that are usually analysed in time and frequency domains. I developed two iphone apps to perform HRV analysis, and used them to collect data during trainings and morning orthostatic tests (HRV Logger & HRV4Training).
The first app is Heart Variability Logger, a general purpose app which connects to any bluetooth low energy heart rate sensor (bluetooth 4.0, also called SMART) and extracts time and frequency domain features from RR intervals (configurable time window). The app allows all data to be exported (HR, RR-intervals, features, events).
HRV Logger extracts and stores the following features:
The second app is HRV4Training, which is the app I use for short-term HRV analysis. HRV4Training measures Heart Rate and Heart Rate Variability features while guiding you through an orthostatic test (lying down and standing right after waking up), which can be used to determine your training state and prevent overtraining.
Reference device: I used imec's ECG Necklace as reference device, since it can record the full ECG and not only ECG-derived events such as RR-intervals. The ECG Necklace is a low power wireless ECG platform. The system relies on an ultra-low-power ASIC for ECG read-out and achieves up to 6 days autonomy on a 175 mAh Li-ion battery. For these tests, the ECG Necklace was congured to acquire one lead ECG data at 256 Hz. Two gel electrodes were placed on the chest, in the lead II configuration. Data were recorded on the on-board SD card to ensure integrity.
Other sensors: I used three commercially available bluetooth low energy heart rate monitors:
Before starting with short and long-term HRV analysis, I compared three different sensors in order to determine the accuracy and reliability, compared to a reference device able to collect the full ECG. This analysis was performed mainly because I found no up do date study comparing commercially available heart rate sensors to holter monitors, and I had the feeling some of these sensors were not completely reliable for heart rate variability analysis (e.g. optical, wrist based sensors).
Measurements at rest
Most of the features I will be looking at to measure training load/recovery and fitness level are measured at rest (either while lying down or during an orthostatic test). Thus, I first recorded data from the three sensors and the reference system during sedentary activity, to assess the monitor's accuracy.
The following plot shows the recorded RR-intervals. As expected, optical wrist-based monitors (mio alpha) are unable to capture RR-intervals with high accuracy. Quite a significant number of RR-intervals is lost even for a short 10 minutes recording. Another issue with the mio alpha is the smoothing effect, probably necessary to deal with motion artifacts. The recording with the Polar H7 is about 5 minutes, while the one using Under Armour's Armour39 is approx 22 mins.
Detail over 30 RR-intervals:
Correlation (full recording):
Mio alpha: 0.77
Polar H7: 0.99
The following plot shows the square root of the mean squared difference of successive RRs (rMSSD), computed over 30 seconds windows. rMSSD is one of the most discriminative features for training load/recovery. As expected, the smoothing effect on the mio alpha significantly reduces the feature's variability (lower rMSSD value, top plot). The other two monitors perform quite well compared to the reference:
EDIT: the data shown here is actually SDNN.
Mio alpha: 0.23
Polar H7: 0.97
Bottom line: a chest strap is necessary for accurate heart rate variability analysis. The mio alpha is still a good heart rate monitor, as shown from the last plot, where I used it for about 60 minutes while running (the bottom plot is about 25 minutes from a Polar H7, showing almost perfect overlap with the reference):
Mio alpha: 0.97
Polar H7: 0.99
All data used for these comparisons (reference ECG @ 256Hz and RR intervals recorded from the sensors) is available for download here.
 Aubert, André E., Bert Seps, and Frank Beckers. "Heart rate variability in athletes." Sports Medicine 33.12 (2003): 889-919.
 Tulppo, Mikko P., et al. "Effects of aerobic training on heart rate dynamics in sedentary subjects." Journal of Applied Physiology 95.1 (2003): 364-372. Al-Ani, M., et al. "Changes in RR variability before and after endurance training measured by power spectral analysis and by the effect of isometric muscle contraction." European journal of applied physiology and occupational physiology 74.5 (1996): 397-403.
 Kiviniemi, Antti M., et al. "Endurance training guided individually by daily heart rate variability measurements." European journal of applied physiology 101.6 (2007): 743-751.
 Esco, Michael R., et al. "The relationship between resting heart rate variability and heart rate recovery." Clinical Autonomic Research 20.1 (2010): 33-38.
 Lee, C. Matthew, and Albert Mendoza. "Dissociation of heart rate variability and heart rate recovery in well-trained athletes." European journal of applied physiology 112.7 (2012): 2757-2766.
 Grant, Catharina C., et al. "Relationship between exercise capacity and heart rate variability: Supine and in response to an orthostatic stressor." Autonomic Neuroscience 151.2 (2009): 186-188.
 Loimaala, Antti, et al. "Controlled 5-mo aerobic training improves heart rate but not heart rate variability or baroreflex sensitivity." Journal of Applied Physiology89.5 (2000): 1825-1829.
 Boutcher, Stephen H., and Phyllis Stein. "Association between heart rate variability and training response in sedentary middle-aged men." European journal of applied physiology and occupational physiology 70.1 (1995): 75-80.
 Garet, Martin, et al. "Individual interdependence between nocturnal ANS activity and performance in swimmers." Medicine and science in sports and exercise 36 (2004): 2112-2118.
 Pichot, Vincent, et al. "Relation between heart rate variability and training load in middle-distance runners." Medicine and science in sports and exercise 32.10 (2000): 1729-1736.
 Altini, Marco, Julien Penders, and Oliver Amft. "Personalizing Energy Expenditure Estimation Using a Cardiorespiratory Fitness Predicate." Pervasive Health 2012.
 Altini, Marco, Julien Penders, Ruud Vullers, and Oliver Amft. "Automatic Heart Rate Normalization for Accurate Energy Expenditure Estimation: An Analysis of Activities of Daily Living and Heart Rate Features." submitted to MIM.