Marco Altini
  • Home
  • Research & Publications
  • Apps & Projects
  • Blog

Heart Rate Variability using the phone's camera

2/1/2014

22 Comments

 
No doubt the main inconvenience of HRV-based applications is the need for a heart rate monitor. Even when you have one, performing your daily measurement can be a burden (sensor needs to be wet and comfort is clearly an issue). 

Can smartphones come to the rescue? Current generation phones include both a camera and a light emitting diode, which can be used for reflection based bio-optical imaging. 
The technique is called photoplethysmography (PPG for short) and consists in detecting changes in blood volume during a cardiac cycle, by illuminating the skin and measuring changes in light absorption. PPG has become quite a popular non-invasive method for extracting physiological measurements such as heart rate and oxygen saturation. However, most applications today focus simply on heart rate, and it is not clear from literature if HRV features can also be reliably extracted using a phone's camera [1]. 
Picture
Here is the good news: it is indeed possible to achieve good accuracy in HRV measurements using this technique, but the methods needed are slightly more complicated than acquiring a video and computing peak detection on the PPG signal (which is sufficient for heart rate measurement). This post covers the steps involved in the implementation of Camera HRV, the iPhone app I developed to measure HRV using the phone's camera. The algorithms are part of HRV4Training since version 3.2.

Later in 2017, we have also validated these methods and published the results with respect to ECG and Polar straps, showing equivalency. You can find the full paper here.

Overview

1 - Data acquisition from the phone's camera
2 - Filtering & smoothing
3 - Resampling with cubic spline interpolation
4 - Peak detection
5 - Artifact removal and features extraction
6 - Comparison with heart rate monitors (Polar H7)
7 - Tips
Signal processing pipeline:
Picture
All algorithms were first tested in MATLAB and then implemented in iOS.

1 - Data acquisition from the phone's camera

The iPhone's camera can be configured to acquire videos at 30 frames per second, which means a 30 Hz signal. This is the first issue to face, since accurate HRV measurements require sampling frequencies around 200 Hz [2]. Camera HRV gets the RGB values  from the camera frame, averages them over the whole frame, and finally converts RGB to HSV values. HSV values are often used in image processing since they better separate the intensity from the color information. In our case, we are really interested only in intensity, so we will use the hue value of the HSV color space:
Picture

2 - Filtering & smoothing

The acquired PPG signal needs some further processing. More specifically, I applied a fourth order Butterworth band pass filter. The band pass filter helps in both removing the DC component, due to finger movement or changes in venous pressure, and also high frequency noise. The result of the filtering is shown in the next plot: 
Picture
The final plot includes a five samples moving average to further smooth the signal and ease peak detection.
Picture

3 - Resampling with cubic spline interpolation

As mentioned before, one of the biggest issues for HRV analysis is the low resolution of the signal, due to the limited frame rate we can obtain from the phone's camera (i.e. 30 Hz). For Camera HRV I implemented cubic spline interpolation at 180 Hz, which is fast enough for the iPhone to compute, and significantly improves the resolution of the signal. The following plot shows the resolution issue and the improved signal after cubic spline interpolation:

4 - Peak detection

Peak detection is performed using a simple slope inversion algorithm, plus a local peak search to avoid detecting double peaks due to the nature of the PPG signal:
Picture

5 - Artifact removal and features extraction

Peak to peak intervals are corrected for artifacts, before features extraction. Intervals correction prevents artifacts due to ectopic beats or motion from affecting features computation, as often reported in literature for HRV analysis. Especially for the PPG signal, intervals correction is very important since the signal is very sensitive to motion artifacts, and HRV features risk to get completely unreliable even when heart rate is quite stable.

Interval correction is performed by analyzing intervals in windows of 10 seconds, and discarding 1) all intervals differing more than 20% from the median of the intervals over the current window, 2) all intervals belonging to a window where we have less than 4 intervals detected or more than 50% intervals removed (i.e. too much noise for a reliable signal). 

After interval correction, features extraction is performed in standard ways, as described in a previous post.

6 - Comparisons with heart rate monitors

Comparisons are against a commercial heart rate monitor, Polar's H7. I used polar's H7 because I already validated it's accuracy with respect to a device able to capture the full ECG (see this post, hardware section), and it also allowed me to directly compare the output with the one of Camera HRV, by developing an application able to record both at the same time.

Heart rate during rest and paced breathing
The following plot shows heart rate recorded by the H7 and Camera HRV. This is the instantaneous HR received every second by BLE HR monitors, however it is not clear if the value is derived only from the last RR-interval or if it is an average of the last few seconds. In Camera HRV I average the last three seconds of data and update the information every second. This might explain small differences between the two. However, both during rest and paced breathing (second part of the plot, showing much greater variations in HR), the correlation between the two signals is high:
Picture
Correlation coefficient: 0.97

The following video shows both the PPG signal and the instantaneous HR as received from the Polar H7 (left) and computed by Camera HRV (right). The recording is not the same shown above in the heart rate plot.
Heart rate post exercise
The following plot shows 2 minutes and a half of heart rate post exercise (25 minutes of biking (average heart rate ~130 bpm)). As expected, heart rate can be acquired with high accuracy, even at higher intensities:
Picture
Correlation coefficient: 0.92

RR-intervals

In the next plot we can see RR-intervals from the rest, paced breathing and recovery sequences (they did not happen in this order, I just put together all the data in a single time series). Peak to peak intervals are plotted without and with artifact correction. The third plot highlights some successful corrections (interval correction is always enabled in the commercial version of Camera HRV):
Picture
Features comparison
Features extraction is the last step to confirm the validity of Camera HRV as a tool able to assess not only heart rate but also heart rate variability. Features were extracted over 30 seconds windows, using the rest, paced breathing and recovery data. The following plots show the relation between HRV features (AVNN, SDNN, rMSSD and pNN50) computed using Polar's H7 and Camera HRV. Features computed using Camera HRV include RR-interval correction. All plots include a time series of the features and a comparison of the values under different conditions.
Picture
Picture
Picture
Picture
These plots show that Camera HRV is able to measure heart rate variability under different conditions, capturing small changes in HRV and dealing well with artifacts.

Correlation between Polar features (already validated in a previous experiment) and Camera HRV features is 0.99 for AVNN, 0.98 for SDNN, 0.78 for rMSSD and 0.87 for pNN50 (outperforming what I could obtain with other optical measures, for example the mio alpha - correlation for rMSSD was only 0.23).

7 - Tips

  • use your fingertip
  • place your fingertip exactly on the camera, you need to cover it all
  • do touch the camera, but limit pressure. It's important to touch it gently, otherwise too much pressure will impede blood flow (which is what needs to be measured)
  • try to record using 60 seconds windows. Research on ultra-short term HRV showed that all features can be reliably measured in 50 seconds [3]
Follow @marco_alt

References

[1] Jonathan, E., and Martin Leahy. "Investigating a smartphone imaging unit for photoplethysmography." Physiological Measurement 31.11 (2010): N79.
[2] Voss, A., et al. "Influence of low sampling rate on heart rate variability analysis based on non-linear dynamics." Computers in Cardiology 1995. IEEE, 1995.
[3] Salahuddin, Lizawati, et al. "Ultra short term analysis of heart rate variability for monitoring mental stress in mobile settings." Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th Annual International Conference of the IEEE. IEEE, 2007.
22 Comments
Paul Lee MD
12/1/2014 03:42:05 am

I am a cardiologist exploring new ways to use smartphone as a diagnostic tool. This app is one of the best example of “iPhone as diagnostic tool” I have found. The design of this software is more sophisticated than other heart rate devices in the app store. The smoothing algorithm with cubic spline method is ingenious. The programmer validated his HRV measurement with validated clinical instrument and this enhanced the trustworthiness of his method. One small piece of advice: Bland-Altman plot and paired t-test should be used to compare the camera HRV with the clinical gold standard measurement. Overall, I am really impressed by its sophistication. I wish you good luck in your future project.

Reply
Marco Altini
12/1/2014 04:00:45 am

Thanks a lot Paul, I really appreciate it. Hope you can find my app useful in your work.

Reply
Ralph Harvey, MD
13/4/2016 07:42:54 pm

Dear Dr. Lee- You are the first cardiologist I have heard talking about HRV. I am a family practice doc, trying to figure out how to measure and incorporate HRV evaluation, and HRV training into my clinical practice. I would appreciate an opportunity to talk with you directly.
Ralph Harvey, MD East Lansing, MI

Reply
Wendy Fite
12/3/2014 12:15:40 am

I am a 56 yr old marathoner getting ready for the 2014 Boston Marathon. Tracking HRV provides input for my daily decisions on training intensity, in my quest to train to peak but not over-train by race day. This is a great tool. Easy to use and the differences between the camera and a Bluetooth heart rate monitor for this purpose is minor. Recommend to any endurance athlete looking for data to help make decisions to avoid overtraining.

Reply
Lotus
27/3/2014 09:01:00 am

Got your app. I've had 4 massive heart attacks. Congestive heart failure. Renal fail. Internal bleeding. And soon to get a pacemaker.

Am still using your program. But how do I delete the big history I am building. ????

Reply
Marco Altini
27/3/2014 05:18:47 pm

Hi, just go to the recordings list and swipe to delete (right to left).

Reply
Eric
6/5/2014 02:12:10 am

I am looking to start measuring my HRV & have downloaded this app...it is not reading my HRV as shown in the above video, it just shows my heart rate. Is there a minimum recordings before HRV is shown?

Reply
Marco Altini
6/5/2014 02:38:46 am

HRV is computed over configurable windows of 30 seconds to 5 minutes. You will see features in the real-time window (after starting a recording), based on what you configured. Otherwise you can wait the end of the recording and look at the history.

Reply
Eric Jain link
20/1/2015 09:19:42 am

The camera in the iPhone 6 supports 240 fps; would be interesting to see how much further that improves accuracy...

Reply
Peter G
5/2/2015 09:03:06 pm

Hi Marco - I have just purchased the Camera HRV app to do some personal research into HRV. I notice that the downloaded HR and RR do not have regular time stamps - is there any way of making them regular - ie, one HR reading for each and every second of session?
Thanks - its a great app!
Peter

Reply
Marco Altini
5/2/2015 11:51:29 pm

Hi Peter,
thanks for your message. HR should be in a separate file with information reported per second, so at regular intervals. However RR intervals are stored based on when they actually happen (in case you want to compute your own features from the csv file), so the interval can't be regular. Hope this helps.

Reply
Johni
10/2/2015 01:23:51 am

I like the interface and the data collected. I'm trying to export the RR values to Kubios. Any thoughts on that? Thanks in advance, john

Reply
Marco Altini
10/2/2015 02:11:31 am

hi Johni,
you can probably rename the file as txt and use only the rr column if thats all that is needed by Kubios (ive never used that software). Since it's a common request also for the HRV Logger, I will be looking into it in the near future. I'll try to add it in the next update, but it will probably be in a couple of weeks.

Reply
John Goodfellow
10/2/2015 07:21:02 am

Marco,
Thank you for your quick reply. I was able to remove everything but the RR data and Kubios imported it fine as a .csv file. A text file would certainly work as well.
Continued success with your HRV applications. John

Jared
27/2/2015 04:53:48 am

Very nice work!
Are you developing an Android version? You have at least one sale if you do.

Reply
Freda
5/4/2015 09:03:50 am

Does this app work with an iPhone 4S??

Reply
Marco Altini
5/4/2015 09:46:56 am

yes, 4S or later

Reply
Shelby Bertsch
25/8/2015 03:15:54 am

Hi I am a psychophysiological researcher.
I am hoping to use the app in tandem with other HRV tools to test validity. I hope to add this measurement to my research as a coherence variable.
I'm impressed with the app, it is sophisticated and I can tell much thought has been put into the design.
I have a few questions though on the data export - is it possible to receive a feed of the raw data in any way?

Reply
Lisa A Pfost
12/9/2015 04:44:31 am

Brilliant....android version?

Reply
Gregory Hall link
26/2/2016 02:01:49 pm

Thanks for making an HRV app (Camera HRV) that uses the camera in the iphone! One less piece of hardware, and more convenient to use than Heartmath or Wild Divine.
One request--would it be possible to graph the HRV itself rather than the pulse/heartbeat? I have taught myself to breathe with a graph, and though I can use the heart rate numbers, I really like the graph. It shows what is happening in real-time with variability. Just a suggestion--thanks for a great app!

Reply
Sophie
13/3/2017 05:25:20 pm

Hi Marco, I love to use your camera hrv app for an experiment at a workshop for accuracy over other apps. There will be 25 people in the workshop. I would to demonstrate how HRV changes with mental and physical stress. Is this app going to be useful for that if I ask them to use it for 3 min. Is there a free version that I can get the participants to download at the workshop to use it for demonstration purpose?

Reply
Marco Altini
13/3/2017 05:42:09 pm

hi Sophie,
unfortunately no free version. Yes, you can definitely show / capture different levels of stress even with a 1-2 minutes test following acute stressors (intense activity, paced breathing, etc.). Recordings can also be compared in the app. Sometimes shorter measurements are better with this app, as movement can create problems, and 3 minutes are a bit long.

Reply

Your comment will be posted after it is approved.


Leave a Reply.

    Marco ALtini

    Founder of HRV4Training, Advisor @Oura , Guest Lecturer @VUamsterdam , Editor @ieeepervasive. PhD Data Science, 2x MSc: Sport Science, Computer Science Engineering. Runner

    Archives

    December 2022
    August 2022
    June 2022
    April 2022
    March 2022
    February 2022
    January 2022
    December 2021
    November 2021
    October 2021
    September 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    May 2020
    April 2020
    March 2020
    February 2020
    January 2020
    November 2019
    October 2019
    May 2019
    April 2019
    March 2019
    November 2018
    October 2018
    April 2018
    March 2018
    June 2017
    December 2016
    July 2016
    March 2016
    September 2015
    August 2015
    May 2015
    March 2015
    February 2015
    January 2015
    December 2014
    May 2014
    April 2014
    January 2014
    December 2013

    RSS Feed

  • Home
  • Research & Publications
  • Apps & Projects
  • Blog