Artificial Light Texture (ALT) for Non‑Contact Vital‑Sign Monitoring (Heartbeats, Respiration)

“A motion‑capture suit made out of light.”

Quick links

1. A Python script that can be used to capture and save infrared frames from the D400‑series Intel RealSense cameras is available here. Additionally, a companion script that analyzes the metadata file generated by the frame capture script can be found here. The metadata file contains information about each captured frame, including its frame number and timestamp. The companion script identifies any gaps in the sequence of frame numbers (indicating dropped frames), calculates the effective frame rate (FPS), and provides a detailed report on frame continuity and performance. Instructions for these scripts are available here and here, respectively.

2. [GitHub] ALT technology based pulse and respiration monitoring using Intel RealSense cameras.

3. [LinkedIn] ALT technology based non‑contact vital‑signs monitoring on mobile devices: an iPad + Structure Sensor example.

4. [Medium] Articles on Medium related to contactless sleep monitoring using ALT tech.

   A video (YouTube; Tip: set playback to 1440p60 for better quality) from one of the Medium articles showcasing the use of ALT for monitoring respiration and heartbeats during sleep.
   A screenshot from the video:
   

---

What is ALT?

The Artificial Light Texture (ALT) technology uses a video camera and a special light source to obtain heart‑rate and respiration‑rate information for a person without contact and in real time.

The person’s skin does not need to be exposed for ALT to work [1–4]. ALT works for any pose and even when the person is covered by a very thick blanket (or two) or wears loose‑fitting clothes. ALT does not use depth data to obtain vital‑sign information [1–4].

---

How does ALT work?

ALT can use inexpensive computing and image‑capture devices such as a Raspberry Pi single‑board computer [5] and Pi NoIR camera [6] and is compatible with light‑emitting elements of various consumer devices such as the projectors inside Microsoft Kinect ([7]), Intel RealSense cameras [8–10], and the Occipital Structure Sensor ([11]).

Three key hardware components of an ALT system are:

A computer, a video camera, and a light source that generates an artificial light texture.

The light source illuminates parts of a person’s body, adding a patterned “artificial light texture” (ALT) to what the camera sees. We call the distinct illuminated areas of the added texture its elements.

You can imagine ALT as many light spots projected onto the person’s body (as shown above).

Applying ALT increases illumination contrast in the camera’s view. Small body movements related to respiration and heartbeats cause time‑varying changes in the elements’ intensity, shape, size, position, or count. A computer processes the captured video frames to convert those changes into numeric estimates of heart rate and respiration rate.

You can think of ALT as putting a person in a motion‑capture suit made out of light.

The added artificial light texture acts like an amplification medium for small body movements. Applying ALT can increase the signal components related to heart activity and respiration by orders of magnitude compared to the case with no artificial light texture (e.g., when the ALT emitter is off) under otherwise identical capture/processing.

ALT can also cover objects in contact with the body (e.g., chair, blanket, bed, floor). Motion transferred to such objects can be picked up in the ALT data too. This is why ALT can detect heartbeats and respiration even when a person is completely hidden under a thick blanket [2].

---

An Example ALT system: Kinect + Raspberry Pi + Pi NoIR camera

In one implementation, the light source is the infrared projector of a Microsoft Kinect for Xbox 360, the computing element is a Raspberry Pi single‑board computer, and the video camera is a Pi NoIR camera. Both Kinect and the Pi NoIR camera are connected to and controlled by the Raspberry Pi.

ALT data that captures heartbeats and respiration can be obtained as follows:

1. The Kinect infrared projector illuminates the scene observed by the Pi NoIR camera.
2. Video is encoded to H.264 [12] using the Raspberry Pi and the Picamera library [13].
3. For (ideally) each encoded frame, compute sum of absolute differences (SAD) values [14].
4. Compute the per‑frame sSAD value (sum of the SAD set).
5. Analyze the sSAD time series to estimate respiration and heart rates (e.g., via Fourier analysis/STFT [15]).

Python code for capture and processing on a Raspberry Pi with a Pi NoIR camera is available here. The Microsoft Kinect can be connected and its emitter controlled by the same Raspberry Pi.

The sSAD values contain information about respiration, heartbeats, and other movements (e.g., arms, legs) over the recorded period. Numeric values representative of respiration and heart rates can be obtained, for example, by performing Fourier analysis [15] of the sSAD stream.

Note: the sSAD baseline can be on the order of hundreds of thousands, while heartbeat/respiration signals may be only a few percent of that baseline—even when ALT is applied.

Practical setup. As a starting point, place the Kinect roughly 5 ft from the person with the Pi NoIR camera nearby. Distance affects how pronounced the heartbeat signal is during breathing; generally, closer Kinect/camera placement reduces the heartbeat component during inhale/exhale. An example of a similar distance effect for Intel RealSense cameras can be found here. At a large enough distance there can be little or no discernible pulse or respiration signal. Adjust positions while observing data visualizations.

Example ALT data (see figure below [1]): real‑time collection at ~49 samples/s with simultaneous HD (720p) video recording; subject at 1.5 m (5 ft); camera viewing ~2/3 of the body.

• Respiration rate: 0.24 Hz (≈14 breaths/min)
• Heart rate: 1.12 Hz (≈67 bpm)

Lighting tip. To avoid flicker under mains‑powered lighting, select an fps that divides the flicker frequency (e.g., in 60 Hz regions: 60, 30, 20, 15, 12, 10, 6 fps; in 50 Hz regions: 50 or 25 fps).

Implementations other than the Kinect+Pi described above have been demonstrated. For example, ALT systems can use Intel RealSense cameras that generate both static (R200 [9], D415 [16], D435 [16]) and dynamic (F200 [10]) light patterns [4]. ALT can use emitters at visible or infrared wavelengths, depending on application needs.

Questions? Comments? info(at)lvetechnologies.com · https://lvetechnologies.com/

Tip jar: bc1qg7j26tlw60mtt9vsczu6rzxlvuyql78a0xf3pr (BTC)

PS: Making heartbeats audible using ALT tech (YouTube video)

---

References (some links may require login and some vendor pages may change over time)

1. “Non‑contact real‑time monitoring of heart and respiration rates using Artificial Light Texture.” https://www.linkedin.com/pulse/use-artificial-light-texture-non-contact-real-time-heart-misharin
2. “What has happened while we slept?” https://www.linkedin.com/pulse/what-has-happened-while-we-slept-alexander-misharin
3. “When your heart beats.” https://www.linkedin.com/pulse/when-your-heart-beats-alexander-misharin
4. “ALT pulse and respiration monitoring using Intel RealSense cameras.” https://www.linkedin.com/pulse/alt-pulse-respiration-monitoring-using-intel-cameras-misharin
5. https://www.raspberrypi.org/
6. https://www.raspberrypi.com/products/pi-noir-camera-v2/
7. “Kinect.” https://en.wikipedia.org/wiki/Kinect
8. https://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html
9. “Intel® RealSense™ Camera R200 — Product Datasheet (PDF).” https://www.intel.com/content/dam/develop/external/us/en/documents/realsense-camera-r200-product-datasheet.pdf
10. “Support for Intel® RealSense™ Camera F200 (official hub).” https://www.intel.com/content/www/us/en/support/products/92255/emerging-technologies/intel-realsense-technology/intel-realsense-cameras/intel-realsense-camera-f200.html
    Note: At the time of writing, a publicly accessible Intel‑authored F200 product datasheet could not be located. For a formal Intel datasheet covering the F200’s successor in the same coded‑light family, see “Intel® RealSense™ Camera SR300 — Product Datasheet (PDF).” https://www.intel.com/content/dam/develop/external/us/en/documents/realsense-sr300-product-datasheet-rev-1-0.pdf
    For operating specs of the F200 developer‑kit unit, see Creative “Intel RealSense 3D Camera (VF0800) — Technical Specifications.” https://support.creative.com/kb/ShowArticle.aspx?sid=124661
11. “3D scanning, augmented reality, and more for mobile devices.” https://structure.io
12. “H.264/MPEG‑4 AVC.” https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC
13. https://picamera.readthedocs.io
14. “Sum of absolute differences.” https://en.wikipedia.org/wiki/Sum_of_absolute_differences
15. “Short‑time Fourier transform.” https://en.wikipedia.org/wiki/Short-time_Fourier_transform
16. “Intel® RealSense™ D400 Series Product Family Datasheet.” https://www.intel.com/content/www/us/en/content-details/841984/intel-realsense-d400-series-product-family-datasheet.html

ALT by Alexander Misharin, LVE Technologies LLC is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The full text of the CC BY‑NC‑SA 4.0 license can be found at https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode. Contact LVE Technologies LLC if you would like to obtain a commercial license: info(at)lvetechnologies.com