Skip to main page content
U.S. flag
See More

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Mar 5:8:181.
doi: 10.3389/fbioe.2020.00181. eCollection 2020.

3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision

Matteo Zago  1 , Matteo Luzzago  2 , Tommaso Marangoni  2 , Mariolino De Cecco  3 , Marco Tarabini  2 , Manuela Galli  1
Affiliations

3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision

Matteo Zago et al. Front Bioeng Biotechnol. .

Abstract

The design of markerless systems to reconstruct human motion in a timely, unobtrusive and externally valid manner is still an open challenge. Artificial intelligence algorithms based on automatic landmarks identification on video images opened to a new approach, potentially e-viable with low-cost hardware. OpenPose is a library that t using a two-branch convolutional neural network allows for the recognition of skeletons in the scene. Although OpenPose-based solutions are spreading, their metrological performances relative to video setup are still largely unexplored. This paper aimed at validating a two-cameras OpenPose-based markerless system for gait analysis, considering its accuracy relative to three factors: cameras' relative distance, gait direction and video resolution. Two volunteers performed a walking test within a gait analysis laboratory. A marker-based optical motion capture system was taken as a reference. Procedures involved: calibration of the stereoscopic system; acquisition of video recordings, simultaneously with the reference marker-based system; video processing within OpenPose to extract the subject's skeleton; videos synchronization; triangulation of the skeletons in the two videos to obtain the 3D coordinates of the joints. Two set of parameters were considered for the accuracy assessment: errors in trajectory reconstruction and error in selected gait space-temporal parameters (step length, swing and stance time). The lowest error in trajectories (~20 mm) was obtained with cameras 1.8 m apart, highest resolution and straight gait, and the highest (~60 mm) with the 1.0 m, low resolution and diagonal gait configuration. The OpenPose-based system tended to underestimate step length of about 1.5 cm, while no systematic biases were found for swing/stance time. Step length significantly changed according to gait direction (p = 0.008), camera distance (p = 0.020), and resolution (p < 0.001). Among stance and swing times, the lowest errors (0.02 and 0.05 s for stance and swing, respectively) were obtained with the 1 m, highest resolution and straight gait configuration. These findings confirm the feasibility of tracking kinematics and gait parameters of a single subject in a 3D space using two low-cost webcams and the OpenPose engine. In particular, the maximization of cameras distance and video resolution enabled to achieve the highest metrological performances.

Keywords: artificial intelligence; computer vision; gait analysis; markerless motion capture; movement measurement.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Laboratory setup, schematic (left) and pictorial (right) view.
Figure 2
Figure 2
Stick diagrams as returned by the marker-based optical system (top, left) and OpenPose model (top, right); corresponding 3D reconstruction of the skeletal structures during walking (bottom).
Figure 3
Figure 3
Extraction of gait phases from the trajectories of ankle nodes' velocity, explanatory example taken from a straight gait test. OP, OpenPose-based system; MB, marker-based optical system.
Figure 4
Figure 4
Sample trajectories of a landmark (position of the right ankle) obtained from the reference marker-based (black) and markerless, OpenPose-based (blue and red) systems (top); corresponding RMS distance (bottom).
Figure 5
Figure 5
Boxplots of the RMS distance (measurement error) for the straight (left) and diagonal (right) walking tests; cam: cameras, res: resolution; OP, OpenPose.
Figure 6
Figure 6
Effects plot for the RMS distance, according to the gait type (walking direction), cameras distance and resolution (left). Gait type×distance interaction plot (right). Cam: cameras, res: resolution; OP, OpenPose.
Figure 7
Figure 7
Bland-Altman plot of the pooled (selected) gait parameters, comparison between the OpenPose- and the marker-based systems.
See this image and copyright information in PMC

References

    1. Abbondanza P., Giancola S., Sala R., Tarabini M. (2016). Accuracy of the microsoft kinect system in the identification of the body posture, in International Conference on Wireless Mobile Communication and Healthcare (Milan: ), 289–296. 10.1007/978-3-319-58877-3_37 - DOI
    1. Barone V., Verdini F., Burattini L., Di Nardo F., Fioretti S. (2016). A markerless system based on smartphones and webcam for the measure of step length, width and duration on treadmill. Comput. Methods Programs Biomed. 125, 37–45. 10.1016/j.cmpb.2015.12.003 - DOI - PubMed
    1. Cao Z., Simon T., Wei S.-E., Sheikh Y. (2016). Realtime multi-person 2d pose estimation using part affinity fields. arXiv Prepr. 1:arXiv1611.08050. 10.1109/CVPR.2017.143 - DOI - PubMed
    1. Cappozzo A., Della Croce U., Leardini A., Chiari L. (2005). Human movement analysis using stereophotogrammetry. Part 1: theoretical background. Gait Posture 21, 186–196. 10.1016/S0966-6362(04)00025-6 - DOI - PubMed
    1. Ciprandi D., Bertozzi F., Zago M., Ferreira C. L. P., Boari G., Sforza C., et al. . (2017). Study of the association between gait variability and physical activity. Eur. Rev. Aging Phys. Act. 14, 1.4: 19. 10.1186/s11556-017-0188-0 - DOI - PMC - PubMed