Nature Neuroscience

Magnetic resonance-based eye tracking using deep neural networks

  • 1.

    Anderson, T. J. & MacAskill, M. R. Eye movements in patients with neurodegenerative disorders. Nat. Rev. Neurol. 9, 74–85 (2013).

    Article 

    Google Scholar
     

  • 2.

    Morrone, M. C., Ross, J. & Burr, D. Saccadic eye movements cause compression of time as well as space. Nat. Neurosci. 8, 950–954 (2005).

    CAS 
    Article 

    Google Scholar
     

  • 3.

    Berman, R. A. et al. Cortical networks subserving pursuit and saccadic eye movements in humans: an fMRI study. Hum. Brain Mapp. 8, 209–225 (1999).

  • 4.

    Petit, L. & Haxby, J. V. Functional anatomy of pursuit eye movements in humans as revealed by fMRI. J. Neurophysiol. 82, 463–471 (1999).

    CAS 
    Article 

    Google Scholar
     

  • 5.

    McNabb, C. B. et al. Inter-slice leakage and intra-slice aliasing in simultaneous multi-slice echo-planar images. Brain Struct. Funct. 225, 1153–1158 (2020).

    Article 

    Google Scholar
     

  • 6.

    Voss, J. L., Bridge, D. J., Cohen, N. J. & Walker, J. A. A closer look at the hippocampus and memory. Trends Cogn. Sci. 21, 577–588 (2017).

    Article 

    Google Scholar
     

  • 7.

    Tregellas, J. R., Tanabe, J. L., Miller, D. E. & Freedman, R. Monitoring eye movements during fMRI tasks with echo planar images. Hum. Brain Mapp. 17, 237–243 (2002).

    Article 

    Google Scholar
     

  • 8.

    Beauchamp, M. S. Detection of eye movements from fMRI data. Magn. Reson. Med. 49, 376–380 (2003).

    Article 

    Google Scholar
     

  • 9.

    Heberlein, K., Hu, X., Peltier, S. & LaConte, S. Predictive eye estimation regression (PEER) for simultaneous eye tracking and fMRI. In Proc. 14th Scientific Meeting, International Society for Magnetic Resonance in Medicine 14, 2808 (2006).


    Google Scholar
     

  • 10.

    Son, J. et al. Evaluating fMRI-based estimation of eye gaze during naturalistic viewing. Cereb. Cortex 30, 1171–1184 (2020).

    Article 

    Google Scholar
     

  • 11.

    Alexander, L. M. et al. An open resource for transdiagnostic research in pediatric mental health and learning disorders. Sci. Data 4, 170181 (2017).

  • 12.

    Nau, M., Schindler, A. & Bartels, A. Real-motion signals in human early visual cortex. Neuroimage 175, 379–387 (2018).

    Article 

    Google Scholar
     

  • 13.

    Polti, I., Nau, M., Kaplan, R., van Wassenhove, V. & Doeller, C. F. Hippocampus and striatum encode distinct task regularities that guide human timing behavior. Preprint at bioRxiv https://doi.org/10.1101/2021.08.03.454928 (2021).

  • 14.

    Nau, M., Navarro Schröder, T., Bellmund, J. L. & Doeller, C. F. Hexadirectional coding of visual space in human entorhinal cortex. Nat. Neurosci. 21, 188–190 (2018).

    CAS 
    Article 

    Google Scholar
     

  • 15.

    Julian, J. B., Keinath, A. T., Frazzetta, G. & Epstein, R. A. Human entorhinal cortex represents visual space using a boundary-anchored grid. Nat. Neurosci. 21, 191–194 (2018).

    CAS 
    Article 

    Google Scholar
     

  • 16.

    Ehinger, K. A., Hidalgo-Sotelo, B., Torralba, A. & Oliva, A. Modelling search for people in 900 scenes: a combined source model of eye guidance. Vis. Cogn. 17, 945–978 (2009).

    Article 

    Google Scholar
     

  • 17.

    Wolfe, J. M. Visual search: how do we find what we are looking for? Annu. Rev. Vis. Sci. 6, 539–562 (2020).

  • 18.

    Hebart, M. N. et al. THINGS: a database of 1,854 object concepts and more than 26,000 naturalistic object images. PLoS ONE 14, e0223792 (2019).

    CAS 
    Article 

    Google Scholar
     

  • 19.

    Duchowski, A. T Eye Tracking Methodology: Theory and Practice 3rd edn (Springer International Publishing, 2017).

    Book 

    Google Scholar
     

  • 20.

    Brodoehl, S., Witte, O. W. & Klingner, C. M. Measuring eye states in functional MRI. BMC Neurosci. 17, 48 (2016).

  • 21.

    Coiner, B. et al. Functional neuroanatomy of the human eye movement network: a review and atlas. Brain Struct. Funct. 224, 2603–2617 (2019).

    CAS 
    Article 

    Google Scholar
     

  • 22.

    Keck, I. R., Fischer, V., Puntonet, C. G. & Lang, E. W. Eye Movement Quantification in Functional MRI Data by Spatial Independent Component Analysis. In International Conference on Independent Component Analysis and Signal Separation Vol. 5441 (eds Adali, T., Jutten, C., Romano, J. M. T. & Barros, A. K.) 435-442 (Springer Berlin Heidelberg, 2009).

  • 23.

    Franceschiello, B. et al. 3-Dimensional magnetic resonance imaging of the freely moving human eye. Prog. Neurobiol. 194, 101885 (2020).

  • 24.

    LaConte, S. M. & Glielmi, C. B. Verifying visual fixation to improve fMRI with predictive eye estimation regression (PEER). In Proc. 15th Scientific Meeting, International Society for Magnetic Resonance in Medicine, Berlin 3438 (2007).

  • 25.

    Sathian, K. et al. Dual pathways for haptic and visual perception of spatial and texture information. Neuroimage 57, 462–475 (2011).

    CAS 
    Article 

    Google Scholar
     

  • 26.

    O’Connell, T. P. & Chun, M. M. Predicting eye movement patterns from fMRI responses to natural scenes. Nat. Commun. 9, 5159 (2018).

  • 27.

    Esteban, O. et al. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat. Methods 16, 111–116 (2019).

    CAS 
    Article 

    Google Scholar
     

  • 28.

    Tagliazucchi, E. & Laufs, H. Decoding wakefulness levels from typical fMRI resting-state data reveals reliable drifts between wakefulness and sleep. Neuron 82, 695–708 (2014).

    CAS 
    Article 

    Google Scholar
     

  • 29.

    Naselaris, T., Kay, K. N., Nishimoto, S. & Gallant, J. L. Encoding and decoding in fMRI. Neuroimage 56, 400–410 (2011).

    Article 

    Google Scholar
     

  • 30.

    Kriegeskorte, N. & Douglas, P. K. Interpreting encoding and decoding models. Curr.Opin. Neurobiol. 55, 167–179 (2019).

    CAS 
    Article 

    Google Scholar
     

  • 31.

    Sonkusare, S., Breakspear, M. & Guo, C. Naturalistic stimuli in neuroscience: critically acclaimed. Trends Cogn. Sci. 23, 699–714 (2019).

    Article 

    Google Scholar
     

  • 32.

    Lim, S.-L., O’Doherty, J. P. & Rangel, A. The decision value computations in the vmPFC and striatum use a relative value code that is guided by visual attention. J. Neurosci. 31, 13214–13223 (2011).

    CAS 
    Article 

    Google Scholar
     

  • 33.

    Koba, C., Notaro, G., Tamm, S., Nilsonne, G. & Hasson, U. Spontaneous eye movements during eyes-open rest reduce resting-state-network modularity by increasing visual-sensorimotor connectivity. Netw. Neurosci. 5, 451–476 (2021).

  • 34.

    Murphy, K., Birn, R. M. & Bandettini, P. A. Resting-state fMRI confounds and cleanup. Neuroimage 80, 349–359 (2013).

    Article 

    Google Scholar
     

  • 35.

    Frey, M. et al. Interpreting wide-band neural activity using convolutional neural networks. eLife 10, e66551 (2021).

  • 36.

    Shen, D., Wu, G. & Suk, H. I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017).

    CAS 
    Article 

    Google Scholar
     

  • 37.

    Misra, D. Mish: a self regularized non-monotonic neural activation function. Preprint available at https://arxiv.org/abs/1908.08681 (2019).

  • 38.

    Biewald, L. Experiment tracking with weights & biases. http://wandb.com/ (2020).

  • 39.

    Kingma, D. P. & Ba, J. L. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).

  • 40.

    Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).


    Google Scholar
     


  • Source link

    Related Articles

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Back to top button