108 – Human-Computer and Brain-Computer Interfaces
Mansi Bhatnagar; Vivek Dwivedi; Ivan Minárik; Radoslav Vargic.
Annotation

This course provides a comprehensive overview of human-computer and brain-computer interfaces, focusing on both the theoretical and technological foundations of modern interactive systems. Students will explore traditional and emerging user interface modalities, including command line, graphical, speech-based, and sensory-driven approaches. Special emphasis is placed on technologies such as audio and visual sensors, microphones, cameras, displays, haptic devices, olfactory and gustatory interfaces, and biosensors used for brain-computer communication. The course covers signal acquisition, processing, hardware and software integration, and multimodal data fusion, alongside the design and evaluation of interactive systems. Practical aspects include the development of user interfaces, application of MEMS and digital signal processing, and the integration of artificial intelligence in XR environments. The curriculum highlights accessibility, usability, and ethical considerations in the design of next-generation interfaces.

Objectives

Upon successful completion of this course, students will understand the key concepts and mechanisms behind human-computer and brain-computer interfaces. They will be able to describe the physiological and technical foundations of sensory interfaces for vision, hearing, touch, smell, and taste. Students will gain competence in the analysis, design, and evaluation of user interfaces, including practical skills in working with audio and visual signals, image and speech processing, and XR systems. Additionally, they will develop critical thinking regarding the integration of multimodal inputs, accessibility, usability, and the ethical and societal impacts of modern interface technologies.

Keywords
Human-computer interaction; Brain-computer interface; User interface design; Sensory interfaces; Audio signal processing; Visual perception; Haptic technology; Extended reality (XR); Accessibility; Usability; Multimodal interaction; Ethics in technology
Date of Creation
21.09.2024.
Duration
4 hours
Language
English
License
Creative Commons BY-SA 4.0
ISBN
Literature
  1. M. Mikuláštík, Komunikační dovednosti v praxi, 2. dopl. a přeprac. vyd. Praha: GRADA Publishing, a.s., 2010.
  1. J. A. Hall and M. L. Knapp, Nonverbal Communication. in Handbooks of Communication Science. De Gruyter, 2013.
  1. R. Williams, The Non-Designer’s Design Book, 4th Edition. Peachpit Press, 2014.
  1. https://www.slideshare.net/slideshow/sound-acoustics-psychoacoustics/32315432
  1. https://www.slideshare.net/slideshow/acoustics-131749521/131749521
  1. https://www.slideshare.net/slideshow/speech-signalspptx/254112278
  1. https://en.wikipedia.org/wiki/Audio_signal#:~:text=Audio%20signals%20may%20be%20characterized,be%20single%2Dended%20or%20balanced.
  1. Sylvain Marchand, RESPECT: A FREE SOFTWARE LIBRARY FOR SPECTRAL SOUND SYNTHESIS, SCRIME / LaBRI – CNRS, Université Bordeaux 1 351 cours de la Libération, 33405 Talence cedex, France.
  1. https://www.raypcb.com/audio sensors/#:~:text=An%20audio%20sensor%20can%20be,processing%20circuitry%20and%20a%20microphone.
  1. https://mynewmicrophone.com/how-to-plug-a-microphone-into-a-speaker/
  1. https://www.britannica.com/technology/condenser-microphone
  1. https://www.thomannmusic.com/onlineexpert_page_dynamic_microphones_what_is_a_dynamic_microphone.html
  1. https://robu.in/sound-sensor-basics-pin-configuration-working-applications-and-interfacing/
  1. https://www.britannica.com/technology/crystal-microphone
  1. Horowitz S, Nishida T, Cattafesta L, Sheplak M. Development of a micromachined piezoelectric microphone for aeroacoustics applications. J Acoust Soc Am. 2007 Dec;122(6):3428-36. doi: 10.1121/1.2785040. PMID: 18247752.
  1. https://en.wikipedia.org/wiki/Electrostatic_loudspeaker
  1. https://www.researchgate.net/publication/37413321_Loudspeaker_behaviour_under_incident_sound_fields/citations
  1. https://en.wikipedia.org/wiki/Audio_interface
  1. https://www.techtarget.com/searchcustomerexperience/definition/speechrecognition#:~:text=Speech%20recognition%2C%20or%20speeh%2Dto,convert%20them%20into%20readable%20text.
  1. https://en.wikipedia.org/wiki/Speaker_recognition
  1. Nilu Singh, R.A. Khan, Raj Shree, Applications of Speaker Recognition, Procedia Engineering,Volume38,2012,Pages3122-3126,ISSN1877-7058,https://doi.org/10.1016/j.proeng.2012.06.363.
  1. https://research.aimultiple.com/speech-recognition/
  1. https://www.shaip.com/blog/voice-recognition-overview-and-applications/
  1. https://www.slideshare.net/niranjankumar47/speaker-recognition-61526019
  1. https://www.uaudio.com/blog/audio-compression-basics/#:~:text=Compressors%20are%20one%20of%20the,also%20injecting%20coloration%20and%20tone.
  1. Sánchez López de Nava A, Somani AN, Salini B. Physiology, Vision. [Updated 2023 May 1]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2024 Jan-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK538493/
  1. https://www.wikilectures.eu/w/LIGHT,_EYE_AND_VISION
  1. https://reflective.info/reflective-tape-technical-articles/the-difference-between-reflectivity-and-refraction-of-light/
  1. https://my.clevelandclinic.org/health/articles/21204-vision
  1. https://chromatone.center/theory/color/perception/
  1. https://ntemid.com/binocular-vision-how-do-we-see-the-things-we-see/
  1. Polatoğlu, A., & Özkesen, İ. C. (2022). Working principles of ccd and cmos sensors and their place in astronomy. Journal of Anatolian Physics and Astronomy, 2(1), 51-59.
  1. https://www.electroniclinic.com/display-panels-lcd-led-oled-amoled-and-s-amoled/
  1. https://www.dynamsoft.com/blog/insights/image-processing/image-processing-101-image-enhancement/
  1. https://www.cloudflare.com/en-gb/learning/performance/glossary/what-is-image-compression/
  1. https://www.kaspersky.com/resource-center/definitions/what-is-facial-recognition
  1. https://blog.roboflow.com/object-detection/
  1. https://en.wikipedia.org/wiki/Haptic_perception
  1. https://www.researchgate.net/profile/AlbertoBoem/publication/326390381/figure/fig2/AS:648123797999618@1531536163436/Two-VRMIs-represented-through-the-Volflex-interface.png
  1. Adrian David Cheok, Kasun Karunanayaka, Digital Smell Interface, Virtual Taste and Smell Technologies for Multisensory Internet and Virtual Reality, 2018, ISBN : 978-3-319-73863-5.
  1. https://newatlas.com/computer-rendering-taste-experience-mixed-reality-lab/29948/
  1. Abiri R., Borhani S., Sellers E. W., Jiang Y., Zhao X. (2019). A comprehensive review of EEG-based brain–computer interface paradigms. J. Neural Eng. 16:011001.
  1. Hohaia, W., Saurels, B.W., Johnston, A. et al. Occipital alpha-band brain waves when the eyes are closed are shaped by ongoing visual processes. Sci Rep 12, 1194 (2022). https://doi.org/10.1038/s41598-022-05289-6
  1. Simard, F., BCI Paradigms Cheat sheet, https://medium.com/@re-ak/bci-paradigms-layer-cheat-sheet-ccc60a03d302, online
  1. FaceFusion, https://github.com/facefusion/facefusion, online