Introduction. The special education or education with special needs (special-needs education) is the education of students that takes into account their individual limited abilities (emotional, behavioural, cognitive disorders or intellectual, hearing, visual, speech, motor disabilities). Also the special education controls special needs learning style (visual learning, verbal learning, auditory learning, and physical learning) compared to the statistical majority of students for whom regular education is designed with its usual means of learning [1, 2].
For many years, the personal computer has provided students with three main information channels in the form of a computer keyboard, a mouse-type manipulator for transmitting information to the computer, and a computer monitor for receiving information. The provision of computer mobility in the form of smartphones has added an information channel based on a touch panel with finger touches. The emergence of hardware and software available for mass use, which allows recognition of human actions (gestures, voice) and generates voice messages, has significantly increased information channels. But the maximum computerization of the educational process through a significant increase in the share of remote online classes can significantly limit the educational accessibility of students with permanent physical impairments (disabilities), for example, visual, hearing, speech and motor disabilities of the upper limbs.
Regardless of students' physical disabilities, gamification of the educational process can partially compensate for the presence of these disabilities by increasing students' motivation to overcome such obstacles. Computer gamification of the educational process allows to partially reduce the teacher's workload in a large group of students. Modern computer games take into account available hardware and software and make it possible to reduce the dependence on the constant presence of the teacher as an assistant during remote online classes. During online classes, special attention should be paid to students with the specified persistent physical impairments (disabilities). Computer games should also hide the specified features of online classes.
The purpose. The course introduces students to computer technologies of human-machine interaction for people with permanent physical impairments (vision, hearing, speech, and musculoskeletal disorders) and is meant to provide them with knowledge and skills in creating prototypes of educational computer games using human gesture recognition software libraries, human voice, and text.
Target audience of the course: 1) first of all, teachers and psychotherapists who want to implement the latest computer technologies to transfer knowledge and skills to students with special educational needs; 2) students with special educational needs
Skills and competencies. Online communication skills during the education process, creativity, physical activity and applying machine learning.
Content. Basic types of human actions/sensations and the use of human-machine interaction computer interfaces by people with persistent physical impairments, such as visual, hearing, speech and musculoskeletal impairments. Basics of computer game design: game plot, game mechanics/dynamics/aesthetics, game multimedia content (graphics, sound). Design of educational games and adaptation of game design to the education of people with persistent physical disabilities. Block visual programming in the Scratch environment and rapid prototyping of computer games. Human gestural interface in the Scratch environment based on the Webcam and the PoseNet neural network. Images of real-world objects in educational computer games based on the Google Teachable Machine, Web Camera and neural network. Sound and voice interface in educational computer games.
- Special Education: Everything You Need to Know. URL: https://www.youtube.com/watch?v=H90Po8tHbOU
- Special Education Teacher Interview Questions & Answers. URL: https://www.youtube.com/watch?v=x7PHYvW2ryM
- Hewett; Baecker; Card; Carey; Gasen; Mantei; Perlman; Strong; Verplank. "ACM SIGCHI Curricula for Human–Computer Interaction". ACM SIGCHI. Archived from the original on 17 August 2014. Retrieved 15 July 2014.
- NVDA - NonVisual Desktop Access. URL: https://www.nvaccess.org/
- Orca Screen Reader. URL: https://gitlab.gnome.org/GNOME/orca
- Emacspeak. URL: https://github.com/tvraman/emacspeak
- American online video sharing platform owned by Google. URL: https://www.youtube.com/
- Luiz Cláudio Silveira Duarte, André Luiz Battaiola, Distinctive features and game design, Entertainment Computing, Volume 21, 2017, pp. 83–93.
- Ben-Sadoun, G., & Alvarez, J. Gameplay Bricks Model, a Theoretical Framework to Match Game Mechanics and Cognitive Functions. Games and Culture, 18(1), 2023, pp. 79–101.
- The Scratch programming environment. URL: https://scratch.mit.edu/
- Experimental Scratch server. URL: https://stretch3.github.io/
- Blazhko, Oleksandr, Podhorna, Vіktoriia, Kokotieieva, Anastasiia and Ivanov, Oleksii. Scratch PoseNet Exergame Prototyping for Learning Process Support in Physical Education. Proceedings of International Conference on Applied Innovation in IT 2024/03/07, Volume 12, Issue 1, pp.59-63. DOI: https://doi.org/10.25673/115642
- Blazhko, O., Podhorna, V., Kokotieieva, A., Bashavets, N. (2024). Scratch-Based Exergame-Educational Approach in Teaching the Course “Physical Education” for IT-Specialties. In: Auer, M.E., Langmann, R., May, D., Roos, K. (eds) Smart Technologies for a Sustainable Future. STE 2024. Lecture Notes in Networks and Systems, vol 1027. Springer, Cham. https://doi.org/10.1007/978-3-031-61891-8_32
- RAISE Playground Scratch server. URL: https://playground.raise.mit.edu/main/
- Google Teachable Machine. URL: https://teachablemachine.withgoogle.com/