Table of Contents

THEORY. Introduction. Input Interfaces. Output Interfaces and Displays. Computer Graphics. 3D Interaction. 3D Navigation. Descriptive and Predictive Models. Multi-Touch. Multi-Touch for Stereoscopic Displays. Pen and Multi-Touch Modeling and Recognition. Using Multi-Touch with PetriNets. Eye Gaze Tracking as Input in Human–Computer Interaction. Brain–Computer Interfaces: Considerations for the Next Frontier in Interactive Graphics and Games. ADVANCED TOPICS. 3D Input Math Basics. Introduction to Digital Signal Processing. Three Dimensional Rotations. MEMS Inertial Sensors and Magnetic Sensors. Kalman Filters. Quaternions and Sensor Fusion. HANDS-ON. Hands-On: Inertial Sensors for 3D Input. Simple Hands-On Project with Unity3D and Oculus Rift. Hands-On Approach with Leap Motion. Hands-On Approach with Kinect Sensor v2. Creating Home-Brew Devices with Arduino Microcontrollers. Autonomous Bicycle with Gyroscope Sensor. Input Implementation Details. CASE STUDY: SPEECH AS INPUT. Multimodal Human-Like Conversational Interfaces. Adaptive Dialogue Systems for Health. CONTENTS OF ONLINE SUPPLEMENTARY MATERIAL. The World of Gesture Recognition. Input Frameworks and Development Considerations for Input Technologies. Hands-On: Multi-Touch and Pen Using Microsoft Windows 8 and Windows 10 with Microsoft Surface Pro. Hands-on: Intel RealSense SDK. APPENDICES. Displays. Creating Your Own Virtual Reality Headset.

About the Author

Francisco R. Ortega, PhD, is a postdoctorate research fellow at Florida International University, Miami, where he received his PhD in computer science. He is the current director of the Open Human-Interface Device Laboratory at Florida International University (http://openhid.com). He was a member of the Digital Signal Processing Laboratory at FIU, and has over 17 years of experience in software development and systems integration. His interests are in 3D user interfaces, input devices, human–computer interaction, 3D navigation, and input modeling, among other interests. He has multiple publications in journals, lecture notes, and conference proceedings.

Fatemeh Abyarjoo, PhD, received her PhD in Electrical engineering from Florida International University, Miami, where she was also a research assistant in the Digital Signal Processing Laboratory, focusing on sensor fusion for human motion tracking. She is currently a Fraud Risk Data Scientist, focusing on financial data analyzing. Her research interests are data mining, data analysis, statistical modeling, sensor fusion and wearable devices. She is a former Open Science Data Cloud PIRE National Science Foundation Fellow.

Armando Barreto, PhD, is a faculty member of the Electrical and Computer Engineering Department at Florida International University, Miami, as well as the director of FIU’s Digital Signal Processing Laboratory. He earned his PhD in electrical engineering from the University of Florida, Gainesville. His work has focused on applying DSP techniques to the facilitation of human–computer interactions, particularly for the benefit of individuals with disabilities. He has developed human–computer interfaces based on the processing of signals and has developed a system that adds spatialized sounds to the icons in a computer interface to facilitate access by individuals with "low vision." He is a senior member of the Institute of Electrical and Electronics Engineers and the Association for Computing Machinery.

Naphtali Rishe, PhD, is Eminent Chair Professor of Computer Science at Florida International University, Miami. He has authored three books on database design and geography and has edited five books on database management and high performance computing. He holds four US patents on database querying, semantic database performance, Internet data extraction, and computer medicine. He has also authored 300 papers in journals and proceedings on databases, software engineering, Geographic Information Systems, Internet, and life sciences. His TerraFly project—a 50-terabyte database of aerial imagery and Web-based GIS—has been extensively covered by worldwide press.

Malek Adjouadi, PhD, is a professor with the Department of Electrical and Computer Engineering at Florida International University, Miami. He received his PhD from the Electrical Engineering Department at The University of Florida, Gainesville. He is the founding director of the Center for Advanced Technology and Education funded by the National Science Foundation. His earlier work on computer vision to help persons with blindness led to his testimony to the US Senate on the committee of Veterans Affairs on the subject of technology to help persons with disabilities. His research interests are in image and signal processing with applications in neuroscience and assistive technology research.