SmartKom: Foundations of Multimodal Dialogue Systems

Dialogue Systems Go Multimodal: The SmartKom Experience.- Facts and Figures About the SmartKom Project.- An Exemplary Interaction with SmartKom.- Multimodal Input Analysis.- The SmartKom Architecture: A Framework for Multimodal Dialogue Systems.- Modeling Domain Knowledge: Know-How and Know-What.- Speech Recognition.- Class-Based Language Model Adaptation.- The Dynamic Lexicon.- The Prosody Module.- The Sense of Vision: Gestures and Real Objects.- The Facial Expression Module.- Multiple Biometrics.- Natural Language Understanding.- The Gesture Interpretation Module.- Multimodal Dialogue Processing.- Modality Fusion.- Discourse Modeling.- Overlay: The Basic Operation for Discourse Processing.- In Context: Integrating Domain- and Situation-Specific Knowledge.- Intention Recognition.- Plan-Based Dialogue Management for Multiple Cooperating Applications.- Emotion Analysis and Emotion-Handling Subdialogues.- Problematic, Indirect, Affective, and Other Nonstandard Input Processing.- Multimodal Output Generation.- Realizing Complex User Wishes with a Function Planning Module.- Intelligent Integration of External Data and Services into SmartKom.- Multimodal Fission and Media Design.- Natural Language Generation with Fully Specified Templates.- Multimodal Speech Synthesis.- Scenarios and Applications.- Building Multimodal Dialogue Applications: System Integration in SmartKom.- SmartKom-English: From Robust Recognition to Felicitous Interaction.- SmartKom-Public.- SmartKom-Home: The Interface to Home Entertainment.- SmartKom-Mobile: Intelligent Interaction with a Mobile System.- SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment.- Data Collection and Evaluation.- Wizard-of-Oz Recordings.- Annotation of Multimodal Data.- Multimodal Emogram, Data Collection and Presentation.- Empirical Studies for Intuitive Interaction.- Evaluation of Multimodal Dialogue Systems.