Mudra is a novel multimodal gesture interaction framework that introduces a software engineering methodology for the implementation of gestures. While existing frameworks support application developers by providing a set of predefined gestures (e.g. pinch or rotate), our framework also enables the implementation of new and composed gestures. This approach highlights how a declarative rule-based language description of gestures in combination with a host language increases the extensibility and reusability of gestures.
While the essential complexity in dealing with advanced gestural interfaces cannot be reduced, we offer the appropriate concepts to limit the accidental complexity. We feel confident that our approach will support the HCI community in implementing and investigating innovative gestures that go beyond the current state-of-the-art.