Chapter 7.3 Spoken Dialog Systems

Front 1

What are the tasks of a speech dialog system?

Back 1

Maintaining the interaction, distributing the initiative (system-initiative, user-initiative, mixed-initiative)

Aggregation of information which is relevant for the task

Keeping track of input modalities and respective confidences

Provision of the desired information to the user

Meta communication: Correction of misunderstandings, feedback (explicit, implicit), etc.

Provision of help to the user

Resolution of complex dialogue phenamena (ellipses, references, etc.)

Link to the application (database, booking protal, etc.)

Front 2

Name four types of dialog systems and explain what they consist of.

Back 2

1. Command systems: one command -> reaction
2. Menu oriented systems: navigating in menu to find the command
3. Speech dialog systems: speech recognition engine, speech understanding component, dialog control module, interface to application system, system response generator, speech output
4. Multimodal dialog systems: multiple modalities

Front 3

Explain the two types of speech dialog system architectures.

Back 3

1. Sequential system: speech recognition and speech output lines separately, controlled by dialog manager (semantic frame) that accesses the database
2. Hub system: hub receives and stores information, checks syntax. An example: TRIPS architecture

Front 4

Name four basic blocks of speech dialog system.

Back 4

1. Speech recognition
2. Semantic extraction
3. Dialog management
4. Speech output

Front 5

What is Wizard-of-Oz-Experiment? When and why is it used?

Back 5

Human substitutes missing speech modules in a dialog system, acts like a machine. Used when prototyping, to design the system (define vocabulary etc). People don't like talking to a system, they use different vocabulary, speak slower etc.

Front 6

Attribute-value pairs (slots) are used to store semantics. Give an example of it.

What is the name of the process that fills the slots?

Back 6

Destination; München Hbf
Time of Departure; 11 a.m.
Parser fills the slot by breaking speech into components, labelling etc.

Front 7

Give an example of contextual analysis by a speech dialog system.

Back 7

User: Next Sunday. This utterance needs to be converted to a date.

Front 8

What are the tasks of a dialog manager?

Back 8

1. Maintaining the interaction, distributing the initiative (system-initiative, user-initiative, mixed-initiative)
2. Aggregation of information which is relevant for the task
3. Provision of the desired information to the user
4. Meta communication: Correction of misunderstandings,
feedback (explicit, implicit), etc.
5. Provision of help to the user
6. Resolution of complex dialogue phenamena (ellipses, references, etc.)

Front 9

Explain three ways of implementing dialog systems.

Back 9

1. Dialogue grammars: Top-down approach, e.g. with finite state machines or declarative grammars
2. Plan-based approaches: Modelling individual dialogue goals; assumption is that system and user have common goals
3. Collaborative approaches: Modelling of motivations and methods for reaching the dialogue goals, e.g. via adjustment of reciprocal assumptions and goals

Front 10

What models does dialog manager use to monitor dialog flow?

Back 10

1. Dialogue History: Sequence of propositions from the user and the system
2. Task Record: Representation of the task and the necessary information
3. World Knowledge Model: Background information necessary for resolving the task
4. Domain Model: Description of the task domain
5. Conversational Model: Communicative capabilities of the system
6. User Model: Preferences, goals, assumptions about and and intentions of the user

Front 11

What is a finite state automaton?

Back 11

A machine that is in one state at a time. Some trigger makes the machine go from one state to another.

Front 12

What are the three conversation model in a speech dialog system? What is the problem of user-initiative?

Back 12

1. System-initiative
2. User-initiative (improbable that task completed)
3. Mixed-initiative: both ask questions

Front 13

What is the difference between explicit and implicit feedback in speech dialog system? What are the risks?

Back 13

Explicit asks separately, whether information correct. Implicit merges the last information to new questions. Explicit is more reliable but slow. Implicit might make something wrong if the user doesn't notice the merged information.

Front 14

Things to take into account in speech dialog output.

Back 14

Not too much information given by the system. Limited vocabulary, the same vocabulary used by the system is probably used by the user. Parametric or real-recording system? Real-recording hard to update (requires the same speaker).

Front 15

Specific /Properties Requirements for Multimodal Dialogue Management

Back 15

System can change modality in case of errors

System needs to handle semantic representations of input and output, in order to resolve cross-modal references

System may handle a larger variety of users (with specific modality usage preferences)

Timing of user interruptions in different modalities (premature requests, corrections, etc.)

Confirmations in alternative modalities (e.g. non-intrusive feedback on a screen)

Integrate conflicting information chunks from different modalities using dialogue context information