text: Turn-taking is essential to the structure of human teamwork. Humans are typically aware of team members' intention to keep or relinquish their turn before a turn switch, where the responsibility of working on a shared task is shifted. Future co-robots are also expected to provide such competence. To that end, this paper proposes the Cognitive Turn-taking Model (CTTM), which leverages cognitive models (i.e., Spiking Neural Network) to achieve early turn-taking prediction. The CTTM framework can process multimodal human communication cues (both implicit and explicit) and predict human turn-taking intentions in an early stage. The proposed framework is tested on a simulated surgical procedure, where a robotic scrub nurse predicts the surgeon's turn-taking intention. It was found that the proposed CTTM framework outperforms the state-of-the-art turn-taking prediction algorithms by a large margin. It also outperforms humans when presented with partial observations of communication cues (i.e., less than 40 of full actions). This early prediction capability enables robots to initiate turn-taking actions at an early stage, which facilitates collaboration and increases overall efficiency.
  @cite_0: Today there are solutions for some specific turn-taking problems, but no general model. We show how turn-taking can be reduced to two more general problems, prediction and selection. We also discuss the value of predicting not only future speech silence but also prosodic features, thereby handing not only turn-taking but “turn-shaping”. To illustrate how such predictions can be made, we trained a neural network predictor. This was adequate to support some specific turn-taking decisions and was modestly accurate overall.
  @cite_16: Deciding when to take (or not to take) the turn in a conversation is an important task. It has been stressed in the descriptive literature that such decisions must involve prediction ,a s they often seem to be made before a transition place has been reached. In computational systems, however, turn-taking is normally a reaction to parameters like pause length. In this paper, we report on experiments that try to bridge this gap. We describe an experiment (using controlled stimuli) that shows human performance at prediction of turn-taking decisions and then show that a model automatically induced from data can reach a similar level of performance. We then describe a series of experiments on spontaneous dialogue data where we combine pause thresholds with syntactic and prosodic information to make turn-taking decisions, successively reducing the pause threshold until reaction becomes prediction. All our classifiers improve significantly over the baselines; prediction however is shown to be the hardest task, and we discuss additional information sources that could improve it. Index Terms: turn-taking, machine learning, prosody.
  @cite_7: This paper reports on the development of a multi-agent simulation of small group discussion that focusses on the interaction and the coordination of turn-taking. We describe the addition of nonverbal behaviours, such as gaze, gestures, posture shifts and head and facial expression, to the model; how the agents in the simulation take these behaviours into account in their decisions to speak, to stop, or to give feedback. The simulation is to be evaluated comparing its statistical profile against the statistics generated by a simpler, base model, one without the nonverbal behaviours, to show that it better approximates the statistics of a real group discussion. The properties to be assessed include mean transition intervals, turn lengths, relation of gaze to speaking order, frequency of simultaneous starts, and of feedback.
  @cite_6: A key mechanism in the organization of turns at talk in conversation is the ability to anticipate or PROJECT the moment of completion of a current speaker's turn. Some authors suggest that this is achieved via lexicosyntactic cues, while others argue that projection is based on intonational contours. We tested these hypotheses in an on-line experiment, manipulating the presence of symbolic (lexicosyntactic) content and intonational contour of utterances recorded in natural conversations. When hearing the original recordings, subjects can anticipate turn endings with the same degree of accuracy attested in real conversation. With intonational contour entirely removed (leaving intact words and syntax, with a completely flat pitch), there is no change in subjects' accuracy of end-of-turn projection. But in the opposite case (with original intonational contour intact, but with no recognizable words), subjects' performance deteriorates significantly. These results establish that the symbolic (i.e. lexicosyntactic) content of an utterance is necessary (and possibly sufficient) for projecting the moment of its completion, and thus for regulating conversational turn-taking. By contrast, and perhaps surprisingly, intonational contour is neither necessary nor sufficient for end-of-turn projection.
summary: In conversational turn-taking analysis, linguistic cues such as pause duration @cite_16 and pitch levels @cite_0 have been found to play a key role in turn-taking regulation. The study of de Ruiter el al. @cite_6 revealed that syntax and semantics cues are also important in projecting the end of a speaker's turn. Non-verbal behaviors such as gaze and posture shifts have been investigated and found to be relevant to turn regulations @cite_7 .