Dialogue Experimental Toolkit

ABOUT



The toolkit is an instant-messaging platform for conducting research on dialogue. The software consists of a chat server and chat clients, written in Java.

The chat-tool makes data-collection and preparation much easier and quicker. Configuration of the server and clients allows experimental designs that are much more standardized and reproducible.

The toolkit can be used "off-the-shelf" to collect and prepare instant-messaging conversations. It is also a powerful, programmable toolkit for conducting experiments on live interaction.

All participants' turns are routed via the server which can be scripted to detect, for example target words, phrases or syntactic constructions. This information can then be used, in real-time to trigger experimental interventions that are sensitive to the interactional context.

The toolkit provides an extensive API for scripting these manipulations, as well as a constantly expanding library of experimental setups that can be reconfigured

FEATURES



Recorded data

The chattool automatically records all keypresses, words and turns, notifications (including typing notifications and read receipts), number of edits, typing speed, typing overlap.

Instead of costly and time-intensive transcription, all data is immediately available for analysis. The data is saved in a variety of formats - to help both qualitative and quantitative analyses (e.g. loading into SPSS, R, Excel, MATLAB).

 


MANIPULATING THE INTERFACE

Existing software (e.g. whatsapp, viber, line, wechat) use subtly different interfaces that have undocumented, ad-hoc and constantly changing features. The effect of these interface features on interaction are currently poorly understood.
The toolkit allows configuration of all aspects of the interface, including the screen dimensions, text (colour, font, positioning), typing synchrony and interleaving of turns, typing status, read receipts, as well as simulating network conditions, e.g. latency.




Manipulating the interaction

Possible kinds of experimental intervention include: Conceptual & linguistic coordination
  • Manipulation of lexical, syntactic and semantic constituents (e.g. by inserting "spoof" clarification requests into the interaction that target a specific element - see image to the right).
  • Manipulation of the specificity of referring expressions.
  • Manipulation of priming & levels of alignment.
  • Procedural coordination
  • Manipulation of the timing and sequencing of turns.
  • Group-membership and identity:
  • Manipulation of participatory status (e.g overhearer vs. bystander).
  • Manipulation of (apparent) identity of participant.
  • Assigning participants to different sub-groups, e.g. to allow group-specific conventions to emerge.
  •  

    SCREENSHOTS


    a couple of questions about the methodology

    • But don't the participants notice that something very strange is going on and realize that the server is manipulating the text?


      Surprisingly, no! Participants tolerate very high levels of disturbance. Even quite unnatural-looking clarification requests are treated as genuine. We have carried out tens of thousands of different interventions, and so far only a handful of participants suspected a turn might have been generated artificially. Of course, the interventions do need to be piloted and designed judiciously.

    • But this isn't the same as real spoken dialogue


      Yes and no. Text-based dialogue is similar to spoken dialogue in many important respects: it exhibits disfluencies, hesitations, interruptions, pauses, fillers, repair, acknowledgments. We have shown that text-based variants of key dialogue tasks, including the tangram task (Clark and Wilkes-Gibbs, 1986), maze task (Garrod and Doherty, 1994) and story-telling task (Bavelas et al., 1992), exhibit local and global patterns in participants' dialogue that are comparable to those observed in the original spoken versions - and are also amenable to experimental manipulation using this technique.

    DOWNLOAD


    There are four different flavours of the toolkit.

    The toolkit requires java 1.7 because of this issue

    EASIEST

    • Single file

    • No installing required
    • Requires Java 1.7

    SAFEST

    • Toolkit & Java libraries

    • No installing required
    • Includes Java 1.7

    DEVELOPER

    • Source code

    • Modify interface(s)
    • Script interventions
    • Contribute to toolkit

    MOBILE

    • Mobile client

    • Coming soon





    DOCUMENTATION


    If you want to use the software "out-of-the-box" without doing any scripting/programming, or you would like to try it out, choose the user manual (to be released February, 2016). Otherwise, follow the link to the developer site which has more detailed instructions on how to run, configure, and script the toolkit.

            Developer site

    PUBLICATIONS

    Please email g.j.mills@rug.nl if you have published research that uses the toolkit.

    Mills, G. J. and Healey, P. G. T. (submitted) A dialogue experimentation toolkit.

    Yanchao Yu, Arash Eshghi, Gregory Mills, and Oliver Lemon. (2017) The BURCHAK corpus: a challenge data set for interactive learning of visually grounded word meanings. In Proceedings of the EACL 2017 workshop on Vision and Language (VL'17), Valencia,

    Liebman, N., & Gergle, D. (2016, February). It's (Not) Simply a Matter of Time: The Relationship Between CMC Cues and Interpersonal Affinity. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (pp. 570-581). ACM.

    Arash Eshghi and Patrick G. T. Healey. Collective contexts in conversation: Grounding by proxy. Cognitive Science, pages 1--26, 2015

    Concannon, S., Healey, P. G., & Purver, M. (2015). Shifting Opinions: An Experiment on Agreement and Disagreement in Dialogue SEMDIAL 2015 goDIAL, 15.

    Nolle, Jonas, Kristian Tylen, and Gregory Mills. (2015) "Environmental affordances shape linguistic coordination in the maze game."

    Jacobi, J., de Rechteren, A., Mills, G., & Redeker, G. (2015). Dutch-speaking children’s co-ordination skills in dialogue. Annual TABU Dag (Taalbulletin), Groningen

    Engbrenghof, M., Mills, G., Redeker, G. (2015). Manipulating evidence of grounding in a collaborative communication game. Annual TABU Dag (Taalbulletin), Groningen

    Mills, G. J. (2014) Dialogue in joint activity: complementarity, convergence, conventionalization New Ideas in Psychology.

    Patrick G. T. Healey, Gregory J. Mills, and Arash Eshghi. Making things worse to make them better: The role of negative evidence in the coordination of referring expressions. In Proceedings of 35th Annual Meeting of the Cognitive Science Society (CogSci 2013), 2013.

    Christine Howes, Patrick G. T. Healey, Matthew Purver, and Arash Eshghi. Whose turn is it anyway? same- and cross-person compound contributions in dialogue. In Proceedings of the 34th Annual Meeting of the Cognitive Science Society (CogSci 2012), 2012.

    Mills, G. J. (2012) Clarifying reference and plans in dialogue . Fifth International Conference of the German Cognitive Linguistics Association. University of Freiburg 10-12 October. Poster

    Christine Howes. (2012) Coordination in dialogue: Using compound contributions to join a party. PhD thesis, Queen Mary University of London, 2012.

    Mills, G.J. (2012) Making and breaking procedural conventions in dialogue . Poster presented at Annual meeting of the cognitive science society (CogSci), Kyoto

    Davidenko, N. and Mills G. J. (2012) Describing faces: Conventionalizing ontologies through dialogic interaction . Poster presented at Annual meeting of the cognitive science society (CogSci), Kyoto

    Christine Howes, Patrick G. T. Healey, Matthew Purver, and Arash Eshghi. (2012) Finishing each other's. . . responding to incomplete contributions in dialogue. In Architectures and Mechanisms for Language Processing, Riva del Garda, 2012.

    Christine Howes, Patrick Healey and Matthew Purver (2012). Whose turn is it anyway? Same- and cross-person compound contributions in dialogue. Poster, in CogSci 2012, Sapporo, August 2012.

    Christine Howes, Patrick G. T. Healey, Matthew Purver, and Arash Eshghi. (2012) Finishing each other's. . . responding to incomplete contributions in dialogue. In Proceedings of the 32nd Annual Conference of the Cognitive Science Society, Sapporo, 2012.

    Mills, G. J. (2011) The emergence of procedural conventions in dialogue In Proceedings of the 33rd Annual Conference of the Cognitive Science Society. Boston. USA

    Howes, C. Purver, M., Healey, P. G.T., Mills, G. (2011) On Incrementality in Dialogue: Evidence from Compound Contributions Dialogue and Discourse. Vol 2, No 1. Special Issue on Incremental Processing in Dialogue.

    Gregoromichelaki, E., Kempson, R., Purver, M., Mills, G., Cann, R. (2011) Incrementality and intention-recognition in utterance processing Vol 2, No 1. Special Issue on Incremental Processing in Dialogue.

    Patrick G. T. Healey, Arash Eshghi, Christine Howes, and Matthew Purver.(2011) Making a contribution: Processing clarification requests in dialogue. In Proceedings of the 21st Annual Meeting of the Society for Text and Discourse, Poitiers, July 2011.

    Mills, G. J. and Gregoromichelaki, E. (2010) Establishing coherence in dialogue: sequentiality, intentions and negotiation. In Proceedings of SemDial (PozDial). Adam Mickiewicz University, Poznan, Poland 16-18th June.

    Arash Eshghi, P. G. T. Healey, Matthew Purver, Christine Howes, Eleni Gregoromichelaki, and Ruth Kempson. Incremental turn processing in dialogue. In Architectures and Mechanisms for Language Processing, York, UK, September 2010.

    Kempson, R., Gregoromichelaki E., Mills, G. Purver, M., Howes, C., Healey, P. (2010) On Dialogue Modelling, Language Processing Dynamics, and Linguistic Knowledge Linguistic Evidence 2010, Tübingen, Feb 2010.

    Howes, C. , Healey, P. G. T., Mills, G. J. (2009) A: an experimental investigation into B:...split utterances In Proceedings of SIGDIAL 2009: the 10th Annual Meeting of the Special Interest Group in Discourse and Dialogue, Queen Mary University of London, Sept. 2009.

    Arash Eshghi and P. G. T. Healey. What is conversation? distinguishing dialogue contexts. In Niels Taatgen and Hedderik van Rijn, editors, Proceedings of the Thirty-First Annual Conference of the Cognitive Science Society, 2009

    Kempson R, Gregoromichelaki E, Purver M, Mills G, Gargett A, Howes C (2009) How mechanistic can accounts of interaction be? In: Proceedings of Diaholmia, the 13th workshop on the semantics and pragmatics of dialogue

    Mills, G. J. and Healey, P.G.T. (2008) Semantic negotiation in dialogue: mechanisms of alignment. in Proceedings of the 8th SIGdial workshop on Discourse and Dialogue, Columbus, OH, US; June 2008.

    Mills, G. J. (2007) Semantic co-ordination in dialogue: the role of direct interaction. PhD Thesis.

    Mills, G.J. and Healey, P. G. T. (2006) Clarifying Spatial Descriptions: Local and Global Effects on Semantic Co-ordination, In Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue. Potsdam. Germany.

    Healey, P.G.T. and Mills, G.J. (2006) Participation, Precedence and Co-ordination in Dialogue. In Proceedings of the 28th Annual Conference of the Science Society. Vancouver. Canada.

    Matthew Purver. The Theory and Use of Clarification Requests in Dialogue. PhD Thesis, University of London, 2004.

    Healey, P.G.T., Purver, M., King, J., Ginzburg, J. and Mills, G. J. (2003) Experimenting with Clarification in Dialogue. in Alterman, R. and Kirsh, D. (eds) Proceedings of the 25th Annual Conference of the Cognitive Science Society. Mahwah, N. J.: LEA pp. 539-544.

    Purver, M., Healey, P. G. T., King, J. Ginzburg, J. and Mills, G. (2003) Answering Clarification Questions. In Proceedings of the 4th SIGdial Workshop on Discourse and Dialogue, pp 23-33, Association for Computational Linguistics, Sapporo, Japan, July 2003.

    CONTACT


    For more information, please email:
    g.j.mills@rug.nl
    p.healey.qmul.ac.uk

    To be notified of updates, join this mailing-list (very low volume).