You are here:  

• Print

DReSSI

‘Understanding New Forms of Digital Record’ was a 3-year research ‘node’ project (2005-2008, Grant No. RES-149-25-0035) that was also funded by the ESRC as part of the National Centre for e-Social Science (NCeSS). Digital Record was an inter-disciplinary project, utilising the expertise of Computer Scientists, Psychologists and Linguists. The project design and development focused on co-designing technical and computational applications and tools which met the requirements of, and were of use to, a wider research community. Digital Record explored the issues involved in the processes of assembling, structuring, refining and analysing digital records for e-Social Science, with the aim to co-design and develop the 3rd generation of corpus tools; corpora that are based upon multi-media renderings of naturally occurring conversation. This was to facilitate the exploration of language-in-use from a more multi-modal perspective, extending the potential for empirical linguistic enquiry. Therefore, to allow for the analysis of relationships between, specific terms and parts of speech (the ‘linguistic’ features), with proxemic movements and gesture and how such relationships contribute to or regulate the message of the utterance (thus building upon the preliminary explorations conducted in HeadTalk). This focus marked a move away from traditional corpus linguistic techniques that focus upon representing only one aspect of discourse, spoken or written language in the form of textual renderings (i.e. the ‘text of talk’, refer to Knight et al, 2006c for more information), giving the analyst the tools for exploring the lexical, grammatical and accompanying gestural elements that are frequent in everyday conversation.

The Digital Replay System, DRS, was developed as part of this project in order to allow us to represent and replay multimodal linguistic datasets. This tool provides users with the capacity for exploring multiple modes of data (extracted from natural language communication contexts), i.e. streamed verbal, visual and audio data, within the same user-friendly interface. It is also integrated with a system for marking-up relevant gestural aspects of the discourse (the foundations of which were the focus of the HeadTalk project), as well as for codifying interesting patterns that emerge, for the development of linguistic research.

The research conducted throughout the Digital Record project demonstrated the potential to extend existing e-Social Science research and develop new approaches for structuring digital records that marry established analytic schemes with vision-based mark-up systems to foster a richer understanding of the embodied nature of language use and its manifold relations to the production of distinctive social contexts. The ultimate aim was to collect ¼ million words of video data, 125,000 words recorded in dyadic academic supervision contexts, and 125,000 words recorded in a series of lectures delivered at the University of Nottingham. This data comprises purely of native British speaking participants, from the same discourse context (Nottingham University), but it is hoped once all aspects of the corpus development and exploration methodologies are refined, these factors can be extended to include a wide range of speakers from a variety of contexts, in future research of this nature (see DReSSII for more details).

More information on the DReSS project, and a link to the current version of DRS pcan be found on the main DReSS website.

Publications

• Carter, R. and Adolphs, S. (2008). Linking the verbal and visual: New directions for Corpus Linguistics. Language and Computers special issue ‘Language, People, Numbers’, 64; 275-291.

• Adolphs, S. (In press, 2008) Corpus and Context: investigating pragmatic functions in spoken discourse. Amsterdam: John Benjamins.
• Knight, D., Evans, D., Carter, R. and Adolphs, S. (Forthcoming, 2008). Redrafting corpus development methodologies: Blueprints for 3rd generation “multimodal, multimedia’’ corpora.
• Knight, D. and Adolphs, S. (In Press, 2008) Multi-modal corpus pragmatics: the case of active listenership. In Romeo, J. (ed) Corpus and Pragmatics. Berlin and New York: Mouton de Gruyter.
• Brundell, P., Knight, D., Tennent, P., Naeem, A., Adolphs, S., Ainsworth, S., Carter, R., Clarke, D., Crabtree, A., Greenhalgh, C., O’Malley, C., Pridmore, T. and Rodden, T. (Forthcoming, 2008). The experience of using Digital Replay System for social science research. Proceedings of the 4th International Conference on e-Social Science, Manchester, 18-20 June 2008.
• Brundell, P., Tennent, P., Greenhalgh, C., Knight, D., Crabtree, A., O’Malley, C., Ainsworth, S., Clarke, D., Carter, R. & Adolphs, S. (Forthcoming, 2008). Digital Replay system (DRS): A Tool for Interaction Analysis. Proceedings of the International Conference for the Learning Sciences 2008 (ICLS), Utrecht, the Netherlands. June-July 2008.
• Knight, D. and Tennent, P. (Forthcoming, 2008). Introducing DRS (The Digital Replay System): A tool for the future of Corpus Linguistic research and analysis. Proceedings of the 6th Language Resources and Evaluation Conference, Palais des Congrés Mansour Eddahbi, Marrakech, Morocco, 28-30th May 2008.
• Knight, D., Adolphs, S., Tennent, P. and Carter, R. (Forthcoming 2008). The Nottingham Multi-Modal Corpus: A Demonstration. Proceedings of the 6th Language Resources and Evaluation Conference, Palais des Congrés Mansour Eddahbi, Marrakech, Morocco, 28-30th May 2008.
• Adolphs, S. & Carter, R. (2007). Beyond the word: New challenges in analysing corpora of spoken English. European Journal of English Studies 11(2). (18 pages).
• Knight, D. (2006d). Corpora: The Next Generation. Part of the AHRC funded online Introduction to Corpus Investigative Techniques, The University of Birmingham.

Conference Papers and Posters

• Adolphs, S. and Knight, D. (2008). Analysing Discourse Markers: A Multi-Modal Approach. Paper presentated at the BAAL annual conference, September 2008, University of Swansea.
• Brundell, P., Tennent, P., Greenhalgh, C., Knight, D., Crabtree, A., O’Malley, C., Ainsworth, S., Clarke, D., Carter, R. & Adolphs, S. (2008). Digital Replay system (DRS): A Tool for Interaction Analysis. Paper delivered at the International Conference for the Learning Sciences 2008 (ICLS), Utrecht, the Netherlands. June-July 2008.
• Adolphs, S. (Forthcoming, 2008). Plenary session on multi-modal corpora. To be delivered at the IVACS (Inter-Varietal Applied Corpus Studies) Conference, June 2008, University of Limerick, Ireland.
• Knight, D. (Forthcoming, 2008). Gesturing power in dyadic conversations: A study of academic supervisory meetings. Paper delivered at the 4th IVACS conference, June 2008, University of Limerick, Ireland.
• Brundell, P., Knight, D., Tennent, P., Naeem, A., Adolphs, S., Ainsworth, S., Carter, R., Clarke, D., Crabtree, A., Greenhalgh, C., O’Malley, C., Pridmore, T. and Rodden, T. (Forthcoming, 2008). The experience of using Digital Replay System for social science research. Paper presented at the 4th International Conference on e-Social Science, Manchester, 18-20 June 2008.
• Knight, D. and Tennent, P. (Forthcoming, 2008). Introducing DRS (The Digital Replay System): A tool for the future of Corpus Linguistic research and analysis. Poster presentation with demo, delivered at the 6th Language Resources and Evaluation Conference, Palais des Congrés Mansour Eddahbi, Marrakech, Morocco.
• Knight, D., Adolphs, S., Tennent, P. and Carter, R. (Forthcoming 2008). The Nottingham Multi-Modal Corpus: A Demonstration. Paper delivered as part of the ‘Multimodal Corpora’ workshop held at the 6th Language Resources and Evaluation Conference, Palais des Congrés Mansour Eddahbi, Marrakech, May 2008, Morocco.
• Carter, R. (2008). 'Breaking new ground in Applied Linguistics'. Colloquium at AAAL 2008, Washington DC, US.
• Knight, D. and Evans, D. (2008). Multi-Modal Corpora, Discourse and Gesture. Paper presented at AAAL 2008, Washington DC, US.
• Adolphs, S., Knight, D. and Evans, D. (2007). Multi-modal corpora. Presentaton delivered as part of the CRAL seminar series, October 2007, University of Nottingham.
• Evans, D. and Naeem, A. (2007). “Using visual tracking to link text and gesture in studies of natural discourse”, Online Proceedings of the Cross Disciplinary Research Group Conference ‘Exploring Avenues to Cross-Disciplinary Research’, November 7, University of Nottingham. www.nottingham.ac.uk/cdrg/
• Adolphs, S., Bertenthal, B., Boker, S., Carter, R., Greenhalgh, C., Herald, M., Kenny, S, Levow, G., Papke, M. and Pridmore, T. (Forthcoming, 2007). “Integrating cyberinfrastructure into existing e-social science research”, Online Proceedings of the 3rd International Conference on e-Social Science, October 7-9, Ann Arbor, Michigan: ESRC/NSF. 
• Tennent, P. and Knight, D. (2007). Multi-modal corpora: Adapting gesture recognition techniques for linguistic analysis. Poster delivered at the 3rd annual International eSocial Science Conference, October 2007, University of Michigan, US.
• Adolphs, S., Carter, R., Knight, D. and Evans, D. (2007). e-Social Science and Applied Linguistics: a multimodal corpus case study. Paper delivered at the New Horizons in Linguistics symposium, September 2007, University of Oxford.
• Evans, D., Knight, D., Carter, R. & Adolphs, S. (2007). Multi-modal corpus design, construction and use. Paper delivered at BAAL 2007, September 2007, University of Edinburgh, Scotland.
• Knight, D., Evans, D., Adolphs, S. & Carter, R. (2007). Approaching the problems: Capturing, coding and analysing gesture in multi-modal communication data. Paper delivered at Corpus Linguistics 2007, July 2007, University of Birmingham.
• Knight, D. (2006). Developing a Multi-Modal Corpus: Data Coding Issues. Paper delivered at the IVACS Annual Research Symposium, February 2006, University of Limerick, Ireland.
• Knight, D. & Bayoumi, S. & Mills, S. & Crabtree, A. & Adolphs, S. & Pridmore, T. & Carter, R. (2006c). Beyond the Text: Construction and Analysis of Multi-Modal Linguistic Corpora. Paper delivered at the 2nd annual international eSocial Science conference, June 2006, University of Manchester.
• Knight, D. & Adolphs, S. (2006b). Analysing Spoken Corpora: Methodological Issues and Technological Challenges. Paper delivered at the BAAL SIG Seminar (Special Interest Group: Corpus Linguistics), April 2006, The Open University, Milton Keynes.
• Knight, D. & Adolphs, S. & Carter, R. (2006a). The Multi-Modal Corpus: Coding and representing data- the issues. Paper delivered at the 3rd IVACS conference, May 2006, University of Nottingham.
• French, A. & Wright, M. & Greenhalgh, C. & Knight, D. & Brundell, P. & O’Malley, C. & Ainsworth, S. & Clarke, D. & Rodden, T. (2006). ‘Replaytool’ software in practice. Poster presented at the 2nd annual international eSocial Science Conference, June 2006, University of Manchester.

Unpublished Reports

• Brundell, P. & Knight, D. (2005). Current Research and Tools to Support Data Intensive Analysis for Digital Records in eSocial Science. Unpublished Node report. University of Nottingham.

CRAL Project Staff

If you would like more information on any aspect of this research project, do not hesitate to contact a member of the CRAL staff involved, as detailed below:

• Professor Ronald Carter - Domain Advocate
• Professor Svenja Adolphs - Project Researcher

 

Back to top