This work package is designed to define the themes in which projects on embodied language can be submitted to ASLAN. Our goal is to propose and develop research questions that are relevant for both computer science and language sciences.

Keywords : multimodal interaction, affective and sensorimotor experience, learning (machine/human/HMI) ; embodiment, cognition, development

First or second language learning and teaching in monolingual and bilingual populations: from sound to discourse

The question of language learning can be addressed in several perspectives taking into account typically and non-typically developing monolingual and bilingual populations in a life-span perspective. The shared goal is to describe and explain the learners’ linguistic developmental trajectories in relation to biomechanical, cognitive, and contextual and environmental constraints. Pursuing this goal can involve modeling a language-based human-computer interaction. Projects can look at, for example, implementing ways to trace this developmental experience, proposing how such a trace can be used to facilitate language acquisition and/or learning more generally, etc.

Concerning first language acquisition, either early or later language development stages can be considered (e.g. babbling, first words, word combinations, narrative competence, writing, etc.). For instance some studies have been conducted (concerning a later stage of linguistic development – children and adolescents) in order to identify writing processes and to characterize writer profiles. These profiles can be documented through traces of interaction and their interpretation.

Concerning second language acquisition, research is ongoing concerning the acquisition of L2 phonology by adult late learners and children who have recently arrived in France and who speak different first language(s). Another project examined the role of the motor system in second language (L2) processing.

The following questions are still to be addressed: what modeling of the developmental and/or socio-cognitive mechanisms of these stages is needed in this context? What use can be made of these descriptive stages for proposing new interaction devices (humans-humans, humans-devices, and devices-devices) for learning, work, or play? What help do these models offer for documenting language embodiment, diagnosis of developmental disorders, manipulation of artifacts and visualizations for making decisions and for collaborative problem-solving? How can research in language learning and teaching inform the sequential learning of languages or tasks by an (artificial) agent, addressing the questions of incremental task learning, catastrophic forgetting and transfer of conceptual learning from one context to another?

Embodied cognition: Motor Action, Language, Emotion, Enaction

Research conducted on embodied cognition considers language as closely intertwined with the body and its interactions with the environment. Accordingly, the way we act with the world is assumed to shape our cognitive representations and affect language processing and use, hence our understanding of words and sentence would developed in parallel with our developing sensorimotor capacities and skills. In this view, language is grounded in the brain cortical and subcortical structures for action, perception and emotions.

Different aspects of language processing or decoding might shape the sensorimotor brain regions, form phonology to semantics and syntax. Experiments pertaining to neurolinguistics have shown the specific activation of brain areas devoted to action planning and execution when processing actions words, phoneme or complex sentences. Words, sound and sentences may also directly impact our overt behaviour. Examining the relations between brain structures and networks pertaining to both language processing and affective/sensory and motor domain will enable to provide new inputs in educational strategy, therapeutic approaches, computer modeling. These domains will in turn propose a new vision of neurolinguistics and will inform the study of language processing to reach new questions such as : how can linguistic structures emerge as a result of sensori-motor learning from interactions with the environment and/or people?

Interacting bodies through multiple modalities

Interaction and the emergence of human and agent experience(s) through social activities generates many research questions in language sciences and AI. For example, how do visualizations and representations play a role in learning and in the evaluation of the quality of interaction? How can emotion tracking, detection, comprehension, and models of intervention be interpreted for different contextualized objectives?

Research on interaction and the emergence of human and agent experience(s)
through social activity from a language sciences perspective is concerned with the analysis of spoken language in a given context. However, in line with recent theories on embodied language and embodied cognition, projects funded by ASLAN highlight how spoken language is produced in relation to how human bodies interact. Indeed, participants mobilize linguistic, corporeal, gestural and postural resources during human interaction to build
meaning and to promote mutual understanding in a given situation. They also use the material resources that are available in their environment.
In what way do interactions (devices-devices, humans-humans, humans-devices) inform our studies of multimodal language development, competencies and phenomenological experience? How can our understanding of fine-grained analyses of human interaction shape computational models in such devices and reciprocally inform us on how human interaction is modified by artificial devices?

The description of languages, embodiment in linguistic systems and the construction of embodied knowledge

Research on embodiment in language description is concerned with the way in which languages represent embodied activity and experience in the lexicon, expressions / phraseology and grammar (morphology, syntactic structures), for example the expression of the experience of spatial activity in verb- and satellite-framed languages or constructions in specific languages. It is also concerned with the embodied experience of linguistic signifiers (words as articulated units, prosodic units as embodied an interactive patterns verbal action) and the way in which they express (encode or enact, depending on the paradigm of reference) semantic representation and joint sense-making experience, at all semantic levels (from low-level phono symbolism grounded in sub morphemic units to high-level expressivity in constructions). Embodiment in language description focuses on natural languages, endangered languages, dialect variation, language contact, language change, is open to comparative and typological approaches, and can be related with usage-based descriptions of embodiment in situated speech and interactions.The question is how linguistic units shaped by embodied processes represent or construct knowledge, how automatic language processing can represent, modelize and analyze the workings of embodied markers (for instance derivation and agglutinations in morphology), and how embodied knowledge can be represented in AI.

Projects (ASLAN 1) : Whether in learning contexts (GEDECO, ISMAEL, ForLAN), medical contexts (IAA, MALICE, REMILART, MILSA, Food in the hospital, THÉSÉE) or contexts mediated by technology (JouES!, ITAC, SENEC, JANUS), researchers have analyzed interacting bodies in previous projects. The large majority of this research was so far positioned within interactional linguistics frameworks (e.g. language use in real time human interaction).