dorkbotSF

This is very complex use of assistive technologies that has the potential to go many places. Basically, it records data about what is going on around a kid and allows them to create a story about it. It is designed for kids with serious disabilities such as Cerebral Palsy that make communication hugely laborious.

This is a great synthesis of sensor data gathering and visualization, storytelling design, natural language studies, assistive technologies, and interface design for kids.

From their grant summary:

Our goal is to develop a computer tool which helps children who cannot speak create a story about their day at school. Story telling is an essential aspect of social interaction, and story-telling skills are developed through practice. It is difficult for non-speaking children to get such practice, our tool will help them.

More specifically, we want to use various kinds of sensors to acquire information about where the child went, what she did, and who she interacted with; write a computer program which automatically creates a draft story based on this data; and create a story editing and narration interface which lets children edit the draft story and then tell it when they are happy with it. Possible sensors include GPS for tracking where children go, RFID tags for tracking what objects children interact with and hence their activities; and barcode scanners for recording who children interact with. The story-generation software will be based on technology for generating English summaries of data which has been developed in other EPSRC-funded projects such as SumTime, RoadSafe, and BabyTalk. The story editing interface will probably be based on a visual timeline metaphor.

Courtesy of ACM courtesy of BBC News, here’s a bit from a recent article:

Dr Ehud Reiter, from the University of Aberdeen’s school of natural and computing sciences, said: “How was school today? uses sensors, swipe cards, and a recording device to gather information on what the child using the system has experienced at school that day.

“This can then be turned into a story by the computer – using what is called natural language generation – which the pupils can then share when they get home.