Banire’s research reflects QF’s commitment to enhancing inclusion and accessibility for those with autism.
It also highlights how QF is at the centre of making Qatar a cradle of collaboration – collaboration that produces discoveries with real-world, human benefits.
Banire’s research uses commercially available facial recognition software and a webcam, to track changes in facial expression and eye gaze that occur while children are engaged in an attention test.
It is pioneering for two reasons. First, it uses a unique combination of facial expressions and eye tracking to gauge the attention span of children with ASD.
Second, it offers a way of measuring attention in a non-obtrusive manner – which is important, as those with ASD can sometimes have a heightened sensory response under certain settings, and may not respond well if attention monitoring is done by gadgets involving physical contact such as goggles or caps.
“Changes in expression normally happen on facial landmarks or ‘hotspots’ on the face, such as eyebrows and cheekbones,” says Banire.
“In our research, these changes in expression and eye gaze were compared and co-related to the outcomes of the attention test.”
The raw data of these facial landmarks, along with eye tracking from the attention test, is fed into an algorithm that will be used to design a model to predict and monitor attention.
The contactless approach will help teachers and instructors to better predict when a student with autism is about to react negatively to a specific task, or needs a break.
Additionally, changes in facial expression and eye-tracking can also trigger a warning that alerts a student with autism, who may be distracted, to refocus on the task at hand.
To carry out her research, Banire needed an attention test suitable for students with autism and Dr Mansoor stepped in.
“As an engineer, I have always been keen to develop a more effective way of delivering my lessons to my undergraduate engineering students,” says Dr Mansoor.
“For me, mixed reality (MR) presents us with so many possibilities. I knew that students would understand the concept better if they wore MR goggles and interacted with a virtual atom. The same principle can be applied in teaching those with autism.
“So, when I heard that Bilikis was looking for someone to help design an attention test like the MR test I had developed at Tamuq, I knew we could help.”
As those with ASD do not react positively to wearable devices — such as MR goggles — Dr Mansoor helped create a similar test that could be viewed on a 3D monitor, providing users with a device-free, virtual experience.
While taking part in the test, the participant’s facial expressions can be monitored through a webcam and analysed by the algorithm in the facial recognition software.
The software to gauge facial expression is already used for various purposes; however, according to both Bilikis and Dr Mansoor, using it to assess the attention span of children with ASD is a groundbreaking step.
Banire also reached out to experts at Renad Academy, a specialised QF school for students with mild to moderate ASD.
They offered suggestions and recommendations on the parameters that need to be measured.
Conversations with therapists and teachers helped her understand and appreciate the complexities of ASD, helping her fine-tune the attention test.
Banire has also been able to involve experts and students diagnosed with ASD from QF and non-QF schools in her study.
Students from Step by Step – a Doha-based school for children with special needs, including ASD and other volunteers – have participated in the attention tests that Tamuq helped Banire develop, providing data, contributing to the success of the research.
Sherri L Miller, director of Renad Academy, said: “It is such a fabulous opportunity for Renad to support people all around the world who have ASD, by collaborating with researchers in Education City. The work they are doing is commendable, as it is an innovative means to gather data without disturbing students with ASD.”