—Wa’ed Hakouz and Nour Musa, Team of I Copy You project
Qatar’s pride: Nour Musa, left, and Wa’ed Hakouz. Photos by the author.
By Kamran Rehmat
SEATTLE — The girls overcame a strenuous 3-day, 23-team, 12-country Middle East and North Africa competition in June to win, alongside Bahrain, the Pan-Arab Imagine Cup and qualify for the World Championships here.
Qatar University’s Wa’ed Hakouz and Nour Musa, who sparkled at the epoch-making 12th edition of the global student technology competition with their I Copy You project even if it did not quite land them the coveted prize.
I Copy You is a low-cost educational software to build communication skills for autistic children. Using the Microsoft Kinect camera, it detects the child’s motion and provides feedback to a RoboSapien — a low-cost humanoid robot — that mimics the child’s movement (See Page 6).
They began their presentation before a five-member judge panel at the Microsoft-sponsored championships in this emerald city.
They explained how RoboSpaien works and the whole gamut of interaction that draws in autistic children into becoming more attentive and focused in learning to carry out simple tasks.
“We have done many tests in class environment with autistic children and with success. We have gone to schools with special children, and engaged them in fun activities. The teachers say the system was easy for them to use,” Wa’ed and Nour explained during the presentation with the help of videos.
“We have seen how the kids interact more engagingly, which improves their ability to mimic actions, keep eye contact and retain focus for longer periods,” they said.
“The best part of this creative project is that customized needs can be addressed through it. We can change the game, we can change the colours, we can change the sounds. We will be continuously engaged with our customers to harness the goal, which is to help autistic children meet their individual needs,” they told the judges.
“We are the only team or company who will have these products to buy off the shelf. Not only will this be a relatively low cost product but easy to use as well,” they promised.
The girls say they have a feasible marketing strategy to make the cut. The product will cost US$687. Addressing questions about autistic kids trying to lunge at the robots and twisting its arms etc, they said — citing one autistic kid’s similar reaction — that at this developing stage it is a risk they are willing to take. They said the idea is to get used to the robot environment and when familiarity takes hold, the kids don’t behave adversely.
One of the judges inquired if there had been any evidence of success in the social stream since the idea was to enable them to perform better in terms of social skills.
The Qatari qualifiers said that in several experiments they conducted in class environment and otherwise, looking at the robot’s movements of greeting, shaking and waving hands and sharing stuff, the kids responded similarly with them (Wa’ed and Nour).
Similar other social actions, they said, gave them hope that these kids could really improve in a social setting.
Wa’ed said from the experience she and Nour concluded it was important that the settings were changed in terms of even games because the idea was not just making the kids familiar with one set of things but different things to be able to keep their attention and interest alive.
In response to a query, they said they have been in constant touch with the parents, teachers and care-givers of autistic kids to factor in the whole gamut of autistic children’s responses to different situations to develop the designs they are working on and have actual stats that feed into programs that will eventually help them.
But they said their idea was not limited to just games even if these helped kick start the idea of helping an autistic child. Rather, they had an eye on taking into account physical disabilities that could be overcome as much as possible with technology that is evolving.
This could mean employing better technology and more sophisticated robots, moving away from the current very basic needs to engage the autistic children in a helpful environment.
In an exclusive interview with Gulf Times at the Microsoft campus, Wa’ed and Nour opened up on a host of questions. Their responses have been collated here for clarity.
How did it all begin? Whose idea was it and why choose World Citizenship program category over Games and Innovation?
We had seen children with disabilities, especially autistic kids, and thought about how we could contribute in a meaningful way to help them tide over their basic difficulties. Our professor Uvais Qidwai had tested the same robot with autistic kids before and got a good feedback. Our concept was different; it was the same robot but a different idea.
As for why chose the World Citizenship category, we wanted to be useful to people in need, to come up with a solution that helped autistic children. To improve their lives.
How long did it take for the project to be tabled as a decent proposal?
We started out early this year — it took us about six months.
How did it feel to be one of the teams on which rested the Arab world’s hopes at the world’s premier student technology competition?
It felt good, but this also meant it brought a certain responsibility to work hard as representatives of the Arab world. As well as giving us pride, it also means that we work towards a goal that helps society in that it improves people’s lives, like those of autistic children in our case.
Why do you think there are lesser number of teams coming out of the Arab world at events such as the high profile Imagine Cup?
Frankly, we think it has to do with a general lack of awareness about these competitions; while we knew about it (Imagine Cup) for some time, not everyone does — not even students. There needs to be a greater effort to make this happen particularly in the educational arena.
This does not however mean there is a lack of potential. Surely, there is no dearth of talent going around in the Arab world to spur ideas and make these work.
Another point we want to highlight here is that it is not enough to just have talent. Students need all-round support to be able to come out with creative ideas and develop these.
While we had support and guidance, that is not the case everywhere. Support is key to harnessing ideas.
Does your success signal breaking the glass ceiling — there were six female and just four male students out of the teams that competed at the national level and you two won? Did you face any stereotyping as being told technology is an arena for the boys?
(Laugh) we can’t put a finger on why this is so (females outdoing boys); we were just too busy with our own project. However, we were fortunate to receive complete backing of our families right from the inception of the I Copy You idea to winning the Pan-Arab round of the Imagine Cup and qualifying for the World Finals.
Will it hurt not to win the Imagine Cup? Will you still plan to move ahead with your project?
It is always nice to win, but that is not all there is to it. We were here to present an idea that we are convinced is useful. When we go back home we plan to put this project to good use. We did a lot to get here.
Have you approached anyone to help get your idea off the ground?
We’ve had a lot of meetings with organizations and people to take this forward. We’ve been told this is a very good idea and been assured support. There have been promising interactions with ictQatar, for instance.
Tell us a bit about your families. Who would you credit for being able to get all the way to here?
Wa’ed: My father Adel Jamal Hakouz is a photographer at the Ministry of Administrative Development and my mother Eman Mohammed Ali Hakouz a housewife. I have two school going sisters and a brother, who is a Qatar University student like me.
Nour: My father Abdelmajid Ayoub Musa is a mechanical engineer at the Qatar Airways and mother Nawal Ali Khalil a housewife. I have three brothers — one is a mechanical engineer and the other two school going and two sisters, one of whom I have advised to also target next year’s Imagine Cup.
Both the girls thanked their parents; mentors Dr Uvais Qidwai, Associate Professor at the Qatar University’s KINDI Research Center, and Engineer Mohamed Shakir, Research Assistant at the Qatar University, who also accompanied them at the World Championships in Seattle; Reem al Mansoori at ictQatar; Step by Step (school for children with special needs) Principal Olcay Bilge Connor; Aejaz Zahid and David Banes at Mada (Qatar Assistive Technology Center) and Ruba Hachim, Country Marketing Manager at Microsoft, Qatar.
Wa’ed Hakouz and Nour Musa explain how RoboSapien works and the finer details of their project
Today, humanoid robots are used by different groups for different purposes. Two of the most important places where these robots are of great use are in educational institutions and in ASD children therapy centres.
Most humanoid robots are expensive, or come with specific software or method for controlling the robot. This limits what can be done with the robot and the software or method used may not allow expansion of possible applications.
Using DC and servo motors, the robot (RoboSapien) chosen to implement our project has a low cost and has decent, if limited, functionalities. The graphical toolkit is used to control the robot, using a general purpose microcontroller board instead of the robots’ original microcontroller, freely or in two modes.
One mode is the joystick mode where the actions of the robot can be controlled by a joystick and a mimic mode where the robot mimics the behaviour of the user.
The toolkit enables the control of RoboSapien. The robot is modified by replacing its microcontroller and adding a servo motor for the head so it can be controlled to move left and right. The toolkit gives the user the freedom to control the movement of separate parts of the robot. The toolkit in this solution is built based on the fact that the original microcontroller of the RoboSapien is replaced with the microcontroller.
The microcontroller is provided with a firmware that was built to match the data that will be sent to it through the toolkit. In other words, the toolkit will generate commands based on the firmware that was built and dumped into the microcontroller. The toolkit provides basic robot functionalities such as moving forward and hand movements.
As an application to the developed toolkit, programs will be made to have the robot working or interacting with autistic children. The application will be divided into two modes of operation based on the desire of the user. On the one hand, the user can choose the joystick mode where he or she can control the motions of the robot using the joystick, and on the other, the mimic mode can be chosen where the robot will mimic the actions of the user.
This can be used in ASD therapy sessions where teachers can use the joystick to make the robot perform certain actions which may help in getting the child’s attention, or that child can interact with the robot where it can mimic their behaviour.
Incremental and iterative development process has been used for this project. This process was chosen because it makes sure that by the end of an iteration, a subsystem is finished and obtained. The goals of the iterations are such that firstly, the robot will have enough space to install a new circuit, then the microcontroller and the communication method will be prepared, the motor drivers tested and readied, full software control of the robot motors and motions achieved, the sensors tested and readied for use.
The robot can function in both modes of operation (joystick control or mimic mode).
The system has two modes of operation; in one mode, the user can control the robot using the joystick, and in the other mode, the robot can mimic the behaviour of a child. In the joystick mode, the joystick is connected to the computer. However, in the mimic mode, the motion sensing technology depicts the behaviour and those behaviours are sent to the computer so they can decode into robotic movements.
The computer then sends the commands through a Bluetooth connection to the Bluetooth module; the Bluetooth module transfers the commands to the microcontroller. In the microcontroller, there is a firmware that translates the commands so appropriate motors are turned on or off based on the commands sent by the computer.
Joystick
The joystick is used in the joystick mode. Based on buttons pressed, the robot behaves in a certain way, such as moving forward and raising arms. The pressed buttons are processed by the computer so they can be translated into robotic movements.
Motion Sensing Technology
This is the technology used to get the movements of the child. The output of this is sent to the computer so it can be translated into robotic movements.
RoboSapien robot
This is the robot controlled by the user.
PC or laptop
The PC has the main running program which decodes the commands sent from the joystick or interprets the child’s movements into robotic movements. The program sends the commands to the Bluetooth module so they can be sent to the microcontroller.
Microcontroller
The microcontroller gives the signals to turn on/off motors based on the commands it receives.
Bluetooth module
This is used to connect the computer to the microcontroller wirelessly. It transfers commands from the computer to the microcontroller.
Motor drivers
These are used to supply the motors with appropriate current and voltage levels. They are also used to control the direction of the motion of the motors.
What is RoboSapien?
RoboSapien is a biomorphic, multi-functional robot constructed by Mark Tilden and manufactured by WowWee toys. This humanoid robot has multiple functionalities; it has the capability to perform several movements as a human does. For example, RoboSapien can walk forward, turn left, turn right and pick up objects.
RoboSapien is controlled by an Infrared remote control which has two working modes with up to 67 easy to access commands. It is an interactive robot which can react accordingly with the surrounded environment using its different type of sensors. It has six touch sensors on its fingers, toes and heels; also, it has a sonic sensor to sense sounds when the robot is in listen mode.
The following table shows RoboSapien specifications. Also, it offers the users and developers the ability to program its sensors in the way they want. And one of the reasons that makes RoboSapien suitable for this project is that RoboSapien is a low cost, affordable humanoid robot. In addition, RoboSapien is designed in a way which makes the hacking and modification process easy and doable.
Height: 34.3 cm
Weight: 2.27 kg
Length: 31.75 cm
Width: 15 cm
Remote control Batteries: 3 x “AAA”
Body Batteries: 4 x “D” size Alkaline
Motors: 7 DC Motors
The microcontroller board is designed based on the ArduinoTM open-source hardware prototyping platform. This microcontroller has a 32-bit MIPS processor core running at 80 MHz, 128k of flash program memory and 16k of SRAM data memory. The Multi-Platform Integrated Development Environment is a developed version of the original Arduino. The board can be powered by external power source or via USB serial port interface which is used to connect the microcontroller with I/O pins provided and supports different peripheral functions like UART, SPI, I2C ports and PWM output.
In this project, the Bluetooth module is used to transmit data between the toolkit and the microcontroller. This Bluetooth module has an encrypted connection.
With the usage of this motor driver, one can control up to 2 DC motors at a constant current of 1.2A. This driver can provide a Pulse Width Modulation (PWM) output to control the motor’s speed if needed up by two pins (IN1 and IN2) for 4 modes: Counter ClockWise (CCW), ClockWise (CW), short brake and stop.
The programming language used here to build the toolkit is G-language, which is a dataflow programming language. Execution is determined by the structure of a graphical block on which the programmer connects different function-nodes by drawing wires. These wires propagate variables and any node can execute as soon as all its input data becomes available.
—Info provided by Engineer Mohamed Shakir, the mentor of the Qatari team