Improving human-robot interactions

Industry 4.0, the Smart Factory – however you refer to the development of more connected factory floors, the use of robots in shops is on the rise and will continue to grow. How we program and interact with those robots should also become simpler than it currently is. A lab at University of British Columbia (UBC), in partnership with DLR Space Administration, Germany’s space program, is doing its part to simplify these interactions in part through the use of augmented reality technology.

Advances in robotic programming by the likes of KUKA and ABB have simplified the use of robots in welding operations particularly. Shops have gone from point-to-point construction of welding programs on the factory floor to being able to program welds in the office, saving time on the production line.

But as UBC graduate and lab researcher Sarah Li noted, programming a robot is still very difficult. “This is particularly true for large robots, which you can’t really just move from position to position with your hands,” she said. “With many of these robots, to get them to do any simple thing, you usually need to learn how to do inverse kinematics with the robot and then you move it around with a controller. That controller might have 40 buttons on the teaching pendant that you have to work to make the robot do something simple like picking up an object.”

With technological developments that have occurred over the past few years, researchers and companies have started looking at simpler ways to interact with robots. The question that Li believes is key is, “How does a machine shop operator who isn’t skilled in programming learn how to program a robot?” The UBC lab she worked in this year sees this as the ultimate goal – making programming easy enough that anyone can do it.

This is where the Microsoft HoloLens technology comes into play. The HoloLens is an augmented reality technology. It projects a hologram into real space and allows you to interact with that object in different ways.

“You can use both speech and gestures to interact with the hologram,” said Li. “That allows an employee to more intuitively say, ‘Pick this up and put it there,’ and the employee can then give it all the spatial parameters to say exactly where ‘here’ and ‘there’ are.”

The UBC team has been looking at human-robot interactions for some time, asking questions like, “What is the natural way to interact with a robot?” One PhD student is looking at gestures, examining how some robot gestures are perceived by humans, and how human gestures are perceived by robots.

“Another thing that the lab is looking at is how do you hand things from a robot to a human?” said Li. “For humans, it’s very natural; we know exactly how to hand an object to another person without thinking about it, but there are a lot of subtle cues in what we do. For example, you wouldn’t put something right in someone’s face, nor would you push something right into someone’s hands. Usually you are a certain distance from that person and you pass it halfway and look at them, and through eyes and head movement we signal how we are going to hand it over. It is very natural, but when it comes to robots, we have to think that through completely. How do we do it? And, for instance, where do we put a camera on the robot if there is a camera on it? Where a person’s head might be? How would people react to that?”

The UBC researchers want to use the HoloLens’ augmented reality technology to push into “mixed reality” work situations.

“Mixed reality has your hologram interacting with the real space around it, and vice versa,” said Li. “Industrial robots can be big, intimidating objects. They are scary when you push the wrong button and it does something you don’t want it to do. With mixed reality, before you send any commands to the robot, you can do a simulation of what it’s going to do in real space in real time. For us, that’s important because it plays into the safety features of the industrial robot.”

The partnership with the DLR Space Administration is looking at automating the movement of composite layers on parts of a spaceship. Its application on shop floors in the future could be very helpful in making the integration of robot technology less intimidating and the programming of those robots much quicker.

To learn more about the research work being done at UBC, visit the lab’s website at http://caris.mech.ubc.ca/.