Above: Working with a group of local entrepreneurs and other UAB research teams, the ETLab is developing sensitive motion-capture devices that could allow patients to safely practice physical therapy exercises at home. An early demonstration is shown above.

You can see the future in UAB’s Enabling Technology Lab. Lots of futures, in fact, from the highly probable to the wildly speculative. Between the Google Glasses and Oculus Rifts, behind the vials of fake blood and $40,000 motion scanners and cutting-edge smartwatches and the disco balls, you will encounter virtually every promising device that has generated buzz in the tech world over the past decade – and plenty of overhyped duds as well.

Even more exciting, though, are the custom-built projects coming out of this corner of the UAB School of Engineering’s Department of Mechanical Engineering. Pick up an iPad and you can play an action game designed to train security staff at one of America’s high-profile terrorist targets. Another game, designed for harried emergency room doctors, teaches cutting-edge strategies to save patients from the aftermath of heart failure.

Miniature sensors, created by UAB’s Engineering and Innovative Technology Development (EITD) group, are being connected through code that can read and react to subtle limb movements, potentially creating a paradigm shift in physical therapy. The technical breakthroughs developed here are powering a Birmingham startup – not the first to emerge from the lab. And the technology could eventually power a battle simulator for the U.S. military that achieves unprecedented realism.

ETLab member Eva Dennis at work on one of the team’s digital creations.

The common link in all these projects, and the true goal of the ETLab, is to find new ways “to pump information into a person and out of that person,” lab Director Corey Shum said. “We want to close the loop, to create a world that reacts instantly to you.”

Shum is the Astro Teller of UAB – equal parts inventor, entrepreneur, futurist and technology evangelist. Like Teller, the famous leader of Google’s moonshot-launching X lab, Shum is a big believer in projects that sound like science fiction.

They all start with a question, from a clinician or researcher with a problem, or from Shum himself: What if we could use the much-maligned Google Glass to help doctors see through the charred skin of burn victims, quickly separating the parts that can be saved from the sections that are dead? How about equipping drones with 3-D cameras and using them as advance scouts for security teams and other first-responders?

“We’re not interested in building cool toys just to build them – although we do really like building things,” Shum said. “We don’t want to invent something that already exists. We want to do really neat things that haven’t been done before – and make a difference.”

Shum describes the ETLab crew as a “cross-functional A-team.” It’s a group that includes programmers, 3-D artists and software developers. (Meet the members of the team in this UAB Magazine article.) “We’re not looking for expertise in a particular coding language or anything like that,” Shum said. “Things change too fast. It’s more important to be the kind of person who is ready to learn something new, someone with that gleam – someone who wants to build the future.”

The ETLab team pilot-tested a new security training game for a high-profile government facility with this intricate re-creation of a Paris street scene.

Here are three projects Shum’s team is working on – and some way-out directions they’d like to explore next.

1. Outthinking the enemy

Highly visible government facilities are a natural target for terrorist attacks, which means that security and protection teams need to be on their toes at all times. “Our major project last year was a series of security training apps for a high-profile facility here in the United States,” Shum said. (He is not yet at liberty to publicly divulge its location.)

The ETLab team crafted a highly detailed model of the facility, and the steps that visitors and staff must take to enter. Their simulation can be projected in a room-sized 3-D simulator, like the ETLab’s VisCube; but it is designed to be used primarily on tablets or smartphones. New security officers play through the “game” over and over again, learning the correct responses to common situations – and dangerous scenarios as well. The facility’s administrators “want to train people, but they don’t want to train them on the job,” Shum said. “They need to have zero mistakes.”

Shum’s team traveled to the site, taking pictures and accessing architectural plans to create a “high-fidelity” version of the facility. “We built this app to the specifics of the location, with all the equipment looking exactly the same as it does when you’re there,” said Shum.

Like “Grand Theft Auto,” this is an “open-world” game, which means players can do whatever they like. But in the game’s tutorial phase, they are shepherded through the entire security process, “and it grays out everything other than what you should be doing,” Shum said. “We’re just trying to burn those neural pathways.”

In the next phase, players must remember the correct steps to take, but the game “doesn’t let you do anything wrong,” said Shum. “You’re trying to recall what to do, but it refuses to let you go down bad paths and learn those.”

Finally, players are allowed to do whatever they want. “After you’re all done, it tells you what you did right, what you didn’t do at all, and what you did out of order,” Shum said. “It gives you a grade, but more importantly it gives you a qualitative assessment of what you forgot.”

After successfully delivering a phase-one version of the training app, which is now in use at the facility, Shum’s team is expanding it into a “complete cause-and-effect simulation,” he said. “That means if an adversary plants a bomb and it explodes, it may knock out the water main, and knock out the computers, so when you go over to the computer to report what’s happened, as you’ve been trained to do, that computer isn’t working.”

The goal is to build a “completely dynamic system where we can do team play,” Shum said. “One group is trying to break in or otherwise cause havoc, and the other is trying to block them.” The game would also allow “competing-hypothesis testing,” Shum said, “where you can sit there and say, What if somebody did this, what would happen?” That would help the facility’s security forces get “inside the aggressor threat curve,” said Shum. “That’s their goal.”

One step beyond: Drone power

By combining depth-sensing cameras and highly maneuverable, inexpensive drone copters, security teams and first-responders such as firefighting crews could eventually build simulations on the fly, Shum says.

Imagine you’re a firefighting unit about to enter a burning building. “You could send in drones or robots to see what’s there, and then virtually run into the building and do a dry run with a head-mounted display before you actually do it for real,” Shum said. “Our goal is to bring these things together.”

2. Re-creating a three-day sprint

A cardiac arrest brings two rounds of injury. The first is when blood flow is shut down throughout the body. The second is when the flow is suddenly turned back on. “It’s like when the water company accidentally breaks a water main,” said Henry Wang, M.D., vice chair for Research in the UAB School of Medicine Department of Emergency Medicine. “The first inconvenience is when the water gets cut off to your whole house. The second inconvenience comes after they fix it – dirty water comes through the pipes for hours.” That same thing happens inside the body, Wang explained. “When you restart the heart, the ‘crud’ of cellular reperfusion damage flows through the blood vessels, injuring the brain, heart and other organs.”

That lack of blood flow, followed by an influx of detritus, often leads to brain damage. But emergency physicians such as Wang now have a powerful weapon to fight back: a technique known as therapeutic hypothermia. “It consists of dropping the body temperature down to around 90 degrees Fahrenheit to help the body recoup and prevent brain damage,” Wang said. UAB has been using therapeutic hypothermia for more than five years, with great success, he adds.

But it isn’t an easy technique to learn. Practitioners must choose between up to four different machines and a range of medications to achieve safe cooling. The right choice depends on a number of factors, including a patient’s body weight. “Someone who is overweight is very hard to cool,” Wang said. “And because it’s a relatively new technique, there’s not a lot written about therapeutic hypothermia in textbooks.”

Wang sought out the ETLab, looking for a new way to teach UAB’s well-tested cooling protocol to trainees and new hires. “This isn’t a graphically intensive simulation – it’s just a series of decisions that happen over 72 hours,” Shum said. “And the decision you make at 8 in the morning may not be felt until tomorrow afternoon. There is no feedback there.” So Shum and his team designed a proprietary formula “that lets you speed up time in a repeatable way,” he explained. “The idea is that doctors and nurses can put this on their cellphones or tablets and just do continuing education,” said Shum. “They can re-create a process that takes 72 hours in 12 minutes, making these decisions and seeing how their curve of temperature variance over time matches the ideal. And then do it again and again, and do it on their own schedule with a patient who isn’t going to suffer any consequences.”

That may not sound difficult, but it is actually very complex. “Corey has invented the secret sauce that makes the time-lapse happen in a consistent way,” Wang said. “If you don’t do it in this special way, you’ll get very different results depending on how fast you do the time-lapse.”

Shum and Wang are now refining the simulation, including adding more scenarios. They are also exploring partnerships with makers of medical education software. And the underlying code could help train practitioners on other time-sensitive procedures, Wang noted. “This idea can be generalized to a whole bunch of health care applications; particularly applications where time-lapse education could be really helpful.”

3. 

   No interest in reinventing the wheel: “We want to do really neat things that haven’t been done before – and make a difference,” said ETLab Director Corey Shum (above, with UAB student Hannah Hellwig).

   A video game that helps you heal

By now, there’s a good chance you’ve heard of the Fitbit, a digital-age pedometer that tracks your steps – and sleep patterns. You may well be wearing one yourself. Shum is working with two local entrepreneurs, and the Engineering and Innovative Technology Development (EITD) group, to create a hyperaccurate Fitbit that doesn’t just watch you exercise – it makes sure you’re doing it right. It could also transform the process of rehabilitation after an injury.

Orthopedic surgeon Ken Jaffe, M.D., a 1982 graduate of the UAB School of Medicine and a former UAB faculty member now in private practice, was inspired by the high-tech motion-capture devices in the golf industry. “In the golf world, they have sensors you can put on your club to look at your club head speed and swing path,” Jaffe said. “I had the idea of using this technology for medical rehabilitation.” He also knew where to go for help turning the idea into reality. “I was introduced to some of the talent at UAB when I got my MBA” at the UAB Collat School of Business in 2003, Jaffe said. A former faculty member in the business school, Sanjay Singh, Ph.D., connected Jaffe and his business partner in the startup Management of Motion to Shum and the ETLab.

“They had the idea and they have business connections for commercializing it,” Shum said. “They came to us for a technical solution and the potential for schematics that they could then use to make a bunch of these devices.”

After an injury, patients are prescribed physical therapy exercises to help restore function, Shum explained. “It’s very important that you do these exercises correctly, so patients have to visit the therapist’s office several times a week to be evaluated.”

But if patients could instead take home a high-tech sensor vest packed with a number of motion sensors, they could get immediate feedback on their exercise performance. “You put these sensors on and then you do the exercise, and it’s a game – it tells you when you’re doing it wrong, and when you’re doing it right,” said Shum. “Then it uploads that information to a server so the clinician can review it and see how well you did. Instead of driving into town, you’re at home looking at a laptop. Again, we’re closing that loop quickly so that you’re not at home doing the exercise wrong and getting worse; you’re at home doing it right.”

Sports medicine specialists – and video game makers looking to re-create elite athletes’ moves – currently use ultrasonic technology for accurate motion capture. “But you have to be in a specially designed space, and it’s very expensive,” Shum said. “Those systems cost tens of thousands of dollars, and you’re not going to send that to somebody in a FedEx box.”

Engineers in the EITD are adapting off-the-shelf accelerometers, electronic gyros and magnetometers to build the sensor vest, and the ETLab is combining the data captured by the vest with graphics software to present this feedback to users in an intuitive way. “That’s one of the wonderful things about being here at UAB,” Shum said. “We can work with the folks at the MPAD (UAB’s Materials Processing and Applications Development lab), and EITD can fabricate the actual materials to NASA quality.”

The current system works well as a proof-of-concept test, with the sensors in small 3-D-printed boxes. “The goal is to get that sized down to something more like a Fitbit, and then maybe even smaller so you can have several of them in an article of clothing,” Shum said. “In the next phase, we want to get it to a place where it’s as small as it needs to be and all they need to do is send the schematics off to be mass-produced.”

Breaking down walls: The ETLab has specialized in creating highly realistic 3-D simulations in its VisCube for clients such as Air Force pararescue teams (above). Sensor vests, wireless 3-D goggles and other advances could soon bring these simulations out into the real world.

One step beyond: Simulation in the real world

Several years ago, the ETLab designed a hair-raising 3-D simulation to help prepare U.S. Air Force pararescue teams for the terrors of battle. (Hence the vials of blood – and the disco ball. “We put in everything we could to disorient them,” Shum said.) With the development of highly accurate sensor vests and 3-D goggles like Microsoft’s new Hololens, Shum is close to realizing his dream for the next step in that simulation: taking it outside.

“We want to create an immersive, team-based training system that’s field-deployable – as in, you can do it out in a field, not in a lab,” Shum said. Accomplishing that goal, without having someone run into a real-world tree, requires plotting the positions of users in extreme detail. That’s where the sensor vests come in. In addition to positional sensors, the vests could also include biometric gauges such as heart rate and skin conductance. “We discovered in the first phase of our military simulation that it’s all well and good to put stressors on people and use their own self-reports on how they reacted,” Shum said. “But this would let us understand what’s happening at another level.”

Closing the Loop

Shum’s office is a shrine to the latest in technology, but it’s also a graveyard of failed silicon dreams. He’s under no illusion that every hot new object will take off. Will Microsoft’s Hololens beat the Oculus Rift in the coming virtual-reality wars – or will Magic Leap overtake them all? Who knows? But in order to keep up, he explains, you have to be willing to jump in and explore. “What we’re trying to do is assume that all these guys are going to succeed at what they’re doing,” said Shum, “and have applications ready when people at UAB need them.”

He’s also busy recruiting for his own team of futurists. “I’m not looking for people with a specific skill set – who know a specific programming language, for instance,” he said. “I’m looking for people who are interested in the future. And I want to make sure those people want to come work at UAB.”