Bay Area engineers pack AI into smart hardware gadgets

At the Fort Mason workshop, which overlooks the Golden Gate Bridge, most workers rarely look up to take in the spectacular view.

Instead, on the second floor of the Founders technology incubator, more than a dozen young inventors are hunched over lines of code and shoving wires into circuit boards, trying to build the next generation of amazing electronic gadgets powered by artificial intelligence.

Littered with wires, industrial drills and battery parts, the lab is dotted with devices in various stages of construction or repair, from smart goggles that can interpret the world for the blind to portable devices that could help humanity survive an end-of-life scenario.

Far from being software chatbots that can converse like humans, answer questions or create images out of thin air, these experiments are efforts to pack AI into hardware, creating smart devices that interact with the world around them in ways never before possible.

Starting in mid-April, Founders, a company that finds and funds promising innovations and startups, invited more than a dozen technologists from around the world to its spacious hardware lab at Fort Mason for six weeks, with the best ideas potentially receiving a six-figure seed investment.

It’s no wonder entrepreneurs and investors are chasing this opportunity: Market analytics firm Precedence Research estimates that the market for AI-enabled electronics has already topped $50 billion this year and could reach nearly $500 billion within 10 years, including use in computer chips, storage, networking and other end markets.

Some early attempts at AI-powered consumer devices, such as San Francisco-based Humain’s Ai Pin, have not taken off, likely because they were pushed to market too early. Safwan Khan, head of capital at Founders, is looking for innovations that are here to stay.

“It’s not just about assembling hardware,” he says. The training encourages builders to learn from the smart people around them and solve problems faster.

One of the inventors still trying to work out the kinks in his machine is Shubh Mittal, who wears thick black glasses with a camera mounted on the bridge of his nose and a broad smile. He’s easy to spot.

Combined with AI programs like OpenAI’s ChatGPT and Anthropic’s Claude, Mittal’s camera can scan images and text and provide a computerized voice description in about three seconds. “The idea is to help the visually impaired become more independent by using the glasses to read menus and describe their surroundings,” Mittal said.

But making complex AI software work well with evolving hardware can be extremely challenging.

For one thing, when Mittal recently demonstrated the latest software update to a crowd — his traditional Friday ritual to highlight the week’s progress — the results were mixed.

Mittal asked the glasses to identify a pink stuffed elephant in front of them, but at first there was no response. Eventually the program denied the elephant’s presence and instead depicted a forest scene that had nothing to do with the hardware lab.

With some embarrassment, Mittal says that moving his head blurred the image and confused the program, but the glasses then painted a surprisingly accurate picture of the lab, pinpointing which areas were research and which were industrial.

Founders Lab and Mittal aren’t the only ones working on smart glasses. Engineers at the SRI International Research Center in Palo Alto are investing millions of dollars in government grants into head-mounted AI technology that could one day help untrained soldiers heal wounds or fix broken engines on the battlefield. The goal is to create a headset that uses AI to walk users through procedures with visual displays and audio instructions.

Those plans are still a ways off, for now this will help me make lunch.

SRI researcher Bob Price recently donned a Microsoft HoloLens headset connected to a bank of computers and demonstrated how he uses the device to teach people how to make tortilla wraps.

The headset requires users to press virtual buttons in the air projected on a head-up display, and a robotic voice guides the user through the process, while a built-in camera can reliably detect when a tortilla has been spread with peanut butter and jam, rolled, and sliced.

Project director Charles Ortiz said the hardest part will be training the system to recognize mistakes and respond appropriately: Even a user’s hand getting in the way can confuse the machine.

Another project is underway in a lab a few miles away at Stanford University, where Gordon Wetzstein, an associate professor of electrical engineering who runs the university’s Computational Imaging Lab, and his team are working to perfect a pair of glasses that will display 3-D images through a lighter, slimmer frame than the cumbersome HoloLens.

Wetzstein used algorithms and laser etching to model how light projected from a side-mounted light source bounces around the lens and into the user’s eye, creating a 3D holographic effect.

Wetzstein hopes that future AI goggles will look more like his lighter device, which uses optical tricks to create augmented reality, rather than being headsets that project flat images.

“The convergence of AI and hardware really enables new things,” he said.

While glasses and headsets are an obvious early use for AI hardware, Founders Lab engineers are pursuing more esoteric opportunities.

Engineer Adam Cohen Hillel is tweaking a product he calls “Ark,” a handheld device designed to help users survive Armageddon.

Assumes that networks and communications will be disrupted, and Ark comes preloaded with maps and uses Meta’s Llama 3 AI engine to tell users where to find water, shelter, how to administer first aid, how to start a fire, and more.

For now, it’s just a screen on a circuit board with a stylus attached, and Cohen-Hillel jokes that it resembles a makeshift bomb.

During a demonstration in front of the group, the map ran smoothly, but the voice chat feature was down. Cohen-Hillel hopes to one day do away with the stylus and run the device solely through voice commands, add solar panels to power the Ark when the power grid goes off, and even add a special casing to help it survive the electromagnetic pulse of a nuclear bomb.

“The hardest part is packing so much data onto a small board,” he said. Making it run fast enough is just as difficult.

But if they actually work, devices like Mittal’s and Cohen-Hillel’s offer a glimpse into the future.

It’s easy to reach that “magic moment” when something works, says Hubert Thieblo, a general partner at Founders Inc. “It’s much harder to get something to work every time.”

Contact Chase DiFeliciantonio at [email protected]; Twitter: @ChaseDiFelice