Today’s smart home is a reality thanks to the iPhone

Today’s smart home is a reality thanks to the iPhone

The iPhone as the “Ultimate Digital Device” as coined by Steve Jobs in 2007 became much, much more than even he could have predicted. Today, the iPhone (and modern smartphones) has powerful apps, voice control, proximity automation, the list goes on. And all of those features, based in a hand-held device, have largely solved the decades-old challenges of creating a (real) smart home.

“Hey Siri, good night!”

That’s what I say to my iPhone every night, right before bed. And while that simple command executes a few other commands that I’ve hacked together outside of Apple’s walled-garden of HomeKit, it starts within the realm of Siri and HomeKit. Long ago, I wished for a way to use my voice to control the devices in my home. But voice control was far more difficult to set up back then compared to today’s voice assistants like Siri and Alexa. And, location-based control or having the ability to precisely control the brightness of a lamp from a pocketable device were essentially impossible.

Pushing Many Buttons

An X10 plug-in module, wireless controller and the JDS TimeCommander central X10 controller

Before the iPhone—and to be specific—smartphones with useful, Internet-connected apps—the choices for consumer, DIY-style home automation were pretty limited. The ubiquitous X10 standard, developed in the 1970s and eventually marketed by Radio Shack and Sears with plug-in devices, master controllers and the like, was good but had reliability issues and almost no security. Later protocols like Zigbee (1998), Z-Wave (2003) and Insteon (2005) started to make their mark on the smart home in the 2000s with more reliable, secure communications technology that could work with both line-powered and battery-operated devices. And while these systems improved the experience of setting up and operating a smart home, there was still the issue of controlling it easily.

Control of the “smart home” in those days was limited to wired and wireless push button controllers, manual device control, or, if you got fancy and spent the time to set them up, PC-connected controllers or “macro devices”. In the mid-90s, I was frustrated by “manual” X10 control of my lighting, so I acquired a JDS TimeCommander to automate the lights in my half of my college dorm room…much to the amusement (or was it bewilderment?) of my roommate and other dorm residents. (I even got the controller to perform music-synchronized light shows, but that’s another story altogether.) But even with my “advanced” level of home automation, it was still pretty awful. Macros didn’t work all the time, commands were missed, or everything would “lock up” and had to be completely reset. And I was still triggering scenes and other actions with push buttons (and sometimes, schedules). Voice control or precise control from remote were out of the question.

The Sci-Fi Influence

A "PADD" from Star Trek. Image from Paramount Pictures / CBS Studios

A "PADD" from Star Trek. Image from Paramount Pictures / CBS Studios

I was an avid viewer of Star Trek: The Next Generation growing up, and I was always fascinated by the portable “PADD” devices often seen on the show, along with the incredibly interactive computer that would respond to voice queries that started with the word “computer”. I always wanted a way to control my electronic devices and/or computer(s) via voice, and to have some sort of “ultimate remote control” that resembled a PADD that could fit in my pocket. In the 1990s, the gadgets that closely represented the Star Trek device were the Apple Newton and Palm Pilot series of PDAs. Of course, they had no Internet connectivity…which no one really complained about back then. So neither of these PDA devices could help me automate my home, since they had no wireless connectivity.

Limited Remote Control

Even if you had just wanted web-based control of your “smart home” devices in the 1990s with X10 products, this was pretty difficult. Most of the solutions required you to have some sort of server setup on your PC or Mac, which at the time was usually a pretty involved project. Of course, as a web developer back then, I setup some basic access to control and get the status of my X10 devices. In 1998, with a PC with an early DSL connection, I had interfaced my home automation controller via its serial port to my PC’s web server executing Perl scripts. The system didn’t work right all the time, but it was pretty cool…and I could annoy my housemates from anywhere in the world, assuming I had access to the web.

For voice control, this was equally challenging. While Microsoft built speech recognition into Windows 95, it performed terribly. Apple was working with speech for a long time, and even had built it into “classic” Mac OS in the early 1990s but it had to be setup specifically by users during OS install. Some companies, like Dragon (who makes NaturallySpeaking), released their own standalone software packages for speech. But unless these were paired with automation systems, they weren’t necessarily useful for for controlling your lights and appliances.

HomeVoice software. Image from HomeToys.com

One company, Applied Future Technologies, created HomeVoice, a speech recognition system that could execute commands and macros on PC-connected automation controllers, like my JDS TimeCommander. AFT even sold microphones disguised as standard toggle light switches. While they couldn’t provide voice control from a portable device, you could place their mics around your home, wire them back to a mixer and feed that combined mix into your PC’s sound card. If properly setup, you instantly had whole-house voice control of your home automation system. Certainly not as easy as plugging-in an Amazon Echo Dot in every room of the house! (And none of these speech systems improved automatically like we’re used to with today’s modern voice assistants, you usually had to wait for a software update to arrive on disk or CD…how medieval!)

Wireless Communications Proliferate

Alright, let’s fast forward a bit to 2006, right before the iPhone was announced. Two technologies had blossomed since the 1990s, Wi-Fi and Bluetooth. Wi-Fi had made its mark on notebook computers and some desktops, and many smartphones of the time had Bluetooth and some even had limited Wi-Fi. Users were getting acquainted with wireless earbuds and headsets for making calls…even though we all looked like we were talking to ourselves with those things! But largely, aside from Wi-Fi routers and access points, there really weren’t many “headless” (no display) Bluetooth or Wi-Fi devices…home automation or otherwise. There just wasn’t a reliable way to control those kinds of things via a convenient means. It’s the complete opposite of today where we configure so many devices with our phones.

“These are not three separate devices”

Steve Jobs introducing the iPhone. Image: genius.com

Steve Jobs introducing the iPhone. Image: genius.com

And then, at 9:41am PST on January 9, 2007, Steve Jobs began the introduction of the iPhone at Macworld. The rest, as they say, is history. When I saw the iPhone keynote, I literally lost my mind. I re-watched the iPhone portion of the Macworld event many, many times. I could not wait to get an iPhone, despite the high initial price. (I actually didn’t get mine until October of that year.) But while the original iPhone was amazing, it was still limited by the fact that it had no native apps. I, of course, knew that the potential of the device was nearly unlimited. In 2008, Apple switched away from its “web apps” idea and introduced the App Store. Many apps were free, some were just 99 cents. Apps like Shazam brought us the ability to identify a song anywhere, just by having our iPhone “listen” to it. Facebook’s mobile app began the social network’s rapid rise to becoming the default social network. And any home automation company with the least bit of foresight began working on plans to launch an app for their platform.

The ability to have a device in your pocket that you could use to control your home’s lighting, appliances and climate control, from potentially anywhere in the world, was novel at the time but would become the norm in just a few short years after the iPhone’s release. But in 2008, there were few Internet-connected home automation platforms. And for those that did exist, they generally had rudimentary web-based interfaces. Smart hubs like SmartThings and Wink, and today’s familiar products like LIFX and the Ecobee thermostat were either engineer’s ideas or not even thought of yet.

Home Control Debuts On The Small Screen

Indigo Touch on iOS

One of the first home automation control apps for the iPhone was Indigo Touch, made for the Indigo home automation platform on Mac OS X. Indigo is an excellent home automation server for the Mac that is plugin-based. With it, you can automate nearly anything. And if there isn’t an existing plugin for it, there’s a framework available so you can create one. I had read about Indigo for years, but hadn’t made the switch to the Mac platform until late 2009. After jumping on-board the Apple train, I’ve been an Indigo user ever since. It helps, of course, that Indigo has tight integration with Insteon devices, which I’ve used extensively since 2006. Indigo let me have far more control and automation over my Insteon lighting and appliance controllers, and gave me (and many others) true remote control of my devices for the first time via Indigo Touch.

Entrepreneurs and companies, both tech and non-tech-related, started to see the immense benefits of consumers carrying always-connected devices with them at all times. At the same time, the rapid miniaturization of powerful technology and chips—thanks to products like the iPhone—enabled home automation products to gain new capabilities via tiny hardware that could be integrated while keeping to an acceptable product size. (This is, if you’re wondering, the same reason why consumer drones “became a thing”, seemingly out of nowhere, a few years ago.) At the same time, protocols like Zigbee and Z-Wave were being enhanced, and their products were getting more powerful with more capable chipsets and wireless controllers. In 2010, the insanely-popular iPhone 4 featured a GPS chip for the first time, ushering in the personal-geolocation era. And in 2011, the iPhone 4s launched with Bluetooth 4 (Low-Energy), giving the green light to device makers wanting to create low-power devices. (Apple’s inclusion of nearly any technology tends to ensure its success and proliferation.)

And I can’t leave out modern voice control, right? The first cloud-connected, relatively-usable virtual assistant debuted with the above-mentioned iPhone 4s. It had sarcastic responses and could perform various tasks like getting the weather or booking a dinner reservation, and even had some contextual abilities, too. But because of how Apple usually implements new technologies, it had zero abilities beyond what Apple gave it. And home automation control was not included, unless you count control of your music via your iPhone. It would be several more years before Apple’s chatty assistant could control connected-home devices.

The IoT Revolution Begins

It certainly didn’t take long for these individual innovations to coalesce into what we now call “The Internet of Things”. Nest launched its Learning Thermostat in 2011, Philips Hue color-changing lighting debuted in 2012 and crowdfunded favorites like the SmartThings hub and LIFX smart bulbs hit the market in 2013. And in 2014, Apple announced HomeKit at its annual WWDC event, promising to bring integrated home automation to iOS.

The Nest Learning Thermostat

Nest was co-founded by “the father of the iPod”, Tony Fadell and was founded in 2010 after Fadell left Apple in late 2008. The Nest Learning Thermostat debuted in 2011 with a familiar round shape and innovative round screen with a stainless steel outer ring that users would turn and push to interact with the thermostat. The Nest had always-on Wi-Fi and communicated to Nest servers instead of directly to a smartphone app. This constant connection, along with sensors previously unseen on thermostats enabled features like Auto-Away for automatic energy saving, and monthly e-mail reports to let users easily track their usage and see how they rank compared to others in their area.

Philips Hue bulbs gave both precise and fun lighting control to anyone with an Internet connection and an iOS or Android smartphone, without re-wiring their home. The white and color-adjustable bulbs could work with other apps to sync the lights to music, TV and to automatically adjust based on time, user location and more. The lights use Zigbee and commands are translated via the Phililps Hue Bridge, an Internet-connected device which now connects with nearly every home automation device and platform on the planet.

SmartThings empowered consumers to control and automate “all of their things”, regardless of brand or whether they were Zigbee, Z-Wave or Wi-Fi devices from an always-connected hub with a relatively-easy-to-use app. Many, many devices would become compatible with SmartThings, and power users tend to love the platform for the SmartApps that can be written to extend the hub’s functionality. SmartThings would eventually launch a line of its own smart devices, including sensors, a second-generation hub, and later be acquired by Samsung in 2014.

Apple Goes “All-In” On Home Automation

Apple's Home app on iOS

And I couldn’t round out this history lesson without HomeKit. Apple’s HomeKit promised connected-home control right from within iOS, bringing home automation to nearly every iPhone user. While it officially debuted with iOS 8 in 2014, there were no HomeKit devices available until mid-2015. Apple had very strict requirements for HomeKit devices, including a special authentication chip. Thus, companies had to launch entirely new devices, or add HomeKit functionality via special smart hubs. A mixture of devices started to trickle out, initially as upgraded products with hubs, like the Lutron Caseta lighting control system or the Insteon Pro hub.

But HomeKit didn’t really gain traction until iOS 10 in 2016, when Apple debuted its own Home app that was included in the mobile operating system. Along with its straightforward app, easy-to-access home device controls were added to Control Center, so with a few swipes and taps, your home would respond to your every whim. HomeKit would bring powerful yet simple home automation to anyone with a recent iPhone. The mere presence of the Home app on the iOS Home Screen staring back at users could entice them to try out a home automation device for the first time. And people are generally delighted when they enable automations such as “Arrive Home” and their lights, thermostat and other parts of their homes respond like magic. Once you automate, you usually never go back.

Control of smart home devices in Apple’s ecosystem now extends from the iPhone to Apple Watch to Apple TV, with a generous buffet of apps to augment functionality that isn’t available in the default Home app. Of course, command of your home is possible through the omnipresent Siri virtual assistant—although she’s still not great at understanding what you want her to do. Many times these days, she still thinks I want to turn the lights to “cream” when I say “green”. Dearest Siri, why do you still hate me?

The ecosystem of HomeKit devices and categories has grown considerably since 2015. And Apple’s new option for devices to authenticate via software instead of the previously-required special chip will undoubtedly vault the number of HomeKit-compatible devices into the stratosphere. This is ultimately good news for consumers, as Apple has started to face fierce competition in recent years from other home automation “assistants” like Amazon’s Alexa and the Google Assistant (which have less stringent policies for device integration.)

What A Difference A Decade Makes

And speaking of services like Alexa, which run on a device like the Amazon Echo… Today, we largely use smartphones to configure what I earlier referred to as “headless” devices. In fact, many of us do it all the time. Setting up a Z-Wave smart lock? There’s an app for that. Change the backdrop image on your Chromecast? Use the Google Home app. Adjust the schedule on your Thread-enabled toaster? And the list goes on. Nearly every device in today’s smart home, no matter what language it speaks, is configurable via an app. No more clunky web interfaces, odd combinations of buttons to push or bizarre sets of blinking lights to analyze…just power it up and your phone (maybe via your smart hub) should see it.

Many of today’s smart homes are largely built around control via our smartphones, along with other devices that became popular after the iPhone’s success…like modern tablets. Our homes can begin adjusting climate control, music and blinds/shades when we’re “on-approach” to arriving home. Lights and appliances can detect our presence in a room and adjust automatically via Bluetooth. If we’re sitting on the couch and the connected wall switch is just too far away to reach, no problem. We just use our voice, talking to a smartphone or one of today’s “voice-first” devices like Google Home. And, if the voice control fails or it’s not appropriate at that moment, don’t fret! There’s an app for that, probably on your phone and potentially even on your wrist. No matter what you need to adjust in your home, there’s a way to do it precisely and easily…from anywhere. My my, how times have changed.

The Future

But with all of this control and automation seemingly already being accomplished, where does the smart home (powered by the iPhone) go from here? It gets even smarter.

AI

Companies like Vivint are introducing artificial intelligence to home control, meaning that your home will start helping you create automations you didn’t even know you needed…but without being creepy about it. And with Apple’s current pursuit of making Siri more useful and intelligent, while focusing on user privacy, it’s only a matter of time before that intelligence finds its way into HomeKit. Because as great as HomeKit is, there’s still the newbie question of “I have these lights and things, but what should I do with them?” If Siri can ultimately suggest ideas for automations, and then actually perform them based on your consent, that will make for a truly smart home.

AR

Augmented reality is still a new topic, but the iPhone is about to let hundreds of millions of people dive right into the world of AR. (And not just a new version of Pokemon Go.) With the new ARKit that’s part of iOS 11, IKEA is planning an app where you can see how some new furniture might fit into your humble abode. Other developers could take this a step further and let you visualize how smart home devices could automate your home. You could potentially preview how new lighting would look, with adjustable scenes based on new and existing lights in your home. And perhaps see if that smart lock you’ve been eyeing would actually fit on your front door.

Some people, myself included, are convinced that augmented reality is more useful (at least right now) than virtual reality. If you’re trying to think of what AR could actually do for you, check out the video above of a measuring tape app demo. And this is just the tip of the augmented reality iceberg.

The Next Ten

I’ve personally witnessed the transformation of the smart home over the past ten years from being rather difficult to use and inaccessible to most people, to being relatively simple to setup and actually useful today. I can barely imagine what the next decade of innovations will bring to our lives, both in and out of our homes. The modern connected home largely owes its very existence to the technologies made popular, easy-to-use and accessible to people across the globe by one magical device: iPhone.

Conversation of Things #1 - Allison Sheridan

Conversation of Things #1 - Allison Sheridan

Can the Drift Light help you sleep better? (Review)

Can the Drift Light help you sleep better? (Review)