Designing Connected Products: UX for the Consumer Internet of Things (2015)
Chapter 10. Interoperability
BY CLAIRE ROWLAND
Much of what we currently call the “Internet of Things” is not yet much like the rest of the Internet: a network of networks based on open standards. The proliferation of different technical standards means that getting devices to work together is hard. Many devices are locked away in proprietary ecosystems, because frequently that is the easiest way to get them to work. But their lack of interoperability with other devices and systems is seriously limiting its potential value and usability. Users will expect devices to work together, but right now many do not.
Building on the networking concepts discussed in Chapter 3, this chapter explores the technical challenges of interoperability (and some emerging solutions) and their impact on UX.
This chapter introduces:
§ Why the current state of the Internet of Things has been dubbed the “CompuServe of Things” (see The CompuServe of Things)
§ The concept of interoperability and the problem it poses for IoT (see What Is Interoperability and Why Is It a Problem?)
§ Ways devices can interoperate (see Interoperable Data)
§ Emerging solutions for improving interoperability (see How Can We Improve Interoperability?)
§ The UX of interoperability (see The UX of Interoperability)
This chapter addresses the following issues:
§ Why closed ecosystems of devices are not a true “Internet of Things” (see The CompuServe of Things)
§ How incompatible networking standards and data formats limit interoperability (see Network Interoperability)
§ How devices can talk directly to each other, via gateways, or via the Internet (see Interoperability at Different Levels of System Architecture)
§ Why “functioning together” is different from true interoperability (see Functioning Together Versus True Interoperability)
§ How web standards may be used to improve interoperability (see How Can We Improve Interoperability?)
§ The burden on users to figure out which devices can interoperate and get them to coordinate (see Knowing which devices will talk to each other)
§ The pros and cons of specific versus generic UIs for controlling devices (see Generic versus specific UIs)
§ The pros and cons of concentrating functionality in single apps, versus having multiple apps (see Organizing devices and accessing controls)
The CompuServe of Things
Being able to control a device in your home via a mobile app is still quite novel. Most of us have very few connected devices, and perhaps a different app for each type of device. This works OK for now.
But you might, in the next few years, own perhaps 20 connected products. Imagine you have 20 separate apps on your mobile: for the heating, lighting, energy monitoring, security alarm, cameras, baby monitor, smoke alarm, washing machine, slow cooker, and more. Just locating the right app for the action you want there and then becomes a pain.
And the more connected devices you have, the more likely it is that you will want to get them working together. You might want the hallway light to turn on when the alarm goes off to provide illumination for the camera. But right now, many systems don’t talk to one another. Each one may be sitting on a separate manufacturer’s software platform, with its own frontend app, and often no API to talk to any of the others.
If the systems don’t talk to one another, the user has to bridge the gap. Say you want to make more efficient or cost effective use of energy in your house. That requires a combination of energy monitoring data, perhaps other contextual data (like when the house is occupied) and control setting, such as turning off appliances when they are not needed, adjusting heating and hot water schedule, and perhaps running appliances when energy is cheaper. All these data points and controls may sit in separate systems, creating a lot of work for the user to understand the whole picture, figure out what to do, and make changes to the way things work. (We’re not necessarily advocating automation or an autonomous system here, just the ability to offer insights and recommendations alongside user controls.) In the words of the scientist Alasdair Allan, this turns the user into a “mechanical Turk for other people’s software.”[138]
This isn’t much like the Internet as we know it: an open network based on common standards. It’s close to the state of online services in the 1980s and early 1990s. Most people who were online then weren’t really using the Internet, but a walled garden network provided by an online service provider, such as CompuServe or AOL (see Figure 10-1). (The IoT engineer Adrian McEwen nicknames the current state of IoT the “CompuServe of Things”[139]). These services were closed networks and did not interoperate: for example, if you were an AOL subscriber, you couldn’t access content from any other provider, and you could only send email to other AOL subscribers.
Standards for Internet communication developed. By 1994, the World Wide Web was taking off in popularity and public websites were proliferating. One by one, the online service providers became Internet service providers: offering open access to any public website, and the ability to email whoever you liked, whichever their ISP.
Walled gardens briefly reemerged in the early days of the mobile Internet, with mobile operators offering access only to their own portals (see Figure 10-2). But this time it didn’t last as long: customers had tasted the open Web and weren’t prepared to put up with their operators’ bland content portals when the real fun was clearly happening elsewhere.
It’s unimaginable today that we would tolerate being restricted to proprietary web content again. Yet the consumer Internet of Things is going through a very similar process. You buy edge devices from a company (often but not always the hardware manufacturer). They provide you with a gateway device (if needed), a mobile/web app, and an Internet service.
Figure 10-1. 1980s CompuServe advert (image: AOL)
Figure 10-2. Vodafone live!, a mobile portal (image: Vodafone[140])
This is how product-based platforms like the Mimo baby pajamas (mentioned in Chapter 7) work. Home automation systems, like SmartThings and Lowes Iris, may support devices from third-party manufacturers. But development work is required on the platform to build in support for each device. You can’t use unsupported devices with the system, nor can you control it with anything other than the app the provider gave you. Things may use the Internet to connect, but they are not internetworked.
In a real Internet of Things, any Thing should be able to connect as a peer to any other Thing (or online service), as computers on the Internet can do. But right now, in technology terms, most Things are vertical stacks of integrated products: they don’t interoperate.
What Is Interoperability and Why Is It a Problem?
In information technology, interoperability refers to “the ability to transfer and render useful data and other information across systems, applications and components.”[141] In other words, it’s the ability for devices, applications, and services to interact with other devices, applications, and services from different vendors, regardless of hardware architecture or type of software.
The authors John Palfrey and Urs Gasser, in their book Interop: The Promise and Perils of Highly Interconnected Systems[142] propose a four-layer model of interoperability.
In their model, the two lowest layers are concerned with the technology (in this chapter, we will focus largely on these).
If two devices can interoperate on the technology layer, that means a signal can physically get from device A to device B. If the devices cannot transmit messages to each other, they cannot interoperate in any way.
But the ability to pass messages between device A and device B is not useful unless the two devices can understand each other. This is interoperability on the data layer: the devices can speak the same language. (If you’ve ever had a printer spew out pages of garbage because you used the wrong print driver software, you’ve seen a data layer interoperability problem.)
In IoT, the technology and data layers equate roughly to devices being able to speak the same network and application protocols (these are discussed in Chapter 3). In more advanced systems, the use of shared data models (see Chapter 15) may enable devices to work together in smarter ways: this is also the data layer in Palfrey and Gasser’s model.
Palfrey and Gasser extend their theory of interoperability beyond technology to consider the people and institutions which that technology supports.
Interoperability in the human layer means that the people involved in the exchange of information can understand and act effectively on the information. An IoT example might be the different user needs that have to be taken into account when a product designed for suburban homes in North America is launched in different markets (e.g., differences in language, culture, size of home, and daily activities). (Chapter 5 contains many examples of issues that could be classed as human interoperability challenges.)
At the most abstract, there is the institutional layer: the extent to which societal systems can engage effectively around the system. In IoT, an example might be the need to negotiate differing privacy laws in different markets (Chapter 11 focuses on privacy in depth).
This holistic view of interoperability (see Figure 10-3) is a useful reminder that technology isn’t just hardware and software, but that it serves human and societal purposes. But as most of this book is about human and societal concerns, this chapter will focus on the technical side: network and data layers.
Figure 10-3. Palfrey and Gasser’s model of interoperability
In IoT, network and data interoperability means that IoT devices and services can easily discover, communicate, and coordinate with other IoT devices and services, regardless of manufacturer. It also means that devices (or other components) can be substituted: replaced with an equivalent device from a different vendor, as you might replace your web browser or web hosting service with an alternative, without any significant impact on the functioning of the overall system. So whichever manufacturer’s connected light switches or bulbs you buy, you can be sure they will work with your other light switches and bulbs, your choice of connected home control application, and even share information on energy consumption with your metering service. Right now, some devices will work with some other manufacturer’s devices. But unless they are specifically configured to do so, there is often a sacrifice in terms of functionality (see How Can Devices Interoperate? for more details).
Most people involved in the field share a long-term ideal vision of an open, interoperable Internet of Things, but the industry as a whole has not figured out the standards to achieve this. Different standards have often evolved for good reasons: a network protocol that is ideally suited to the home or a smartphone peripheral is not necessarily going to be a good fit for a sensor network. A data model created to describe industrial heating/cooling systems is not necessarily going to be appropriate in the home. (Note that Palfrey and Gasser are careful to differentiate between interoperability and shared standards. Interoperability need not require that everyone adheres to the same standards: that every device uses the same radio or application protocols, that all humans speak the same language, or that all countries adopt the same laws. It only requires that ways are found to translate or mediate between them.)
Security is also a barrier: devices on the open Web are highly exposed, and securing IoT devices is challenging (for reasons set out in Chapter 11).
And then there are business models: a major factor that often mitigates against interoperability.
Interoperability is great for users. It broadens the choice of products, letting them mix and match products and services from multiple vendors to suit their own needs.
But businesses often think that their interests are best served by keeping their technology proprietary and noninteroperable. They are often right—at least in the short term. Companies that perceive they have an initial competitive advantage, such as a differentiated product, or a strong brand or reputation, know that their products will sell (Apple is an obvious example). They may try to turn that into a longer-term advantage by creating products that don’t interoperate with those of their competitors, locking customers into buying further products and services from them (of course, they may also argue that their technology is better and that interoperating with inferior systems would degrade the product). Interoperability (e.g., open standards) makes it easier and cheaper for competitors to enter the market. This is good for users and competitors, but bad for the early innovators. If a company is in a position to create a hugely popular set of products and dominate the market, interoperability is not in their interest.
A big company might also choose to collaborate with another big company, so that their products are interoperable with each other’s but not with anyone else’s. This is effectively forming a cartel to crowd others out of the market.
But in the long term, interoperability helps grow the overall size of the market, which is good for everyone. Many IoT companies are moving toward openness and increased interoperability to create ecosystems of devices and services (e.g., “Works with Nest” and “Works with SmartThings”). These are ultimately more valuable to consumers than the limited product set any one of these companies could create alone.
IoT is on track to become more interoperable over time, but the process of getting there will take time, due to both technical and competing corporate interests.
Network Interoperability
In Internet networking and the World Wide Web, interoperability is enabled through universally supported open standards for network protocols (such as the IP stack) and data formats (such as HTML).
Right now, some open standards are used in IoT (such as the HTTP and MQTT application protocols and Bluetooth, ZigBee, or WiFi link networking). But these are not universally supported. Devices and services vary enormously and no single standard is a good fit for every type of system. The diversity of IoT devices and systems is perhaps the biggest barrier to developing standards. As we saw in Chapter 3, some run on IP networks like WiFi, some on local networks like Bluetooth or ZigBee that don’t support IP (at least not yet in commercial products). Some devices need to be constantly connected and exchange lots of data, some need to spend as much time as possible asleep and minimize network activity to maximize battery life. There is no single set of standards that provides the perfect solution to all these needs. Many commentators argue that there never will be.
The lack of standardization exists at different levels of the technology. Proprietary networking, protocols, and APIs or data formats mean that some devices can only work with those of the same manufacturer. Some devices won’t even work with different or older versions of the same device from the same manufacturer, if the software or hardware has been upgraded and changed. Where standards theoretically exist (e.g., ZigBee), different implementations of them can mean that some devices can only work with other devices that speak exactly the same version of that standard. Of course, none of them can talk to the many devices that don’t speak ZigBee.
Within specific markets, there can be differing networking and data standards too, such as those historically used within commercial HVAC and lighting systems. These were optimized around the needs of the particular market, at a time where there was no conceivable need to share data with other types of system.
And finally, there are limitations at the client device end: the native app that only runs on iOS or Android or Windows. This can be addressed through providing web apps that can run on multiple devices, although these don’t always offer the sophisticated UX of a native app.
This lack of interoperability is important to designers because it creates a huge gulf between our idealized expectations of the Internet of Things and the reality of what can currently be achieved.
Users, of course, shouldn’t have to care about this stuff. If they buy a connected light fitting, it ought to work just fine with all their other connected light fittings. You don’t have to buy all your dumb light switches from the same company because light switches are light switches. Connectivity shouldn’t change that, but for now, it does.
First, you have to know what networking technology your home gateway supports, such as ZWave or Insteon. Then you have to find light fittings that support that. That’s an added layer of complexity that most consumer users could well do without (see Figure 10-4).
Figure 10-4. The SmartThings website lists compatible devices (shown here: the home and family category; image: SmartThings)
The simplest UX designs right now come from single platform experiences, such as the Philips Hue bulbs or Withings health and fitness ecosystem. But this limits users to devices from a single manufacturer if they want things to work together. Do you want to have to buy everything in your house from Apple, Samsung, or Philips? It also makes the cost of switching to a different platform very high. Old devices won’t work with the ones from the new manufacturer and must be replaced. Data is at stake, too: a user who replaces their activity monitor or bathroom scales with a device of a different brand may lose their data because it will almost certainly be tied to the original vendor’s platform.
All of this is hampering the potential of the IoT through limiting the exchange of data and functionality. People and devices should be able to communicate and work together regardless of manufacturer, model, or network technology.
Part of the problem is that there is currently a financial disincentive for vendors to create interoperable products. The business models of big players such as Apple, Google, and Microsoft depend on proprietary operating systems and/or retaining users and their data. There is no incentive to enable devices to work with others outside the walled garden: as it stands, this would undermine the business model.
There is also a design disincentive, especially for a company that considers great UX a differentiator. Interoperating with third-party devices often results in a clunkier user experience and often a reduced feature set. This can happen because different devices use different formats for data, APIs, or network device profiles (as used in ZigBee). Third parties may not be able to access all the data or functionality on the system.
As an example, for a while, the home automation platform Insteon unofficially supported the Nest thermostat, but offered only basic controls. Users who wanted to control their Nests alongside other connected home devices either had to sacrifice using some of the more advanced functionality, or switch to the Nest app for that (and then risk conflicts when two apps are trying to control the device). Insteon now works fully with Nest, but this type of integration can be effortful to do.
As the original equipment manufacturer (OEM), you may also want to discourage the user from using third-party apps that you think are not as well designed as your own or may cause unintended problems.
Devices may be linked at the Internet-service level via APIs. If this is the case, all interactions must go via the Internet where responsiveness could be an issue. For more details on different system architectures for interoperability, see How Can Devices Interoperate?
It’s widely acknowledged that the current state of affairs is a mess. Some organizations are trying to solve it by creating platforms that consolidate various network standards and devices. For example, the Wink gateway (see Figure 10-5) and app platform supports multiple networking standards (e.g., WiFi, ZigBee, Z-Wave, Bluetooth, and Lutron Clear Connect).
Figure 10-5. The Wink hub (image: Wink)
It can be time consuming to create systems that allow different devices to work together. For a connected home platform, every single third-party device on the platform must be individually configured to work optimally with the system. It is not enough to have the correct radio type on the hub; the device must also be associated with a software profile, which specifies the feature set it supports. There are standard profiles, but they do not necessarily support the range of functionality that modern devices offer. For example, ZigBee offers separate contact and temperature sensor standard profiles. A device cannot be associated with more than one profile. So even a simple device that can sense both open/closed status and temperature will need a proprietary device profile. Similarly, the ZWave profile for a thermostat is based on old-fashioned devices that only support manual temperature control for heating. Devices with timers or air conditioning require a proprietary profile.
The more proprietary profiles there are, the less interoperable the system. But if you only support standard device profiles, you may well not be able to support the full range of functionality users expect from a device. All ZWave devices are mandated to support basic on/off functionality, so any ZWave device connected to a gateway will in theory do something. But this may not be very useful: a ceiling fan, for example, might only be able to be switched on to 100% power or off. A ZigBee device will do nothing unless it has a suitable profile.
Testing and reliability also becomes a headache as there are so many potential combinations of devices that the user could have. Any combination could cause unexpected errors, but it is unfeasible to test them all.
Interoperable Data
Getting devices to connect is only part of the problem. Are they able to understand each other, or are they talking gibberish? Local networking protocols designed for specific purposes (like home automation) may provide a basic vocabulary of commands: so, for example, any Z-Wave device can understand an “on” or “off” instruction.
But there are no standards for more complex data. Fitness trackers may measure activity in a number of ways: perhaps minutes of activity or steps or distance traveled over time (for running). Nike+ has a proprietary measure of activity called “FuelPoints” (see Figure 10-6). Or lighting brightness may be measured in lumens or candela across different sensors and systems. So, combining and comparing data about the same things when they are measured or organized in different ways is yet another interoperability challenge. This is the realm of metadata. It is possible to program devices to translate where the units have an equivalence, for example, where the system knows that 5,000 steps is the equivalent of 2,000 Nike FuelPoints. But if one system is measuring heart rate and the other is counting steps, the relationship between the two is much more complicated. Metadata models are discussed in more detail in Chapter 15.
Figure 10-6. Nike+ measures activity in FuelPoints (image: Nike)
How Can Devices Interoperate?
In this section, we’ll look at the different technical levels on which devices can work together, which has an impact on UX. We’ll also explore the distinction between enabling devices to work together in a closed system and making them truly interoperable.
Interoperability at Different Levels of System Architecture
Different approaches to interoperability can have different implications for the user experience. As we saw in How System Architecture Affects UX in Chapter 3, devices can connect and share data on different levels. Engineers have sought to create interoperability options at all of these levels. For the same reasons that system architecture can have implications for UX, so can the level on which devices interoperate. We set out some examples of these in the following sections.
One edge device connects directly to another
Two devices supporting a common networking technology (e.g., WiFi, Bluetooth) can connect directly to each other over a local network link. In a simple case, one device may only need to register the presence of another in order to trigger an action. In the Ninja Sphere home lighting demo from Ninja Blocks (see Figure 10-7), the lighting detects the presence of a Fitbit’s Bluetooth LE profile and turns lights on and off as the user moves around the house.
Figure 10-7. “Follow Me” lights demo (images: Ninja Blocks)[143]
In a more complex case, actual data or instructions will be exchanged. AllJoyn[144] is an open source project initiated by Qualcomm. It provides a software framework, networking standard, and set of core system services to enable edge devices of different manufacturers to talk to each other and other software applications, as long as they all support AllJoyn. AllJoyn services allow devices to announce their own capabilities on a network, and thus discover and connect to other nearby devices with the capabilities needed to fulfill a particular task, such as using screened devices to act as control panels for nearby headless (nonscreened) devices, or routing audio (such as a doorbell alert) to the most appropriate speaker (see Figure 10-8). For example, an August door lock can broadcast a text notification around the house that the door has been opened so other devices can greet you. It can also translate this to audio and send it to Panasonic speakers. Even if the manufacturers of two devices have never worked together, the shared AllJoyn framework allows for unexpected interactions between devices.
Figure 10-8. Image illustrating AllJoyn scenario
If you can trust the devices to pair reliably, device-to-device connections should be quick and responsive. In an example such as AllJoyn, there is the potential to streamline how user inputs and outputs are handled around the home. This could reduce the number of devices in the home that require specific capabilities: for example, instead of having to build speakers into every device that needs to output audio, you can have one good set of speakers in every room.
One edge device connects to another via local gateway/hub
In this scenario, devices that use different networking technologies connect to a gateway device or hub that contains multiple radios. This allows instructions to be passed between the two. This setup could be used to control devices of the same type running on different network types together via a single app. For example, you could turn on both Insteon and ZWave light switches with a single master switch or lighting rule. The Wink hub (gateway), described earlier, is an example of this, as are SmartThings, Staples Connect (see Figure 10-9), and Lowes Iris. One of the most ambitious was Revolv, whose hub had seven radios.[145]
Figure 10-9. The Staples Connect hub
If there’s some intelligence in the hub, more complex instructions can be passed from one device to another. If a ZWave door lock is opened, instructions could be sent to ZigBee light bulbs to flash in red or green. In this example, some system intelligence and configuration is required in the hub to know that the door lock opening should trigger the bulbs to flash, as the two edge devices aren’t smart enough to handle this themselves. But the fact that the two edge devices run on different network types isn’t a barrier to using them as part of the same system. This is the typical approach in the home automation world: allowing users to access and control heating, lighting, security, cameras, and many more types of home system from a single platform, and a single web/mobile app.
A consumer setup like this would be intended to be used mostly with an Internet connection. So the intelligence that tells devices how to work together might be partly or wholly in the cloud (so would not work if the connection was lost). But if there is appropriate intelligence in the gateway, then instructions can be passed between the two devices without relying on the Internet. So, for example, the automated rule that turns on some lights after dark if no one is home could still run even if the Internet goes down.
Device-hub-device interactions should be quick and responsive, as they run over local networks. But sometimes joining two devices that support different networking technologies can lead to reduced functionality.
One gateway connects to another
It’s possible for one gateway to connect to another. This allows two systems to be integrated over local networking, for a responsive experience, but also for the devices to continue to connect to their own manufacturer cloud. The SmartThings gateway can connect directly to a Philips Hue bridge. Any connected Hue bulbs will appear directly in the SmartThings app and continue to be available to the dedicated Hue app.
Edge devices connected via a cloud platform
In this scenario, data and controls for edge devices are integrated via an Internet service. For example, a fitness tracker (connected via Bluetooth to a mobile device) and a WiFi bathroom scale (like Withings) would both connect to the Internet via different means but share data with the same Internet service (so the user would access the data from both via the same web or mobile app). The same fitness tracker could even display a friend’s step count (from their fitness tracker) that day as a spur to competition (if you both agreed to share data). Both devices connect to different mobile devices, the integration happens via the manufacturer’s cloud.
Interactions with devices connected via a cloud platform will inevitably be subject to the inherent latency of the Internet. Data can’t be shared between devices if either loses Internet connectivity. So this method best suits devices with a reliable connection, or services (like weight tracking) where data doesn’t necessarily have to be live, just updated regularly.
Service to service over Internet
In this scenario, the integration is between Internet services. A service provider makes an API available to allow other services to retrieve data and perhaps send commands. These APIs might be totally open—for example, the Xively public API for open sensor data. Or they might be accessible only to others participating in a developer program, as is the case with Nest and Philips Hue. Connected Mercedes-Benz cars can now notify a Nest heating controller that the driver is on the way home, triggering the heating to come on automatically (see Figure 10-10). The notification goes via the Mercedes-Benz Internet service to the Nest Internet service. Or the Nest Internet service can notify the LIFX lighting service that the Protect has detected smoke, triggering the bulbs to flash red to indicate danger.
Figure 10-10. The Mercedes-Benz dashboard app showing Nest integration (image: Mercedes-Benz)
Cloud integration works well in the cases where Internet-grade responsiveness is acceptable. Interactions with interconnected services will not only be subject to Internet latency, but have more potential points of failure (devices, gateways, Internet services). APIs may impose loads on how frequently other services can poll (check in) for changes, which might also create potential delays. For example, it can take up to 15 minutes for IFTTT to check in with Gmail. If you were hoping to notify yourself of emails from someone important by making a Hue light bulb change color, you might find it’s not the fastest way to find out. If you need things that are near each other to connect immediately, a local network is more reliable.
Nest seeks to mitigate this by promising “real-time” communication between their APIs and their devices, so in theory, service-to-service interactions can be quick. However, in many cases, particularly where an API is public, the provider will have imposed limits on how often it can be polled so this will not always be the case.
As noted, services may capture, store, and handle data in quite different structures (e.g., a RESTful API for pollution sensors might be structured as country/area/sensortype or sensortype/country/area, and fitness trackers may measure activity in minutes, steps, or heart rate), leading to APIs that, despite doing similar things, may be structured in quite different ways. This means that any developer who wants their system to access a third-party API will need to spend time understanding what data and functions it offers and how it is structured, and program their own system to request data and pass commands in the right way. Right now, the inconsistency of data structures means that it’s rarely easy for a service automatically to discover other services and request data.
Products and services can interoperate on several levels
There are currently lots of companies seeking to become the de facto gateway/cloud integration platform for the home, through a combination of gateways that support multiple networking technologies and cloud services that talk to other cloud services. It’s quite possible for a system to combine both approaches.
Manufacturers may wish their devices to continue to connect to their proprietary cloud first, even if those devices could technically connect to a third-party gateway. This allows them to maintain their relationship with the user, retaining their data and providing their own services.
Sometimes integration providers (such as SmartThings) will take a fairly open approach to third-party devices, making developer kits and APIs available to third parties to integrate their own devices onto the system. Approval may be required before the device is officially declared “supported,” but in principle the process is fairly open. Other companies (such as Staples Connect or Lowes Iris) may only integrate products made by manufacturers with whom they have a business partnership.
This latter approach may mean that the experience of adding and using a supported device will be more consistent with the general platform UX. But it means that other devices won’t work (or won’t work well) even if they use a protocol that the hub supports.
Functioning Together Versus True Interoperability
Platforms that connect devices from different manufacturers together or support different radio protocols are an improvement on wholly closed manufacturer ecosystems. But they aren’t anywhere nearly as interoperable as, say, the Web. You can’t plug any light switch into any connected home system and expect it to work. But you can point any browser at any HTML-based website and (barring inconsistent support for some advanced tags) it should do something with it.
There are advantages to closed systems: for one, they are easier to secure. Security for connected devices poses very complex challenges and solutions are immature (see Chapter 11 for a user-centric introduction to security). A closed system allows you to control who and what can connect to the system, and what they can do. This also improves reliability. Devices can conflict in unexpected ways, causing odd behaviors. Standards (like device profiles) that are ostensibly the same may have been implemented slightly differently. If you control the devices that can work with the system, you can configure them to work together sensibly, and QA test them together. (However, as you add support for more devices, the potential combinations that need to be tested will rapidly increase and it will at some point become unmanageable to test everything.) Finally, a closed system gives you more control of the UX. If the user adds a heating controller you support, you can offer them the perfect UI for that specific controller. A generic heating controller UI, designed by a third party to support a range of devices, might be a poor fit for your specific device. For example, the UI might use different terminology such as “schedule” instead of “timer” or have only a subset of the functionality that the controller supports.
Some platforms are semi-closed ecosystems, where access is granted to selected third parties to build services that integrate with the platform. The Nest developer program mentioned earlier is an example of this, sharing data over APIs with other manufacturer clouds, such as Jawbone and Mercedes-Benz. This provides some interoperability—more devices and services are supported. But both parties retain enough control to ensure security, reliability, and UX aren’t compromised.
Some platforms are entirely open, like Xively: anyone can access any data on the Xively public API without requiring permission. Devices connected to Xively are generally at lower risk: the most you can do with many of them is read data that the sensor owners have chosen to share publicly (see Figure 10-11). Xively is interoperable in the sense that the data is publicly available and well structured, so it’s easy for third-party developers to work with. You can program your system to retrieve data from Xively. But that code will be specific to Xively. If you want to retrieve temperature data from sensors on Xively and also sensors on another service, you’ll probably need slightly different code for each service, because the structure of the data might be different.
Figure 10-11. The Air Quality Egg map of sensors (image: Air Quality Egg)
How Can We Improve Interoperability?
Standards are like toothbrushes. Everybody agrees you should have one, but no one wants to use yours.[146]
Interoperability can be improved through addressing inconsistencies in networking (ensuring devices can connect) and data (ensuring they can communicate and coordinate). Inconsistencies could be ironed out by everyone conforming to a common standard, such as using ZigBee networking in the home or using a common semantic structure for data, but this probably isn’t realistic given the wide range of applications covered by IoT. A network of several thousand solar-powered air quality sensors will run on low-bandwidth connections and have a high tolerance for latency and reliability issues and missing data points. A medical monitoring system for a critically ill patient and a home lighting system will have very different needs.
A more sensible approach is to accept that different standards will continue to exist and work around that.
At the network level (of getting the things connected), gateways may contain more radio chips so that more edge devices can connect to them. Earlier, we mentioned the Revolv home automation gateway that has seven radio chips. But additional radios (like Bluetooth and ZigBee) could also be built into devices most of us already have in the home, such as broadband and WiFi routers.
More powerfully, methods are emerging for devices and services to communicate that are independent of the type of network they use. The 6LOWPAN project defined mechanisms for IPv6 to be transported over low-bandwidth wireless networks, meaning that IP networking could be extended to low-powered edge devices. This makes them capable of being directly addressable on the Internet and enables the use of application protocols such as HTTPS or MQTT. So your low-powered light switches can speak the same “language” as your WiFi-enabled speakers, making it much easier to coordinate the devices around you. 6LOWPAN support is beginning to appear in commercial consumer products, such as the Tado heating controller:
Fitting a building for “IoT” shouldn’t mean fitting a hundred different custom gateways. It should be more like WiFi deployment.
—TOBY JAFFEY, CTO OF 1248[147]
Within a service or platform, you might still want to treat devices of the same type that were made by different manufacturers or run on different network types as equivalents. Device abstraction is an engineering approach that maps actual devices to virtual representations of devices. Your door and window sensors may be made by different manufacturers, but the system can recognize and treat them as instances of the same thing. When you add a new door or window sensor, the system recognizes it as a door/window sensor and groups its data with your other door/window sensors no matter who made it or whatever network it runs on.
To be truly interoperable, services also need to be able to discover useful data and functionality within other services. That’s not easy when APIs, security, and application data formats are inconsistent. Imagine you were staying in an Airbnb apartment with connected devices such as the door lock, heating controller, entertainment devices, and lights. To control these using current technology, you’d have to download one or more dedicated smartphone apps provided by the manufacturer(s) of the devices in the home and then log into the homeowner’s accounts (or special guest accounts they set up) in order to use them. You might think this was too much hassle. Wouldn’t it be better if your own home control app could automatically discover the devices around you and offer you the home controls you needed for the duration of your stay? Or you could just discover the devices around you using a web app?
Or you’re walking back to the apartment late at night in an unfamiliar city and want to find a well-lit route where there are likely to be other people. Traditional mapping apps may tell you the route, but don’t provide any contextual information to help you decide how safe you may be (see Figure 10-12).
Local live data on city street lighting would be useful. It would also be useful to have data on current road traffic density—are there people driving around or is this street deserted? Historic crime levels might be useful, too (see Figure 10-13). You may want to know whether it’s safer to try to get a bus, but you have no idea of the routes or when the next one might be due. This is data that may be available publicly but isn’t easily accessible. Each city has its own, separate, public transport apps, but you have to know where to find them and then download them. The data on street lighting, traffic, and crime could be online somewhere but probably in separate places, and anyhow, it won’t be easy to get the information you need, which is “which roads on my way home are well lit, not deserted, and not crime hotspots?” What you need is one service that you can rely on to retrieve local data, wherever you are, and crunch it all to give you the answer you need. And what that service needs is a way of finding all this information.
Figure 10-12. Google Maps doesn’t yet provide information that would help you understand a route’s safety (image: maps.google.com)
Figure 10-13. The London Metropolitan Police maps show historical crime rates by local area (image: maps.met.police.uk)
The HyperCat standard (based on HTTPS, REST, and JSON) seeks to enable this kind of discovery. It provides a very simple, light API catalog layer that allows apps to search on a hub or IoT service to discover what resources (devices, data) are available that the app can use, or to look for very specific data types (even if they don’t understand any of the other data). So an app that is simply looking for lighting brightness or traffic density data can find that, even if it’s buried in a load of other irrelevant information about lighting maintenance schedules or traffic light cycles that the app can’t understand.[148] HyperCat does not aim to solve all the problems of interoperability at every level (such as common semantics or data formats), but just to address the basic problem of enabling apps/services to discover useful resources on other services.
Ultimately, a complex system will need ways of discovering and dealing with very different, and even unfamiliar devices—devices that may be connected to it directly, and things it discovers out there on the network. Sensors may have different sampling rates or capture different measures, and actuators (even those that do similar things) may have different capabilities. Knowing what something can do, and how it captures/structures data is one challenge. Coping with changing network resources is another. Things can go online or offline or cease to exist. New devices, that the system doesn’t know anything about, may appear. In Chapter 15, we cover the use of data models: a loose group of methods seeking to enable interconnected systems to understand each other and coordinate.
The UX of Interoperability
Currently, consumers are buying devices to fulfill specific needs: lighting (e.g., Hue or LIFX), music (e.g., Sonos), heating or cooling (e.g., Nest or Tado). People may have trouble understanding networks (as we saw in Chapter 9), but in our experience it does seem obvious to them that devices should work together. If the heating system knows I’m on vacation, why wouldn’t the lights? They will also expect devices to work with equivalent devices from other manufacturers. You may have a few full-featured Hue or LIFX light bulbs, but you also want these to work with cheaper, more basic bulbs from other manufacturers, or connected light switches or smart plugs on lamps.
Interoperability is really about reducing the amount of human intervention involved in getting things (or services) to work together: to return to Alasdair Allan’s quote, to avoid both programmers and users becoming “mechanical Turk for other people’s software.” Right now, it’s often not possible to get devices working together unless they have been programmed to do so by the manufacturer. Where it is possible, it’s often difficult to do so and the UX and functionality of the devices may be compromised. Open source alternatives may exist, such as Open HAB open source home automation software (see Figure 10-14), and fulfill a need for those who do not want to entrust their homes or lives to a corporation. But they require a degree of technical skill that most consumers will not possess.
Figure 10-14. Open HAB home automation software (image: Open HAB)
UX Issues in Interoperability
In this section, we outline the key challenges that interoperability poses for both users and designers. We’ll also look at why, despite those challenges, interoperability is ultimately in the user’s interest.
Knowing which devices will talk to each other
Just knowing which things could potentially work together can be hard. For an iPhone or Android phone user buying a wearable, it’s fairly simple to find out whether the device supports your phone, as there’s likely to be an easily recognizable Apple/Android icon on the box to tell you whether the app will work on your phone. You might need to know whether your phone supports Bluetooth LE, but most modern smartphones will.
If you want to add a device to a connected home system, it’s a bit more complex: unless a device has been explicitly sold to work with your system, users have to know what type of radio a device uses (e.g., Insteon, ZigBee, ZWave, Lutron Clear Connect). This is more of a pain: the icons for these systems are less easily recognized (see Figure 10-15).
Figure 10-15. Icons for different radio types
With 6LOWPAN bringing IP networking to edge devices, more and more will be able to talk, but this is a long way off becoming standard. And you’ll still need to make sure there’s a suitable radio for your device to connect to.
Getting devices to coordinate
Even if you can add two devices to the same gateway or Internet service, that doesn’t necessarily mean that they will work together in clever ways. The example given previously of the Nest Protect causing LIFX bulbs to flash red when the alarm is sounding has been configured in the service. If the user has the devices set up, they will most likely only need to enable this option in the app. But on another connected home platform that supports a wider range of hardware you might have to configure your own rules in order to achieve a similar effect. This is a much more complex activity, more like programming.
To make it simple for users to configure interoperable devices, system designers either need to build in lots of preconfigured options, and/or build more intelligence into the system so that it “knows” what devices do, and what kinds of coordination between them may make sense.
See Chapter 15 for more information on designing programming-like experiences, options for making the system smarter, and the pros and cons of different approaches.
Generic versus specific UIs
A specific app for your device allows the app UI to be optimized to exactly the functionality your device offers. You can offer a rich experience, specialized to the needs of your devices. An integration platform allows many devices to be accessible via the same app, with a generic UI for lots of devices, regardless of manufacturer. This may not provide access to sophisticated extras that are unique to your device (such as firmware upgrades).
For example (at the time of writing), the Staples Connect app allows users to view video from D-Link cameras, but not to zoom, adjust brightness, take a snapshot, or view the image in landscape mode,[149] all possible in the dedicated D-Link app (see Figure 10-16). The Hue lighting app offers a party mode, but the generic UI in the Revolv app (which supports Hue) does not. This example shows how you may have to consider how to show status information in the generic UI if it does not support functionality the user activated in the proprietary app.
If you’re designing a generic UI, you need to be flexible enough to cope with unexpected devices and handle them as much as possible. For example, all Z-Wave devices are mandated to work somewhat even with controllers that have no idea how to handle them. Most of the time that is just setting a single state that will hopefully do something useful. But while this is helpful for an advanced user, it will confuse everyone else.
Figure 10-16. D-Link’s dedicated camera app works in landscape mode and allows users to take snapshots (image: mydlink.com)
Organizing devices and accessing controls
If you have a lot of connected devices, you may, as things stand, end up with lots of apps. It can be hard to find the app you want. And as devices will be tied to the manufacturer’s platform, it’s likely that they won’t necessarily interoperate or will do so in only limited ways. If you have lighting from several manufacturers, you’ll have several apps.
Having one app per manufacturer effectively means that your devices are organized by manufacturer. But mostly, users don’t care who made a device as much as they care about what it does. Grouping devices by manufacturer isn’t a great fit for their needs. In some situations, users may want to see all the devices that do the same thing—for example, all the devices that measure temperature or all the devices that light the home. Or they might want to see all the things that support the same activity, such as leaving home in the morning. And often they may want to see things that work as systems, such as the heating controller, boiler, and hot water tank. It’s very rare that they will want to see them organized by network protocol: all their ZWave devices in one place, and all their ZigBee devices in another. But these are the kinds of groupings that poor interoperability unintentionally creates. (Sometimes the right UI will not be device centric at all. Chapter 15 discusses ways of organizing large sets of devices in more detail.)
The idea of a single app to control many devices, especially around the home, is superficially appealing. This approach can work quite well with a moderately sized set of devices.
But the UI may be less rich, as already discussed. And when you have a lot of devices, a single app could also become an unwieldy way to access them. The user could easily end up digging around several levels deep to find the control they need in a mobile app. Four taps to turn off the hall light is rather a lot. Of course, you can provide quick access to key functionality on a homescreen, but there will be a point where this approach won’t scale to all the controls the user might need frequently.
So there may be an advantage in different frontend apps: more key functionality can be made quickly accessible. In an ideal world, apps need not be manufacturer groups: you might have one for lighting, one for security, and one for HVAC and hot water. But if you want to coordinate functionality across multiple apps, where does that go? If you want the security alarm to turn on certain lights if the alarm goes off, does that sit in the security app or the lighting app? This is an issue right now, in our manufacturer-centric app world. You may go into your Hue app to authorize several other services to access this, your SmartThings app to authorize some other services, including Hue... and so on. There is no centralized way to manage authorizations. This can lead to problems keeping track of which service can do what, and might lead to problems, where one service is receiving conflicting instructions from two different services.
One option could be to offer separate control apps for lights, heating, security, and so on, then a “settings” app that controls how they all work together and handles overall preferences. But the user would then have to install and maintain several apps—which can be confusing at point of setup. And doing this across manufacturers would depend on a high level of integration.
Finally, consider whether you need to set the user’s expectations about how things may work. In an ideal world, users should not have to care whether devices in the same place may be on the same local network or connected via the Internet. But you might choose to differentiate them in the UI if there are likely to be latencies in the experience, to set expectations and provide an explanation of any delays reasonably.
Interoperability puts users in control
Closed ecosystems of closely integrated hardware and software may tend to offer better UXes for now. But many people feel strongly that closed ecosystems lock users off from doing whatever they like with their own data and devices, and prevent developers offering them more choice. Fierce proponents of Android often take this view of iOS in the mobile sphere. Many consumers don’t yet feel strongly about this when it comes to their mobile phones, but this may change as more and more of the everyday things around them become connected. True interoperability means that we can choose to be in control of our things without being beholden to an authority that limits what we can do with them. It means you don’t have to buy all your devices from the same manufacturer, or throw away perfectly working devices just because they are incompatible with your new home. It means you’re not dependent on your car’s manufacturer to keep it running, so they can’t stop you using it if you miss a subscription payment. And it means that your heating controller need not stop working because the company that made it went out of business.
Summary
The Internet as we know it is based on open standards, but many IoT devices are currently locked away in closed ecosystems.
Interoperability is the ability of devices, applications, and services to interact with other devices, applications, and services from different vendors, regardless of hardware architecture or type of software. As things stand, the diversity of networking standards and data formats used in IoT systems means that many systems have limited interoperability. This limits the potential value of IoT and inhibits the quality of the UX.
Devices can interoperate in different ways. Edge devices can connect together directly, via a gateway that supports different radio technologies, via gateways that talk to each other, or via an Internet service. Or two Internet services can talk together. But manufacturers collaborating to make their devices work together is not quite as open as true interoperability.
Interoperability can be improved through the use of web standards, but there are still many issues to overcome, such as ensuring devices remain secure.
Users will expect devices to interoperate. But right now, onus is on the user to research which devices will talk to each other, and it can be hard work to get them to coordinate.
Platforms designed to support a wide range of devices may not offer full support for the functionality of each individual device. They may of necessity have rather generic UIs designed to support multiple devices of the same type. While these support interoperability, this can come at the cost of a compromised UX. And while it may seem superficially appealing to control many devices through a single app, this may not scale.
Currently, closed ecosystems are likely to offer a more mature consumer experience, but in the long run interoperability gives users more control and freedom.
Case Study 4: LOOP: Connected Pelvic Floor Exerciser
BY ADAM SCHEURING
LOOP is a smartphone-connected, interactive pelvic floor muscle exerciser for women. It’s a modern, designful take on a medical device used in physical therapy practice for a long time. The story of its invention started in February 2013 when a company called Thalmic Labs released the Myo armband, a human–computer interface device that presented a consumer-friendly wearable implementation of a commonly used medical sensor technology called EMG (electromyogram). This sparked our imagination and we started further researching EMG, which then led to the discovery of various vaginal and rectal muscle sensors and stimulators used by physical therapists and doctors for the treatment of various medical conditions related to the pelvic floor muscle group, such as stress urinary incontinence. These devices were rigid cylindrical objects with metal pads carrying bio-signals from the most intimate parts of the human body to clunky white-pink plastic boxes with a low-contrast $1 LCD display (Figure 10-17). In the age of iDevices, this lack of design appeal was shocking and sparked the idea to reinvent this product.
Our team set out to do this using our expertise in design and technology. Half of the team (Anders and Adam) previously worked as engineers at a semiconductor company making low-energy microcontrollers and Bluetooth radios. Many of the customers using these chips were developing fitness trackers, wearables, and digital health products. The other half of the team (Karianne and Line) worked for more than six years on designing high-end ergonomic sexual wellness products, and therefore were well acquainted with creating products that work well for the female body. The main idea was to create a better pelvic floor muscle exerciser for women by applying the technology and the concepts of behavior change used in quantified self products like Fitbit combined with the design principles and high-quality, body-safe silicone materials used in high-end sexual wellness products.
Figure 10-17. Most pelvic floor exercisers look like medical devices (images: Savantini, Laborie, Aquaflex)
To better understand the entire problem space, we started researching the medical issues related to weak pelvic floor muscles. Although the researchers’ opinions on how to evaluate the self-reported data from patients are not completely aligned, the prevalence of these issues is surprisingly high. Studies estimate that 75% of women have some issues related to pelvic floor weakness throughout life. More than 30% of women suffer from some form of incontinence and the majority of them from stress urinary incontinence, the involuntary leakage correlated with coughing, sneezing, or other physical effort. We found stories of women in their early 50s who stop going out with their friends and become socially isolated because of the fear of leaking when laughing or sneezing. We found that more than 50% of those suffering from incontinence never actually talked to a physician, and many don’t even talk to their spouses about the problem. In addition, people often described their experience with the existing medical products as “undignified and embarrassing.” While incontinence pads comprise a $7 billion global business, the adverse effects of the disease cost healthcare systems around the world even more. Although many women suffer from sexual dysfunctions that are not broadly researched in medicine, they remain mostly the discourse of psychology and not mainstream population health. The subject is too touchy for our culture. The good news is that science also showed that exercising the pelvic floor muscles regularly can prevent or diminish stress incontinence and has positive sexual health effects for women of all ages. Yet many women don’t know about it and even if they do, they don’t do these exercises. We felt that we stumbled upon a big problem that has little awareness and poor existing solutions. We decided to create a product that is positioned in the realm of health and fitness and removes the negative connotations from both the adult toy and the medical device industry (see Figure 10-18).
Figure 10-18. The LOOP device and app
We identified two main problems with related pelvic floor muscle exercises. People were asking:
“How do I know that I’m doing it right?”
Lack of feedback of the correct muscle contraction
“How do I know that I’m getting stronger?”
Lack of strength tracking, and therefore lack of motivation over time
We knew that we needed a solution with a feedback loop.
We started sketching the design and made working prototypes within a few months during the summer of 2013. We started researching medical papers, and talked with physical therapists, gynecologists, midwives, yoga and Pilates instructors, new mothers, and women of all ages. We made a basic prototype: a purple pillow made out of rubbery plastic material containing a circuit with a Bluetooth LE chip and a pressure sensor. We made a very simple app that was demonstrating the concept and the accuracy of the sensor. It wasn’t body safe, therefore not testable. But it was good enough for starting discussions around the idea.
Based on this research, we made a few key observations early on:
§ Pelvic floor muscle exercises work well and people don’t like electric stimulators.
§ Wireless changes the game because it allows people to do the exercises in various places. No need to be locked up in the bathroom anymore while doing the exercises.
§ People want to know if they are doing the exercises correctly: 1 out of 3 women can’t contract their pelvic floor muscles correctly without feedback. Accurate, real-time visual feedback is important because the muscles are a hidden part of the body.
§ Motivation is key. Simple metrics, reminders, and accurate tracking of strength improvements go a long way to facilitate the behavior change.
§ Peer support and coaching are effective forms of maintained motivation.
§ Charging is a terrible user experience for intimate products. Remembering to keep them out of sight (e.g., hide them from their kids) puts extra mental load on people.
§ Button switches for power and Bluetooth pairing are difficult for less dexterous people.
§ Most people wanted the experience of the “weight-scale” and the “personal trainer.” The use case for a wearable, always on sensor didn’t seem very compelling.
§ Muscle strength measurements are hard to reproduce.
§ Medical professionals (physical therapists, gynecologists) were reluctant to recommend legacy medical devices that are aesthetically unappealing.
As the next step, we created a body-safe prototype and a simple smartphone app with only one screen that featured a line going up and down along the horizontal axis based on the force exerted by the pelvic floor muscles (see Figure 10-19). We recruited a handful of testers who were willing to give us early feedback about the prototype of such an intimate product. We asked them to insert the body-safe device in the vagina, try to contract the pelvic floor muscles, and report their experiences in a one-on-one interview with one of our female cofounders. After this first testing round, we figured that people found the visualization itself quite motivating and informative. What we identified from the early feedback was that people would have benefited from seeing how consecutive contractions measured up against each other because this allowed them to quickly improve their contractions as part of the exercise. They also wanted simple statistic and timing along with instructions on when to contract the muscles and for how long.
Figure 10-19. LOOP app screens
We added these features and organized another round of test. This time we gathered more than a dozen testers and split them into groups. One group continued with giving feedback in one-on-one interviews, one group provided feedback through a form, and a few people allowed our female cofounders to observe them while they were using the product. We confirmed that the circular visualization allowing users to see consecutive contractions worked well. But we were surprised to see that the guiding line moving up and down designed to give instructions to the user on when to contract and relax confused some people.
In the next iteration, we will add a vibrating buzz on the phone to trigger the user to contract the pelvic floor muscles and we will simply use an interactive text as an instruction instead of a moving line. Some testers complained that our white background made their phones light so brightly that it disturbed their partner when they tried the product in bed at night. So we decided to introduce a night mode.
People really wanted a fitness/wellness product that is fun to use, that doesn’t make them feel sick but doesn’t look like a common sex-toy either. They were validating our hypothesis that this product fits into a new product category within health and fitness.
We wanted to create a fitness product that is visually appealing and fits into the daily life of women. LOOP has no wires or buttons, and does not require charging. It’s simple and ergonomic, and the sensor is accurate and runs on a replaceable coin cell battery for up to one year. It has an accelerometer that detects posture to make it easier to reproduce measurements. People perceived the app as clean, easy to use, and motivating.
We received great feedback from testers and healthcare professionals. We believe that LOOP is more than just a tool. It’s a new way of thinking and talking about the health aspects of one of the most important parts of the female body. We are proud to work on a meaningful cause even if it means that we have to challenge social norms and fight prudery, but we’ll keep winning those battles. This is our contribution to a healthier, happier future.