Designing and Developing for Google Glass (2015)

Part III. Develop

Chapter 15. Reflections on the Future

The most intriguing thing about Google Glass is that it’s just getting started. The promise of all the things Google’s flagship wearable computing platform already is and the exciting prospects of what it can become are nothing short of awe-inspiring. Do a search for “Google Glass use cases” and it’ll take you an hour just to go through the results. From general purpose to profession-specific, from the ultra-cool to the downright ludicrous, the sky’s the limit for how people are going to put Glass to work.

As we’ve discussed, a big portion of society speculated, dreamed about, praised, criticized, and condemned the product for more than two years before it even came out. Few products in history have achieved this sustained level of interest that far out from launch.

Throughout this book we’ve presented concepts, code, design tips, and our philosophy to help ensure you enjoy a long and healthy life with your technology investment and give your Glassware a powerful stage on which to succeed. This final chapter discusses some advanced uses for Glass and what lies ahead for the platform.

Corporate Glass

One of the questions that invariably pops up when considering Glass is how businesses will deal with their employees using it in the workplace. These days the lines between high-performance business machines and off-the-shelf consumer tech are completely blurred, and companies are outfitting their staffers with the very same smartphones, tablets, and laptops that everyday consumers stand in line for and pick up at popular retailers. The BYOD (“bring your own device”) concept, thought to be anathema 15 years ago, is gaining popularity these days, and is even encouraged in many other operations across industry. Staffers are free to source their own gear that gets outfitted with corporate access controls and internal applications, distributed and running safely and securely outside of the public Glass ecosystem.

Glass can be a boom for productivity since it’s so nonintrusive; or it can serve as the bane of an IT department’s existence in being yet another node joining a wireless network, using resources and distracting staffers from actually getting stuff done. It’s a legitimate quality control concern. Many organizations—notably government agencies and the military—restrict access to several types of applications they allow on their networks like videoconferencing, instant messaging, various streaming media formats, and many other services.

So a big challenge—and opportunity—is clearly on the horizon for integration and Glassware development. Since all syncing traffic for services based on the Google Mirror API takes place in quasi-real time (after a request/response roundtrip to the service), how can this be filtered to allow some Glassware, like news updates, stock prices, and messaging, to persist throughout the workday, while disallowing notification distractions from other Glassware like games, memes, and porn? Large organizations may prefer the closed model of installed apps written with the GDK so that they control all aspects of the communications chain and won’t have to rely on Google’s system as middleman for things like company-wide messaging. Or, they may prefer the managed, cloud-oriented environment of the Mirror API with its simple UI, inherent security, and known procotols, and then outright block native Glass apps.

Google itself even created the Glass at Work initiative to help spread knowledge about how Glass is being used by workers in different professions, to demonstrate how developers are creating custom solutions within industry, and to stimulate businesses integrating wearable electronics into their operations.

Will a new generation of wearable computing devices spearheaded by Glass assist workers and let them complete more tasks more quickly, therefore being a valuable tool, or will it be yet another time vampire, preventing work from actually getting done? That part’s up to you. Google’s done all the engineering for the product; the social rules of how it can be used within offices, schools, churches, buildings, and other places need to be developed, put into practice, and enforced. Remember, a key element of how to Think for Glass is getting others to understand what it is, and is not.

Streamlining Operations

Pennsylvania-based Fiberlink extended its MaaS360 mobile monitoring and administrative application to support Glass by writing Glassware, empowering IT managers to control wireless devices on their networks hands-free and at any time by seeing all connected staffers’ devices across their network in timeline cards. The service includes a slick feature that uses a wearer’s headset to locate a lost device and display its whereabouts on a map, giving an administrator the option to lock it down or wipe the device completely.

Glass also has great potential for the laborer not bound to a desk. Augmate, a development shop in New York City, is using Glass to connect enterprise backends with digital eyewear to streamline workflows and business processes for field workers, with applications developed using the Glass UX for the automotive, manufacturing, construction, and aviation industries, among others. Imagine your UPS deliveryman scanning your package as he drops it off with Glass. What could Brown do for you then?

Glass in Medicine and Education

Practically every industry and sector is at least considering how Glass might work for them. While opportunity abounds for the startup community and existing brands looking to add another cash cow to their stables by creating can’t-miss Glassware, the most emerging areas with inarguably the largest implications on society are healthcare and education. Glass as a communications tool for students and faculty, both for remote learning and for enhancing in-class instruction, is booming at the elementary, secondary, and collegiate levels.

Northeastern University in Boston created a college course on Glass as a driver for healthcare innovation. Similarly, USC has developed coursework in how Glass can transform journalism, and the Glass Creative Collective is a partnership between Google and several design and film schools to advance the craft of visual storytelling. Several daring instructors have used Glass for telepresence and advanced videoconferencing, such as Michigan physics teacher Andrew Vanden Heuvel, who remotely took his students on a once-in-a-lifetime virtual tour of CERN via the Hangouts Glassware.

Healthcare practitioners are pushing the ecosystem forward by diagnosing illnesses, treating patients, and collaborating on research, with several surgical procedures now documented from the Glass first-person POV as a teaching tool to extend the audience normally restricted to the surgical gallery and demonstrate live cases with colleagues. Healthcare technologist Dr. Rafael Grossman broke new ground when streaming his work during the insertion of a feeding tube into a patient of his via Hangouts. The Ohio State University soon after used Glass for telemedicine when Dr. Christopher Kaeding performed reconstructive knee surgery, sharing the live video with colleagues via Hangouts and chatting with students and colleagues about the technique in real time, while not endangering the patient or requiring the doctor to constantly step away from the operating room. Whereas videoconferencing tools historically involved stationary cameras and monitors at awkward angles, participants literally saw the procedure through his eyes. In addition to Hangouts and the ability of Glass to facilitate video calls, surgeons have been able to review x-rays, radiology reports, and other forms of medical imaging pushed to their HUDs mid-procedure.

It wasn’t too long before other physicians followed suit and started sharing their knowledge, too. As of the time of this writing, Glass has been used in a number of anatomical procedurres and on three continents.

And in Australia, Glassware is being developed to provide hands-free tutorials for new mothers for proper breastfeeding.

Other physicians and clinics have been asking for Glass to help eliminate the need for clipboard-based patient charts by digitizing data and making it contextual as hospital caregivers make their rounds. Similarly, consultations are easier with video calls. Further still are the healthcare providers that are working on ideas to use Glass to communicate with patients, letting people in trouble make video calls, or being able to relay vital signs to medical responders through timeline cards through sensors. Philips Healthcare is putting a lot of R&D effort into using Glass for telemetry applications. Pharmacists are looking to Glass as a way to expedite the filling of prescription orders, which is one of several concepts that Glass Explorers Chris Vukin and Thomas Schwartz are building with evermed, a suite of products to assist practitioners and patients alike. A touching video showed how Children’s Memorial Hermann Hospital used Glass to let patients take a virtual tour of the Houston Zoo. And we’ve covered the booming space that is fitness Glassware.

On that note, much work is being done as to using Glass as a part of sports medicine and for the treatment of athletic injuries, as well as for broader use in trauma and triage situations for ER professionals, EMTs, and possibly even field combat medics.

Further, how might this also be used for other medical disciplines like chiropractic care, physical therapy, dentistry, plastic surgery, psychology, or psychiatry? Could a marriage counselor analyzing clients on the couch possibly use a Glassware-based polygraph, essentially a wearable lie detector app, to pick up on patterns in the couple’s voices to assess elevated stress levels, which would aide in their treatment? Could a person recovering from an injury going through exercises have their biolevels sent in real time to a telemetry center for analysis to track progress? We certainly hope so.

But of course, the implementation of these ideas in industries isn’t trivial. Concerns over privacy and security, HIPPA stipulations, issues involving interacting with young people, endangerment, and skepticism from the medical and educational communities will have to be addressed. Many organizations involved in health have taken the Glass hardware and removed all system software, installing their own proprietary Android forks based on the source Google released (as detailed in the previous chapter), to comply with federal regulations about confidentiality of data between provider and patient.

But the general consensus seems to be that the application of these new tools is a positive step in either line of work. Both the healthcare and education sectors have the resources, brainpower, and motivation to really push the platform far ahead in very rewarding ways. Many believe these to be the most noble of all pursuits when applying Glass in real-world scenarios, with each having ample coverage in the mainstream media and multiple Communities on Google+ actively discussing the pros and cons of Glass and wearable electronics overall. The Stanford MedicineX series of Hangouts On Air prominently featured Glass to critique the platform and its potential as a tool. It’s a stirring, pragmatic discussion about the future.

If you feel as strongly or just want to discover some of the many ideas being floated, we encourage you to listen in and speak up.

Accessibility

Another area relative to healthcare where the Glass experience can make great strides is for people with limited hearing, use of their limbs, motor skill impairments, or other forms of handicap. Google shared the inspirational story of Alex Blazczuk, a law student who after a car accident doesn’t have full use use of her hands due to paralysis. She documented going on a camping trip with her friends, negotiating the system controls on Glass by voice commands.

Spoken input and audible output are wonderful ways for people living with disabilities to interact with the system—voice in, voice out. There’s also work ongoing to make Glass work for people with autism, where the feedback loop in Glass might not be so natural.

Several companies are also heavily involved working on gesture-based program controls, including eye-tracking. And our friend Mike DiGiovanni, whose work we’ve rightfully cited before, is cobbling a GDK app that controls timeline navigation without voice or touch at all, using only head gestures. It’s very inspiring work—and very worthwhile.

Home Integration

One of the announcements in 2013 that really shook up Glass devotees was the revelation that Google had submitted and had been approved for patent applications positioning Glass as a hub for home integration, to be used as a remote control for things like controlling settings on your refrigerator and opening your garage door. The patent described Glass employing several communications methods, including RFID, QR codes, Bluetooth, infrared, and the curiously generic “visual identification” over which to wirelessly communicate with connected devices.

This finally gave a viable use case to the promising Android@Home initiative, which admittedly was a space that, after being announced at the Google I/O 2011 developers conference, saw next to no public traction exhibited outside of Google’s own Nexus Q project, which coincidentally was scuttled in 2012 mere months after its announcement. This bold revelation really got people starting to think about how Glass could interface and control objects in the real world. If you’re a hardware manufacturer looking to add a slick new dimension to your products, this is worth having your R&D team investigate and tinker with.

Revolv is already doing some impressive work with home automation by extending its own platform for Glass, using the HUD as a frontend controller for wireless remote domestic robotics (“domotics”), empowering users with the ability to access their lights, curtain blinds, garage door openers, and locks. Cutting-edge developers at the Nodebot conference were able to control a Parrot AR.Drone quadcopter with Glass using Node.js libraries and Glass gestures—a feat that was repeated by a team led by technologist Dave Martinez at the Breaking Glass hackathon.

And the Glass community cheered loudly when the news broke in early 2014 as Google announced its acquisition of Nest, the company that produces Internet-aware thermostats and smoke detectors. The much-speculated initiative for Google to have relevance with appliances, controllable by smart frontends including Glass, is incredibly cool. Commercials for more than two decades have hinted at consumers being able to monitor and enable/disable security systems and home gadgets with a phone call, touchscreen tap, or voice command. Imagine doing so on Glass in the middle of a jog or while at the store or in the middle of a meeting or while in class—you’d get a timeline card relaying the temperature of the room, and you could speak and/or tap and manipulate the device back home.

You could even set thresholds that would proactively push card updates to you if your living room got too cold or too hot, giving you the ability to raise/lower your IP-controlled device.

Chromecast and Home Entertainment

Not since the inevitable marriage of chocolate and peanut butter have two platforms been so fated to meet and join in union as Glass and Chromecast. Clearly Glass is a ripe platform for casting content through the HDMI dongle either as a possible sender and/or receiver and as a controller or mirroring source. To this end, LynxFit, which we’ve mentioned a few times before, was the first Glassware to incorporate Chromecast, allowing the users to fling their workout guide videos through its GDK app onto a large display. It’s a tremendous cross-platform feature that makes using the app incredibly sticky.

The Mirror API could also possibly be leveraged to generate timeline cards as a relay for a second-screen utility, like “now showing” or “coming up next” screens or something a little more static. Cable providers could let users manage their TV programming in their DVRs and library of media in their home entertainment systems. This might be THE feature that services like Netflix, Hulu, Pandora, and YouTube need to do things like provide Glass wearers access to their queues-at-a-glance, without actually trying to use Glass as a viewing/listening station. Glass could certainly be used to add further value to the leanback experience of enjoying Internet multimedia content on big home displays.

Obviously, we’ve framed this just in the context of entertainment media, but there are a ton more use cases—and issues—to consider. How could you combine the two?

Android Wear

The escalation in activity for development of applications incorporating Android Wear is also incredible. Glass plays a pivotal role in Google’s place in wearable computing, so integrating the Mirror API and the GDK with the Android Wear SDK and sharing notifications across wearable devices is exciting new territory. But just as we laid out in the chapters concentrating on design for Glassware, the litmus test is going to be having wearable devices like Glass, smartwatches, fitness trackers, and others not compete for a user’s attention, bandwidth, and notification attention—but to have all of them peacefully coexist.

These platforms shouldn’t be seen as rivals, but as partners. So the software shops and developers who master the art of having their applications and services work seamlessly across desktop, web, mobile, and wearables will quickly separate themselves from the pack. We hope to see you break new ground and write about your achievements in a future edition of this book!

Hardware Hacking and the Internet of Things

Just because there’s not the inherent ability to do certain things with Glass out of the box doesn’t mean someone won’t try. It’s the magic of having the innovative spirit—you’re bound only by your own imagination. Many first-generation ideas like turning Glass into a miner’s helmet or a construction worker’s scale sound corny…but so did turning an iPhone into a flashlight. And some forward-thinking people have made pretty nice chunks of change from those little endeavors. So because a hacker’s work is never finished, let’s now shift to moving outside the scope of rooting your device and get into extending Glass to communicate with other platforms entirely.

The community of hackers, builders, and makers using Arduino and Raspberry Pi microcontrollers is staggering, and they’re doing some really amazing work for very little money. With the rise in hardware startups, the Accessory Development Kit is a great starting point to create new electronics based on Android. The documentation is worth checking out, as is the Android Open Accessory protocol, on which the ADK is based.

But what if you’re not a budding electrical engineer with an entire garage full of spare LEDs, breadboards, soldering irons, and potentiometers? (First, we pity you. You’re really missing out—because until you’ve wired up a circuit that actuates a servo over the Internet based on motion detection, you’ve not truly lived.) But fear not, there’s hope.

Commercial kits let you take advantage of the Internet of Things, the ambitious concept that uses the monstrously expanded address range provided by IPv6 in tandem with smart devices to make them accessible and controllable online. Kits like Ninja Blocks, WeMo sensors, FitBit, Philips hue lighting rigs, and Lego Mindstorms give builders a prefab-yet-customizable platform on which to build Internet-aware electronics and see what data they’re gathering as its gathered. We’re really happy to see one of our favorite services, IFTTT, trigger actions from APIs from other well-known services in IFTTT’s ever-expanding stable of channels and connect those events to the Mirror API’s event framework. The possibilities are nearly limitless. And this requires developers to really get creative, while sticking to the Think for Glass foundations we’ve laid out.

A shining example of this type of foresight is the work of Sahas Katta, a young software engineer from California. Over a weekend in June 2013 he essentially rebuilt the Android native app for his relative’s Tesla Model S electric car using Glass APIs so he could remotely open/close the sunroof, unlock doors, check the battery’s charge, manage climate control settings, and turn on the headlights. He succeeded in making the car hackable. (Mercedes-Benz and Hyundai have since announced plans to integrate Glass into their automobiles as well.)

Even Google cofounder Sergey Brin himself has stated that at some point he’d like to see Glass function as a viewfinder for his DSLR camera, implying really cool cross-device communication. So the true scope for Glass as an extensible platform reaches far outside the box…and prism.

Peripherals/Accessories

This last topic is already seeing some great traction, as tricking out your gear has always been a big thing with techies. With Glass being a modular platform, there’s lots of room to make new things to work with it, make it even more personalized, or make it look even more unique. As slick as Glass’s design is, some people will want to customize the device and wear it out and about with style and personalization.

GPOP, a design company cofounded by San Francisco-based Glass Explorer David Lee and featuring the designs of artist Virginia Poltrack (creator of the Word of the Day Glassware), wasted no time in putting together snazzy sticker designs giving you a range of alternative frame colors for Glass, using cool visuals like snakeskin patterns, camouflage, a matte carbon fiber texture, iridescent dots, your favorite sports team’s logo, crazy stripes, and other ways to make your wearable computing device truly stand out in a crowd.

Custom themes have been a staple of operating systems since Windows 95, and who among us hasn’t downloaded a cool skin for our browser or rocked a neat case with our alma mater’s logo on our tablet, set a neat ringtone on our phone because we were tired of everyone using the same polyphonic version of Canon in D, or modded the heck out of our Xbox?

The market for Glass peripherals is expected to be huge, giving you yet another avenue to look even more distinct as you wear your computer. Charging platforms, docking stations, holsters, carrying cases, and gear with colors and designs matching your phone and tablet to your Glass headset are expected. We also fully anticipate visual artists to create downloadable templates the community can use for timeline cards to expand the set available in the Google Mirror API Playground.

And of course, there’s the merchandise, both officially from Google and knockoff fan products—never discount the importance of good swag.

And the space for accessories for Glass is expected to bloom. No, check that—explode. Alternative frames alone should be a big hit. Neoprene or chain lanyards to hang your headset from your neck while not actively using it from companies like Croakies would sell like hotcakes. You can even build upon the ecosystem itself, as the folks over at Remotte at doing with their handheld remote control device, which uses a variety of sensors, programmable LEDs, and a tactile keypad to control Glass and other Bluetooth-connected smart devices. Neat idea!

In Closing

You now have the tools to do great things with Glass and really push the envelope, so if you’re willing and daring, go for it! We’d love to hear, see, and use what you come up with, and we’ll be happy to share your ideas in a future edition of this book.

Remember, the first prototype of Google Glass started out as a pair of modified circuit boards taped to either side of an off-the-shelf pair of sunglass frames for counterbalance, with ribbon wires all over the place and a small prism. But with a whole lot of work and thought, the final product didn’t turn out too bad (and two guys from opposite sides of the planet even wrote a book about it together).

So we encourage you to use the platform to its fullest. Keep tinkering with your own ideas, keep learning the development frameworks, and commit the design guidelines to memory. There’s a lot to be done to make the Glass ecosystem grow and prosper. We can’t wait to see what you do with it, and we’ve both got many more creative ideas that we want to start using with Glass, too.

It’s been our pleasure to share our philosophy with you to help you get the most out of the platform. We hope you’ve enjoyed reading our book as much as we have putting it together. We invite you to interact with us at Glass meetups and events and get in touch with us on Google+. And remember, always Think for Glass!