Designing and Developing for Google Glass (2015)
Part III. Develop
Chapter 12. Context Is King: Using Location and Other Signals
What are the three most important things about buying a house? Location, location, and location. There may be a lot more relative to building great Glassware, but location is certainly one of the most important cues that we have about what someone wearing Glass will want to know about at any given moment. It is a core principle to the third Noble Truth: Keep it Relevant. And the one unifying force that binds not only location data, but also signals relative to time, the user’s activity, identity, proximity, scheduled events, and presence of other devices is context.
Location isn’t the only tool we have at our disposal to help read the mind of someone using our Glassware, but it does serve as a good foundation upon which to base our other contextual inputs. We’ll be discussing how Glass directly supports location-aware services, some unexpected ways we can use this information, and how it serves as a model for other context-based data that we may wish to incorporate in our Glassware.
We’re going to start with a warning, however, which you’ll see echoed a few more times in later chapters. Although all of Glass is evolving, location services are still somewhat immature. You should expect that it will only improve as we move forward, but what we describe here will serve as a cornerstone for what you can expect in the future.
WHY AREN’T WE THERE YET?
With location being so important and a huge feature on Android, why does it seem to have so many problems on Glass? We can only speculate, but there are probably a few reasons why. Checking for GPS signals is extremely resource intensive, especially on the battery, so Glass is pretty conservative about how frequently it tries to get your whereabouts when it isn’t explicitly navigating. Although there have been some improvements on this front with recent versions of Android, as of the time of this writing Glass was still using an older Android build—expect some improvements when Glass gets an update.
You might wonder why Glass isn’t just using the location services that have been a part of Google Maps for a while now. They may have anticipated some of the changes with Maps that moved location services to Google+ and changed how location tracking was done for mobile. As mobile location tracking continues to shake out from this change, we might expect Glass to adapt as well.
Enabling Location
You may be surprised to learn that you actually need to do something to enable location services with Glass. After all, if our users expect us to read their minds, and where they are is a big factor to this, wouldn’t it just be assumed that we should know where they are? Certainly location services on Android tend to work behind the scenes without the user having to be very involved.
As we explored early on, Glass has had to tread lightly when it comes to things that could be seen as invading a person’s privacy…and your location is certainly one of the things that a person may most wish to guard. While it is easy to turn your location on and off on your phone, you don’t have the same kinds of controls with Glass. So any application that wishes to know where you are must explicitly ask for permission to find out.
Can you guess how it asks for the rights to use geodata? The very same way our Mirror API–based Glassware asks for permission for anything—it has its own OAuth scope that indicates we want to view a user’s location. If you’ve forgotten what an OAuth scope is, take a quick look again atChapter 8. (Don’t worry, we won’t notice.)
You should already be requesting the https://www.googleapis.com/auth/glass.timeline scope, and probably at least one other scope such as profile so you can get the user’s information. To these we will need to add the https://www.googleapis.com/auth/glass.location scope, which will instruct Glass to subscribe to location updates, provide location information along with timeline events, and give us the ability to make queries for a user’s location at any specific time.
Where Do You Think I Am?
You might remember from Chapter 10 how when we talked about subscribing to timeline events we mentioned that we would talk about subscribing to location events in this chapter? The time has come! And the subscription itself is amazingly simple. For theSubscriptions.collection property, we can specify “locations” to get alerted whenever there is new location information for our user.
When the information is available, we’ll get a callback similar to ones we’ve seen for timeline events:
{
"collection": "locations",
"itemId": "latest",
"operation": "UPDATE",
"userToken": "12345678901234567890",
"verifyToken": "shibboleet"
}
We might be tempted to use the same logic to process this callback as we did for timelines, but there are a few things we need to do differently. Most notably, we’ll need to call a different method to get the location event than we did to get the timeline item. Instead of calling Timeline.getwith the ID specified by itemId, we’ll be calling Locations.get.
We know…it’s going to be difficult to remember the difference between the two. We think you’re up to the task, however.
Calling Locations.get returns some JSON not unlike this fragment:
{
"kind": "mirror#location",
"timestamp": "2014-02-02T17:57:06.770Z",
"latitude": 37.4038194,
"longitude": -122.081267,
"accuracy": 22
}
Referencing the Locations documentation, we find the fields are pretty straightforward:
timestamp
When the user was recorded at this location. Note that although we’re requesting the “latest” position, it doesn’t mean it is very recent. We’ll discuss this a bit more later on.
latitude
A floating-point number indicating degrees north of the equator if the number is positive or south of the equator if the number is negative.
longitude
Similarly, this is a floating-point value indicating degrees east of the Prime Meridian (if it is positive) or west of the Prime Meridian (if negative).
accuracy
The best estimate of accuracy, in meters, of this reading at the time it was taken.
YOU WANT IT WHEN???
How often will we receive a location change event? Currently, no more frequently than every 10 minutes. That seems like a lot, but we can’t even assume that we’ll get it that frequently all the time. For a variety of reasons that may be out of our control, Glass may not update our location on the schedule we expect.
Fortunately, Google both knows about these issues and understands how important location is. We can expect improvements in frequency as Glass matures. In the meantime, we can start working with what we have with the knowledge that it will only get better.
Getting these periodic updates is a great way to judge when and what to deliver to our users. Are they still at work, are they getting close to quitting time, and does traffic in their area look bad? Perhaps now might be a good time to warn them. Have they not moved very far since the last time we got their location? Sounds like they don’t need new driving reminders about where the next gas station is.
That last suggestion raises a good question. We know where a person is now—can we find out where they were recently? Well…yes and no. Yes, there is the Timeline.list method that will let you get the most recent timeline events. Unfortunately, at the moment it only ever returns one event—the event with the ID of “latest.” Expect this to change in the future as well.
Location change events are all well and good, but can we tie them to events we’re already familiar with—things shared with us or new notes that are taken? This is easier than you might think.
Location as Part of Timeline Events
It turns out that once you request location information via the glass.location scope, you’ll get this information attached to every timeline item shared with your Glassware.
You may be encouraged by this—if your user is sharing lots of things with your contacts, your Glassware will get lots of location updates, and this will get past the problems of updates only every 10 minutes. Your mind is probably racing with a way to convince people to check in with you specifically so you can get the most accurate location possible.
Sorry to dash your hopes, but location is still only updated every 10 minutes, and this doesn’t offer you a way around this. The location attached to a timeline item is just the latest location at the time the item is shared. If your user is sharing an item that was created significantly earlier, the location won’t match the place where the timeline event was originally generated.
So why is this included with a timeline item when we could just request it? It saves us a function call (and thus some of our quota for our daily Mirror API calls), and if we’re requesting location information at all, it makes sense that we would find it useful when a user shares something.
WHAT HAPPENED TO GEOTAGGING IMAGES?
One of the issues that popped up in the early days of Glass was noting how images captured with the camera didn’t get EXIF location metadata. Once uploaded to Google+ via Instant Upload the location could be entered, but the manual nature of having to do so irked Explorers.
Geotagging, now a staple of photography in the Social Era, wasn’t available for Glass when either tethered to a smartphone via the MyGlass app, or even when on WiFi. This seems tied to the general location issues we’ve encountered in this chapter. As with everything else having to do with location, it seems safe to assume that this will eventually be resolved.
What sort of uses can we put the location to for a timeline event? Attaching the share to a marker on a map is the most obvious, and there is already some great Glassware that illustrates some of these concepts, but what else is location and a share good for?
It may seem flippant, but it all depends on what your Glassware needs to do. Keep in mind that locations can determine many things beyond the obvious point on a map. Your location may determine what language the local citizens are speaking or what currency they’re using, and if your Glassware is involved in translation or currency conversion, you may be able to take advantage of this information. If you know a person’s location and the current time, you may be able to determine things such as the current weather, the current tides, the sunset and sunrise times, or how bright it is outside. Timeline events are all marked in UTC time, but the event location may let you convert that into the local time because you’ll know the user’s current time zone.
Setting Things Straight
This previous concept of using the user’s location to determine the current time zone is a good one, but it has a flaw: to get the time zone for a transaction, you’ll take the location and probably forward it to a time zone service—for every single message that comes in. That can be a lot of extra server roundtrips and hits against your API quota. Since Glass already has this information, wouldn’t it be great if it could provide it for us? It turns out that it can.
The Mirror API provides us with the Settings.get method to retrieve this sort of information. For certain key values (Google calls them “IDs”), calling this method will return the value the user’s device is set for. The Settings resource accesses data that the user configures in MyGlass, which lives in the cloud, is accessible through the MyGlass mobile app and on the Web, and is exposed to both the Mirror API and the GDK.
To get the current time zone, we can request it with the timezone ID using a URL such as this (and providing the correct auth token):
https://www.googleapis.com/mirror/v1/settings/timezone
…which might return a JSON object looking something like:
{
"kind": "mirror#setting",
"value": "America/New_York"
}
(Clearly this is the example from Allen. Jason’s time zone, given that he lives in a place that’s 17 hours ahead of Eastern Standard Time, is literally more futuristic.)
With this, the time zone database that most languages support can convert the UTC time Glass provides us into a time format more useful to the user.
Location Becomes Localization
But just knowing the time zone isn’t necessarily enough to format a date and time correctly. Different countries and languages have different styles—not to mention different ways of saying the exact same thing! Localization, which displays content such as text, images, and videos in a format that’s appropriate for a particular region, and has always been a big part of Android and web development, is an effective solution for this problem. By supporting multiple languages, your Glassware broadens the reach of your products and adds great value. This can be applied not only to content within timeline cards, but also to menu items and voice commands.
MYGLASS SPEAKS YOUR LANGUAGE
Did you know Glass displays a different voice command prompt for capturing images depending on what the user has set as her default language in MyGlass? It’s true! For users having “English (United States)” as their preference, the familiar “Take a picture” will appear. But for our friends over in the UK, “Take a photo” will be accepted when they have “English (United Kingdom)” as their language (Figure 12-1).
Additionally, time formats and spelling also adjust—for example, the UK setting applies the 24-hour clock to the home card and renders the “Recognize” command as Recognise.”
Figure 12-1. The Language setting dialog in MyGlass
Fortunately, the Mirror API provides us with a locale setting to get this information. So we can ask for the following:
https://www.googleapis.com/mirror/v1/settings/timezone
…and get back a standard locale string as part of the response:
{
"kind": "mirror#setting",
"value": "en-US"
}
Subscribing to Locale Changes Saves API Calls
These fragments of data are useful to get a user’s current preference when they set up your Glassware, but they don’t really make things easier in the long run. We still might need to issue another API call periodically to get the time zone in case the user is traveling. (It’d be far less likely they’d change their locale, at least.) It won’t change that often, but it would be helpful to know when it does.
Fortunately, we can leverage the Subscription object that is already part of the Mirror API to do this. Instead of subscribing to the Timeline collection, we can subscribe to the Settings collection. This will send us notifications when any of the settings change. If you need a reminder about how subscriptions and notifications work, check out Chapter 10 on that topic again. (And you may be wondering if you can do this sort of thing for location—take a look at the documentation about Subscription and you may find what you’re looking for, but remember some of our caveats earlier.)
Your backend can then use whatever templating framework you want (Rails, Django, ASP.NET, Node.js, etc.) to serve the appropriate resource string, which is then pushed to the Mirror API and delivered to your users.
It’s a very good idea to proactively have your Glassware support multiple languages, even though as we write this the only languages available in MyGlass are the US and UK flavors of English (or is it flavours?). When you submit your Glassware for review, you’ll get helpful pointers from Google on how to best implement localization, if you require it.
You can find more tips from the Glass team on best practices for localizing your Glassware for both the Mirror API and the GDK.
Other Contextual Signals
In all of these cases, location plays an important contextual role—but it’s not the only type of signal you can use in your applications. This approach emphasizes the personal network consisting of the user’s surroundings, the presence of others, what the user is doing at that moment, times and dates, and what’s happening both in the user’s vicinity or based on events they may have through other platforms like calendar or to-do applications. These signals tap a constant stream of inputs swirling around the user at any given moment.
A fantastic practical example of a contextual action is what a buddy of ours, E. John Feig, did with Talkray, a calling and messaging app with Android Wear support. John and his team at TiKL created an auto-reply feature using activity recognition, based on the user’s real-life action. If a user receives a message and then chooses the Auto-reply busy button, the device running Talkray checks to see how fast the user is moving; if it detects that the user has a certain velocity, it’ll send “I’ll get back to you later, I’m driving” as a canned response. It’s the perfect contextual microinteraction!
Let this be a springboard for how you might use contextual signals in your own wearable applications.
ANDROID WEAR AND THE PERSONAL NETWORK
The Android Wear SDK is built in such a way so it detects all the devices the user is signed into under the same Google account—including Glass. This means it can read data from, push notifications to, and sync information between all connected devices within a user’s network seamlessly.
Context and the Future
You’re probably looking incredulously at this chapter—that’s it? That’s all there is to using location with Glass? Well, no. But that’s all there is to say about how to use it. The concepts are very simple and build on the things we already know how to do with Glass. The power will be in how you use it.
Location provides an example of how we can use other information in context. We will often need the user’s permission to get access to additional sources of data. Some information we will want to act on as it changes and is delivered to us, while other bits of data are only relevant when the user shares something with us. We need to be prepared to handle both to make great contextual Glassware. Location is an incredibly valuable signal that contributes to the overall goal of understanding the users—where they are, what places and things are around them, and who is nearby—in addition to who they are, what their interests are, what social connections they have, and what they’re currently doing. But, it’s still just one piece of a very large puzzle.
What other sorts of information signals should we be thinking about as Glassware architects? Consider a service delivering information about Major League Baseball—clearly we’ll want in-game events delivered so we can pass them on to our users, which is a pretty straightforward scenario, seeing as how America’s pastime is ripe with infinite data with insatiable demand. What other contextual signals might we need to craft messages with high utility?
What if we could see that they were having an afternoon meal with a friend, and the profile information for that companion indicated they were a devout fan of the local ballclub? We might send schedules, scores, and stats about their team before (or during) the lunch meeting to provide conversation fodder. Or maybe information from within the division to which the team belongs, which means including regional teams. Or perhaps we might respond to a picture of a sports logo by providing some trivia about that team.
This is Think for Glass in action, as we’re leaning hard on relevance to life experience, which the user will surely appreciate.
Continuing our example, what if we know that our user is planning an airplane flight, perhaps because we have access to her calendar or because she told us explicitly. Once we have this information we might use her current location to determine which side of the flight she is on. If she hasn’t taken off yet, we might give her gate information and flight status. But if we know she is at the destination airport, we might send her information about where the baggage claim is and let her call her ride to let them know she is on the ground.
Our application needs to be smart enough to know the difference and to have all the information the user needs (departure gate, baggage claim, and ride contact phone—all of which come from different sources) when she needs it. At the same time, if she is still at the source location when the flight lands, we may need to apply some extra logic to figure out if she missed her original flight, or if she is just waiting to make sure a family member has reached their destination safely.
In some ways, this discussion about data sources and context is a fitting conclusion to our chapter on location and also to our section about working with the Mirror API. Glass isn’t about what Google gives you—it is about what you do with those tools. You’re bringing information to the table that your users are interested in. Really think about how to blend the idea of enriching the user’s experience with location data to the Glass paradigm. Make the data and the ways it is delivered to the user part of the experience, not just lumping in a mapping feature.
The Mirror API is all about how you deliver that information to your users and what you’re giving them to interact with it. What we’ll be talking about going forward are other places with Glass you may wish to explore.