Sharing Resources with Glassware - Develop - Designing and Developing for Google Glass (2015)

Designing and Developing for Google Glass (2015)

Part III. Develop

Chapter 11. Sharing Resources with Glassware

We’ve done some exciting things with timeline items so far by using the Mirror API—creating them, adding menus to them, deleting and updating them, and replying to them, but the real power behind the Mirror API ends up being attached to contacts and being able to share things with them. We’ve mentioned a lot so far in this book about the ability to share information, and this chapter drills down into what it takes to manipulate this powerful feature in your services. The concept of sharing timeline cards that contain content in a variety of media formats created by the wearer—or even cards distributed by other Glassware—is as important as subscriptions. In fact, the latter couldn’t exist without the former!

In this chapter, we’ll take a look at the basics behind sharing, but also learn how Google’s implementation of sharing takes mobile computing a step further, making for some really interesting opportunities for your projects and the destinations your data can have. This includes both the Share menu item, as well as leveraging voice commands that are present on the main Glass menu.

The Share Menu Item

In our hypothetical chat Glassware from the last chapter, we covered replying to a message. If we think about other operations we tend to do on messages, we can see that we’re missing a forward-like command. If you looked at the list of available values for theTimeline.menuItems.action property, you’ll have noticed that SHARE hasn’t been discussed yet. You put the two together…and there you have it (Figure 11-1)!

This introduces an interesting problem. It suggests that not everything can be shared—it’s up to the Glassware that creates the card to offer the opportunity to share it. You may be tempted to hoard your precious card for yourself—but resist the temptation. Users are going to expect to be able to share cards with other services—this is, after all, one of the core features of Glass. And this is the fourth Noble Truth from Chapter 5Avoid the Unexpected.

The Share action for Glassware is a powerful facility, allowing resources to be passed onto other Glassware to be processed

Figure 11-1. The Share action

We’re not going to show you how to add the Share action item—you should be able to figure it out from everything you learned in the last chapter and by referencing the documentation about the Timeline. But you may be wondering how to specify who or what your users will be able to share their cards with. We’re not going to include a contact as part of the card—that works for replies, but doesn’t make sense for shares.

Instead, contacts will need to be registered in their own dedicated collection.

Share Contacts

Looking at the Contacts resource representation, you should have a sense of familiarity. We have the expected operations: inserting new ones, deleting old ones, getting or listing our contacts, or updating or patching them.

Creating and managing contacts in this way is different than including a contact as part of a timeline item. Contacts attached to a card are useful for just that card; contacts added by calling Contacts.insert are available as sharing targets and for voice commands. We sometimes refer to these contacts as “sharing contacts” to highlight their role.

Sharing contacts have a number of attributes you need to set. A few of the properties listed on the resource page aren’t necessary for a sharing contact, but these are the basic ones:

id

Unlike a timeline ID, you will be setting this value yourself and Glass ignores it except to echo it back to you when you get an event involving this contact. If you have multiple contacts, and we’ll explore why you may or may not in a bit, you should set this to something meaningful to you (for example, the ID in your own database of the user represented by this contact).

displayName

The name to show for this contact. It should not include the name of your application, since a listing of the contact will show your name before the contact name. This will be shown in a list when triggered from any “OK Glass” audio prompt or on a full card (with the imageUrlsspecified—see next item) when selected from a tap.

Let’s look at some examples in Figures 11-2, 11-3, and 11-4 of how a resource in Glass—a photo, a video, even a link—can be shared with an entity in the Glass ecosystem.

The Share voice command displays all of the available Glassware that can handle the resource a user is sharing

Figure 11-2. The Share voice command

Resources can be shared with Glassware, in this case a Google+ circle...

Figure 11-3. Glassware sharing options

...or even a Google+ Communities group!

Figure 11-4. More sharing options

imageUrls

This will be a list of URLs to show as the background for this contact. Images should take up the entire 640×360 display and have a nonblack, nontransparent background.

acceptTypes

You should only have your contact listed for content types that you’re prepared to handle. It doesn’t make any sense for someone to share a video with you if all you know how to handle is a picture, so this is where you would list all of the MIME types that you can handle. By default, if you leave this property blank, Glass assumes support for all MIME types. Wildcards are supported, letting you specify image/*, video/*, and audio/*, in addition to individual formats like video/mp4. Glass natively records video in video/mp4 format and stores images in image/jpeg format, but remember that other Glassware may have their contents in other formats.

THE KINDA-SORTA-NOT QUITE RELATIONSHIP GLASS HAS WITH AUDIO FORMATS

While you can list supported audio formats for the acceptTypes property, Glass at the moment doesn’t allow you to attach audio-only files to timeline items. For instance, you can’t put an MP3 on a card and push it to a user. The media player Glass uses assumes video, so in cases where you absolutely need an audio track to be distributed, create a video with a static background and then send the URL as a link in a card or upload the clip to YouTube, and let users stream the file from a server, as we covered in Chapter 10.

If we were creating some Glassware that did photo editing, we might create a contact with the following attributes:

{

"id": "invert",

"displayName": "Invert Colors",

"imageUrls": [

"https://dl.dropboxusercontent.com/u/12019700/invertcolors.png",

],

"acceptTypes": [

"image/jpeg"

]

}

Most Glassware will want to set up at least one contact when the user first logs in—at the same time you set up the subscription. This makes a great deal of sense since we’ll be told about shares through the callback mechanism.

When someone shares an item with the contact for our Glassware, Glass does a few important things that aren’t always obvious:

1. It creates a copy of the item being shared and inserts it at the front of the user’s timeline to the right of the home card. If you’re watching Glass, you’ll see this happen.

2. It gives ownership of the copied item to our Glassware. Remember back to Chapter 10—each card is owned by one, and exactly one, Glassware. This is how it makes it ours to control.

3. It then notifies us about this new card via the subscription we have set up for this user.

So using a subscription to the Invert Colors contact as we laid out above, we might get a notification body such as:

{

"collection": "timeline",

"itemId": "f733a4bd-7a9a-405f-a820-624ad3fe6db5",

"operation": "INSERT",

"userToken": "12345678901234567890",

"verifyToken": "shibboleet",

"userActions": [

{

"type": "SHARE"

}

]

}

We’ll follow our usual procedure to get the info about the user, verify the verifyToken, and get a copy of the new timeline item that has been generated on our way to processing it. The item should look familiar, but with some added touches (we’ve removed some elements for clarity):

{

"kind": "mirror#timelineItem",

"id": "f733a4bd-7a9a-405f-a820-624ad3fe6db5",

"recipients": [

{

"kind": "mirror#contact",

"source": "api:1029263551083",

"id": "invert",

"displayName": "Invert Colors",

"imageUrls": [

"https://dl.dropboxusercontent.com/u/12019700/invertcolors.png",

],

"acceptTypes": [

"image/jpeg"

]

}

],

"attachments": [

{

"id": "ps:5997490649262926546",

"contentType": "image/jpeg",

"contentUrl": "https://www.googleapis.com/mirror/v1/timeline/f733a4bd-7a

9a-405f-a820-624ad3fe6db5/attachments/ps:5997490649262926546?alt=media"

}

],

"menuItems": [

{ action: "DELETE" }

]

}

We have a lot of information we’ll need from these two objects. We know who is doing this, of course, from the user ID. We know that this is a share from the Timeline.userActions.type attribute. We can tell that this is a share that wants to invert the colors from the value ofrecipients.id matching that for the invert contact we created. Finally, we have the Timeline.attachments.contentUrl attribute, which gives us a URL we can fetch to get the image itself. Notice that there is no HTML or text attribute for this item.

Before we go and fetch it, however, there are two things we need to pay attention to.

First, if we look at the Timeline.attachments resource, we can see that one of the possible attributes is isProcessingContent. We don’t have that value set here, so we can assume it is set to false and the content of the attachment is ready for us to fetch, but especially when we’re dealing with video, we might find it set to true. So what do we do? Since we’re handling this as part of a job queue, we might just defer the job for a few more seconds and run it again later. When that time comes, we can pull the timeline item again and see if the attachment has been processed yet.

Secondly, when it is time for us to get the content, we still need to authenticate to access the contentUrl. Fortunately, most of the libraries we’re using provide utility methods to do an authenticated URL fetch, but keep in mind that you will either need to use them to get the data, or you will need to include the auth token yourself as part of the HTTP headers.

Once we have the image, we’ll do our color inversion on it so we have a new image. What do we do with it now? We have a number of options.

We could update the original item that was shared with us, replacing the existing attachment with the new image that we just generated. If we do this, we might also want to update the menu items so the user can reshare this image with some other Glassware or send it to a friend or something.

We could just create a new timeline item with this image. This preserves the original image shared with us, but supplements it with the new image. We should probably provide both the DELETE and SHARE menu items, while we’re at it.

While both of these might be good, depending on your exact situation, one great alternate solution would be to put both of the images into the same bundle. This would involve creating a new item, just as we suggested, but also including a bundle ID. We would also edit the item shared with us to add that bundle ID as well. Depending on our needs and desires, we may want to add some text to mention which filter was used or explicitly pick which image is the bundle cover.

This works great if we have just one filter to apply, but what if we have more than one to offer? We could do both color inverts and image rotation. If we’re rotating the image, we may need to provide rotation of 90, 180, and 270 degrees, not to mention image flip. And then we can start getting fancy by providing all sorts of filters. Would we need to create different Glassware for each one?

Hopefully you’ve realized that we don’t. Each option can be a contact. We can distinguish which filter is being requested by examining the id on the recipients of the new item shared with us and then route it to the correct function to do the processing.

One thing we should keep in mind, however, is that wearers won’t want to swipe between dozens of contacts to determine which filter they want to apply. That would create a very poor experience. We have a couple of approaches toward resolving this dilemma.

Firstly, when they sign up (and at any point they return), we can let them configure exactly which filters they want available. Depending on the settings they choose, we can add and remove contacts representing those filters. Remember that we’re designing for Glass here—we need to simplify their options on the device as much as possible and leave the more difficult options to mobile or desktop configuration.

Another option, however, which we might want to use in connection with the first, might be to let them select which filter(s) they want to apply for most of their shares—either individually or layered on each other. So they might create a “Frequent filters” sharing option that takes the picture being shared and creates three new pictures: one with faux HDR treatment applied, one that is brighter, and one that applies both a darker filter as well as a desaturation filter. Each of these new pictures would be saved in the same bundle with the original and labeled with a footer indicating which filter(s) were applied.

YOU CAN’T SHARE TO MULTIPLE ENTITIES

In case you were curious, you’re not able to share to multiple entities at the same time. You’ll need to perform and handle multiple transactions if you want to share a resource with a Google+ Communities group, a Google+ circle, and another Glassware service. This is by design, as not all entities will handle a resource the same way. It also deters senders from being spammy and prevents Glassware producers from finding a sneaky way around using their API quota.

While the footer with filter information might be interesting, it might be even more interesting to have the wearers annotate the picture with some text describing what the picture is. We can do this by allowing them to add a caption to the picture through the sharingFeatures property.This would change our contact to something like Figure 11-5:

{

"id": "invert",

"displayName": "Invert Colors",

"imageUrls": [

"https://dl.dropboxusercontent.com/u/12019700/invertcolors.png",

],

"acceptTypes": [

"image/jpeg"

],

"sharingFeatures": [

"ADD_CAPTION"

]

}

“The “Add a caption sharing feature

Figure 11-5. Adding a caption to a shared resource

Now, after they share the image, they’re prompted to say “OK Glass” and add a caption if they wish. The callback we get is the same, but the timeline item that was inserted now also contains a text attribute with the text that was transcribed:

{

"kind": "mirror#timelineItem",

"id": "f733a4bd-7a9a-405f-a820-624ad3fe6db5",

"text": "beach at sunset",

"recipients": [

{

"kind": "mirror#contact",

"source": "api:1029263551083",

"id": "invert",

"displayName": "Invert Colors",

"imageUrls": [

"https://dl.dropboxusercontent.com/u/12019700/invertcolors.png",

],

"acceptTypes": [

"image/jpeg"

]

}

],

"attachments": [

{

"id": "ps:5997490649262926546",

"contentType": "image/jpeg",

"contentUrl": "https://www.googleapis.com/mirror/v1/timeline/f733a4bd-7a

9a-405f-a820-624ad3fe6db5/attachments/ps:5997490649262926546?alt=media"

}

],

"menuItems": [

{ action: "DELETE" }

]

}

Captioning adds some powerful new options for what you can do with share contacts, but there are a couple of things you should be sure not to do. Don’t use this as a way to issue voice commands and don’t use this as a way to create a text-only document that is devoid of the media it was shared with. For the former, just create another share contact—the voice system will handle it much more reliably. For the latter, we have voice commands.

Voice Commands

We can take pictures or video with Glass and then share it to our apps, but how do we do the same thing if we want to send a text message? For this, Glass has provided two voice commands off the main menu, and has indicated that additional voice commands are forthcoming.

We can set up contacts so that they are valid targets of the “Take a note” command, the “Post an update” command, or both. If we think back to our simple text message application, we had Reply and Share commands…and now we’re seeing how to create a new message.

VOICE YOUR IDEAS

If you’d like to see other voice commands as part of Glass, Google is taking requests. Make sure your suggestions fill a generic need, and not just a need that your Glassware alone can fill, and are distinctively pronounced from other commands. Review some examples at the Google’s Voice Command Checklist page, which also has a link to the form to submit your request.

Contacts need to be registered with the voice commands they’re willing to accept when we add them; we do this with the Contacts.acceptCommands attribute. So if we’re setting up our chat client contacts, we might create one such as:

{

"displayName": "Abe",

"id": "user-prez16",

"imageUrls": [

"https://mirror-api-playground.appspot.com/links/lincoln.png"

],

"sharingFeatures": [

"ADD_CAPTION"

],

"acceptCommands": [

"POST_AN_UPDATE",

"ADD_A_NOTE"

]

}

Did you notice that we didn’t explicitly list an acceptTypes attribute? In this case, we want any multimedia format to be shared to our pal Abe. A far cry from him sitting at the telegraph station, huh?

If we send him a message by announcing “OK Glass…Take a note…Abe” our Glassware will receive a callback with the notification body containing something like:

{

"collection": "timeline",

"itemId": "09ac1ca0-2de6-40f9-9c20-c1b36f665970",

"operation": "INSERT",

"userToken": "12345678901234567890",

"verifyToken": "shibboleet",

"userActions": [

{ type: "LAUNCH" }

]

}

…which is similar to what we’ve seen before, but with the userActions.type of "LAUNCH" indicating this was launched from the home card menu. After we go through our usual verification steps and fetch the item, it might look something like:

{

"kind": "mirror#timelineItem",

"id": "09ac1ca0-2de6-40f9-9c20-c1b36f665970",

"text": "got your theater tickets for this evening",

"created": "1865-04-14T18:06:37.118Z",

"updated": "1865-04-14T18:06:37.118Z",

"displayTime": "1865-04-14T18:06:37.118Z",

"recipients": [

{

"displayName": "Abe",

"id": "user-prez16",

"imageUrls": [

"https://mirror-api-playground.appspot.com/links/lincoln.png"

],

"sharingFeatures": [

"ADD_CAPTION"

],

"acceptCommands": [

"POST_AN_UPDATE",

"ADD_A_NOTE"

]

}

]

}

This timeline item is pretty basic and pretty much what we’ve seen before—we have text, but that’s about it. So when we process this message we should add it into our database, then update the card so it contains our internal identifier as the sourceItemId. We also need to add the creator field, so the REPLY menu command will work correctly…oh, and some menu items such as DELETE while we’re at it, too!

We have two notes of caution here, and both should sound somewhat familiar by now. You may be clamoring at this point to submit your application name as a voice command so you can corner the market on its use. Don’t bet on this option. Although you have control over the name of the contact used by a voice command, the commands themselves will be set by Google, and the company has indicated that it will be focusing on fairly general trigger phrases using imperative verbs that many apps will be able to use. Remember—actions, not apps!

Secondly, you’re probably best off treating the text the user has dictated as an opaque message. Don’t try to parse it, or prepare to suffer the consequences if you do. In particular, you may notice that when transcribing the spoken word Glass sometimes chooses to use the numeric “2” or other times uses “two,” “too,” or “to.” Android’s speech-to-text engine has powerful contextual features that try to discern what form of a homonym is implied by the user, but it’s not perfect, so don’t bank on it. Being prepared to handle these isn’t always fun, and may give you and your user unexpected results, particularly when you are expecting to use the text in a particular context.

On that note, as we’ve discussed many times—Glass is all about context. We’ll look a bit further at that context and one spot of context in particular, the user’s location, in our next chapter.