Hello App Inventor!: Android programming for kids and the rest of us (2015)

Chapter 11. Using speech and storing data on your phone

In this chapter, we’ll look at the different ways you can change words (text) into the Android voice and how you can record your own voice and change that into text. We’ll also examine how you can store items such as text and pictures on your phone.

What is the difference between text to speech and speech to text?

In recent years speech recognition has improved a great deal, but it still isn’t perfect. Text to speech is more reliable and effective but still has some funny quirks, such as mispronouncing words. Technical words are often mispronounced—for example, megapixel is pronounced “maygaypixel.” You’ll also have problems with Roman numerals: King George V is pronounced “King George Vee” by text to speech. But speech to text and text to speech both have the potential to allow you to produce some fun apps, so let’s give them a go.

The possibilities presented by text to speech are massive, from creating apps for visually impaired people to including spoken words in a game. The most basic example would be text added to a button so that when the user clicks the button, the text is read aloud. But text can be input by the user, too, or received via an SMS text from someone contacting the user, and automatically read aloud.

Learning Point: What is SMS?

SMS means Short Messaging Service. It’s the system that lets mobile phone users send and receive texts. The messages are limited to 160 characters (letters, numbers, or symbols). Longer text messages are sent as multiple messages and put back together by the receiving phone service.

Once you’ve seen the basic functionality of speech recognition and text to speech, you can use this knowledge to create other apps. We have included in this chapter the functionality to save and reuse data in App Inventor’s very useful local (stored on the phone) database called TinyDB.

Have you ever woken up in the morning after a crazy dream and discovered that by the time you see your friends you’ve forgotten the details, so you can’t have them analyze it for you? Perhaps it’s just me (Paula), but when I wake up I can hardly hit the button on my huge, cartoon-like alarm clock, never mind type onto a phone keyboard to write down dream details, but I can just about speak. The following app enables you to use Google speech to text to record your dream.

The first app you’ll put together is a simple dream-recorder page using speech to text. To ensure that your dream recorder is kept safe, you’ll include a password so that only you can access your speech-to-text notes. As we mentioned earlier, Google Voice is a bit dicey (or maybe it’s my accent that it can’t understand), so what you say won’t always be what appears onscreen. But hey, that could make the dream more interesting!

My Dream Recorder app

PURPOSE OF THIS APP

This app uses the speech-to-text facility to record your dreams. It can only be accessed through a password-protected screen, leading to a dream-recorder screen. To create this app, make sure you have a device to test it (emulator doesn’t support two screens). And note that you need to be online to use speech recognition.

APP RATING

ASSETS YOU’LL NEED

Cloud image background.

Note

This is the first time you’ve used two screens in an app, and it does make a difference in how you put the app together. It makes sense to first create the password-protected screen, next complete the blocks for that screen, then add the Dreamscreen and complete its blocks, and finally test the whole thing on a phone. You can toggle between the two screens from the Designer, which then changes the blocks in the editor.

Once you’ve set up the user interface, you can get down to the nuts and bolts of how the app works. You start by putting together the blocks for Screen1 (which is effectively a password interface to access Dreamscreen). So, first you need to define the password: you’ll check the password and either open the Dreamscreen or give an error message. Then you need to create Dreamscreen.

1. Defining the password

Before you use the password in the programming of this app, you have to define it as a variable as shown here.

2. Checking the password and opening the second screen

When the user clicks SubmitButton, you want them to access a control event that you can imagine says, “If the password the user has input matches the one you’ve stored, open Dreamscreen. If the password doesn’t match the stored password, show an error message that tells the user they have input the wrong password and should try again.”

In the open another screen screenName block, you insert a Text Dreamscreen block even before the second screen exists. It’s possible to do this because you have to type the name you’re going to call the second screen in the text box; then, once the screen exists and the user clicksSubmitButton, the app will open the other screen.

3. Adding the second screen

In the App Inventor Designer, when you click the Add Screen button, you’re asked to give your new screen a name. Once submitted, it can’t be changed, so be careful! Call the added screen Dreamscreen. When you’ve added it, two buttons appear at upper left on your screen in the Designer, with the names of the two screens.

The second screen can be simple: it has a button with the text “Tell me your dream”. The text produced by the speech recognizer will appear below the button in a label. Putting this part of the app together means you need to know how to store data in the TinyDB database; take a look at the Learning Point “Storing data in TinyDB.”

Learning Point: What exactly is a database?

A database is a collection of related data that is organized so as to allow quick retrieval of the data. Most apps and programs have a database. Can you think of all the databases you come into contact with every day? Your MP3 playlist, Facebook, your television channels, your running apps, Google, and your phone contacts list are just a few, before you’ve even gotten to work or school!

Learning Point: Storing data in TinyDB

The introduction of the TinyDB component takes things to another level and allows you to store information when the app is closed. Nearly every app or program has a database of some kind that needs to be added to. For example, it could be used to keep track of the highest score for a game.

App Inventor has a great component called TinyDB that allows you to use some of the advanced functions of a database without having advanced knowledge. In the My Dream Recorder app, for example, the TinyDB component allows you to store some text (your dream description). Then, when you next open the app, the text is still there. This is called persistent data—it’s obstinate and stays there despite you switching off the app! This is in contrast to non-persistent data, which gives up easily. For example, in the Cheeky Hamster app that you created in chapter 6, as each seed lands in the bowl, you score a point. But even if you run up a huge score, when you reopen the app, the score returns to 0. This is because although you dutifully use a variable to store and add to the score while the app is open, there is nowhere to store it once the app is closed; so the score resets to its default of 0.

USING TINYDB

You don’t have to set up a database to use TinyDB, but you do need to add a TinyDB component to your interface. It will laze around with the other nonvisible components until you give it a job to do.

TAG-VALUE

In order to use a TinyDB component, once you’re into the Blocks Editor you have to assign it a tag-value. A tag-value is like a variable name, but each piece of data that lives in TinyDB has its own tag-value and stays in your Android device’s memory—a variable doesn’t. As mentioned earlier, when you quit an app, anything temporarily stored in a variable disappears, but anything stored in TinyDB sticks around. For example, in My Dream Recorder, you need to store the result of the speech recognizer so it’s there the next time you open the app.

You could think of the TinyDB database as a table of tag value pairs. The tag is the title of the item to be stored, and the value is the item to be stored.

In this case, when you ask TinyDB to retrieve the value associated with the tag Dreamdescription it will retrieve the result of the Google Voice recognizer. The tag is always, always, always text, because it’s the name of the tag; but the value can be either a single piece of data (image, number, or text) or a list. Once you write over it, it’s gone.

This may seem a bit difficult. But after you’ve seen TinyDB in action in the My Dream Recorder app, you’ll practice using it again right away in the Inspiration Scrapbook app, part 2.

Now that you’ve created the password screen and the second screen, you can get down to the business of using speech recognition. You’ll first choose a DreamButton.click event handler that calls the SpeechRecognizer component, and then you’ll set the text box to display the result of the speech recognizer—that is, the text the user has spoken aloud. Almost at the same time, you’ll store the text in TinyDB and make sure the next time you open the app, the text (the description of your dream) appears in the text box.

4. Opening the SpeechRecognizer component

Opening the SpeechRecognizer component is as simple as activating the DreamButton.Click event handler and slotting in the SpeechRecognizer1 .GetText block.

5. Using the SpeechRecognizer text

After the speech has been recorded by Google Voice, you want the result of that recognition to appear onscreen. The next event handler enables you to take the result of the speech recognizer and do two things: show the text onscreen in Dreamscreen, and save it to TinyDB so it’s there when you next open the app.

6. Initializing Dreamscreen (or, setting the rules for Dreamscreen)

When you first open the app, it has an empty label in which the text from the Speech-Recognizer component appears. In order to make this text available when you reopen the app, you need to set Dreamdescription.Text to the contents of the TinyDB component, as in the following blocks:

Happy dreaming!

Taking it further

1. Extend the My Dream Recorder app so the password is also kept in a TinyDB and can be changed by the user.

2. Add to the My Dream Recorder app so you can record more than one dream per day and browse through them.

Inspiration Scrapbook app, part 1

PURPOSE OF THIS APP

This app is designed to help spur you on when you need some inspiration. The user clicks the photograph of someone who inspires them, and the Android voice reads a quotation from that person.

APP RATING

ASSETS YOU’LL NEED

Images of inspiring people (we chose J.K. Rowling, Nelson Mandela, Michael Jordan, and Oprah Winfrey) and a list of quotations from these people.

Most of you will know who our inspirational people are, but for any of you who don’t or who want a refresher, here is a little about each:

· Nelson Mandela was South Africa’s first black president. He is a peacemaker who strove to abolish apartheid in South Africa after being imprisoned for 27 years for planning to sabotage the then government. He won the Nobel Peace Prize in 1993 and has become a symbol of determination and forgiveness.

· Oprah Winfrey is an American talk-show host, an actress, a business woman, and the founder and supporter of many charities. She has overcome many difficulties to become one of the most successful and influential women of the 21st century.

· Michael Jordan is an American former professional basketball player, a hugely successful entrepreneur, and a contributor to many charities.

· J.K. Rowling is the British author of the hugely popular Harry Potter series of books. They’re the best-selling books in history, and the resulting movies are the highest-grossing film series in history. She overcame poverty as a single parent and within five years became a millionaire

On the companion website, we have placed three quotes per person. Here are a few examples:

This app is relatively simple but effective. If you follow the guidelines in the properties table and the images of the interface, you’ll produce a very personalized and enjoyable app. We imagine it being used this way: when you need a bit of inspiration, you can click a photograph once and enjoy the random quote that is read out. Having the quote be randomly accessed leaves an element of chance and will give the user a feeling of serendipity if the quote suits their particular situation.

Learning Point: Choosing how your app will sound

If the default Android voice sounds too much like your bossy gym coach, you can adjust it on your phone. To access the optional defaults, go to Settings > Language & Input > Text-To-Speech Output, click the gear icon next to Google Text To Speech, and then click Language on the settings screen. If you want to see illustrated instructions, go to www.wikihow.com/Change-the-Android-Voice.

When digging around trying to figure out how to do this, we found a very funny American thread in which the contributors agreed that mobile devices should have UK voices because they sound more natural and make the users feel like secret agents receiving instructions. We prefer a friendly female American voice, making the text sound more like a suggestion than a command.

Learning Point: Who inspires you?

Don’t forget, these are people we find inspirational. You may be inspired by other people. Customize this app for yourself. A good site for quotes is www.brainyquote.com. You can search by name or topic area, such as motivational quotes, quotes about love, and quotes about education.

For this app, you’re using the TextToSpeech component. First you need to create an event handler that responds to the user clicking a button to read out a message. Then you’ll define a variable that becomes the list of quotes. Next you’ll insert the quotes into the list and add a quote button so that when the user clicks it, a random quote is selected from the list.

1. Inserting the title button

When users open the app, they might be puzzled about what it does. So, the title is created to look like a button. The following blocks activate a text button that reads out an inviting message:

2. Defining the list variable

Depending on how many inspirational people you put in your app, you’ll need to repeat this process a number of times. The following example is for the Nelson Mandela list.

Before you establish your list, you have to define a variable. Go to the variable block, click the word variable, and change it to Nelsonquotes.

Now set up your quote list. The call make a list block is in the Built-in > Lists drawer. Insert Text blocks, and type in the quotes.

3. Adding the quote button

You want the Nelson Mandela quotes to be read out when the Nelson Mandela photograph button is clicked.

Inspiration Scrapbook app, part 2

PURPOSE OF THIS APP

This app takes Inspiration Scrapbook, part 1 to another, more interactive level. You keep your two favorite inspirational photographs and lists of quotations but replace the other two with the ability to take a photograph of someone who inspires you and insert quotes from them.

APP RATING

ASSETS YOU’LL NEED

Images of inspiring people (for this part of the app, we’re going to introduce you to a couple of giants of the computing world).

Learning Point: Tim Berners-Lee

Tim Berners-Lee is a British computer scientist who is said to be the inventor of the World Wide Web. Marissa Mayer is an American computer scientist who was a computer engineer for Google and is now the Chief Executive Officer (CEO) of Yahoo.

Note

Don’t forget, this app only works with an Android device that has a camera. You can put it together on your PC, but testing will need to be done on the phone.

In case you’re coming straight to part 2, we have included the entire design table for you to follow. Brace yourself: there are a few new concepts to learn as you put together part 2.

Do steps 1–3 from the Inspiration Scrapbook, part 1 section. Then start here with step 4 to adapt the app.

You need to add a button to enable the user to store the submitted quote and then take a picture of their own inspirational person using the Camera component. Next, you’ll send the picture produced by the camera to TinyDB. The last bit checks whether the user has already added a quote and a photo before reading out or displaying the result.

4. Storing the text from the text box in TinyDB

To store the text from the text box, you have to create a tag for QuotesTinyDB. You want this event to be triggered by the user clicking Submitquotebutton. Then it’s just a case of choosing the StoreValue block, giving the tag a name, and saying what will be stored in the tag—in this case, the text the user inputs into Submitquotetextbox.

5. Taking the photograph

This is the first time you’ve used the camera as part of an app, so it’s worth explaining a little about using the Camera component. You use Takepicturebutton to take the picture with the app. You then slot in the Camera1.TakePicture block, which instructs the app to take a picture. In this app, this gets you as far as opening the camera—the user has to click the “shutter” and save the picture.

6. Submitting the photo taken with the camera to TinyDB

But where does the picture go? Well, once you use the Camera1.AfterPicture event handler, you’ll see that image is available as a variable. Click it, and choose to get the image.

Now you want to move it to PhotoTinyDB. You also want to resize it, because the user’s camera may well take a photo in the range of 2048 x 1536 pixels, which is massive. The space you allocate on the screen for the photo is 100 x 100 pixels.

Learning Point: Megapixels

Previously we’ve looked at pixels—but cameras are often described as having megapixels (for example, 3.2 megapixels). A megapixel is roughly 1 million pixels (1,048,576, to be precise). Don’t forget that this is the total number of pixels in the image, so an image taken with a 3.2 megapixel camera produces an image with the dimensions 2048 x 1536: 3,145,728. You multiply the width by the height of the image to get the total number of pixels; round down and divide by a million to get the answer in more manageable megapixels.

7. Activating Ownpicturebutton

One lovely feature of this app is that it’s very personalizable. The next feature allows the user to hear their own quote read aloud when the photo button is clicked. It first checks whether there is anything in the QuotesTinyDB and, if there isn’t, reads aloud a standard message to encourage the user to take a photo and add a quote.

8. Understanding how to check TinyDB

Often when you’re dealing with variables and databases in a program you want to check whether there is anything in the database or variable. In this case, when the user first opens the app, the button that becomes the photograph needs to do something to prompt the user to use the app, and the text box needs to contain a hint to encourage the user to submit a quote. Following is a table that describes the functions of these features, depending on whether the user has just opened the app or is already using it.

You can see that the function of the text box and the Your Picture Here button changes once the user has submitted the quote and the photo to the respective TinyDBs. This is important, because you need the app to check whether anything is in the TinyDBs before deciding what to display or do.

When the app starts, you want to set the rules for it (this is called initializing it) to make sure everything is working correctly. You need to check whether the tag-values in the TinyDBs are 0 (whether there is nothing in them).

Your users will enjoy being able to personalize the app using inspirational people from their own lives. By learning how to take words and pictures that are input by the user and add them to a database (TinyDB), you’re able to create something individual for each user.

Taking it further

1. Completely redesign the interface of the Inspiration Scrapbook app. Change the buttons, change the inspirational people, and tailor it to make it your own.

2. Create an Inspiration Scrapbook app for a friend or relative, choosing people who inspire them and including a photo of yourself saying something encouraging. Then email it to them to install on their Android device!

3. Try combining what you’ve learned in My Dream Recorder and Inspiration Scrapbook, part 2, to develop an app that requires the user to input their inspirational quotes orally rather than typing them in.

4. See if you can figure out how to extend Inspiration Scrapbook, part 2 to create a list of new quotes from the user to be randomly read out rather than just one quote.

5. Use your knowledge (gained from previous chapters) to add an index to Inspiration Scrapbook to enable the user to listen to each quote in turn rather than being given a random one.

6. Create a Don’t Bug Me, I’m Studying app that sends a message telling the texter when you’ll be free to respond. It should also read out texts to save you from trying to multitask while ensuring that you don’t miss any essential gossip that might need an instant follow-up.

What did you learn?

In this chapter, you learned the following:

· How to use a second screen in your app

· How to password-protect apps

· How to use the TextToSpeech component

· How to use the SpeechRecognizer component

· What TinyDB does

· About tag pairs used in TinyDB

· How to use the Camera component

Test your knowledge

1

If you don’t want someone to see what you’re typing, what type of text box do you use?

2

What can’t you do to the name of a screen once you’ve created it?

3

How do you reference one screen in the blocks of another?

4

Why don’t you need an index with the lists for the Inspiration Scrapbook app?

5

What type of data can be stored in TinyDB?

6

What type of data does the tag name always have to be?

7

What did you dream last night? I dreamed that this book was the wallpaper of my bedroom and that I was sleeping inside a sleeping bag made out of the pages of the book. Analyze that!