Programming Android (2011)

Part IV. Advanced Topics

In Part IV we cover Android APIs that are important to many applications, but that are not part of the core Android Framework and that not every application is likely to make use of.

Chapter 14. Multimedia

In today’s world of converging technologies, the mobile phone is used for a variety of tasks beyond simple voice calls. Multimedia capabilities, or the playing and recording of audio and video, is one such significant task that many users find to be of great value. Take a quick look around and you will find people using the phone as a means to enjoy a variety of programs as well as share self-recorded media among friends. Android provides the APIs to easily access this capability as well as embed multimedia and its manipulation directly within an application.

Audio and Video

Android supports playback of most popular audio and video formats. It also lets you record some formats. Recordings are stored in files, and can optionally be put in a persistent media store. The MediaStore is the content provider within Android that enables the storing and sharing of media data such as images, video, and audio. Once placed within this content provider, metadata associated with the media files becomes available for other applications to use.

As of this writing, most Android devices currently on the market support the following audio and video formats. Note that device makers can add support for other formats not listed here:

Audio

Video

The asterisk (*) indicates the formats for which encoding is available. For all others, only decoding is possible.

Check the Developers site at http://developer.android.com/guide/appendix/media-formats.html for further details and changes.

Playing Audio and Video

Android provides a standard means to play audio or video: the MediaPlayer class. For audio content, you can also play back raw data, which is useful in sophisticated applications where you generate the audio dynamically.

A MediaPlayer goes through several states during its life cycle:

Idle

The MediaPlayer is instantiated.

Initialized

The media source is set.

Preparing

The MediaPlayer is preparing the media source for playback.

Prepared

The MediaPlayer is prepared for playback.

Started

Playback is in progress.

Paused

Playback has been paused.

Playback complete

Playback of source is done (the playback can be started again).

Stopped

The MediaPlayer is no longer prepared to play the source.

End

The MediaPlayer is no more, and all associated resources are released.

For details on these states, view the state diagram provided on the Developers site at http://developer.android.com/reference/android/media/MediaPlayer.html#StateDiagram. To get started with MediaPlayer, it’s useful at this point to view it as a series of steps in your application:

1. Create a MediaPlayer instance through the create() method (idle state).

2. Initialize the MediaPlayer with the media source to play (initialized state).

3. Prepare the MediaPlayer for playback through the prepare() method (preparing and prepared states).

4. Play the MediaPlayer through the start() method (started state).

5. During playback, if desired, you can pause, stop, or replay the MediaPlayer (started, paused, playback complete, and stopped states).

6. Once playback is finished, make sure to release the MediaPlayer’s associated resources by calling release() (end state).

The following sections provide more detail.

Audio Playback

Audio can be played through two methods, MediaPlayer and AudioTrack. MediaPlayer is the standard, simple way to do playback. Its data must be in a file or be stream-based. AudioTrack, in contrast, provides direct access to raw audio in memory.

MediaPlayer audio playback

When you first start using the MediaPlayer, you should determine whether a file placed within the application’s resources is to be used. If so, MediaPlayer has a convenient static method that will set up the data source and prepare the player:

MediaPlayer mediaplayer = MediaPlayer.create(this, R.raw.example);

If you are not using an application resource, such as referencing an audio file residing on the filesystem (SD card and the like) or on a website (e.g., http://SomeServer/SomeAudioFile.mp3), you’ll have to manually set up and call your data source. You can take the data from a URI through a call to:

setDataSource(context, uri)

The context in the first argument is a means for the MediaPlayer to access the resources of the application itself, and thus be able to resolve the URI. Either the application or activities context will do.

The alternative is to specify an absolute file path through:

setDataSource(path)

API version 9 lets you attach some auxiliary effects (such as reverb) to the player. Set any effects you want while setting the data source, before calling prepare():

MediaPlayer mediaplayer = new MediaPlayer();

// Uri mediaReference = "http://someUriToaMediaFile.mp3";

// mediaplayer.setDataSource(this, mediaReference);

// use absolute path

mediaplayer.setDataSource("/sdcard/somefile.mp3");

// prepare mediaplayer

mediaplayer.prepare();

Once the MediaPlayer is prepared, you can play it:

mediaplayer.start();

During play, the player can be paused or stopped. When in the paused state, it may be unpaused simply by calling start() again. Once the MediaPlayer is stopped, you can’t start it again without resetting it through the reset() method, reinitializing it with the data source as shown earlier, and issuing prepare(). However, look at the following:

mediaplayer.pause(); // pausing

mediaplayer.start(); // going from pause to play

mediaplayer.stop(); // stopping

...

// to be able to play again reset must be called

mediaplayer.reset();

// now the media player must be reinitialized to play again

While the MediaPlayer is playing, you can track its current position in the file through getCurrentPosition(). This returns the amount of time played through in the file, in millisecond units:

mediaplayer.getCurrentPosition();

Once the MediaPlayer is no longer needed, make sure to release it so that the resources are cleaned up and made available for the system:

mediaplayer.release();

AudioTrack audio playback

AudioTrack provides a much more direct method of playing audio. The following example shows the parameters required to set up an AudioTrack:

File mediafile = new File(mediaFilePath);

short[] audio = new short[(int) (mediafile.length()/2)];

// read in file and fill up audio[]

AudioTrack audiotrack = new AudioTrack(

// stream type

AudioManager.STREAM_MUSIC,

// frequency

11025,

// channel config—mono, stereo, etc.

AudioFormat.CHANNEL_CONFIGURATION_MONO,

// audio encoding

AudioFormat.ENCODING_PCM_16BIT,

// length

audio.length,

// mode

AudioTrack.MODE_STREAM

);

The AudioTrack method provides the type of audio stream (music, ringtone, alarm, voice call, etc.), the sample rate in Hertz (44100, 22050, 11025), the audio configuration (mono or stereo), the audio format/encoding, the length of the audio in number of bytes, and the mode (static or stream). Android’s AudioTrack, once configured, will automatically know how to interface with the hardware on the device, thus providing a painless experience.

To play the audio, issue the play() method and write the data out to the hardware:

// start playing state

audiotrack.play();

// write audio to hardware

audiotrack.write(audio, 0, audio.length);

To pause the track, utilize the pause() method:

// pause

audiotrack.pause();

To stop playing the track, set it to the stopped state. If you don’t need the track anymore, release it. Otherwise, to replay the audio, you must reinitialize it:

// stop

audiotrack.stop();

// release all resources

audiotrack.release();

Video Playback

Video playback, unlike audio playback, can use only the MediaPlayer. There is no video equivalent to AudioTrack. Video uses the MediaPlayer similarly to audio files, but you must additionally specify a view (called a surface) on which the video can be displayed. Android offers a convenient control that includes its own surface: the VideoView view. An example of its use follows. It includes the addition of an optional controller that lets the user control the playback through a simple interface that includes buttons to start, stop, and pause the playback, as well as a seek bar to skip forward or back within the video’s playback progress:

// create the view (in this case it is already included in the layout resource)

VideoView videoview = (VideoView) findViewById(R.id.videoview);

videoview.setKeepScreenOn(true);

// used if streaming

if (videouri != null) videoview.setVideoURI(videouri);

// absolute path if it is a file

else videoview.setVideoPath(videopath);

// let's add a media control so we can control the playback

mediacontroller = new MediaController(this);

mediacontroller.setAnchorView(videoview);

videoview.setMediaController(mediacontroller);

if (videoview.canSeekForward())

videoview.seekTo(videoview.getDuration()/2);

// start the playback

videoview.start();

Recording Audio and Video

The standard class that supports recording is the MediaRecorder. Much like the MediaPlayer, it passes through various states during its life cycle. The states are as follows (for more details, view the state diagram provided by the Developers site, athttp://developer.android.com/reference/android/media/MediaRecorder.html):

Initialize

The MediaRecorder class is instantiated.

Initialized

The MediaRecorder is ready to be used.

DataSource configured

The media source (where the output will be placed) is configured.

Prepared

The MediaRecorder is prepared to record.

Recording

Recording is underway.

Released

All resources are released.

To utilize the MediaRecorder, some permissions may need to be set in the manifest:

§ To enable video recording, enable RECORD_VIDEO and the CAMERA:

§ <uses-permission android:name="android.permission.RECORD_VIDEO"/>

<uses-permission android:name="android.permission.CAMERA"/>

§ To record audio, enable RECORD_AUDIO:

<uses-permission android:name="android.permission.RECORD_AUDIO"/>

Audio Recording

There are three methods to record audio. The MediaRecorder is the standard method; using an Intent is the simplest method; and the AudioRecorder can be used to record directly from hardware buffers.

MediaRecorder audio recording

First, initialize the MediaRecorder. Then set the data source information (the audio input source, the output format, the encoding type, where the file is to be recorded to, etc.). Starting with version 8, you can set the bit rate and sampling rate. Once all this is done, call the prepare() method:

// initialize the MediaRecorder

MediaRecorder mediarecorder = new MediaRecorder();

// configure the data source

// the source of the audio input

mediarecorder.setAudioSource(MediaRecorder.AudioSource.MIC);

// output format

mediarecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);

// encoding

mediarecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);

// use absolute path to file where output is stored

mediarecorder.setOutputFile("/sdcard/audiorecordexample.3gpp");

// prepare to record

mediarecorder.prepare();

Then when the recoding needs to start, call the start() method:

mediarecorder.start();

When the recording needs to be stopped, call the stop() method. If you want to continue recording after this, call reset() to force the MediaRecorder back to the idle state. Then reconfigure the data source to prepare the MediaRecorder again:

mediarecorder.stop();

...

mediarecorder.reset();

Once the MediaRecorder is no longer needed, make sure to release it:

mediarecorder.release();

The following example is a convenient little app that uses the code we developed to provide a “record” button for the user. When the button is clicked, the record method executes with the file path already referenced. A “stop” button is then made visible and the “record” button becomes invisible. When the “stop” button is clicked, the stopRecord method is called and the “record” button comes back:

public class AudioRecorder extends Activity {

private MediaRecorder mediarecorder;

public void onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);

setContentView(R.layout.audiorecorderlayout);

ImageButton recordbutton = (ImageButton) findViewById(R.id.record);

recordbutton.setOnClickListener(new OnClickListener() {

public void onClick(View v) {

record("/sdcard/audiorecordexample.3gpp");

}

});

ImageButton stopbutton = (ImageButton) findViewById(R.id.stop);

stopbutton.setOnClickListener(new OnClickListener() {

public void onClick(View v) {

stopRecord();

}

});

}

private void record(String filePath) {

try {

File mediafile = new File(filePath);

if(mediafile.exists()) {

mediafile.delete();

}

mediafile = null;

// record button goes away

ImageButton button = (ImageButton) findViewById(R.id.record);

button.setVisibility(View.GONE);

// stop button shows up

ImageButton stopbutton = (ImageButton) findViewById(R.id.stop);

stopbutton.setVisibility(View.VISIBLE);

// set up media recorder

if(mediarecorder == null) mediarecorder = new MediaRecorder();

mediarecorder.setAudioSource(MediaRecorder.AudioSource.MIC);

mediarecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);

mediarecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);

mediarecorder.setOutputFile(filePath);

// prepare media recorder

mediarecorder.prepare();

// start media recorder

mediarecorder.start();

} catch (Exception e) {

e.printStackTrace();

}

}

}

private void stopRecord() {

// stop media recorder

mediarecorder.stop();

// reset media recorder

mediarecorder.reset();

// record button shows up

ImageButton button = (ImageButton) findViewById(R.id.record);

button.setVisibility(View.VISIBLE);

// stop button goes away

ImageButton stopbutton = (ImageButton) findViewById(R.id.stop);

stopbutton.setVisibility(View.GONE);

}

}

Intent audio recording

Recording via Intent is the easiest of the methods. Just construct the MediaStore.Audio.Media.RECORD_SOUND_ACTION intent, and start it using the startActivityForResult() from within the Activity. This will launch the default audio recorder that is provided in most Android devices and proceeds to record some audio:

Intent intent = new Intent(MediaStore.Audio.Media.RECORD_SOUND_ACTION);

startActivityForResult(intent, 1); // intent and requestCode of 1

Once the recording is complete and the audio recorder finishes, your Activity that originated the call to startActivityForResult() will be brought back to the fore. When that occurs, your Activity’s onActivityResult() method will be triggered with the requestCode you provided (in this case, 1), a result code (OK or error), and an intent carrying the URI referencing the recorded audio file:

protected void onActivityResult(int requestCode, int resultCode, Intent intent) {

// is it our requestCode?

if (requestCode == 1) {

// is the resultCode OK?

if (resultCode == RESULT_OK) {

// lets get the uri

Uri audioUri = intent.getData();

// lets play the uri or do something with it.

playAudio(audioUri);

}

}

}

AudioRecorder audio recording

In parallel with AudioTrack, AudioRecorder provides a much more direct recording experience:

short[] buffer = new short[10000];

recorder = new AudioRecord( // source to record from

MediaRecorder.AudioSource.MIC,

// frequency

11025,

// channel config—mono, stereo, etc.

AudioFormat.CHANNEL_CONFIGURATION_MONO,

// audio encoding

AudioFormat.ENCODING_PCM_16BIT,

// buffer size

buffer.length

);

The AudioRecord method provides the type of source to record audio from (Mic, Camcorder [mic facing the same direction as the camera], or VoiceCall), the sample rate in Hertz (44100, 22050, or 11025), the audio configuration (mono or stereo), the audio format/encoding, and the length of the buffer in number of bytes. Note that the size of this buffer determines how long an AudioRecord can record before “over-running” data that has not been read yet. Data should be read from the audio hardware in chunks of sizes less than the total recording buffer size. Android’s AudioRecord, once configured, will automatically know how to interface with the hardware on the device, thus providing a painless experience.

To start recording, set the AudioRecord’s state to the Record state and read data repeatedly from the hardware buffer:

recorder.startRecording();

while(recordablestate) {

try {

// read in up to buffer size

int readBytes = recorder.read(buffer, 0, buffer.length);

// do something with the bytes that are read

} catch (Exception t) {

recordablestate = false;

}

}

To stop recording, set the AudioRecord’s state to Stop. If you no longer wish to record, do not forget to release all resources associated with the recording. Otherwise, you may call startRecording() to start recording again:

// stop recording

recorder.stop();

// release recording resources

recorder.release();

Video Recording

You can record video in two ways: by using the MediaRecorder or by using an Intent. Raw recording is not supported, as it is for audio.

MediaRecorder video recording

The process for recording video with the MediaRecorder is much the same as that for recording audio: initialize the MediaRecorder, prepare the data source, and start the MediaRecorder. You can offer the user a preview window so that he can preview the video being captured, by providing a surface as shown earlier for playing back video. Generally, a VideoView is used:

// initialize

MediaRecorder mediarecorder = new MediaRecorder();

// set data source

mediarecorder.setAudioSource(MediaRecorder.AudioSource.MIC);

mediarecorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);

mediarecorder.setOutputFormat(MediaRecorder.OutputFormat.DEFAULT);

mediarecorder.setAudioEncoder(MediaRecorder.AudioEncoder.DEFAULT);

mediarecorder.setVideoEncoder(MediaRecorder.VideoEncoder.DEFAULT);

mediarecorder.setOutputFile("/sdcard/someexamplevideo.mp4");

// provide a surface to show the preview in. in this case a VideoView is used

videoview = (VideoView) findViewById(R.id.videosurface);

SurfaceHolder holder = videoview.getHolder();

mediarecorder.setPreviewDisplay(holder.getSurface());

// prepare

mediarecorder.prepare();

// start recording

mediarecorder.start();

Intent video recording

Intent-based video recording is like using an intent to record audio. The intent to use is MediaStore.ACTION_VIDEO_CAPTURE, and the resultant data is the URI of the video file.

Stored Media Content

Even when media is saved to a file (as in the case of recording), the media file is not immediately available to other applications. To make the file available, you must insert it into the MediaStore. The MediaStore is a content provider dedicated to the storage and retrieval of media data (images, video, audio) with the device. To store a reference to the file, create a ContentValues object and insert it into the appropriate MediaStore content provider. The following example inserts an audio file with appropriate metadata, such as title and artist:

// generate ContentValues and add appropriate metadata values

ContentValues content = new ContentValues();

// VERY IMPORTANT! Must reference the absolute path of the data.

content.put(MediaStore.MediaColumns.DATA, "/sdcard/AudioExample.3gpp");

content.put(MediaStore.MediaColumns.TITLE, "AudioRecordExample");

content.put(MediaStore.MediaColumns.MIME_TYPE, "audio/amr");

content.put(MediaStore.Audio.Media.ARTIST, "Me");

content.put(MediaStore.Audio.Media.IS_MUSIC, true);

// get the Content Resolver

ContentResolver resolve = getContentResolver();

// insert into the content resolver

Uri uri = resolve.insert(MediaStore.Audio.Media.EXTERNAL_CONTENT_URI, content);

// announce to everyone that cares that it was inserted

sendBroadcast(new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE, uri));