cosc 5/4730 mobile media api (mmapi) v1.1 jsr 135 and android media part 1: audio media

34
Cosc 5/4730 Mobile Media API (MMAPI) v1.1 JSR 135 and Android media Part 1: Audio media

Post on 19-Dec-2015

255 views

Category:

Documents


1 download

TRANSCRIPT

Cosc 5/4730

Mobile Media API (MMAPI) v1.1 JSR 135 and Android media

Part 1: Audio media

PLAYING AUDIOBlackberry MMAPI

Mobile Media API (MMAPI) background

• The MMAPI is built on a high-level abstraction of all the multimedia devices that are possible in a resource-limited device. – This abstraction is manifest in three classes that form

the bulk of operations that you do with this API. These classes are the Player and Control interfaces, and the Manager class.

– Another class, the DataSource abstract class, is used to locate resources, but unless you define a new way of reading data you will probably never need to use it directly.

Background (2)

• you use the Manager class to create Player instances for different media by specifying DataSource instances.

• The Player instances thus created are configurable by using Control instances.

• For example, almost all Player instances would theoretically support a VolumeControl to control the volume of the Player

Manager

• This class is used for creating players and it provides three methods to indicate the source of media– These methods are all static– createPlayer(DataSource source)– createPlayer(InputStream stream, String type)– createPlayer(String locator)

• where locator is a URI

– As note, with Eclipse, you need to import manager implicitly or it gets confused…

CreatePlayer• The JavaDocs are very lengthy for createPlayer syntax, but here are some

examples:

Media Type Example syntax

Capture audio"capture://audio" to capture audio on the default audio capture device "capture://devmic0?encoding=pcm" to capture audio on the devmic0 device in the PCM encoding

Capture video"capture://video" to capture video from the default video capture device"capture://devcam0?encoding=rgb888&width=100&height=50" to capture from a secondary camera, in rgb888 encoding mode and with a specified width and height

Start listening in on the built-in radio

"capture://radio?f=105.1&st=stereo" to tune into 105.1 FM frequency and stereo mode

Start streaming video/audio/text from an external source

"rtp://host:port/type" where type is one of audio, video or text

Play tones and MIDI "device://tone" will give you a player that you can use to play tones"device://midi" will give you a player that you can use to play MIDI

playing audio/video from the web

"http://www.yourwebsite.com/audio/song.wav""http://www.yourwebsite.com/video/video.mpg"

Player State Model

• Players are created in the UNREALIZED state

• and are typically realized• then prefetched, and

then started. • When they reach the

end of their media file, or stop() is called, they return to the PREFETCHED state.

Supported A/V

• To get a list of supported protocols and Content you can use• String[] Manager.getSupportedProtocols( String contentType)• String[] Manager.getSupportedContentTypes( String protocol)• Example: (storm2 9550)

– String capture[] = Manager.getSupportedContentTypes("capture");• returns video/mpeg and audio/x-wav

– String wav[] = Manager.getSupportedProtocols("audio/x-wav");• returns file, http, https, capture

– String file[] = Manager.getSupportedContentTypes("file");• returns audio/midi audio/mpeg audio/x-wav audio/amr audio/x-gsm

audio/mp4 audio/aac audio/x-ms-wma video/mp4 video/3gpp video/3gpp2 video/x-msvideo video/quicktime video/mpeg video/x-ms- sf video/x-ms-wm video/x-ms-wmv

Playing audio• Playing Tonesimport javax.microedition.media.*;import javax.microedition.media.Manager;import javax.microedition.media.control.*;… Player player = Manager.createPlayer(Manager.TONE_DEVICE_LOCATOR);player.realize(); //Must realize() before getting tonecontrolToneControl tc = (ToneControl) (player.getControl("ToneControl"));

tc.setSequence(new byte[]{ToneControl.VERSION, 1, ToneControl.C4, 8, ToneControl.C4 + 2, 8}); // D4player.start(); //Now the two tones will play, NON-BLOCKING!//Note no GUI interface is needed to play audio

Playing Tones

• See the java docs ToneControl (in the control package) for more information on tones

• They can be stored in a file– The ToneControl interface specifies a

tone sequence format to be used for programming a tone sequence. In addition, this tone sequence format can also be used as a file format for specifying tone sequences.

– A file with the ".jts" extension must be used to store tone sequences in this format. "audio/x-tone-seq" designates the MIME type for this format.

Playing Tones (2)

InputStream in = getClass().getResourceAsStream("/file.jts");

Player player = Manager.createPlayer(in, "audio/x-tone-seq");

player.realize(); player.prefetch(); player.start(); // non-blocking.

Playing streaming media

Player p = Manager.createPlayer( "http://www.cs.uwyo.edu/~seker/courses/4730/example/MEEPMEEP.WAV");– NOTE, Takes the network parameters as well,

so ;deviceside=true;interface=wifi if you are running via wifi, The above requires the MDS-CS to on for the simulator.

p.prefetch(); //this calls realize(), so we don’t have too. p.setLoopCount(2); //play it twicep.start();

Listener

• Since playing audio (and media) is controlled via a non-blocking method– The listener allows you program to know when

something "has happened"• such as the audio has finished playing.

– implement PlayerListener• playerUpdate(Player player, String event, Object

eventData)– event is the one that tells you want is going on.

playerUpdate Events– BUFFERING_STARTED: Posted when the Player enters into a buffering mode.– BUFFERING_STOPPED: Posted when the Player leaves the buffering mode.– CLOSED: Posted when a Player is closed.– DEVICE_AVAILABLE: Posted when the system or another higher priority application has

released an exclusive device which is now available to the Player.– DEVICE_UNAVAILABLE: Posted when the system or another higher priority application has

temporarily taken control of an exclusive device which was previously available to the Player.– DURATION_UPDATED: Posted when the duration of a Player is updated.– END_OF_MEDIA: Posted when a Player has reached the end of the media.– ERROR: Posted when an error had occurred.– RECORD_ERROR: Posted when an error occurs during the recording.– RECORD_STARTED: Posted when recording is started.– RECORD_STOPPED: Posted when recording is stopped.– SIZE_CHANGED: Posted when the size of the video is changed either because the source

video size or the display size is changed.– STARTED: Posted when a Player is started.– STOPPED: Posted when a Player stops in response to the stop method call.– STOPPED_AT_TIME: Posted when a Player is stopped as responding to the setStopTime call

using the StopTimeControl.– VOLUME_CHANGED: Posted when the volume of an audio device is changed.

PlayerListener Example

public void playerUpdate(Player player, String event, Object eventData) {if (event == PlayerListener.END_OF_MEDIA) {

player.deallocate(); player.close(); }}

• Where player is created… p.addPlayerListener(this);

PLAYING AUDIOAndroid android.media

Android

• android.media is similar in nature to MMAPI• Get a MediaPlayer• use the MediaPlayer class, with the create() method to get

a "player".– States: prepare() start()

• You can pause(), then start() without calling prepare()– Also seekTo() without needed prepare() again.

• with stop() and reset(), then you must prepare() again• See next slide for states and where to go…

– supported media formats http://developer.android.com/intl/zh-CN/guide/appendix/media-formats.html

mediaplayer states

Example

• When it is stored as a Resource (res/raw directory) MediaPlayer mp = new MediaPlayer.create(getBaseContext(),

R.raw.laser);mp.start(); //create calls prepare• For media store somewhere elseMediaPlayer mp = new MediaPlayer();mp.setDataSource("http://www.cs.uwyo.edu/~seker/courses/

4730/example/MEEPMEEP.WAV");• OR on filesystem, “/sdcard/file.mp3”mp.prepare();mp.start();

As a note.

• You can also play the audio from video files with the method as well.– There just won’t be any picture, just the audio.

Audio Capture

• This can not be tested with the android or blackberry simulators– Android simulators may through errors, saying device doesn’t

exist.– Blackberry seems to just ignore the call to record.

• So at least you can test the interface and make sure everything else is working.

• You have to test all the code on the phone.– All code examples on the website were tested and worked on

the phones.• From eclipse, plugin the phone with USB. Click Debug AS

– Blackberry: on device– Android: eclipse will ask you if you want to use the phone or emulator.

CAPTURING AUDIOBlackberry MMAPI

Recording Audio

• You can use the following supported locator strings in the createPlayer( str)– AMR: capture://audio OR

• capture://audio?encoding=amr• capture://audio?encoding=audio/amr

– PCM: capture://audio?encoding=pcm OR • capture://audio?encoding=audio/basic

– GSM: capture://audio?encoding=gsm OR• capture://audio?encoding=audio/x-gsm

– QCP: capture://audio?encoding=audio/qcelp

Audio Capture

• To record via the micPlayer player =

Manager.createPlayer("capture://audio");player.realize();• Get the RecordControl and configure the record streamRecordControl rc = (RecordControl)

player.getControl("RecordControl");• Setup a ByteArrayOutput to record the audio streamByteArrayOutputStream output = new ByteArrayOutputStream();rc.setRecordStream(output);rc.startRecord();

• Now start it recording, and this is blocking, so do all this on a thread.player.start();

Audio Capture (2)• Stop recordingrc.commit();• Get the databyte data[] = output.toByteArray();• Now close down everythingplayer.close();output.close();• You can now save the data in a file, play it back, etc…

• To play it backByteArrayInputStream recordedIS = new ByteArrayInputStream(data);Player player2 = Manager.createPlayer(recordedIS, "audio/amr");player2.prefetch();player2.start();

Blackberry

• Blackberry adds– net.rim.device.api.media• which provides interfaces to bluetooth devices and

mediakeylistener.– MediaKeyListener maps key events to media actions identical

to those implemented by the built-in media player» So volume up/down, etc…

• These do NOT require a the app to be signed to use the APIs.

CAPTURING AUDIOAndroid android.media

Recording Audio

• Simpler then Blackberry, but also less options.• Get a new instance of the MediaRecorder()– Configure it for recording the mic, set the audio

type– And configure the output FILE. (no outputStream)– Like playing, you prepare() and start().• No thread needed.

– Stop() when done recording and then release().– The audio is now located in the file.

Recording Audio (2)

• You need to set the following permissions in the AndroidManifest.xml file– android.permission.WRITE_EXTERNAL_STORAGE– android.permission.RECORD_AUDIO

Example Code private void startRecording() {• get MediaRecorder mRecorder = new MediaRecorder();• configure it mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC); mRecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP); mRecorder.setOutputFile(mFileName); mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);• Prepare to record try { mRecorder.prepare(); } catch (IOException e) {}• start recording mRecorder.start();}private void stopRecording() { mRecorder.stop(); mRecorder.release(); mRecorder = null;}

Audio settings• mRecorder.setAudioSource();

– MediaRecorder.AudioSource.MIC– MediaRecorder.AudioSource.CAMCORDER

• Audio source with same orientation as camera, otherwise mic.

– MediaRecorder.AudioSource.VOICE_CALL• Voice call uplink + downlink audio source

– http://developer.android.com/reference/android/media/MediaRecorder.AudioSource.html • mRecorder.setOutputFormat();

– MediaRecorder.OutputFormat.THREE_GPP• Recommended setting by Andriod.

– Also, AMR_NB, AMR_WB, RAW_AMR– MediaRecorder.OutputFormat.MPEG_4

• Using an MPEG-4 container format may confuse some desktop players. • http://developer.android.com/reference/android/media/MediaRecorder.html#setOutputFormat%28int%29

• mRecorder.setAudioEncoder();– MediaRecorder.AudioEncoder.AAC

• AAC audio codec

– MediaRecorder.AudioEncoder.AMR_NB• AMR (Narrowband) audio codec

– MediaRecorder.AudioEncoder.AMR_WB • AMR (Wideband) audio codec

Recording incoming phone calls

• As of the current time (2010)– It doesn't appear to be possible via

JavaME/Blackberry/Android to record the audio "from the other end" of the conversation.

– Some phones allow you record the mic during a phone call (Blackbery and Android), but not the "headset speaker“• But Android has a setting for recording, so it may work (as of

2011)

– Many JavaMe phones, simply don't run any other programs during a phone call and so you can not record any part of the phone call.

References• JavaME/Blackberry

– http://java.sun.com/javame/reference/apis/jsr135/– http://

docs.blackberry.com/en/developers/deliverables/11942/CS_record_video_without_using_the_camera_app_734825_11.jsp

– http://www.forum.nokia.com/document/Java_Developers_Library_v2/?content=GUID-9CAAF2D4-D395-4021-BBE6-602DFD275B03.html

– http://www.forum.nokia.com/document/Java_Developers_Library_v2/GUID-8F9BE454-FC5E-4B87-B805-765DEFA66C99//overview-summary.html

– http://www.forum.nokia.com/document/Java_Developers_Library_v2/GUID-8F9BE454-FC5E-4B87-B805-765DEFA66C99//javax/microedition/media/control/ToneControl.html#tone_sequence_format

– http://developers.sun.com/mobility/midp/articles/mmapioverview/– http://www.mobiquil.com/jme_mmapi.html

• Android– http://developer.android.com/intl/zh-CN/guide/topics/media/index.html– http://developer.android.com/guide/topics/media/index.html

QA&