http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.

html

Sponsored by:

This story appeared on JavaWorld at http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

Program multimedia with JMF, Part 2

Jump into the Java Media Framework's important classes and interfaces
By Budi Kurniawan, JavaWorld.com, 05/04/01 If you know Java, multimedia programming is a piece of cake. Last month, in Part 1, you saw how many parts of the Java Media Framework (JMF) classes and interfaces can integrate together. In this article, you'll see working JMF code performing multimedia functions. Read the whole "Program Multimedia with JMF" series:

Part 1: Go multimedia by learning how the Java Media Framework compares to your stereo system Part 2: Jump into Java Media Framework's important classes and interfaces

However, before you start coding with JMF, you must install the reference implementation and the supporting applications as discussed below. You should also be aware of the hardware and software requirements. Examples in this article were tested using a Pentium III 800-MHz PC on Windows 2000 with a Logitech USB camera and a built-in microphone. Therefore, the solution presented here could be Windows-specific. Even though the code should not change when deployed to other platforms, the JMF reference implementation is not available for all Java platforms. The reference implementation is not currently available for Linux users, for instance.

Hardware and software requirements

To use JMF, you won't find it difficult to meet the hardware and software requirements. Your old 166-MHz Pentium proves sufficient as long as it has at least 32 MB of RAM. JMF can also run on 166-MHz or greater Power PC or UltraSparc systems. You need a SoundBlaster-compatible card for audio playback if you want sound. If you happen to use an AIX machine, you can use an Ultimedia-compatible sound card. On Windows, the software requirements comprise:

Windows 95/98, Windows NT 4.0, or Windows 2000. JDK 1.1.3 or later for Windows (from Sun). If you want to be safe on the Y2K compatibility issue, you should use JDK 1.1.6 or later.

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 1 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

JMF classes and native libraries. A modern browser if you plan to use applets to display your video. Modern here means at least Netscape Communicator 4.05 with JDK 1.1 patch or Microsoft Internet Explorer 4.0 or later.

Get the reference implementation

Yes, you definitely need to download the Java Media Framework 2.1 reference implementation. Once at the download page, you can choose from the following:

"Cross-platform Java" "Windows Performance Pack" "Solaris SPARC Performance Pack"

As a Java developer, you may be tempted to download the "Cross-platform Java" version. However, select the "Windows Performance Pack" if you are working with Windows. The all-Java version isn't ideal, as it doesn't support audio and video capture. (Also, you may wish to check out the list of known platform-specific issues.) In contrast to the all-Java version, JMF's Windows version supports all video capture devices that employ the VFW (Video for Windows) interface. Moreover, with this version, some cameras have been tested by Sun engineers. These include:

Alaris QuickVideo DVC1 QuickCam Home USB WinTV on 95/98

In fact, JMF's Windows version supports almost any capture device. For the list of supported capture devices, go to: http://java.sun.com/products/java-media/jmf/2.1/formats.html#Capturers.

Set up and run JMF 2.1 for Windows

After you download the JMF 2.1 into a directory, double-click the jmf-2_1-win.exefile to install the reference implementation with its classfiles and native libraries for running JMF players. Afterward, you need to make sure that InstallShield properly configured your CLASSPATH and PATH during installation. If not, you can do it manually with the following command: set CLASSPATH=%WINDIR%\java\classes\jmf.jar;%WINDIR%\java\classes\sound.jar;.;%CLASSPATH%

Your PATH should be set to include the JMF library files: set PATH=%WINDIR%\System32;%PATH% (on Windows NT) set PATH=%WINDIR%\System;%PATH% (on Windows 95/98)

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 2 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

If you later have a problem with the installation, Sun provides the JMF Diagnostics applet to verify that JMF is set up properly on your system.

JMFRegistry
If you plan to use a capture device such as a camera, you must make sure you have installed the correct device driver for that capture device. In addition, you need the JMFRegistryapplication. The JMFRegistry, a standalone Java application, registers new DataSources, MediaHandlers, PlugIns, and capture devices with JMF so that you can use them with your JMF 2.1 installation. You need to run JMFRegistryonly once to register all capture devices available on the system. After you run JMFRegistryfrom the command line or inside your Java tool, you should see a window similar to Figure 1.

Figure 1. Register your capture device with JMFRegistry Click on thumbnailto view full-size image. (71 KB)

Click the "Capture Devices" tab, and then click the "Detect Capture Devices" button. A list should display all the audio and video capture devices on the system. As seen in Figure 1, JMFRegistryfound JavaSound audio capture, a Logitech USB Video Camera, and Microsoft WDM Image Capture. Clicking a list item will display all formats that can be used for that capture device. For example, the Logitech USB camera in Figure 1 supports RGB color with a resolution of 352x288, 160x120, and so on. If JMFRegistryfails to detect your capture device, it means there is something wrong with the device driver, installation process, or both. JMFRegistryalso enables you to rearrange the search order and remove registered extensions.

The JMF API specification

When programming with JMF, it's helpful to have the JMF API specification. Note that JMF 2.1 is an implementation update, which means the implementation features new updates and fixes, but uses the same API as defined in JMF 2.0. Also helpful are the sample JMF applications. These applications, especially JMStudio, contain the source code that you can consult when you want to learn more or when you have a problem with your code.

Our JMF project

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 3 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

Due to space limitations, it's impossible to implement all of JMF 2.1's features in our JMF project. I therefore chose three basic functions that you will always need with JMF:

Play multimedia files Capture devices Capture audio and video

The capture devices function must be run prior to capturing any audio and video data, but it's not needed to play multimedia files. Our project comprises two classfiles: JMF.java(Listing 1 in the source code) and CaptureDeviceDialog.java(Listing 2). Play multimedia files In our project, we have a play()method in the JMFclass. This method plays a multimedia source file of our choice. As discussed in Part 1, to play a multimedia data source, you need a Player. In the code, dualPlayeris the object reference for Player: Player dualPlayer; //for merging audio and video data sources

dualPlayer, as the name implies, can play both audio and video data. In the play()method, you get the filenamestring as the data source by using a FileDialogto browse your computer to choose a file: try { FileDialog fd = new FileDialog(this, "Select File", FileDialog.LOAD); fd.show(); String filename = fd.getDirectory() + fd.getFile(); // . . . } catch (Exception e) { System.out.println(e.toString()); }

You then create a Playerindirectly through the media Manager. You can use the Manager's createPlayer()or createProcessor()to get a new Player. In our play()method, we use the createPlayer()method: dualPlayer = Manager.createPlayer (new MediaLocator("file:///" + filename));

Sometimes, in situations like ours, you might want to use a Playerto control other Players or Controllers. A single controlling Playercan invoke start(), stop(), setMediaTime(), and other methods on the entire group. The controlling Playermanages all of the state transitions and event postings. We then use the addControllerListener()to specify a
http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 4 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

ControllerListenerto which a Controllerwill send an event. The Controllerinterface provides resource-allocation state information, event generation, and a mechanism for obtaining objects that provide additional control over a Controller: dualPlayer.addControllerListener(this);

Lastly, we need to call the start()method, which starts the Playeras soon as possible. The start()method attempts to transition the Playerto the Started state. If the Playerhas not been Realized or Prefetched, start()automatically performs those actions. (For more information about Playerstates, refer to Part 1.) dualPlayer.start();

Since the JMFclass implements the ControllerListenerinterface, we must implement the controllerUpdate()method, called when a Controller(with which this listener is registered) generates an event: public synchronized void controllerUpdate(ControllerEvent event) { if (event instanceof RealizeCompleteEvent) { Component comp; if ((comp = dualPlayer.getVisualComponent()) != null) add ("Center", comp); if ((comp = dualPlayer.getControlPanelComponent()) != null) add("South", comp); validate(); } }

As soon as JMF generates RealizeCompleteEvent, the controllerUpdate()method adds the visual component and a control-panel component. With the visual component, the video data is played; whereas a control-panel component controls the media, such as play back, stop, and so on. Figure 3 at the end of this article shows a piece of video on the visual component. Notice the control panel lying under it. It's interesting to note the following messages from the console. (These messages have been reformatted for reading convenience): Starting player ...javax.media.TransitionEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78, previous=Unrealized, current=Realizing, target=Started] Open log file: C:\123data\JavaProjects\JMF\JMF\jmf.log javax.media.DurationUpdateEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78,duration= javax.media.Time@2a37a6 javax.media.RealizeCompleteEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78, previous=Realizing, current=Realized,
http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 5 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

target=Started] Adding visual component Adding control panel javax.media.TransitionEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78, previous=Realized, current=Prefetching, target=Started] javax.media.PrefetchCompleteEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78, previous=Prefetching, current=Prefetched,target=Started] javax.media.StartEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78, previous=Prefetched, current=Started, target=Started, mediaTime=javax.media.Time@56a05e,timeBaseTime= javax.media.Time@3a8602] javax.media.EndOfMediaEvent [source=com.sun.media.content.video.mpeg.Handler@71bb78, previous=Started, current=Prefetched, target=Prefetched, mediaTime=javax.media.Time@1d332b]

As soon as you call the start()method, the Playerwill be changed to the Started state from whatever state it is currently in. The message above clearly describes the state transition from the Unrealized state to the Started state. When the EndOfMediaevent is invoked (the Playerhas finished playing the media), the state moves from Started back to Prefetched. Also note that when the RealizeCompleteevent is invoked, it quickly adds the visual component and the control panel. Register capture devices The register capture devices function in our JMF project acts as an internal function that does not register anything to outside the project. Its purpose: When multiple capture devices exist on the user's computer, tell the application what audio or video device the user wants to capture. Further, it tells the application what audio or video format to use when capturing. For example, an audio device may support a linear stereo sound format with a 44,100 times-persecond sampling rate or formats with lower sampling rates. When you select an audio device from the Audio Device(s) list, the Audio Format(s) List control will also get updated because each device can support different formats. Likewise, when you select a video device from the Video Device(s) list, the Video Format(s) values will be updated. It is therefore important to get the device configuration before the capture process. In our JMF project, the CaptureDeviceDialogclass is a dialog that appears when the user clicks "Capture Device" from the "Configure" menu. Since capture device configuration must be done prior to capturing, the same dialog also displays if the user clicks "Capture" from the "Action" menu before any configuration is done. The "Capture Device" dialog is shown in Figure 2.

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 6 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

Figure 2. "Capture Device" dialog Click on thumbnail to view full-size image. (28 KB)

Consider the init()method in the CaptureDeviceDialogclass. After the usual initialization for creating components on the dialog, the code continues by trying to obtain all capture devices available on the user's machine using the getDeviceList()static method of the CaptureDeviceManager: //get all the capture devices devices = CaptureDeviceManager.getDeviceList ( null );

CaptureDeviceManager, a manager class, provides access to a list of the capture devices available on the system. CaptureDeviceManageruses query mechanisms and a registry to locate devices and return CaptureDeviceInfoobjects for available devices. The CaptureDeviceManageralso registers new capture devices. The getDeviceList()obtains a list of CaptureDeviceInfoobjects that correspond to devices that can capture data in the specified Formatargument. This method returns a vector containing CaptureDeviceInfoobjects for the devices that support the specified Format. In our code above, we pass nullas the argument. If no Formatis specified, as in the code above where we pass a nullas the argument, this method returns a list of CaptureDeviceInfoobjects for all of the available capture devices. devicesis a vector defined in the class scope. cdiis a CaptureDeviceInfoobject reference for later use: CaptureDeviceInfo cdi;

If the getDeviceList()method does return non-null values, devices will contain elements of type CaptureDeviceInfo. The graphical design of the CaptureDeviceDialogclass groups all audio capture devices available on the system in one Choicecomponent, and all video capture devices in another Choicecomponent. However, we don't know whether a CaptureDeviceInfois an audio capture device or a video capture device. An indirect way, fortunately, is available. CaptureDeviceInfo's getFormats()method returns a Formatarray, and Formatis directly extended by AudioFormatand VideoFormat. An audio capture device naturally will have audio formats and a video capture device will have video formats. Using the instanceOfoperator that inquires a Formatreturned by a CaptureDeviceInfo, we can then classify all audio capture devices in one group and all video capture devices in another: if (devices!=null && devices.size()>0) { int deviceCount = devices.size(); audioDevices = new Vector(); videoDevices = new Vector(); Format[] formats;

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 7 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

for ( int i = 0; i < deviceCount; i++ ) { cdi = (CaptureDeviceInfo) devices.elementAt ( i ); formats = cdi.getFormats(); for ( int j=0; j<formats.length; j++ ) { if ( formats[j] instanceof AudioFormat ) { audioDevices.addElement(cdi); break; } else if (formats[j] instanceof VideoFormat ) { videoDevices.addElement(cdi); break; } } } ... }

audioDevices()will contain all audio capture devices on the system, while videoDevices()will contain all video capture devices. We can then populate our audio device(s) and video device(s) Choicecomponents: //populate the choices for audio for (int i=0; i<audioDevices.size(); i++) { cdi = (CaptureDeviceInfo) audioDevices.elementAt(i); audioDeviceCombo.addItem(cdi.getName()); } //populate the choices for video for (int i=0; i<videoDevices.size(); i++) { cdi = (CaptureDeviceInfo) videoDevices.elementAt(i); videoDeviceCombo.addItem(cdi.getName()); }

It continues by calling the following methods: displayAudioFormats(); displayVideoFormats();

The displayAudioFormats()method performs what the name implies: display all audio formats supported by the selected audio device in the audioFormatCombo(). It starts by removing all of the audioFormatCombo Listcontrol's items, if any: audioFormatCombo.removeAll();

The displayAudioFormats()method then gets the selected index of the audioDeviceCombo()component into iwhich

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 8 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

will have a value of -1if no device is currently selected: int i = audioDeviceCombo.getSelectedIndex();

If a device is selected, it then gets the corresponding CaptureDeviceInfofrom audioDevices(): if (i!=-1) { cdi = (CaptureDeviceInfo) audioDevices.elementAt(i); ... }

From the CaptureDeviceInfoobject we can then get all formats using the getFormats()method. Populating the audioFormatCombo()component then proves straightforward using the following forloop, bearing in mind that the number of formats is given by the length property: if (cdi!=null) { Format[] formats = cdi.getFormats(); audioFormats = new Vector(); for (int j=0; j<formats.length; j++) { audioFormatCombo.add(formats[j].toString()); audioFormats.addElement(formats[j]); } }

The displayVideoFormats()method performs a similar function for the video formats. With the aforementioned methods, we:

Grouped all audio capture devices in audioDevices() Grouped all video capture devices in videoDevices() Grouped all audio formats of the selected audio capture device in audioFormats() Grouped all video formats of the selected video capture device in videoFormat()

It's now straightforward to obtain the selected audio and video capture devices and the selected audio and video formats using the getAudioDevice(), getVideoDevice(), getAudioFormat(), and getVideoFormat()methods from the JMFclass. The capture()method in the JMFclass (given below) assigns the following object references:

audioCDIwith the selected audio device videoCDIwith the selected video device audioFormatwith the selected audio format videoFormatwith the selected video format

http://www.javaworld.com/cgi-bin/mailto/x_java....world/jw-05-2001/jw-0504-jmf2.html&site=jw_core 9 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

Here it is in code: audioCDI = cdDialog.getAudioDevice(); if (audioCDI!=null) { audioDeviceName = audioCDI.getName(); System.out.println("Audio Device Name: " + audioDeviceName); } videoCDI = cdDialog.getVideoDevice(); if (videoCDI!=null) { videoDeviceName = videoCDI.getName(); System.out.println("Video Device Name: " + videoDeviceName); } //Get formats selected, to be used for creating DataSource videoFormat = cdDialog.getVideoFormat(); audioFormat = cdDialog.getAudioFormat();

Capture data The capture()method captures both audio and video data and plays the audio through the speaker and the video on the screen. It starts by checking to determine whether or not an audio capture device or a video capture device has been selected: if (audioCDI==null && videoCDI==null) registerDevices();

Like the play()method, creating a Playerthat plays streams from a DataSourcecan be done by using the createPlayer() static method of the Managerclass: Player createPlayer(MediaLocator sourceLocator)

In our project, we can obtain a DataSourceobject from both audioCDIand videoCDIby calling their getLocator() methods, which return the MediaLocatorneeded to create a DataSourcefor the device through the Manager. Then, we can add a ControllerListenerto a video Playerand call the start()methods of both Players: videoPlayer = Manager.createPlayer(videoCDI.getLocator()); audioPlayer = Manager.createPlayer(audioCDI.getLocator()); videoPlayer.addControllerListener(this); videoPlayer.start(); audioPlayer.start();

However, with this we end up with two Players. Alternatively, we can use the Manager's createDataSource()method to get an audio DataSourceand a video DataSourcefrom our audio and video CaptureDeviceInfoobjects ( audioCDIand videoCDI), and then call the createMergingDataSource()method to combine both audio and video data sources into
http://www.javaworld.com/cgi-bin/mailto/x_java...orld/jw-05-2001/jw-0504-jmf2.html&site=jw_core 10 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

one DataSource, which we call ds: DataSource[] dataSources = new DataSource[2]; dataSources[0] = Manager.createDataSource(audioCDI.getLocator()); dataSources[1] = Manager.createDataSource(videoCDI.getLocator()); DataSource ds = Manager.createMergingDataSource(dataSources);

We can then use dsas the argument to the createPlayer()method to create a single Playercalled dualPlayer: dualPlayer = Manager.createPlayer(ds);

Next, it's just like any ordinary Player. As in the play()method, we can call its addControllerListener()and start() methods: dualPlayer.addControllerListener(this); dualPlayer.start();

Figure 3 shows a shot captured using my low-resolution home camera.

Figure 3. My low-resolution camera in action. Click on thumbnail to view full-size image. (39 KB)

Conclusion
Java Media Framework is a fascinating reference implementation with which you can do any possible multimedia programming. In this article we've seen some simple examples of JMF in action. If you want to learn more about JMF, take a look at Sun's JMStudio application that comes with the source code. Although not for beginners, JMStudio is a complete solution that not only includes data capturing and media playing, but also allows you to transmit multimedia streams onto the Internet and receive audio and video from the Internet. Also, the source code includes the JMFRegistry application. Once downloaded, you can start probing into its classes to learn how to provide capture device detect features so your application will not depend on it.

About the author

Budi Kurniawan teaches Java at the Centre for Continuing Education at the University of Sydney. His book, Web
http://www.javaworld.com/cgi-bin/mailto/x_java...orld/jw-05-2001/jw-0504-jmf2.html&site=jw_core 11 / 12 2008/10/24 12:18:37

http://www.javaworld.com/javaworld/jw-05-2001/jw-0504-jmf2.html

Tips and Techniques, to be published by APress, features important topics that all serious Web developers should be familiar with.END_BODY:
All contents copyright 1995-2008 Java World, Inc. http://www.javaworld.com

http://www.javaworld.com/cgi-bin/mailto/x_java...orld/jw-05-2001/jw-0504-jmf2.html&site=jw_core 12 / 12 2008/10/24 12:18:37