Android media decoding MediaCodec MediaExtractor learning

AndroidMediaPlayer Player Player Player is provided to play media files, but MediaPlyer only packages MediaCodec and MediaExtractor under Android Media for easy use. But it’s best to understand An.Droid media file decoding, encoding and rendering process.

Useandroid.mediaThe MediaCodec and MediaExtractor under the package implement a simple video decoding rendering.

It has been used to:

• MediaCodec:Responsible for coding and decoding media files. Internal methods are native methods.

• MediaExtractor:Is responsible for finding the track from the file of the specified type of media file and filling it into the buffer of MediaCodec, the internal methods are native methods

• AudioTrack:Responsible for playing audio after decoding.

• SurfaceView:Show the video after decoding.

Video playback is mainly divided into the following steps:

1. Load resources to extractor
2. Get track of video
3. Set extractor to select the track where the video is located.
4. Create MediaCodec, decoder for decoding video.
5. Start cycling until the end of video resources.
6. Fill the resource in extractor with one unit into the input buffer of decoder.
7. decoderPadding the decoded video to the output buffer
8. decoderWhen the output buffer is released, the data in the buffer is rendered to surface.

Audio playback is similar, only more AudioTrack parts, less rendering to surface part.

MediaCodec.releaseOutputBuffer(int outputBufferIndex, boolean render);

• renderbytrueIt will render.surface

---

The broadcast controls, video and audio each have one Thread.

    public void play() {
        isPlaying = true;
        if (videoThread == null) {
            videoThread = new VideoThread();
            videoThread.start();
        }
        if (audioThread == null) {
            audioThread = new AudioThread();
            audioThread.start();
        }
    }

    public void stop() {
        isPlaying = false;
    }

VideoThread

private class VideoThread extends Thread {
        @Override
        public void run() {
            MediaExtractor videoExtractor = new MediaExtractor();
            MediaCodec videoCodec = null;
            try {
                videoExtractor.setDataSource(filePath);
            } catch (IOException e) {
                e.printStackTrace();
            }
            int videoTrackIndex;
            //Get track of videoVideoTrackIndex = getMediaTrackIndex (videoExtractor, "video/");If(videoTrackIndex > = 0) {MediaFormat mediaFormat = videoExtractor.getTrackFormat (VideoTrackIndex);Int width = mediaFormat.getInteger (MediaFormat.KEY_WIDTH);Int height = mediaFormat.getInteger (MediaFormat.KEY_HEIGHT);/ / video length: secFloat time = mediaFormat.getLong (MediaFormat.KEY_DURATION) / 1000000;CalLBack.videoAspect (width, height, time);VideoExtractor.selectTrack (videoTrackIndex);Try {VideoCodec = MediaCodec.createDecoderByType (mediaFormat.g)EtString (MediaFormat.KEY_MIME));VideoCodec.configure (mediaFormat, surface, null)0);} catch (IOException E) {E.printStackTrace ();}}If (videoCodec = = null) {Log.v (TAG, "MediaCodec n"Ull ");Return;}VideoCodec.start ();MediaCodec.BufferInfo videoBufferInfo = new MediaCodec.BufferInfo ();ByteBuffer[] inputBuffers = vIdeoCodec.getInputBuffers ();/ / ByteBuffer[] outputBuffers = videoCodec.getOutputBuffers ();Boolean isVideoEOS = false;Long startMs = System.currentTimeMillis ();While (Thread.interrupted ()) {If ((isPlaying) {ConTinue;}/ / transfer resources to the decoderIf ((isVideoEOS) {IsVideoEOS = putBufferToCoder (videoExtractor, videoCodec, inputBuffers);}Int outputBufferIndex = videoCodec.dequeueOutputBuffer (videoBufferInfo, TIMEOUT_US);Switch (outputBufferIndex) {Case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:Log.v (TAG, "format changed");Break;Case MediaCodec.INFO_TRY_AGAIN_LATER:Log.v (TAG, "decode the current frame timeout");Break;Case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED://outputBuffers = videoCodec.getOutputBuffers ();Log.v (TAG, "output buffers changed");Break;Default:/ / when rendered directly to Surface, less than outputBuffer is used.//ByteBuffer outputBuffer = ouTputBuffers[outputBufferIndex];/ / delay operation/ / if the display in the buffer area is availableInter > the progress of the current video playback is dormant.SleepRender (videoBufferInfo, startMs);/ / / renderingVideoCodec.releaseOutputBuffer (outputBufferIndex, true);Break;}If ((videoBufferInfo.flags & MediaCo)Dec.BUFFER_FLAG_END_OF_STREAM) = = 0) {Log.v (TAG, "buffer stream end");Break;}}//end whileVideoCodec.stop ();VideoCodec.release ();VideoExtractor.release ();}}

Gets the track of the specified type of media file.

    //Gets the track of the specified type of media file.Private int getMediaTrackIndex (MediaExtractor videoExtractor, String MEDIA_TYPE){Int trackIndex = -1;For (int i = 0; I < videoExtractor.getTrackCount (); i++) {/ / get track of video.MediaFormat mediaFormat = videoExtractor.getTrackFormat (I);String mime = mediaFormat.getString (MediaFormat.KEY_MIME);If (mime.startsWith (M)EDIA_TYPE) {TrackIndex = I;Break;}}Return trackIndex;}

Pass buffer to decoder

    //Pass buffer to decoderPrivate Boolean putBufferToCoder (MediaExtractor extractor, MediaCodec decoder, ByteBuFfer[] inputBuffers) {Boolean isMediaEOS = false;Int inputBufferIndex = decoder.deQueueInputBuffer (TIMEOUT_US);If (inputBufferIndex > = 0) {ByteBuffer inputBuFfer = inputBuffers[inputBufferIndex];Int sampleSize = extractor.readSampleData (inputBu)Ffer, 0);If (sampleSize < 0) {Decoder.queueInputBuffer (inputBufferI)Ndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);IsMediaEOS = true;Log.v (TAG, "media EOS");} else {Decoder.queueInputBuffer (inputBuf)FerIndex, 0, sampleSize, extractor.getSampleTime (), 0);Extractor.advance ();}}Return isMediaEOS;}

Part of the audio is similar, complete source code, please move stefanJi/MediaPlaySimpleDemo