从demo分析ijk源码一：视频播放

作者: DON_1007 | 来源:发表于2019-12-13 17:58 被阅读0次

从demo分析ijk源码一：视频播放
从demo分析ijk源码五：视频解码
从demo分析ijk源码四：ijk调用ffplay流程分析
从demo分析ijk源码二：IjkMediaPlayer实现分析
ijk播放器在Android 11上播放出现的问题
从demo分析ijk源码三：IjkMediaPlayer的jni
iOS云点播第三方平台视频点播调研
音视频、聊天
iOS IJKPlayer 支持播放 RTSP
QZIJKPlayer 基于 IJK的简易直播/视频播放器 Sw

ijk Android demo源码的整体结构如下

demo

ijkplayer-example是demo程序的主module，它依赖其它module，并实现一个简单的播放器程序
ijkplayer-java 是ijk库的Java实现代码，它的作用有三个
1、加载ijk的so
2、实现对ijk so的jni调用封装
3、封装IjkMediaPlayer供调用者直接使用
ijkplayer-exo 提供了一个使用 google exoplayer的实现封装IjkExoMediaPlayer，IjkExoMediaPlayer的实现继承了ijkplayer-java中的抽象类AbstractMediaPlayer，在ijkplayer-example调用的时候可与IjkMediaPlayer保持一致。
ijkplayer-arm64等，这类module中存放了与各芯片架构对应c源文件与编译ijk之后生成的so等，无Java实现，ijkplayer-example引入对应的moudle，可将该module所包含的so编译进最终的apk中。

一、ijkplayer-example

ijkplayer-example中的Java代码如下

example/
├── activities
│   ├── FileExplorerActivity.java
│   ├── RecentMediaActivity.java
│   ├── SampleMediaActivity.java
│   ├── SettingsActivity.java
│   └── VideoActivity.java
├── application
│   ├── AppActivity.java
│   └── Settings.java
├── content
│   ├── PathCursor.java
│   ├── PathCursorLoader.java
│   └── RecentMediaStorage.java
├── eventbus
│   └── FileExplorerEvents.java
├── fragments
│   ├── FileListFragment.java
│   ├── RecentMediaListFragment.java
│   ├── SampleMediaListFragment.java
│   ├── SettingsFragment.java
│   └── TracksFragment.java
├── services
│   └── MediaPlayerService.java
└── widget
    ├── media
    │   ├── AndroidMediaController.java
    │   ├── FileMediaDataSource.java
    │   ├── IMediaController.java
    │   ├── IRenderView.java
    │   ├── IjkVideoView.java
    │   ├── InfoHudViewHolder.java
    │   ├── MeasureHelper.java
    │   ├── MediaPlayerCompat.java
    │   ├── SurfaceRenderView.java
    │   ├── TableLayoutBinder.java
    │   └── TextureRenderView.java
    └── preference
        └── IjkListPreference.java

播放器的实现在VideoActivity中，其余的Activity都是用来配合VideoActivity的

FileExplorerActivity 文件浏览器，可选择本机视频播放
RecentMediaActivity 记录最近的播放信息
SampleMediaActivity 提供了示例视频url，可直接播放
SettingsActivity 播放器参数设置
VideoActivity 播放器

二、如何使用`ijkplayer-java`封装的`IjkMediaPlayer`

Android系统播放器的使用是MediaPlayer + Surface，Surface可以通过SurfaceView或TextureView获取。
ijkplayer-example中封装了一个类IjkVideoView,IjkVideoView中演示了三种播放器实现的调用

IjkExoMediaPlayer在Ijkplayer-exo中对google exoplayer的调用封装
AndroidMediaPlayer对android系统播放器MediaPlayer的调用封装
IjkMediaPlayer在Ijkplayer-java中对ffmpeg的调用封装

1、创建IMediaPlayer

在使用ijk之前先加载so

IjkMediaPlayer.loadLibrariesOnce(null);
IjkMediaPlayer.native_profileBegin("libijkplayer.so");

IjkExoMediaPlayer、AndroidMediaPlayer、IjkMediaPlayer都实现了接口IMediaPlayer

    public IMediaPlayer createPlayer(int playerType) {
        IMediaPlayer mediaPlayer = null;

        switch (playerType) {
            case Settings.PV_PLAYER__IjkExoMediaPlayer: {
                IjkExoMediaPlayer IjkExoMediaPlayer = new IjkExoMediaPlayer(mAppContext);
                mediaPlayer = IjkExoMediaPlayer;
            }
            break;
            case Settings.PV_PLAYER__AndroidMediaPlayer: {
                AndroidMediaPlayer androidMediaPlayer = new AndroidMediaPlayer();
                mediaPlayer = androidMediaPlayer;
            }
            break;
            case Settings.PV_PLAYER__IjkMediaPlayer:
            default: {
                IjkMediaPlayer ijkMediaPlayer = null;
                if (mUri != null) {
                    ijkMediaPlayer = new IjkMediaPlayer();
                    ijkMediaPlayer.native_setLogLevel(IjkMediaPlayer.IJK_LOG_DEBUG);

                    if (mSettings.getUsingMediaCodec()) {
                        ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec", 1);
                        if (mSettings.getUsingMediaCodecAutoRotate()) {
                            ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec-auto-rotate", 1);
                        } else {
                            ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec-auto-rotate", 0);
                        }
                        if (mSettings.getMediaCodecHandleResolutionChange()) {
                            ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec-handle-resolution-change", 1);
                        } else {
                            ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec-handle-resolution-change", 0);
                        }
                    } else {
                        ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "mediacodec", 0);
                    }

                    if (mSettings.getUsingOpenSLES()) {
                        ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "opensles", 1);
                    } else {
                        ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "opensles", 0);
                    }

                    String pixelFormat = mSettings.getPixelFormat();
                    if (TextUtils.isEmpty(pixelFormat)) {
                        ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "overlay-format", IjkMediaPlayer.SDL_FCC_RV32);
                    } else {
                        ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "overlay-format", pixelFormat);
                    }
                    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "framedrop", 1);
                    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "start-on-prepared", 0);
                    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "http-detect-range-support", 0);
                    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_CODEC, "skip_loop_filter", 48);
                }
                mediaPlayer = ijkMediaPlayer;
            }
            break;
        }

        ...

        return mediaPlayer;
    }

上面的代码根据设置界面的选择，可以创建对应类型的播放器，这里先忽略IjkExoMediaPlayer和AndroidMediaPlayer，重点关注IjkMediaPlayer。

2、创建Surface

Surface可以通过SurfaceView或TextureView获取，在ijkplayer-example中封装了一个接口IRenderView用于将SurfaceView和TextureView的实现和使用统一。
IRenderView

public interface IRenderView {
    int AR_ASPECT_FIT_PARENT = 0; // without clip
    int AR_ASPECT_FILL_PARENT = 1; // may clip
    int AR_ASPECT_WRAP_CONTENT = 2;
    int AR_MATCH_PARENT = 3;
    int AR_16_9_FIT_PARENT = 4;
    int AR_4_3_FIT_PARENT = 5;

    View getView();

    boolean shouldWaitForResize();

    void setVideoSize(int videoWidth, int videoHeight);

    void setVideoSampleAspectRatio(int videoSarNum, int videoSarDen);

    void setVideoRotation(int degree);

    void setAspectRatio(int aspectRatio);

    void addRenderCallback(@NonNull IRenderCallback callback);

    void removeRenderCallback(@NonNull IRenderCallback callback);

    interface ISurfaceHolder {
        void bindToMediaPlayer(IMediaPlayer mp);

        @NonNull
        IRenderView getRenderView();

        @Nullable
        SurfaceHolder getSurfaceHolder();

        @Nullable
        Surface openSurface();

        @Nullable
        SurfaceTexture getSurfaceTexture();
    }

    interface IRenderCallback {
        /**
         * @param holder
         * @param width  could be 0
         * @param height could be 0
         */
        void onSurfaceCreated(@NonNull ISurfaceHolder holder, int width, int height);

        /**
         * @param holder
         * @param format could be 0
         * @param width
         * @param height
         */
        void onSurfaceChanged(@NonNull ISurfaceHolder holder, int format, int width, int height);

        void onSurfaceDestroyed(@NonNull ISurfaceHolder holder);
    }
}

TextureRenderView

public class TextureRenderView extends TextureView implements IRenderView {
    ...
}

SurfaceRenderView

public class SurfaceRenderView extends SurfaceView implements IRenderView {
    ...
}

这里略过了SurfaceRenderView和TextureRenderView的内部实现，它们的目的只有一个，用于生成Surface，供播放器渲染画面。
根据设置界面的选择创建SurfaceView或TextureView

    public void setRender(int render) {
        switch (render) {
            case RENDER_NONE:
                setRenderView(null);
                break;
            case RENDER_TEXTURE_VIEW: {
                TextureRenderView renderView = new TextureRenderView(getContext());
                if (mMediaPlayer != null) {
                    renderView.getSurfaceHolder().bindToMediaPlayer(mMediaPlayer);
                    renderView.setVideoSize(mMediaPlayer.getVideoWidth(), mMediaPlayer.getVideoHeight());
                    renderView.setVideoSampleAspectRatio(mMediaPlayer.getVideoSarNum(), mMediaPlayer.getVideoSarDen());
                    renderView.setAspectRatio(mCurrentAspectRatio);
                }
                setRenderView(renderView);
                break;
            }
            case RENDER_SURFACE_VIEW: {
                SurfaceRenderView renderView = new SurfaceRenderView(getContext());
                setRenderView(renderView);
                break;
            }
            default:
                Log.e(TAG, String.format(Locale.getDefault(), "invalid render %d\n", render));
                break;
        }
    }

3、打开视频

    private void setVideoURI(Uri uri, Map<String, String> headers) {
        mUri = uri;
        mHeaders = headers;
        mSeekWhenPrepared = 0;
        openVideo();
        requestLayout();
        invalidate();
    }

    private void openVideo() {
        if (mUri == null || mSurfaceHolder == null) {
            // 视频地址无效或Surface还未创建完成
            return;
        }
   
        release(false);
        AudioManager am = (AudioManager) mAppContext.getSystemService(Context.AUDIO_SERVICE);
        // 获取音频焦点
        am.requestAudioFocus(null, AudioManager.STREAM_MUSIC, AudioManager.AUDIOFOCUS_GAIN);

        try {
            mMediaPlayer = createPlayer(mSettings.getPlayer());
            final Context context = getContext();
            mMediaPlayer.setOnPreparedListener(mPreparedListener);
            mMediaPlayer.setOnVideoSizeChangedListener(mSizeChangedListener);
            mMediaPlayer.setOnCompletionListener(mCompletionListener);
            mMediaPlayer.setOnErrorListener(mErrorListener);
            mMediaPlayer.setOnInfoListener(mInfoListener);
            mMediaPlayer.setOnBufferingUpdateListener(mBufferingUpdateListener);
            mMediaPlayer.setOnSeekCompleteListener(mSeekCompleteListener);
            mMediaPlayer.setOnTimedTextListener(mOnTimedTextListener);
            mCurrentBufferPercentage = 0;
            String scheme = mUri.getScheme();
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M &&
                    mSettings.getUsingMediaDataSource() &&
                    (TextUtils.isEmpty(scheme) || scheme.equalsIgnoreCase("file"))) {
                IMediaDataSource dataSource = new FileMediaDataSource(new File(mUri.toString()));
                mMediaPlayer.setDataSource(dataSource);
            }  else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH) {
                mMediaPlayer.setDataSource(mAppContext, mUri, mHeaders);
            } else {
                mMediaPlayer.setDataSource(mUri.toString());
            }
            bindSurfaceHolder(mMediaPlayer, mSurfaceHolder);
            mMediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
            mMediaPlayer.setScreenOnWhilePlaying(true);
            mPrepareStartTime = System.currentTimeMillis();
            mMediaPlayer.prepareAsync();
            if (mHudViewHolder != null)
                mHudViewHolder.setMediaPlayer(mMediaPlayer);

            mCurrentState = STATE_PREPARING;
            attachMediaController();
        } catch (IOException ex) {
            Log.w(TAG, "Unable to open content: " + mUri, ex);
            mCurrentState = STATE_ERROR;
            mTargetState = STATE_ERROR;
            mErrorListener.onError(mMediaPlayer, MediaPlayer.MEDIA_ERROR_UNKNOWN, 0);
        } catch (IllegalArgumentException ex) {
            Log.w(TAG, "Unable to open content: " + mUri, ex);
            mCurrentState = STATE_ERROR;
            mTargetState = STATE_ERROR;
            mErrorListener.onError(mMediaPlayer, MediaPlayer.MEDIA_ERROR_UNKNOWN, 0);
        } finally {
            // REMOVED: mPendingSubtitleTracks.clear();
        }
    }

从openVideo中可以看到，IJKMediaPlayer的调用大致分为下面几个部分

设置监听 mMediaPlayer.setOnPreparedListener 等
设置视频源 mMediaPlayer.setDataSource(mUri.toString())
开始加载 mMediaPlayer.prepareAsync()
经过IJKMediaPlayer的封装之后，其调用方法基本与系统播放器MediaPlayer保持一致。

4、响应播放器状态变化

通过上面设置的监听，可以实时监听播放器的状态变化

mMediaPlayer.setOnPreparedListener(mPreparedListener)
监听视频加载，回调视频加载完成状态，需要在这里调用播放器的start方法
mMediaPlayer.setOnVideoSizeChangedListener(mSizeChangedListener)
监听视频尺寸变化，在onVideoSizeChanged中可获取视频的宽高信息，根据视频的宽高信息需要设置SurfaceView或TextureView的大小
mMediaPlayer.setOnCompletionListener(mCompletionListener)
视频播放完成监听，需要释放播放器
mMediaPlayer.setOnErrorListener(mErrorListener)
视频加载失败监听，需要释放播放器
mMediaPlayer.setOnInfoListener(mInfoListener)
通常用来做视频缓冲监听，701表示开始缓冲，702表示缓冲完成
mMediaPlayer.setOnBufferingUpdateListener(mBufferingUpdateListener)
缓冲进度监听
mMediaPlayer.setOnSeekCompleteListener(mSeekCompleteListener)
seek状态监听，通过需要做seek保护，在上次seek未完成之前，不允许做新的seek操作
关于IJKMediaPlayer，需要特殊关注的是setOnInfoListener中回调的信息，
IJKMediaPlayer回调了比系统播放器更多的播放器信息，从demo中可以看到

    private IMediaPlayer.OnInfoListener mInfoListener =
            new IMediaPlayer.OnInfoListener() {
                public boolean onInfo(IMediaPlayer mp, int arg1, int arg2) {
                    if (mOnInfoListener != null) {
                        mOnInfoListener.onInfo(mp, arg1, arg2);
                    }
                    switch (arg1) {
                        case IMediaPlayer.MEDIA_INFO_VIDEO_TRACK_LAGGING:
                            Log.d(TAG, "MEDIA_INFO_VIDEO_TRACK_LAGGING:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_VIDEO_RENDERING_START:
                            Log.d(TAG, "MEDIA_INFO_VIDEO_RENDERING_START:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_BUFFERING_START:
                            Log.d(TAG, "MEDIA_INFO_BUFFERING_START:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_BUFFERING_END:
                            Log.d(TAG, "MEDIA_INFO_BUFFERING_END:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_NETWORK_BANDWIDTH:
                            Log.d(TAG, "MEDIA_INFO_NETWORK_BANDWIDTH: " + arg2);
                            break;
                        case IMediaPlayer.MEDIA_INFO_BAD_INTERLEAVING:
                            Log.d(TAG, "MEDIA_INFO_BAD_INTERLEAVING:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_NOT_SEEKABLE:
                            Log.d(TAG, "MEDIA_INFO_NOT_SEEKABLE:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_METADATA_UPDATE:
                            Log.d(TAG, "MEDIA_INFO_METADATA_UPDATE:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_UNSUPPORTED_SUBTITLE:
                            Log.d(TAG, "MEDIA_INFO_UNSUPPORTED_SUBTITLE:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_SUBTITLE_TIMED_OUT:
                            Log.d(TAG, "MEDIA_INFO_SUBTITLE_TIMED_OUT:");
                            break;
                        case IMediaPlayer.MEDIA_INFO_VIDEO_ROTATION_CHANGED:
                            mVideoRotationDegree = arg2;
                            Log.d(TAG, "MEDIA_INFO_VIDEO_ROTATION_CHANGED: " + arg2);
                            if (mRenderView != null)
                                mRenderView.setVideoRotation(arg2);
                            break;
                        case IMediaPlayer.MEDIA_INFO_AUDIO_RENDERING_START:
                            Log.d(TAG, "MEDIA_INFO_AUDIO_RENDERING_START:");
                            break;
                    }
                    return true;
                }
            };

以上就是IJKMediaPlayer关于视频播放的实现。

从demo分析ijk源码一：视频播放
ijk Android demo源码的整体结构如下 ijkplayer-example是demo程序的主modul...
从demo分析ijk源码五：视频解码
一、播放器的初始化 ijk播放器初始化的时候会先调用native_setup，创建播放器实例 IjkMediaP...
从demo分析ijk源码四：ijk调用ffplay流程分析
以打开一个网络视频的流程为例，我们需要依次调用IJKMediaPlayer的setDisplay/setSurfa...
从demo分析ijk源码二：IjkMediaPlayer实现分析
在module ijkplayer-java中实现了对ffmpeg调用的封装类IjkMediaPlayer，Ijk...
ijk播放器在Android 11上播放出现的问题
项目开发过程中，使用ijk内核播放器在Android 11上播放视频，报如下错误：分析错误日志：signal的错...
从demo分析ijk源码三：IjkMediaPlayer的jni
回顾 mac系统 ijk（Android）编译，ijk的整个编译过程大致分为下载、编译ffmpeg、编译ijk，编...
iOS云点播第三方平台视频点播调研
视频播放@ ijkplayer - 非常赞 B站开源的视频播放器，支持Android和iOS。 iOS中集成ijk...
音视频、聊天
------ijk播放器------YCVideoPlayer，基于ijk播放器~201810317VideoPl...
iOS IJKPlayer 支持播放 RTSP
今天记录一下 ijk ，ijk 默认不支持播放 RTSP 的，下面就说一下如何配置 ijk 让其支持播放 RTSP...
QZIJKPlayer 基于 IJK的简易直播/视频播放器 Sw
QZIJKPlayer 基于 IJK的简易视频播放器，支持切换多个播放源、全屏、跟随屏幕旋转全屏，使用 Swift...