第三方应用视频静态缩略图有角 度旋转且有变形

[DESCRIPTION]
第三方应用在显示视频静态缩略图时发现有角度旋转且有变形，但是默认图库视频缩略图显示正常，且本地录制视频没有该现象，现象如下图所示：

原始视频截图 默认图库 第三方应用
[SOLUTION]
1.首先介绍下默认图库生成video thumbnail的流程
1）php?mod=tag&id=6090" target="_blank" class="relatedlink">Framework通过MediaMetadataRetriever.getFrameAtTime解码出该frame，并且根据orientation值去做相应的旋转
根据文件的拿到正确rotationAngle，并且赋值给frame->mRotationAngle
frameworks\av\media\libstagefright\StafgefrightMetadataRetriever.cpp
static VideoFrame *extractVideoFrameWithCodecFlags(
OMXCLIent *client,
const sp<MetaData> &trackMeta,
const sp<MediaSource> &source,
uint32_t flags,
int64_t frameTimeUs,
int seekMode) {
...
int32_t rotationAngle;
if (！trackMeta->findInt32(kKeyRotation, &rotationAngle)) {
rotationAngle = 0; // By default, no rotation
}
VideoFrame *frame = new VideoFrame;
frame->mWidth = crop_right - crop_left + 1;
frame->mHeight = crop_bottom - crop_top + 1;
frame->mDisplayWidth = frame->mWidth;
frame->mDisplayHeight = frame->mHeight;
#ifdef mtk_HIGH_QUALITY_THUMBNAIL
frame->mSize = frame->mWidth * frame->mHeight * 4;
#else
frame->mSize = frame->mWidth * frame->mHeight * 2;
#endif
frame->mData = new uint8_t[frame->mSize];
frame->mRotationAngle = rotationAngle;
...
}
根据mRotationAngle，在framework做相应的rotate
frameworks\base\media\jni\android_media_MediaMetadataRetirever.cp
static jobject android_media_MediaMetadataRetriever_getFrameAtTime(JNIEnv *env, jobject thiz, jlong timeUs, jint option)
{
...
bool swapWidthAndHeight = false;
if (videoFrame->mRotationAngle == 90 || videoFrame->mRotationAngle == 270) {
width = videoFrame->mHeight;
height = videoFrame->mWidth;
swapWidthAndHeight = true;
} else {
width = videoFrame->mWidth;
height = videoFrame->mHeight;
}
jobject jBitmap = env->CallStaticObjectMethod(
fields.bitmapClazz,
fields.createBitmapMethod,
width,
height,
config);
if (jBitmap == NULL) {
if (env->ExceptionCheck()) {
env->ExceptionClear();
}
ALOGE("getFrameAtTime: create Bitmap failed！");
return NULL;
}
SkBitmap *bitmap =
(SkBitmap *) env->GetLongField(jBitmap, fields.nativeBitmap);
if (bitmap == NULL) {
ALOGE("getFrameAtTime: bitmap is a NULL pointer！");
return NULL;
}
bitmap->lockPixels();
#ifdef MTK_HIGH_QUALITY_THUMBNAIL
rotate((uint32_t*)bitmap->getPixels(),
(uint32_t*)((char*)videoFrame + sizeof(VideoFrame)),
videoFrame->mWidth,
videoFrame->mHeight,
videoFrame->mRotationAngle);
#else
rotate((uint16_t*)bitmap->getPixels(),
(uint16_t*)((char*)videoFrame + sizeof(VideoFrame)),
videoFrame->mWidth,
videoFrame->mHeight,
videoFrame->mRotationAngle);
#endif
bitmap->unlockPixels();
...
return jBitmap;
}
2）app拿到已转90度的图像再做相应的resizeandcropcenter，并不会去query database里的orientation值角度旋转，只会做resizeandcrop。
alps\packages\apps\Gallery2\src\com\android\gallery3d\data\ImageCachedRequest.java
alps\packages\apps\Gallery2\gallerycommon\src\com\android\gallery3d\common\BitmapUtils.java里resizeAndCropCenter()
综上所述，定位framework没有问题，比较怀疑第三方应用有去查询database里视频文件的orientation值做角度旋转。
2.查看database，本地竖屏录制的有角度视频文件的orientation值为0和非本地录制的有角度视频文件的orientation为90，再介绍下orientation值的产生流程：
1）直接录制视频时，是CAMERA ap直接调用MediaProvider的insert方法，将感兴趣的列赋值插入到数据库的，camera app并没有显示的插入orientation列，所以默认database里orientation值为0
2）非本地录制的有角度视频文件，通过蓝牙，nfc或者从PC上直接复制过来，是调用MediaScannerConnection，通过MediaScanner对该文件进行扫描，有列出"rotation"这一列，最终会调用StagefrightMetadataRetriever去读取视频信息，如果视频信息中本身orientation不等于0，那通过MediaScanner扫描出的文件信息也会包含该值，因此在插入数据库时，就会出现orientation的值。
frameworks\av\media\libstagefright\stagefrightMediaScanner.cpp
MediaScanResult StagefrightMediaScanner::processFileInternal(
const char *path, const char * /* mimeType */,
MediaScannerClient &client) {
...
struct KeyMap {
const char *tag;
int key;
};
static const KeyMap kKeyMap[] = {
{ "tracknumber", METADATA_KEY_CD_TRACK_NUMBER },
{ "discnumber", METADATA_KEY_DISC_NUMBER },
{ "album", METADATA_KEY_ALBUM },
{ "artist", METADATA_KEY_ARTIST },
{ "albumartist", METADATA_KEY_ALBUMARTIST },
{ "composer", METADATA_KEY_COMPOSER },
{ "genre", METADATA_KEY_GENRE },
{ "title", METADATA_KEY_TITLE },
{ "year", METADATA_KEY_YEAR },
{ "duration", METADATA_KEY_DURATION },
{ "writer", METADATA_KEY_WRITER },
{ "compilation", METADATA_KEY_COMPILATION },
{ "isdrm", METADATA_KEY_IS_DRM },
{ "width", METADATA_KEY_VIDEO_WIDTH },
{ "height", METADATA_KEY_VIDEO_HEIGHT },
/// M: add for the Gallery 6595 new feature -- fancy Layout
{ "rotation", METADATA_KEY_VIDEO_ROTATION },
#ifdef MTK_AOSP_ENHANCEMENT
....
}
3.KK2后gallery引入fancy ui，database会记录orientation值去决定第一张缩略图是横排还是竖排。若贵司没有用弊司默认的gallery或者说gallery fancy feature没有打开的话，可以直接注释掉kKeyMap里“rotation”这一列，否则请APP做修正，不要考虑orientation值。