YunOS Hardware Abstract Layer --Camera

更新时间：2017/08/13 访问次数：2422

1. 概述(Introduction)
2. 架构(Camera Framework Architecture)
3. 实现(Implement Camera HAL)
4. 测试(Vendor Test Suite)
5. 附录(Appendix)

文档内容拷贝自：codebase_host/yunhal/opendoc/zh/yunhal_doc/camera.md
联系人：@张剑

1. 概述(Introduction)

本文档详细描述了YunOS Camera Framework的整体软件框架，侧重于YunOS Camera Hardware Abstract Layer的介绍，并对厂商如何实现YunOS Camera Hardware Abstract提供了指导。

YunOS Camera Framework会为上层应用提供调用接口建立起相机链路，从而实现用户的需求：如预览，拍照，录像等。同时它也会定义好Hardware Abstract Layer的接口用来抽象底层硬件的控制，厂商通过实现Hardware Abstract Layer的接口，就能接入YunOS Camera Framework中。涉及YunOS Camera Hardware Abstract Layer的部分主要包括下面几点：

VideoCaptureDevice
VideoCaptureCallback
VideoCaptureParam

2. 架构(Camera Framework Architecture)

图2.1展示了YunOS Camera Framework的整体软件架构：

YunOS_Camera_Framework

图2.1: YunOS Camera Framework整体架构图

Camera Framework采用了C/S架构，会有一个常驻内存的Camera守护进程，任何需要访问相机的操作都需要通过Socket与守护进程通信，得到允许后才会返回Camera客户端。

Camera APP首先需要获得相机访问权限，然后就能通过Framework暴露的接口来建立客户端，一旦建立成功就能发起命令控制相机，并能通过注册的回调函数拿到所需要的数据。

Camera Framework会为每个相机建立专门的session，然后分解应用传来的信息，并提供HAL所需要的预览适配模块，相机参数模块和内存共享模块，然后通过HAL接口对底层硬件进行访问，控制摄像头与图像处理器等行为，最后拿到需要的图像数据。

2.1. Camera Framework主要工作线程架构

图2.2展示了YunOS Camera Framework的主要工作线程架构图：

YunOS_Camera_Framework_Thread

图2.2：YunOS Camera Framework工作线程架构图

Camera Framework采用Socket协议建立Inter Process Communication(IPC)通信，并会每个客户端会起两类Socket分别处理正向的命令控制和反向的数据接收，如图中的ClientThread和CallbackThread所示。守护进程接收到客户端过来的消息后会单独分配工作线程来分发请求给HAL，HAL完成后会把结果或者数据返回给守护进程的回调线程，回调线程再回传给客户端，而且这里会根据数据类型分别并行回传，做到互不干扰，客户端接收完数据后再发送确认应答。

3. 实现(Implement Camera HAL)

图3.1展示了YunOS Camera Framework的接口架构

YunOS_Camera_Framework_Intf

图3.1：YunOS Camera Framework接口架构图

厂商着重实现灰色部分模块。

3.1. 如何实现 VideoCaptureDevice

VideoCaptureDevice 定义了所有控制相机硬件的接口，厂商可根据自身特点实现。

下面是 VideoCaptureDevice 的主要接口,控制相机硬件。

Class VideoCaptureDevice: public VendorDevice {
        //inherit VendorDevice
        virtual uint32_t GetVersion() = 0;                                      
        virtual const char* GetID() = 0;                                        
        virtual yunos::VendorModule* GetModule() = 0;                                          
        virtual int32_t Destroy() = 0;
 
        // register callback for VideoCaptureDevice
        virtual bool RegisterCallback(VideoCaptureDeviceCallback* cb) = 0;
         
        // preview related api
        virtual bool StartPreview() = 0;
        virtual void StopPreview() = 0;
        virtual bool PreviewEnabled() = 0;
        virtual bool SetPreviewTarget(PreviewAdaptor* previewAdaptor) = 0;
 
        // recording related api
        virtual bool StartRecording() = 0;
        virtual void StopRecording() = 0;
        // notify HAL can release posted recording frame by index
        virtual void ReleaseRecordingFrame(int index) = 0;
        virtual void SetPreviewCallbackFlag(vcm_callback_t flag) = 0;
         
        // capture related api
        virtual bool TakePicture(vcm_callback_msg_t msgType) = 0;
        virtual bool CancelTakePicture() = 0;
 
        // focus
        virtual bool AutoFocus() = 0
        virtual bool CancelAutoFocus() = 0;
 
        // parameters setting
        virtual bool SetParameters(VideoCaptureParam* params, stream_video_capture_type_e type = STREAM_INIT) = 0;
        virtual std::string GetParameters(stream_video_capture_type_e type = STREAM_INIT) = 0;
 
        // command
        virtual bool SendCommand(vcm_command_t cmd, int arg1, int arg2) = 0;
        // msg
        virtual void EnableMsgType(vcm_callback_msg_t type) = 0;
        virtual void DisableMsgType(vcm_callback_msg_t type) = 0;
        virtual int MsgTypeEnabled(vcm_callback_msg_t type) = 0;
 
    }

下面是 PreviewAdaptor 的主要接口，用于相机与显示系统通信，能方便地完成预览功能。

class PreviewAdaptor {
        // fetch buffer
        virtual int32_t ObtainBuffer(buffer_info_t** buffer, int* syncFd, int* stride) = 0;
        // send filled data buffer back
        virtual int32_t SubmitBuffer(buffer_info_t* buffer, int syncFd) = 0;
        // drop invalid buffer
        virtual int32_t DropBuffer(buffer_info_t* buffer, int syncFd) = 0;
         
        virtual int32_t SetBufferCount(int count) = 0;
        virtual int32_t SetBuffersDimensions(int w, int h) = 0;
        virtual int32_t SetBuffersFormat(int format) = 0;
        virtual int32_t SetBuffersCrop(int l, int t, int r, int b) = 0;
        virtuaal int32_t SetBuffersUsage(int reqUsage) = 0;
        virtual int32_t SetBuffersTransform(int transform) = 0;
    };

下面是 ImageBuffer 的接口， ImageBuffer 是Camera Framework为HAL提供分配内存模块，并且提供把Buffer描述符写入共享内存的接口。

class ImageBuffer {
        // get buffer virtual address
        virtual int Map(int usage, int left, int top, int width, int height, void** vaddr) = 0; 
        virtual int Unmap() = 0;
 
        // authorize buffer for using in other process
        virtual int Authorize() = 0;
        virtual int Unauthorize() = 0;
 
        virtual int GetStride() const = 0;
 
        // get the size of buffer serialization
        virtual int GetSerializeSize() const = 0;
        // serialize buffer descriptor to memory
        virtual bool Serialize(uint8_t* mem, int size) const = 0;
        // unserialize buffer from memory
        virtual bool Unserialize(const uint8_t* mem) = 0;
 
        // get buffer descriptor
        virtual int* GetBufDescriptor(int* count) = 0;
        virtual bool SetBufDescriptor(const int* bufDescriptor, const int count) = 0;
 
        // get native buffer handle
        virtual pf_buffer_handle_t* GetNativeBufHandle() = 0;
    };

ImageBuffer 可以通过如下接口来创建：

VC_EXPORT_API YunOS::ImageBuffer* NewImageBuffer(int width, int height, int format, int usage);

3.2. 如何实现 VideoCaptureCallback

VideoCaptureCallback 定义了数据回调的接口，厂商使用此接口进行消息/数据传递。

class VideoCaptureDeviceCallback {
        virtual void Notify(vcm_callback_msg_t msgType, int32_t ext1, int32_t ext2) = 0;
        //post data back with share memory
        virtual void PostData(vcm_callback_msg_t msgType, VideoCaptureMemory* memPtr,VideoFrameMetaData *metadata) = 0;
        //post data back with timestamp and share memory
        virtual void PostDataWithTimestamp(nanosecs_t timestamp, vcm_callback_msg_t msgType, VideoCaptureMemory* memPtr) = 0;
 
        //request share memory
        virtual VideoCaptureMemory* RequestVCMem(size_t size) = 0;
        //release share memory
        virtual void ReleaseVCMem(VideoCaptureMemory* vcmem) = 0;
    }

3.3. 拍照和录像数据回调的示例代码

下面结合拍照和录像场景贴出示例代码，供厂商参考：

// alloc a new ImageBuffer with width, height, format, usage
    ImageBuffer* imgBuf = NewImageBuffer(width, height, format, usage);
    pf_buffer_handle_t* bufferHandle = imgBuf->GetNativeBufHandle(); // you can get physical address from it
 
    // ===================== send buffer data start ===================
    const size_t dataSize = imageDataSize; //imageDataSize is the size of image data
    VideoCaptureMemory* dataMem = mRemoteCallback->RequestVCMem(dataSize);
 
    // get buffer virtual address
    void* vaddr = NULL;
    imgBuf->Map(usage, left, top, width, height, &vaddr);
 
    // copy buffer data to share memory
    memcpy(dataMem->data, vaddr, dataSize);
 
    // Unmap it after use
    imgBuf->Unmap();
 
    mRemoteCallback->PostData(VCM_CALLBACK_MSG_PREVIEW_FRAME,
                          dataMem,
                          NULL);
    // ===================== send buffer data end ===================
 
    // ===================== send buffer handle start ===================
    // serialize to VCMSHMem
    // 1. calculate share memory size
    const size_t bufferSize = sizeof(data_callback_header_t) + imgBuf->GetSerializeSize();
 
    // 2. create share memory
    VideoCaptureMemory* handleMem = mRemoteCallback->RequestVCMem(bufferSize);
 
    // send buffer handle
    // 3. set header to share memory
    data_callback_header_t* header = (data_callback_header_t*)(handleMem->data);
    uint32_t type = 1/*kMetadataBufferTypeGrallocSource*/;
    header->index = heapIdx; //index which for release recording frame
    header->type = type;
    header->reserved = 0;//reserved for future use
 
    // 4. set ImageBuffer to share memory
    uint8_t* handleInfo = (uint8_t*)(handleMem->data) + sizeof(data_callback_header_t);
    imgBuf->Serialize(handleInfo, imgBuf->GetSerializeSize());
 
    // call camera callback function
    mRemoteCallback->PostDataWithTimestamp(timestamp,
        VCM_CALLBACK_MSG_VIDEO_FRAME_BUFFERHANDLE,
        handleMem);
    // ===================== send buffer handle end ===================
 
    // VCMSHMem and ImageBuffer should be deleted after use
    mRemoteCallback->ReleaseVCMem(dataMem);
    mRemoteCallback->ReleaseVCMem(handleMem);
 
    delete imgBuf;

3.4. 如何使用 VideoCaptureParam

VideoCaptureParam 用来传递相机参数，采用的key-value结构，厂商请直接使用头文件里已有的Key和Value值实现主要的参数配置，特殊的参数配置厂商可以直接在头文件里添进行扩充。
这里要特别注意 VideoCaptureParam 使用的都是std::string类，请尽量避免直接使用c_str()来操作其指针。

4. 测试(Vendor Test Suite)

YunOS Camera Framework开发了底层演示程序和单元测试用来验证HAL的功能，同时也提供了底层显示程序用来方便查看相机运行的预览。

4.1. 如何运行底层演示程序

先运行底层显示程序

1	`camera_native_dispay`

再运行底层演示程序

1	`camera_native_demo`

然后可以根据命令提示操作相机预览，拍照，录像等。

4.2. 如何运行Vendor Test Suite

按照统一VTS test启动方式运行测试

1	`yunhal_tests`

执行VideoCapture相关VTS测试

1	`yunhal_tests --gtest_filter=.TestVideoCapture`

4.3. 如何运行Native Test Suite