I am currently developing a video application which processes video frames from the camera in OpenGL. As this needs to run in realtime I have done my best to remove any copying/uploading of texture data. My solution so far has been to use the EGLImage extension to be able to use the GraphicBuffer from the camera directly as an OpenGL texture with no need to upload the data. The problem here is that this extension seems to be limited to a few pixel formats.
RGBA
This is the format I currently use. I disguise the YUV buffer as an RGBA GraphicBuffer (i.e. no conversion, simply letting OpenGL think the data is in RGBA format) and can use it directly as a texture in OpenGL. However, this involves an unpacking pass where the data in the RGBA texture is moved into two separate textures (one R8 representing Y and one RG88 representing UV) and a packing pass where the data is moved back into the RGBA memory alignment. So the uploading/downloading is replaced by two render passes instead which is better, but still creates unnecessary overhead for high resolutions.
YV12
I am aware of the OES_EGL_image_external extension which can be used to directly sample a YUV GraphicBuffer as an OpenGL texture (I guess this is used to render the preview from the camera). The problem here is that the extension will automatically convert the sample into an RGBA value which is not what I want. This would be a good solution if there was a way to prevent this conversion from happening.
Y8
I tried using Y8 GraphicBuffers but it seems this format is not compatible with EGLImage.
So is there a way to achieve the transfer of YUV buffers to OpenGL without any overhead and without automatic conversion to RGBA? Is there a specification of which GraphicBuffer formats are supported by the EGLImage extension? Are there any Qualcomm-specific solutions for this problem?