Android 4.4 Graphic Architecture

zangcf

Some things that work with Surfaces want a SurfaceHolder, notably SurfaceView. The original idea was that Surface represented the raw compositor-managed buffer, while SurfaceHolder was managed by the app and kept track of higher-level information like the dimensions and format. The Java-language definition mirrors the underlying native implementation. It’s arguably no longer useful to split it this way, but it has long been part of the public API.

一些在surface上工作的东西需要一个SurfaceHolder，尤其是SurfaceView。初始的想法是，Surface代表了raw格式的，被混合器管理的缓冲区，而SurfaceHolder被app管理。这样，app可以在比如大小以及格式等更高层面上来处理问题。Java层定义了一个底层实现的上层镜像。这种分层方法目前已经不再有意义，但是它已经长时间成为了公共API中的一部分。

Generally speaking, anything having to do with a View will involve a SurfaceHolder. Some other APIs, such as MediaCodec, will operate on the Surface itself. You can easily get the Surface from the SurfaceHolder, so hang on to the latter when you have it.

一般而言，对View所做的一切事情都需要通过一个SurfaceHolder。其他的一些api，比如MediaCodec，将直接在Surface上操作。你可以很容易的从一个SurfaceHolder中获取一个Surface。

APIs to get and set Surface parameters, such as the size and format, are implemented through SurfaceHolder.

获取和设置Surface参数的一些API，比如大小和格式，都是通过SurfaceHolder实现。

zangcf

OpenGL ES defines an API for rendering graphics. It does not define a windowing system. To allow GLES to work on a variety of platforms, it is designed to be combined with a library that knows how to create and access windows through the operating system. The library used for Android is called EGL. If you want to draw textured polygons, you use GLES calls; if you want to put your rendering on the screen, you use EGL calls.

OpenGL ES定义了一组Graphic的渲染API。它并没有定义一个窗口系统。为了让GLES可以工作在不同的平台之上，它设计了一个库，这个库知道如何在指定的操作系统上创建和使用窗口。Android上的这个库叫做EGL。如果你想绘制一个多边形，那么使用GLES的函数；如果你想要将它渲染到屏幕上，你需要使用EGL的调用。

Before you can do anything with GLES, you need to create a GL context. In EGL, this means creating an EGLContext and an EGLSurface. GLES operations apply to the current context, which is accessed through thread-local storage rather than passed around as an argument. This means you have to be careful about which thread your rendering code executes on, and which context is current on that thread.

在你使用GLES做事之前，你需要创建一个GL的上下文。具体针对EGL，这意味着创建一个EGLContext 和一个 EGLSurface。GLES的操作作用在当前的上下文之中，而上下文的访问更多的依赖本地线程的存储而不是参数的传递。这意味着你需要关注你的渲染代码执行在哪个线程之上，并且这个线程的当前上下文是什么。

The EGLSurface can be an off-screen buffer allocated by EGL (called a “pbuffer”) or a window allocated by the operating system. EGL window surfaces are created with the eglCreateWindowSurface() call. It takes a “window object” as an argument, which on Android can be a SurfaceView, a SurfaceTexture, a SurfaceHolder, or a Surface — all of which have a BufferQueue underneath. When you make this call, EGL creates a new EGLSurface object, and connects it to the producer interface of the window object’s BufferQueue. From that point onward, rendering to that EGLSurface results in a buffer being dequeued, rendered into, and queued for use by the consumer. (The term “window” is indicative of the expected use, but bear in mind the output might not be destined to appear on the display.)

EGLSurface可以是一个由EGL分配的离屏缓冲区（”pbuffer”）或者一个由操作系统分配的窗口缓冲区。EGL window Surface由eglCreateWindowSurface()函数创建。它持有一个窗口对象做为参数，在Android系统上，这个对象可能是一个SurfaceView，一个SurfaceTexture，一个SurfaceHolder，或者一个Surface—-所有的这些下面都有一个BuffferQueue。当你调用这个函数时，ELG创建了一个新的EGLSurface对象，并且将它连接到一个窗口对象的BufferQueue的生产者接口上。从这一刻开始，渲染到一个EGLSurface上将导致一个buffer经历出队，渲染，入队供消费者使用 这个过程。（这里属于window被使用，但这只是一个预期，实际上，输出可能并不显示在屏幕上）

EGL does not provide lock/unlock calls. Instead, you issue drawing commands and then call eglSwapBuffers()to submit the current frame. The method name comes from the traditional swap of front and back buffers, but the actual implementation may be very different.

EGL并没有提供lock/unlock的调用。你需要调用绘制命令，然后调用eglSwapBuffers()函数去提交当前的帧。这个方法名字的来源是传统的交换前后缓冲区，但是目前实际的实现可能会有很大的不同。

Only one EGLSurface can be associated with a Surface at a time — you can have only one producer connected to a BufferQueue — but if you destroy the EGLSurface it will disconnect from the BufferQueue and allow something else to connect.

一个EGLSurface一次只能关连一个Surface—-一次只能有一个生产者连接到一个BufferQueue上—但是你可以销毁这个EGLSurface，使得它和BufferQueue的连接断开，这样就可以用其他的东西连接这个BufferQueue了。

A given thread can switch between multiple EGLSurfaces by changing what’s “current.” An EGLSurface must be current on only one thread at a time.

一个给定的线程可以通过设置哪个是Current的方法来在不同的EGLSurfaces间切换，一个线程同时只能有一个EGLSurface作为current。

The most common mistake when thinking about EGLSurface is assuming that it is just another aspect of Surface (like SurfaceHolder). It’s a related but independent concept. You can draw on an EGLSurface that isn’t backed by a Surface, and you can use a Surface without EGL. EGLSurface just gives GLES a place to draw.

一个普通的误解是，很多人把EGLSurface当做是Surface的另一种表现（就像是SurfaceHolder）。他们二者之间有关系，但是这是两个独立的概念。你可以在一个EGLSurface上绘制而不需要一个Surface的支持，你也可以不通过EGL而使用一个Surface。EGLSurface只不过给GLES提供了一个绘制的地方而已。

zangcf

The public Surface class is implemented in the Java programming language. The equivalent in C/C++ is the ANativeWindow class, semi-exposed by the Android NDK. You can get the ANativeWindow from a Surface with the ANativeWindow_fromSurface() call. Just like its Java-language cousin, you can lock it, render in software, and unlock-and-post.

公共的Surface类是由Java实现的。在C/C++层对应的是ANativeWindow类，半暴漏在Android NDK中。你可以通过使用 ANativeWindow_fromSurface()从Surface中获得一个ANativeWindow。就像他的java表亲一样，你可以lock，使用软件渲染，然后unlock-and-post.

To create an EGL window surface from native code, you pass an instance of EGLNativeWindowType to eglCreateWindowSurface(). EGLNativeWindowType is just a synonym for ANativeWindow, so you can freely cast one to the other.

为了从本地代码中创建一个EGL window surface，你需要给eglCreateWindowSurface()方法传递一个EGLNativeWindowType实例。EGLNativeWindowType等同于ANativeWindow，所以你可以在二者之间自由的转换。

The fact that the basic “native window” type just wraps the producer side of a BufferQueue should not come as a surprise.

事实上，native window的本质不过是BufferQueue在生产者一侧的包装罢了。

zangcf

Now that we’ve explored the lower-level components, it’s time to see how they fit into the higher-level components that apps are built from.

现在我们已经研究了底层的一些组件，是时候来看下更高层次上组件是如何工作的了。

The Android app framework UI is based on a hierarchy of objects that start with View. Most of the details don’t matter for this discussion, but it’s helpful to understand that UI elements go through a complicated measurement and layout process that fits them into a rectangular area. All visible View objects are rendered to a SurfaceFlinger-created Surface that was set up by the WindowManager when the app was brought to the foreground. The layout and rendering is performed on the app’s UI thread.

一个Android的app Framework层的UI是从视图上而来的基于对象的层次结构。大多数的细节对我们的讨论来说无关紧要，但是理解一下过程依然是对我们有帮助的：即UI元素是如何通过负责的测量和布局过程来将他们部署在一个矩形区域里面的。所有可见的view对象都呈现给了Surfaceflinger—当app由后台转到前台后，通过WindowManager创建了Surface。Layout和渲染都是在app的UI线程里面执行的。

Regardless of how many Layouts and Views you have, everything gets rendered into a single buffer. This is true whether or not the Views are hardware-accelerated.

根据你有多少layout和view，每个对象都在一个独立的buffer中渲染。无论是否使用硬件加速，都是如此。

A SurfaceView takes the same sorts of parameters as other views, so you can give it a position and size, and fit other elements around it. When it comes time to render, however, the contents are completely transparent. The View part of a SurfaceView is just a see-through placeholder.

一个SurfaceView有跟其他view一样的一些参数，所以你可以设置它的位置和大小等等。当它被渲染时，我们可以认为他的所有内容都是透明的。SurfaceView的视图部分只不过是一个透明的占位区域。

When the SurfaceView’s View component is about to become visible, the framework asks the WindowManager to ask SurfaceFlinger to create a new Surface. (This doesn’t happen synchronously, which is why you should provide a callback that notifies you when the Surface creation finishes.) By default, the new Surface is placed behind the app UI Surface, but the default “Z-ordering” can be overridden to put the Surface on top.

当SurfaceView的组件即将变为可见时，Framework层要求WindowManager请求Surfaceflinger创建一个新的Surface（这个过程是异步发生的，这就是为什么你应该提供一个回调函数，这样当Surface创建完成时你才能得到通知）。缺省情况下，新创建的Surface在app UI Surface的下面，但是Z轴顺序可能将这个Surface放在上面。

Whatever you render onto this Surface will be composited by SurfaceFlinger, not by the app. This is the real power of SurfaceView: the Surface you get can be rendered by a separate thread or a separate process, isolated from any rendering performed by the app UI, and the buffers go directly to SurfaceFlinger. You can’t totally ignore the UI thread — you still have to coordinate with the Activity lifecycle, and you may need to adjust something if the size or position of the View changes — but you have a whole Surface all to yourself, and blending with the app UI and other layers is handled by the Hardware Composer.

渲染在这个Surface（SurfaceView的surface）上的内容将由Surfaceflinger来混合，而不是由app。这才是SurfaceView的真正作用：这个surface可以被一个独立的线程或者进程来渲染，和app UI上其他的渲染工作分开，这些缓冲区数据将直接传递给Surfaceflinger。当然你不能完全忽略UI线程—-你依然要和activity的生命周期保持一致，并且一旦view的大小或者位置发生了改变，你可能也需要做些调整—但是你拥有了一个完整的Surface，并且这个Surface和app UI以及其他layer的混合工作将由HWC来完成。

It’s worth taking a moment to note that this new Surface is the producer side of a BufferQueue whose consumer is a SurfaceFlinger layer. You can update the Surface with any mechanism that can feed a BufferQueue. You can: use the Surface-supplied Canvas functions, attach an EGLSurface and draw on it with GLES, and configure a MediaCodec video decoder to write to it.

值得一提的是，新创建的surface实际上是生产者端，而消费者端则是一个Surfaceflinger的layer。你可以通过任何可以填充BufferQueue的途径来更新这个surface。你可以：使用Surface提供的Canvas相关的函数，附加一个EGLSurface然后使用GLES在上面绘制，配置一个MediaCodec 视频解码器直接在上面写数据。

Composition and the Hardware Scaler Now that we have a bit more context, it’s useful to go back and look at a couple of fields from dumpsys SurfaceFlinger that we skipped over earlier on. Back in the Hardware Composer discussion, we looked at some output like this:

我们现在有了更多的上下文知识，所以让我们回去看看前面讲到dumpsys SurfaceFlinger时我们忽略的几个字段。我们来看看如下的几个输出：

    type    |          source crop              |           frame           name------------+-----------------------------------+--------------------------------        HWC | [    0.0,    0.0,  320.0,  240.0] | [   48,  411, 1032, 1149] SurfaceView        HWC | [    0.0,   75.0, 1080.0, 1776.0] | [    0,   75, 1080, 1776] com.android.grafika/com.android.grafika.PlayMovieSurfaceActivity        HWC | [    0.0,    0.0, 1080.0,   75.0] | [    0,    0, 1080,   75] StatusBar        HWC | [    0.0,    0.0, 1080.0,  144.0] | [    0, 1776, 1080, 1920] NavigationBar  FB TARGET | [    0.0,    0.0, 1080.0, 1920.0] | [    0,    0, 1080, 1920] HWC_FRAMEBUFFER_TARGET

This was taken while playing a movie in Grafika’s “Play video (SurfaceView)” activity, on a Nexus 5 in portrait orientation. Note that the list is ordered from back to front: the SurfaceView’s Surface is in the back, the app UI layer sits on top of that, followed by the status and navigation bars that are above everything else. The video is QVGA (320x240).

这个抓取自Nexus 5竖屏模式，当在Grafika里面播放视频的时候。注意这列表是按照从后到前的顺序排列的：SurfaceView的Surface在最后面，app ui layer在上面，然后是状态栏和导航栏。视频是QVGA的。

The “source crop” indicates the portion of the Surface’s buffer that SurfaceFlinger is going to display. The app UI was given a Surface equal to the full size of the display (1080x1920), but there’s no point rendering and compositing pixels that will be obscured by the status and navigation bars, so the source is cropped to a rectangle that starts 75 pixels from the top, and ends 144 pixels from the bottom. The status and navigation bars have smaller Surfaces, and the source crop describes a rectangle that begins at the the top left (0,0) and spans their content.

“source crop”指示了Surface的buffer中要被SurfaceFlinger显示的部分。App UI的surface大小是整个显示的大小（1080*1920），但是由于需要显示状态栏和导航栏，因此从上面裁剪了75个像素，从下面裁剪了144个像素。

The “frame” is the rectangle where the pixels end up on the display. For the app UI layer, the frame matches the source crop, because we’re copying (or overlaying) a portion of a display-sized layer to the same location in another display-sized layer. For the status and navigation bars, the size of the frame rectangle is the same, but the position is adjusted so that the navigation bar appears at the bottom of the screen.

Frame一栏是指最终显示在屏幕上的位置。APP UI因为是完全相同位置的拷贝，因此这个值和前一列完全相同。而对于状态栏和导航栏，大小和前面一列是相似的，但是位置已经发生了改变。

Now consider the layer labeled “SurfaceView”, which holds our video content. The source crop matches the video size, which SurfaceFlinger knows because the MediaCodec decoder (the buffer producer) is dequeuing buffers that size. The frame rectangle has a completely different size — 984x738.

现在我们来看下SurfaceView这个layer，这个layer里面是视频的内容。source crop一列和视频的大小一致，SurfaceFlinger之所以知道这个信息是因为MediaCodec解码器申请的出队的buffer的大小就是这么大。而Frame则有一个完全不同的大小：984*738.

SurfaceFlinger handles size differences by scaling the buffer contents to fill the frame rectangle, upscaling or downscaling as needed. This particular size was chosen because it has the same aspect ratio as the video (4:3), and is as wide as possible given the constraints of the View layout (which includes some padding at the edges of the screen for aesthetic reasons).

SurfaceFlinger处理了这种大小的不同，通过缩放缓冲区数据的方式来填充到frame中，根据需要放大或者缩小。之所以选择这个特殊的大小，是因为这个大小和视频有相同的高宽比，并且在View Layout允许的宽度下尽可能的大（基于美观的考虑，在屏幕边缘也放置了一些填充物。）。

If you started playing a different video on the same Surface, the underlying BufferQueue would reallocate buffers to the new size automatically, and SurfaceFlinger would adjust the source crop. If the aspect ratio of the new video is different, the app would need to force a re-layout of the View to match it, which causes the WindowManager to tell SurfaceFlinger to update the frame rectangle.

如果我们在同一个surface上面播放了一个不同大小的视频，那么底层的BufferQueue会重新使用新的大小重新分配缓冲区，SurfaceFlinger也会调整它的source crop。如果新的视频的高宽比也发生了变化，app需要强制要求View来re-layout来匹配当前大小，这将导致WindowManager通知SurfaceFlinger来更新每个frame矩形的大小。

If you’re rendering on the Surface through some other means, perhaps GLES, you can set the Surface size using the SurfaceHolder#setFixedSize() call. You could, for example, configure a game to always render at 1280x720, which would significantly reduce the number of pixels that must be touched to fill the screen on a 2560x1440 tablet or 4K television. The display processor handles the scaling. If you don’t want to letter- or pillar-box your game, you could adjust the game’s aspect ratio by setting the size so that the narrow dimension is 720 pixels, but the long dimension is set to maintain the aspect ratio of the physical display (e.g. 1152x720 to match a 2560x1600 display). You can see an example of this approach in Grafika’s “Hardware scaler exerciser” activity.

如果你使用这个Surface来做其他用途，比如GLES，那么你可以通过调用SurfaceHolder#setFixedSize() 函数来设置Surface的大小。比如你可以设置一个游戏的大小为1280*720，这样当你去运行在一个大小为2K或者4K的屏幕上时，你可以显著的减少需要填充的像素的数目。显示处理器会来处理缩放。。。。

zangcf

The GLSurfaceView class provides some helper classes that help manage EGL contexts, inter-thread communication, and interaction with the Activity lifecycle. That’s it. You do not need to use a GLSurfaceView to use GLES.

GLSurfaceView类提供一个辅助类，这些类可以帮助我们管理EGL的上下文，线程通信以及与activity生命周期的交互。当然，你不需要GLSurfaceView就可以使用GLES。

For example, GLSurfaceView creates a thread for rendering and configures an EGL context there. The state is cleaned up automatically when the activity pauses. Most apps won’t need to know anything about EGL to use GLES with GLSurfaceView.

举例来说，GLSurfaceView创建了一个用来渲染和管理EGL上下文的线程。当activity pause时，它会自动清空所有的状态。大多数应用通过GLSurfaceView来使用GLES时，不需要了解任何和EGL有关的事情。

In most cases, GLSurfaceView is very helpful and can make working with GLES easier. In some situations, it can get in the way. Use it if it helps, don’t if it doesn’t.

在大多数情况下，GLSurfaceView对处理GLES来说很有帮助。但是在一些情况下，它可能是一种阻碍。仅仅在你需要的时候使用它。