Frame Synchronization

The final scene visible to the user is a combination of elements drawn by the application, the window manager, and the compositing manager. For optimum appearance and performance, the drawing done by the different processes should be properly synchronized. Examples: the window manager frame should only be drawn at a new size after the application contents are also resized to the new size. If the application is making several related changes to its window, such as scrolling and redrawing, the compositor should only redraw the window when all updates are done. The application should not be generating many frames of content when only one of them is drawn to the output device.

Limited synchronization of window manager and client drawing during resizing is possible without a compositing manager, and described below under Basic Synchronization, but the majority of protocols in this section only make sense when there is a compositing manager. These protocols are additionally designed with the assumption that the window manager and the compositing manager are the same process, and are not applicable to the case of a stand-alone compositing manager.

Both forms of synchronization identify "frames" of drawing using XSync extension [XSync] counters. The XSync extension allows creating Counter objects that hold a 64-bit value. Applications can select to get events when the counter object changes or reaches a particular value.

In the synchronization protocols, drawing is timed with reference to the output device's refresh cycle. An output device, such as a monitor, will typically read data from the frame buffer sequentially starting at the top of the displayed area, pause for a period known as the vertical blanking period (or vertical blanking interval) and then repeat. The vertical blanking period provides an opportunity to atomically change the screen contents without the risk of tearing, and is when a compositor using double-buffered drawing will swap buffers. The entire length of the process is called the refresh interval. For example, a monitor updating at 60 frames per second has a refresh interval of 1/60th of a second.

A compositor may be managing multiple output devices with different refresh cycles, and windows may overlap multiple devices. This means when an application draws a frame, the point in time where that frame of drawing actually is displayed to the user may be different on different output devices. For each application frame, the compositor MUST identify a single output device that the frame is primarily displayed upon, and report timing information for the display of the frame with respect to that device. For frame updates that don't include the entire window, the chosen device MAY depend on the particular updated area, and the chosen device may change from frame to frame for this or other reasons.

Basic Synchronization

The goal of basic synchronization is limited to coordinating redraws during interactive resizing. A client indicates that it is willing to participate in basic synchronization by listing _NET_WM_SYNC_REQUEST in the WM_PROTOCOLS property of the client window and storing the XID of a XSync counter in the property _NET_WM_SYNC_REQUEST_COUNTER. This counter is known as the basic frame counter.

Before resizing a window, the window manager sends a _NET_WM_SYNC_REQUEST message to the client window containing a value that the application stores in the basic frame counter when it is done handling the ConfigureNotify event resulting from the resize. This allows the window manager to know that it can move on to the next step of resizing without getting ahead of the client. If the window manager is a compositing manager it may also choose to freeze redrawing of the window until it sees the change in counter value, so that the resized window frame and window are drawn as a single unit.

Extended Synchronization

Extended synchronization provides a more general framework for redraw coordination, including spontaneous application updates as well as resizing. A client indicates its willingness to participate in the extended form of frame synchronization by listing _NET_WM_SYNC_REQUEST in WM_PROTOCOLS and including two XSync counters in the property _NET_WM_SYNC_REQUEST_COUNTER, the basic frame counter and an extended frame counter. A window manager indicates that it will participate by listing _NET_WM_FRAME_DRAWN in _NET_SUPPPORTED.

In extended synchronization, the basic frame counter is unused. The extended frame counter is updated in response to _NET_WM_SYNC_REQUEST messages, but can can additionally be updated spontaneously by the client to indicate the beginning or end of a frame of drawing. The beginning of a frame of drawing is indicated by an increment to an odd number, and the end of a frame of drawing is indicated by an increment to an even value. For each frame marked this way, a client will receive _NET_WM_FRAME_DRAWN and _NET_WM_FRAME_TIMINGS messages from the compositor.

Rationale: using the same counter for basic and extended synchronization would cause problems when switching from a window manager that supports only the basic synchronization to one supporting extended synchronization - since in one case the window manager determines the counter values and in the other case the client determines the counter values, there would be race conditions during the switch. Packing both counters into a single property increases efficiency when initially managing a window.

Compositor frame timing algorithm

When an application is displaying changing content, such as animated transitions, videos, or user interface elements that respond to the user's mouse input, there are three primary measures of quality: Frame rate - the number of frames that are drawn per second. Latency - the time between when a frame is generated and when it is displayed to the user. Moderate latency will cause dragged objects to lag behind the cursor, higher latency will cause apparent discrepencies to the user between when an action is taken and when it is displayed. In some cases, such as the synchronization of audio and video, a known latency can be compensated for. Jitter - the difference in latency across frames. Jitter causes the appearance of stuttering, uneven motion.

The choice of algorithm for scheduling redraws in the compositor will involve tradeoffs between these quality measures. In particular there is a tradeoff between latency and jitter. If a compositor draws as soon as it receives a new frame from any application, this will result in minimum latency with large amounts of jitter - a frame may be processed immediately and drawn with minimal delay, but if the compositor is busy processing a redraw triggered by another client, there will be additional latency. In order to present consistent latency to applications, an alternate approach is recommended: after receiving a new frame, the compositor waits for a fixed point in the refresh cycle before starting the redraw. By making sure that each frame of drawing is finished with some time to spare before this point, an application achieves a latency that is both known and consistent.

The approach of waiting to a fixed point in the refresh cycle can backfire if an application is unable to achieve its target frame rate: the delay before the compositor draws will waste time that could have been spent drawing and cause unnecessary synchronization to a sub-multiple of the system's refresh rate. For this reason, in the protocols described below, frames can be marked as urgent. In response to an urgent frame, a compositor SHOULD draw as quickly as possible and send a _NET_WM_FRAME_DRAWN message so the application can draw the next frame. One approach to deciding which frames to mark urgent is for the application to note whether it sleeps after receiving _NET_WM_FRAME_DRAWN before starting the next frame. If not, the next frame is marked urgent.

The following algorithm is recommended for compositors. It is however, not required.

  • When the compositor receives a XDamageNotify event:

    • If it is part of an urgent frame, schedule an immediate redraw

    • Otherwise, schedule a redraw for the next redraw point. Redraw points occur frame_delay milliseconds after the start of the vertical blanking period.

  • If a redraw is scheduled for time T, and and the swap has not yet completed at time T, redraw immediately when the swap completes

The choice of frame_delay is up to the compositor. The minimum event-handling latency that an application can achieve is application_frame_draw_time + refresh_interval - frame_delay. A larger value will reduce latency slightly, but increases the chance that the compositor won't be done drawing the frame by the vertical blanking period, and it will be displayed a refresh cycle late. Using 0 for frame_delay for delay is not recommended, since that will produce pessimized latency for legacy GL clients that swap buffers during the vertical blanking period. Instead a small value such as 2 milliseconds is recommended.

High precision timestamps

When exchanging timing information about drawing, it is useful for clients to be able to have a higher-precision than the millisecond precision of X server timestamps. To allow for greater precision, intervals and timestamps for protocols in this section are represented in microseconds. For timestamps, the timestamp has the form (x_server_timestamp) * 1000 + microsecond_value.

The window manager MAY produce these high precision timestamps by periodically determining an offset between the server time and a local system time. The server time can be determined by the standard method of changing a dummy property and observing the timestamp in the PropertyNotify event. For the system time it is preferred to use a monotonic time, such as the time given by clock_gettime(CLOCK_MONOTONIC, ...), rather than a real time, such as that given by gettimeofday(). If this method of tracking an offset is used, the accuracy of passing a time between two clients will be limited by the precision of the X server time, and the round-trip time for obtaining the X server time.

Alternatively, a client MAY observe that X server times correspond to a specific method of obtaining the local time, such as using gettimeofday() or clock_gettime(). In this case, high precision times can be generated directly, and two clients such will be able to exchange timestamps with sub-millisecond accuracy. (This technique is only applicable to clients running on the same machine as the X server.)

_NET_WM_SYNC_REQUEST_COUNTER

_NET_WM_SYNC_REQUEST_COUNTER, COUNTER[]/32

The _NET_WM_SYNC_REQUEST_COUNTER property is set by the application on a toplevel window. It contains either one or two XSync counter IDs, depending on whether the client supports only basic synchronization or both basic and extended synchronization.

If the client supports only basic synchronization, then it puts only the basic frame counter ID in the _NET_WM_SYNC_REQUEST_COUNTER. The initial value of basic frame counter is not defined by the specification. The window manager MAY set the value of the basic frame counter at any time, and, if it chooses t use basic synchronization, MUST do so when it first manages a new window.

If the client supports extended synchronization, then the client creates both a basic frame counter and an extended frame counter, and puts both IDs in _NET_WM_SYNC_REQUEST_COUNTER, with the first element of the property being the basic frame counter ID and the second element the extended frame counter ID. The client sets the extended frame counter to an initial value before before changing the window from the withdrawn state, and the client MAY update the value at any time.

To indicate the beginning of a frame, the client increases the value of the extended frame counter to an odd value. At this point, the window is an a "frozen" state and the compositing manager SHOULD NOT redraw the window in response to XDamageNotify events. If the compositing manager needs to redraw for other reasons, such as a change to a different window that overlaps the frozen window, it MAY redraw the window. To indicate the end of the frame the client increases the value of the extended frame counter to an even value. At this point the window is no longer frozen, and the compositor SHOULD redraw the window and send _NET_WM_FRAME_DRAWN and _NET_WM_FRAME_TIMINGS messages. The window manager SHOULD redraw the window and send the messages even if no damage events were received.

Rational: the reason for redrawing the window even if no damage events have been received is so that there is consistent timing for each frame, even if one frame happens to involve no changes to the window. It is recommended to behave as if a damage event was received containing only a single pixel.

To distinguish urgent from non-urgent frames, two different patterns of increases are used. (See the section called “Compositor frame timing algorithm” for the definition of urgent frames.) To mark an urgent frame the odd value is chosen to have a value such that v % 4 = 3, then the counter is increased by 1 to give the even value. To mark a non-urgent frame the odd value is chosen to have a value such that v % 4 = 1, then the counter is increased by 3 to give the even value.

_NET_WM_SYNC_FENCES

On some systems with loose synchronization between different clients using the graphics system, drawing immediately after XDamageNotify events are received does not work properly, and a GL-based compositing manager must also insert GL-level synchronization to ensure correct drawing. Because there is no way of knowing whether that is the case on any particular system, _NET_WM_SYNC_FENCES SHOULD always be exported an applications and compositors MUST synchronize drawing either using _NET_WM_SYNC_FENCES or the alternate method described below.

The _NET_WM_SYNC_FENCES property is set on a toplevel by an application and contains a list of XSync fence objects. Before ending a frame by setting the XSync counter to an even value N, the application uses XSyncTriggerFence() to trigger the fence object stored in the property at index (N / 4) % L, where L is the number of counters in property. If an application waits for the _NET_WM_SYNC_DRAWN message before starting a new frame, 2 is a sufficiently large value for L.

When the window is mapped, the GL-based compositor SHOULD import the fences in _NET_WM_SYNC_FENCES as GL sync objects using the ImportSyncEXT() procedure from [EXT_x11_sync_object]. After receiving a frame update ending with counter value N, the next time that the window is redrawn, the compositing manager calls glWaitSync() on the fence at index (N / 4) % L.

Alternate method

If an application window doesn't export _NET_WM_SYNC_FENCES, or draws outside a frame update, the compositing manager still MUST ensure correct synchronization. If the compositing manager has received XDamageNotify events that are not synchronized by _NET_WM_SYNC_FENCES, then before drawing the screen, the compositing manager MUST use XSyncTriggerFence() to trigger an X fence it has created itself, and then wait for that fence using glWaitSync().

_NET_WM_SYNC_REQUEST

A window manager uses this protocol by preceding a ConfigureNotify event sent to a client by a client message as follows:

type = ClientMessage
window = the respective client window
message_type = WM_PROTOCOLS
format = 32
data.l[0] = _NET_WM_SYNC_REQUEST
data.l[1] = timestamp
data.l[2] = low 32 bits of a frame counter value
data.l[3] = high 32 bits of a frame counter value
data.l[4] = 1 if the the extended frame counter should be updated, otherwise 0

If data.l[4] is 0, then after receiving one or more such message/ConfigureNotify pairs, and having handled all repainting associated with the ConfigureNotify events, the client MUST set the basic frame counter to the frame counter value from the last client message received.

The frame counter value in the client message is determined by the window manager subject to the restriction that it MUST NOT be 0. The number is generally intended to be incremented by one for each message sent.

If data.l[4] is 1, then after receiving one or more such message/ConfigureNotify pairs, and having handled all repainting associated with the ConfigureNotify events, the client MUST set the extended frame counter to a value greater than the frame counter value in the last client message received. This will normally be done as part of indicating the end of a frame of drawing. The exact value used is chosen so that the fences listed in _NET_WM_SYNC_FENCES are used in rotation.

The frame counter value in the client message is determined by the window manager based on the most recent value it has seen for the extended frame counter. The window manager SHOULD retrieve this value when managing the window, and monitor subsequent changes to the value by creating an XSync Alarm object. The update request number in the client message SHOULD be chosen as last_seen_value + 240.

Rationale: if the client is continually redrawing, then the last seen value may be out of date when the window manager sends the message. Picking a number that is 240 later would allow for 1 second of frames at 60fps. In the normal case client will draw only one more frame before waiting for _NET_WM_FRAME_DRAWN.

By using either the Alarm or the Await mechanisms of the XSync extension, the window manager can know when the client has finished handling the ConfigureNotify events. The window manager SHOULD NOT resize the window faster than the client can keep up.

_NET_WM_FRAME_DRAWN

This client message allows a client to know when an update it has created has been drawn to the screen by the compositing window manager. If the compositing manager lists this property in _NET_SUPPORTED, then the compositing manager MUST send a client that supports extended synchronization a _NET_WM_FRAME_DRAWN client message immediately after finishing the redrawing that results from an application frame. If an application draws several frames without waiting for _NET_WM_FRAME_DRAWN, the compositing manager MAY send only one _NET_WM_FRAME_DRAWN for the last frame. The contents of the client message are as follows:

type = ClientMessage
window = the respective client window
message_type = _NET_WM_FRAME_DRAWN
format = 32
data.l[0] = low 32 bits of the extended frame counter value
data.l[1] = high 32 bits of the extended frame counter value
data.l[2] = low 32 bits of high precision timestamp
data.l[3] = high 32 bits of high precision timestamp
data.l[4] = 0

The timestamp indicates that time at which the compositing manager finished submitting drawing for entire scene to the graphics system. (If that time is not available, the compositing manager MAY approximate it by the time at which the message is sent.) The actual completion of drawing will occur at some time later than the indicated time.

In addition to sending one message at the end of each frame, the compositing manager MUST send one _NET_WM_FRAME_DRAWN message for each newly mapped window that supports extended synchronization. If the initial value of the frame counter is odd, then the window starts in the frozen state, and the message is sent as per normal after the frame ends. If the initial frame value is even, the message is sent after the window is first drawn.

_NET_WM_FRAME_TIMINGS

This message provides information about the timing of a previous frame; it is sent subsequent to the _NET_WM_FRAME_DRAWN message for the frame once the window manager has obtained all available timing information.

type = ClientMessage
window = the respective client window
message_type = _NET_WM_FRAME_TIMINGS
format = 32
data.l[0] = low 32 bits of the extended frame counter value
data.l[1] = high 32 bits of the extended frame counter value
data.l[2] = presentation time offset
data.l[3] = refresh interval
data.l[4] = frame_delay

The presentation time offset is a 32-bit signed offset in microseconds from the timestamp in the _NET_WM_FRAME_DRAWN message. It may be 0 if no presentation time is available. The presentation time indicates the time of the end of the vertical blanking period after which the frame contents will be scanned out to the output device. (If this exact time is not available, the composite manager MAY provide a nearby time such as the time at which buffers are swapped.)

The refresh interval a 32-bit unsigned number indicating the refresh interval in microseconds. It may be 0 if the refresh interval is not available.

The frame delay is a 32-bit unsigned number indicating the point in the refresh interval where the compositor will start drawing if it previously received damage, as a number of microseconds after the end of the vertical blanking period. See the section called “Compositor frame timing algorithm” for details. The flag value 0x80000000 indicates that the compositor uses a different frame timing algorithm and no meaningful value is available. All other values with the high-bit set are reserved.