Part 2 -

How X Works, in Detail, and How the User Sees It

So far we have seen an outline of the architecture of X, and the benefits that X can provide for your organization.

In this second Part of the book we look at the internal mechanisms of the system in some detail - how to use the system is covered later in Part 3. You need to understand some of the mechanisms so you can fully appreciate how the system is used, but if you are very keen to look at how the system is used, then jump ahead now, and come back to Part 2 later.

In Part 2 we are building on the quick overview of the system presented in Part I, where you saw that X consists of servers, clients, and communications between them. We are now going to look at the system components in much more depth and will cover:


Chapter 3-

The Server - the Display Control Software

Introduction

This chapter looks in detail at the facilities the server provides, how it is implemented, what it contains, and also what it does not contain but leaves to the clients to handle instead. Most of this chapter deals with output - drawing text and graphics. Input is covered in the next chapter; as we shall see, most of the input processing is handled not by the server itself but by the clients.

3.1 The role of the server

The X server receives requests from client applications and actions them, usually producing output on the display screen. It controls the display's input devices, and sends user input to client applications. It also manages internal resources such as fonts and cursors, manages colour tables, etc. The design goals of X have meant that the server contains as little as possible.

The server is the base system; everything else is in the client and can be relatively easily changed, but whatever is in the server is fixed. (Remember, the server is very analogous to a terminal: you can't change the server, just as you can't change the internal operation of a VT220 terminal after it has been manufactured.)

The main functions of the server (Figure A) are:

Of course, many of these functions require the server to handle the hardware it is running on, or at least interact with the operating system on the server's machine.

Server design considerations

As little as possible of the overall functionality of the system is contained in the server. The advantages of this approach are:

The next module describes the functions you might expect the server to handle but which are in faet left to the client for the reasons above.

Figure A. The functions of the server.

3.1.1 What the server does not include

Many functions such as window management, the user interface, and interpreting keyboard input are not performed by the server but are left to the client. This makes it possible to implement the server on a wide range of machines, and makes the system much more flexible.

In X many system functions are left to the X client program, which in other window systems would usually be handled by the base window system:

In the next module we go back and concentrate on the server and how it is imple- mented.

Figure A. The client redraws freshly-exposed areas of windows as they are uncovered.

Figure B. The window manager client controls size and position of application windows in response to user actions.

3.2 Implementations of the X server

There are many different implementations of the X server for platforms ranging from standard multitasking workstations to simple PCs. Servers can contain extra system administration and usability features to make them easier to use and manage, as well as `extensions' which provide graphics and other capabilities not offered by the basic X system.

The MIT X Consortium produces sample implementations of server and clients. Sample servers are provided for a wide range of systems. (By `sample' we mean they are not `reference implementations' - just because an MIT X program does something, it doesn't mean it is correct. The official standard is the document from MIT that defines the `X Protocol'; if a program disagrees with the Protocol, the program is wrong.)

All the machines supported by the MIT sample implementation are UNIX workstations, but that is only because MIT does not have the resources to port to non-UNIX systems. Hardware manufacturers and third-party vendors have developed servers for other workstations and operating systems, and other systems including X-terminals, DOS PCs and other personal computers such as Atari and Amiga. There are even plugin cards containing server code, graphics controller and display memory, for a variety of systems.

Conceptually an X server is simple, and a schematic is shown in Figure A for the simplest case - a server running on a workstation. However, many implementations are more complex because of special requirements imposed by the target machine, or because of extra facilities added to the basic system. For example, Figune B shows schematically that an X-terminal is complicated by having to provide many of the basic facilities such as network handling which would be provided by the operating system in a workstation or PC.

Extra features in servers

While the X standards do not require it, almost all servers provide some convenient way of st.arting the first client application, so that you can initiate your X session. If the server didn't have this facility, you would have to st.art the application by entering commands on the keyboard of the remote host on which you'll be executing the application. So, effectively these startup mechanisms save you having to walk around your office to the remote machine. (System startup is described in Module 11.1.)

Another common extra is the facility of a virtual screen larger than your physical screen, which is particularly useful on PC X servers with low-resolution screens with relatively few pixels, for example, 600 x 480. The virtual screen may be much larger, say 1600 x 1000 pixels, and your physical window acts as a viewing port onto part of this larger area. This lets you start all the applications you need, but keep some of them off-screen out of the way, so that your limited screen space is kept uncluttered for the applications you are currentIy using. Some window managers provide a similar facility, illustrated in Module 8.1.1. (This facility is also sometimes called a virtual desktop or a virtual root window.)

Extensions to servers

When X was initially developed, its designers realized that certain functionality would have to be added to the system later, either because the technology had not yet matured enough, or because the functionality was not crucially important, or just because as any system develops people want it to do more and more. To handle add-ons like this cleanly, X has a well-defined mechanism for adding extended functionality to the server.

Specialised servers with extensions for live video windows, or PostScript or other special facilities are available. The ext.ension mechanism is also used to incorporate new functionality into new releases of standard servers, for example the support for non-rectangular windows which was added in Release 4. By using extensions rather than adding in new functions directly, compatibility is maintained with earlier releases and consistent operation on servers from different vendors is possible. (See Module 3.8 for more detail.)

Now we look at X-terminals in detail, as they illustrate some important features of the X system as a whole.

Figure A. Schematic structure of an X server

Figure B. An X-terminal has to provide many non-X functions as well as the server code.

3.2.1 Specific server implementations - X-terminals

Special-purpose X-terminals are becoming increasingly popular. An X-terminal is just a package consisting of an X server, the CPU and memory to run it on, mouse, keyboard, and network interface.

X-terminals illustrate some important features of X. In fact, a good way to understand X is to think of the server very much as a terminal. Most of the points mentioned below are also true for many of the X servers that run on PCs: these servers effectively transform the PC into an X-terminal. First we look at how X-terminals resemble ordinary terminals, and then at the important differences.

Similarity of X- and character-based terminals

Figure A illustrates many of the ways in which an X-terminal is like a character-based terminal:

Differences between X- and character terminals

Of course, X-terminals have many advantages over ordinary terminals:

One disadvantage of X-terminals is that they have a dynamic requirement for memory, which is not easily predictable: the more applications you run, and the more fonts and other resources you use, the more memory you need. Well-engineered X-terminals warn you if you are running out of memory, so you can avoid any serious effects. And, some recent X-terminals have virtual memory systems, paging memory over the network using NFS (the network file system), giving you practically limitless memory.

In the next module we look at how many X-terminals provide a lot of extra useful features.

Figure A. Similarity of X-and character-based terminals.

3.2.2 Extra features for X-terminals

Many X-terminals provide a lot of extra functionality, both to make the terminals easier to use and administer, and to improve their performance.

Many innovations in X have been led by X-terminal manufacturers, not least because to survive commercially they must make their product.s as attractive as possible, and by adding features they have been able to differentiate X-terminals from competitive workstations, for example.

X-terminals provide many facilities that make it easy for the system manager to handle large numbers of X-terminals on a network:

Performance extras

Increasingly, X-terminals and PC X servers are allowing local clients. For terminals this is almost a contradiction in terms (if applications run inside it, it's not a terminal) but the clients provided are very restricted. The two most common options are a local window manager and a local telnet or local terminal emulator. These facilities simplify setup and reduce the amount of network traffic, which is especially useful when working over very slow network links such as serial lines.

The local window manager comes into its own where there is a high network latency, that is, the round-trip time from server to client and back again is long, which is often the case with satellite links or long-haul networks. If the window manager is tracking the mouse interactively, as when dragging a rubber-band outline, you get very bad response with high-latency connections; running a local window manager gets over this problem. However, for such networks you would probably be better off using a workstation and running as few applications as possible remotely. (For more about latency, see Modules 4.5 and 13.2.) The terminal manufacturers are also working (in conjunction with the MIT X Consortium) on standards to improve the performance of X over very slow networks (Module 13.2.1). As part of this, vendors are looking at running more applications locally in the terminal, to avoid the bottleneck of the slow network.

As these features develop and become more widespread, they are often adopted by other server vendors as well. This is especially true for PC X servers, because with these a PC is very little different from a true X-terminal.[Footnote: An interesting market prediction is that sales for X servers for PCs will overtake sales of purpose-built X-terminals in the next few years.] In fact, most extras that are generally applicable are implemented on almost every commercial server in time, and many have been adopted as formal standards by the MIT X Consortium, or are included in their release so they become de facto standards.

3.3 How the server handles output to the screen

X outputs graphics and text to windows on the screen. Windows are organized hierarchically in a tree structure.

Windows are the building blocks of X, and all output (drawing text and graphies) is performed in windows. In X, windows are cheap: they are not limited resources, so you can easily have hundreds of windows within an application. (Contrast this with other systems where each window in an application is equivalent to an open file: then you can only have a few tens of windows.)

Windows are arranged hierarchically in a tree structure, as shown in Figure A. The top of this tree is the root window, which covers the whole screen. Each application has its own window which is a child of the root window, and these are called top-level windows as they are at the top of the hierarchy. Application windows appear on top of the root window, and obscure the portions of the root window which are underneath. Within each application window there can be many sub-windows, each of which in turn can have its own sub-windows, and so on. Sub-windows are used to create separate objects or areas within the application. For example, a menu can be created out of one window for the menu background, with several child windows, one for each button on the menu, as shown in Figure B and Figure C. Or, in a simulation package you might use a separate sub-window for each component (valve, gauge, pipe, reactor vessel, etc.) in the model you are building.

You can change the size of windows, and their position relative to their parent. You can explicitly make a window invisible (remove it from the screen) by unmapping it, and bring it back later by re-mapping.

Windows are drawn on the display screen; X allows each display to have up to three screens (assuming the hardware can support this) as shown in Figure D. You still have only one keyboard and mouse; the three screens are controlled by the same server and it is the server, not the screen, that receives input. Multiple screens are very useful where large amounts of information must be displayed simultaneously, such as in mapping or geographical information systems, or in financial dealing rooms. They are also useful when debugging graphical systems; for instance, if you are developing your CAD package on one screen, it is helpful to be able to debug it from a different screen so that the debug windows themselves don't interfere with the application. Some X systems let you have multiple virtual screens: you switch between them by moving the mouse off the side of the screen, causing another virtual screen to be displayed on the physical monitor. (Note that the term `virtual screen' here has a different meaning than in Module 3.2, where it meant a large virtual root window.)

In the next module we look in more detail at how the visibility of windows is controlled, and in the module after next at how the application requests output to the screen.

Figure A. X windows are arranged in a tree hierarchy

Figure B. A sample menu, using sub-windows

Figure C. The window tree for the sample menu.

Figure D. X allows three screens on a display.

3.3.1 Visibility of windows, and clipping

The hierarchy of windows controls the visibility of windows. Child windows can be bigger than their parent, but those parts of them that are outside the parent's boundary are invisible - they are `clipped'.

A child window is clipped by its parent: only the part of the child that is within the parent window is visible (Figure A). A child window can be bigger than its parent, but clipping implies that part of the child will be invisible. You can draw into windows even when they are invisible, or partly so (but what you `drew' is effectively thrown away: if the invisible part is exposed again, the application will have to redraw that area).

This is a convenient feature in several ways. First, it lets the applications programmer create the window of a size best suited to the task at hand, to simplify the programming. For example, it is much easier to write `ABCD' into an oversized window and let the window system itself clip off all of the `D' and part of the `C' as shown in Figure B, rather than having to calculate the size the string would be in the window, omit the `D' and explicitly manipulate the image of a `C' character and then draw the result. Secondly, being able to draw into a window without having to worry if it fully visible or not is essential if applications are to work with different styles of window management. When you are writing your program you have no idea what constraints a window manager will impose - for example, whether windows will be overlapping on the screen, or `tiled' to lie side by side - so the more insulated you are from potential changes in visibility and size of your windows the better. Finally, the facilities of both types of virtual screens (Modules 3.2 and 3.3) rely on applications continuing to work when they are fully or partially off-screen (that is, outside their parent, the root window) and therefore fully or partially invisible.

Figure A. Child windows are clipped by their parent.

Figure B. Drawing into a clipped window

3.3.2 Graphics requests - requests for output

X outputs to the screen only in response to requests from clients. The requests specify what is to be drawn, or how windows are to altered. Graphics requests are usually object-based; for example, a line is specified by its start and end points - not as a set of pixels to be drawn. You can draw graphics to off-screen images in memory (`pixmaps') as well as to windows.

(If technical details aren't so relevant to you, you can skip over this module.)

All X output is the result of requests from clients. There are many different types of request, covering all the operations required for manipulation of screens, windows, text, graphics, and input control. Requests are to X what system calls are to an operating system - the fundamental operations on which everything else is based. This module describes the types of requests X provides, and the context in which they work, and we point you to other modules dealing with the more important points in greater detail.

Figure A. Detailed contents of a request to draw two line segments.

3.4 The server contains the only display hardware

dependence in the system

Clients cannot access the display hardware directly - they can only send requests to the server, which in turn controls the hardware. So clients are independent of the type of hardware of the server, and the server's operating system.

One of the design goals of X was to make clients hardware independent. This is achieved by not giving them direct access to the display hardware, but insisting that they can only send requests to the server, which itself controls the display.

So, only the server knows what hardware is being used. For instance, if the server is running on a DOS PC, is the display EGA, or VGA, or some special graphics board? If the server is on a workstation, does it have a simple frame-buffer, or is there sophisticated graphies aceelerator hardware as well? The server has to receive standard requests from clients and then translate them into the appropriate hardware commands (or deposit appropriate instructions in the hardware registers) to make the particular hardware do what was requested (Figure A).

The server has to know what hardware is present, and support it. As shown in Figure B, the server (or the operating system it is built on) contains device drivers for the display hardware, as well as the keyboard, mouse and any other input devices. Therefore, while the server has probably been designed to be as portable as possible with a modular internal structure, when it is finally implemented it is very hardware dependent, and of course will only run on the CPU-type it was compiled for.

Clients are mostly hardware independent

Clients are completely hardware independent in the sense that any client can operate with any server, no matter what the architecture or operating system the two of them have. But there are a few hardware features that clients have to take account of:

There are fairly easy solutions to these problems. By writing your application correctly you can make it very easy fur the user to customize (access to the source code isn't required) so that they can specify colours, sizes, layouts, etc. which are suited to their configuration, as described in Module 12.1. It is also possible to specify fonts in a resolution-independent way, and to query the server for details of the display so that layouts can take account of the available screen space, etc.

Mouse support - how many buttons?

Mice have different numbers of buttons: the Macintosh has one button, most PC mice have two, and most workstation mice have three. The server has no problem - it handles up to five buttons - but what if the application requires you to press the third button, and you have only one?

Server developers have tried to get over this by simulating the extra buttons: pressing, say, both buttons simultaneously on a two-button mouse simulates pressing a middle button. This is a nasty solution, as the user interface of the application is distorted, making it hard to use. As above, the correct approach is to make the application customizable, so the users can choose how the functions normally invoked with the missing mouse buttons are to be invoked on their systems. (But until all applications are written correctly, the server developers have to continue with their inelegant simulations.)

Figure A. The server translates X requests into commands to the hardware.

Figure B. The server, alone, knows hardware is present.

3.5 Handling text and fonts

X supports multiple fonts and character sets simultaneously on the screen. Text can only be horizontal; fonts are stored as bitmaps, text cannot be rotated, but from Release 5 onwards it can be scaled.

X supports multiple fonts, and allows sophisticated use of English and non-English character sets. Characters in a font may be of fixed width (as you need for terminal emulators, and some spreadsheet applications) or of variable width (which you need for proper typesetting and DTP).

Server vendors usually provide a large set of fonts for use with their server (but there are no standard fonts, so you can't assume a particular font exists). Fonts are normally provided ready for use with your server. They may also be distributed in a portable, plain-text, format called BDF (Bitmap Distribution Format) which you then have to convert into the format your server needs using a font-compiler provided with your server. In Release 5, the format is PCF (Portable Compiled Format): this stores the font information in an efficient internal form, but it is portable across all servers from Release 5 onwards. In Release 4 and earlier, the internal format was SNF (Server Natural Format); in practice this often was portable across platforms, but the specification did allow it to be platform-specific, and BDF was the only guaranteed portable form.

X has a standard naming scheme for fonts, called XLFD (X Logical Font Description). The XLFD name contains information about the type of font this is - the typeface, character size, whether bold or italic, etc. - so a lot of information about the font can be obtained just from the name, without having to load the whole font into the server first. We don't want to load unnecessary fonts, because they consume a lot of memory in the server. The server also accepts font specifications that contain `wildcards', so when you are naming a font, you only have to give specific characteristics you are interested in, instead of having to name a font exactly. This lets you specify font names more or less generically, which makes you less dependent on specific fonts being available on every server (see Module 11.3).

Characters in a font are encoded as one or two bytes, that is characters in some X fonts are represented by 8 bits, and in other fonts by 16 bits. An 8-bit font can contain 2ş = 256 characters, which is adequate for English and European languages, and many sets of symbols. A 16-bit font can contain 2'6 = 65536 characters, and is required to accommodate the very large oriental character sets. The eneoding of a character within a font just means its number or position in the font; for example the ASCll encoding of upper-case B is 66, that is, when a byte containing 66 is interpreted as an ASCII character, it is taken to mean B. Where possible X uses the encodings specified by ISO, the international standards body. In fact the final parts of the XLFD name specify the encoding. `ISO8859-1' specifies `ISO Latin alphabet number 1', which is more or less ASCII with extras covering some symbols and letters with diacritical marks (aceents, etc.) for European (`Latin') languages. Other ISO8859 encodings cover Cyrillic, Hebrew, Greek and Arabic languages, and there are also standard encodings for the oriental languages.

Up to and including Release 4, X fonts were stored exclusively as bitmaps on the server (as `files', usually in ROM on an X-terminal or on disk on a workstation), and in Release 5 most files are still in this form. There are several consequences of this:

With Release 5, a mechanism has been provided for scaling fonts, in three ways:

  1. A scaler for bitmap fonts has been included, which lets you use any bitmap font at any size. But it is a simple mechanism and the results can be slightly ugly unless you are increasing the size to an exaet multiple of the original (Figure A).
  2. The Bitstream Speedo system for scalable outline fonts has been incorporated. Instead of representing each character in the font as a bitmap, it is stored as a set of curves representing the shape of the character, which can be scaled to whatever size is necessary and then converted to a bitmap for drawing the character on the screen.
  3. The user-contributed software release contains a PostScript Type 1 font scaler, plus some Type 1 scalable outline fonts.

In the next two modules we look at the X font server which allows fonts to be managed centrally, and at the requests used to draw text.

Figure A. Scaling a bitmap font can produce ugly results

3.5.1 The X Font Server

Storing fonts on individual X servers makes system administration difficult, especially on a large network. This is overcome by storing the fonts on a `font server'; the X server retrieves fonts from the font server rather than reading them directly itself.

Storing fonts on the X server, especially in a server-specific format, causes many problems:

  1. Even when fonts are not loaded they take up a lot of storage space, on disk or in ROM. For example, on our MIT server, a typical bitmap font for Romanlanguage text takes between 10 and 20 kbytes on disk, an outline font takes about 65 kbytes, and a bitmap Chinese-language font takes between 0.5 and 1.0 Mbytes! So on a network of X stations, you may be using a vast amount of disk space, storing multiple copies of the same fonts for different servers.
  2. Server and application vendors supply different fonts; you may find that when you get a new application which requires a specific font, you have to install this font individually on all your X stations.
  3. Introducing new font formats becomes very difficult, not just because of the difficulty of upgrading all X servers everywhere, but also because the new font format may depend on a proprietary technology not available to all X server developers.
  4. Formats such as outline fonts consume a lot of CPU power and memory: this may make them unsuitable for low end servers such as PCs and small X-terminals.

To get over these problems, the X font server was developed. The basic idea is that the X server, instead of reading font files directly itself, requests them from the font server, which provides them in the bitmap format required by that X server.

Figure A illustrates three important points about the font server design. First, a single font server can provide fonts for many X servers. Secondly, an X server can be connected to more than one font server. And finally, font servers can be chained together: if the first font server has been asked for a font which it does not have, it can request this font from another font server it is connected to. Let's see how these features overcome the problems we listed above.

  1. Fonts no longer need to be duplicated across every X server - the font server can hold a single copy and make it accessible to all the X servers on the network.
  2. Adding a font to the network is easy - you only have to load it onto one font server and it is available to all X servers and therefore to all applications.
  3. A new font format can easily be accommodated - only the font server needs to be enhanced to handle it. If the technology for the new format is proprietary, the vendor can supply a complete font server for the new format; this font server can then be chained to existing font servers without any modification at all (Figure B).
  4. The font server can be run on a powerful compute server, which is able to handle the demands of complex font formats.

These factors are especially attractive for networks of X-terminals, because they reduce still further the amount of (X) terminal-specific information that has to be stored anywhere.

The font server also provides other facilities for administration. It provides licensing and access control, because many fonts cany a licence or usage fee. It supports grouping fonts into subsets called catalogues to help you control who can access which fonts. For example, you might group chargeable fonts into a catalogue and make it available only to specialized DTP users who really need those facilities. The sample implementation of the font server in Release 5 doesn't support these facilities.

Figure A. Relationship of X servers with font servers.

Figure B. Adding a new font format by adding a proprietary font server

3.5.2 Graphics requests for text

X provides requests for inquiring about which fonts are available, for loading specified fonts, for drawing text, and for calculating dimensions of text prior to drawing.

(If technical details aren 't so relevant to you, you can skip over this module. )

There are relatively few graphics requests for handling text and fonts, because X's text handling is straightforward. This module describes the requests and some more detailed aspects of X's text handling.

3.6 Handling colour

X provides support for a very wide range of colour capabilities, from simple mono bitmap frame buffers to elaborate 24-bit colour systems. Colours can be specified by name, in terms of RGB components or device-independent specifications, or as internal values within the server.

In keeping with its design goal of being suited to a wide range of display types, X works across the whole spectrum of hardware, from monochrome to 24-bit colour, as well as grayscale.

Internally - that is,within the server - X colour specifications consist of three 16-bit values, one each for the red, green and blue components (RGB) of the colour to

be displayed. These values are normally written in hexadecimal notation:

pure red ffff / 0000/ 0000 full red,no green or blue

pure green 0000 / ffff / 0000 full green,only

white ffff / ffff / ffff as much of everything as possible

black 0000 / 0000 / 0000 nothing at all

yellow ffff / ffff / 0000 mixture of red and green

Externally,for example when you specify in a text file a default colour for some item, you can use ordinaty colour names like `pink',`orange',etc. And most X programs let you specify foreground and background colours on the command-line when you run them,as in:

xclock -foreground red -background blue

You can also use RGB specifications externally in place of colour names, by adding a prefix of `rgb:'.[Footnote:Release 5 only. Earlier releases had a similar facility, but the rgb: prefix wasn't necessary, because the other types of colour specifications introduced with Release 5 were not supported. In Release 4 and earlier, an explicit RGB colour was prefixed with a hash (#) to distinguish it from a colour name.] The command:

xclock -foreground rgb:ffff/0/0 -background rgb:0/0/ffff

is exactly equivalent to the one above.

The server maintains a simple database of colour names and the corresponding RGB values; internally, using standard X functions, the client can request the server to lookup a colour name, and use the value returned, or it can specify a colour directly by name. The MIT program showrgb lists the colour names your server knows about, with their RGB components; for example, Figure A shows a selection of lines in our colour database.

A given RGB specification can result in markedly different colours depending on the monitor and graphics hardware used. As well as being more natural, colour names allow a degree of device independence, by letting you tune the RGB-values allocated to a colour name for the particular hardware you are using. For example, you (or your system vendor) would ensure that the RGB specification corresponding to `pink' really does give a pink colour on screen! In Release 5, a much more powerful system for specifying colours in a device-independent way was included, and we describe that in the next module. After that we describe some of the detailed mechanisms available for handling multiple applications which together require more colours than the hardware can display at one time.

Figure A. Some of the entries in our RGB colour database.

3.6.1 Device-independent colour specifications

Release 5 introduced Xcms, the X Colour Management System. This lets you specify colours in a device-independent way, so you can be sure the visible colour will be the same whatever hardware it is displayed on.

(If technical details aren 't so relevant to you, you can skip over this module.)

The RGB scheme for specifying colour that we outlined in the previous module relates very elosely to colour display hardware. The individual RGB values specify more or less directly the intensity with which the electron guns in the hardware are to fire at the red, green and blue phosphors on the sereen. While this scheme may be a convenient way of driving the hardware, it has several deficiencies from the point of view of people:

To overcome these problems, Xcms, the X Colour Management System, was introduced with Release 5. This makes use of the international colour standards of the CIE (Commission Internationale de l'Eclairage - International Commission on Illumination). With Xcms, colours can now be specified using the CIE XYZ, CIE uvY, CIE xyY, CIE L*a*b*, CIE L*u*v*, and TekHVC schemes, as well as the RGB values and colour names we have already seen. These schemes were developed to allow colours to be specified in an absolute and reproducible way, and with the exception of TekHVC were not specifically for addressing colour in computer systems (in fact, most of these systems were developed quite a long time ago). By using one of these representations, you can specify a colour irrespective of the hardware you are using, and with Xcms you can reliably reproduce this colour on any hardware. We won't go into the details of all these specification schemes, but will just outline the principles of Xcms.

With Xcms there are no changes in the serşer - it still handles colours internally in terms of RGB specifications. All the changes are on the client side, and consist of enhancements to the standard client library, Xlib, and to the applications that want to use it.

Figure A shows schematically how colour specifications are processed with Xcms. Xcms effectively acts as a translator, changing device-independent colour specifications to RGB values to send to the server, and converting RGB values received from the server into the format required by the application.

In the next module we look at how Xcms takes account of the colour characteristics of the particular graphics hardware that your server is using.

Figure A. Xcms translates device-independent colour specifications on the client side.

3.6.2 Calibrating hardware for device-independent colour

The colour and illumination characteristics of the graphics hardware used by the server must be measured, and then stored so that Xcms can access them and take account of the way that your hardware renders colours.

To convert between device-independent and RGB specifications, we must first perform some measurements on the hardware, to find out what its colour and illumination characteristies are. For instance, the illumination from a screen does not increase linearly with RGB value; at low intensities, you need quite a large increase in RGB value to give a small increase in illumination, whereas at higher intensities this isn't so. Separate measurements have to be camed out for each of the red, green and blue components of the screen. All these characteristics are specific to each type of screen and graphics hardware. Figure A shows the characteristic curve of illumination versus RGB value for the red component of a screen similar to our own.

This process of measurement is called device calibration. The contrib release includes the program xCrtCa, to drive a CRT colour analyser so you can perform this function for yourself. (Very few organizations need to do this - tables of characteristics for particular combinations of graphics hardware and monitors are being made available. The contrib release already contains some of these in a form suitable for use by Xcms, and as Xcms is more widely adopted, hardware manufacturers will provide the necessary calibration information. )

Having got the device characteristics, we must make this information available to Xcms in a form it can understand; this is called device characterization. The colour profile of the screen is stored in two properties on the root window, XDCCC_LINEAR_RGB_CORRECTION and XDCCC_LINEAR_RGB_MATRICES. You load these properties from a file containing the characterization information using the MIT program xcmsdb. Applications using Xcms can now retrieve the characterization information from the properties when they require it, as shown in Figure B.

Figure A. How illumination intensity varies with RGB value on our screen

Figure B. How Xcms uses properties for device characterization.

3.6.3 Colour capabilities and colormaps

The number of colours you can display simultaneously on your screen is determined by your hardware, and is usually quite limited. If you have several applications running, they may require different sets of colours, which would exceed the hardware limit. To get over this, X provides `colormaps' - tables of the colours which are currently to be displayed by the hardware.

(If technical details aren 't so relevant to you, you can skip over this module.)

The most common colour systems are 8-bit. That is, each pixel is represented by an 8-bit value, which means that 28 = 256 distinct colours on the screen are possible at any one time. We say that such a colour system has a depth of 8. On such a system, while only 256 colours are possible at any one time, we can choose which 256 are to be used out of the total palette that the colour hardware makes available, which typically offers 224 =16777216 possibilities. It is the software that specifies which set of 256 colours are to be used, by means of a colormap - a colour lookup table that says what particular combination of RGB values is to be associated with a given pixel value (Figure A). All pixels containing the same value will show the same colour on the screen, because they all reference the same element of the colormap. If you change the RGB values in the colormap element, then all pixels referencing it will immediately change on screen to reflect the new colour.

This manipulation of colormaps can be very useful in imaging applications, where the user needs to set colours frequently, for instance while trying to adjust the contrast to get the clearest view of some image. Speed is obviously important in this case, and in fact colormap manipulation is very fast; as only a few bytes in the colormap are being changed, the colour on screen changes almost instantaneously. (If it was necessary to cycle through a11 the video memory, perhaps as much as a megabyte or more for a full-screen colour image, changing individual pixels to contain a different value, it would be very much slower.) In fact, colormaps are often manipulated interactively by means of scrollbar-like sliders, one each for the red, green, and blue components of a colormap element; as you drag a slider the colormap is updated and you see the effect instantaneously on your whole screen. The contributed program cpicker, shown in Figure B, has this facility.

In fact, colormap handling is considerably more complex and sophisticated than we have described here. Some hardware's colormaps are read-only and cannot be modified; other hardware supports multiple simultaneous hardware colormaps. And different systems either themselves interpret the values in the colormap elements in different ways, or let you specify how they are to be interpreted You may come across the terms `PseudoColor', `DirectColor', `StaticGray', `StaticColor', and `TrueColor': these are specifications of how the colour mechanism works. The term visual or visual class refers to which of these colour mechanisms is used. For example, the visual class for the colour scheme outlined in the first paragraph of this module is 8-bit PseudoColor. However, you're unlikely to have to bother with these low-level aspects unless you are programming the system or working with it at quite a detailed level.

Monochrome systems have a depth of 1, and can therefore only display 21= 2 distinct colours, that is, black and white. Grayscale systems have a depth of greater than one, so they are not limited to black and white, but can also show many different intermediate shades of gray, just like a black-and-white photograph. High-end systems are increasingly providing 24-bit colour, which obviously allows a huge number of distinct colours.

In the next module we see how X handles different applications using different colormaps at the same time.

Figure A. Colormap determines what colour a pixel shows on screen.

Figure B. The CpiCker colormap manipulation tool

3.6.4 Using many different colormaps together

Each application may need to use a set of colours specific to itself, so X allows each window to have its own colormap associated with it. The window manager loads this colormap into the colour hardware when the window becomes active so that it shows its correct colours when you are working within it.

Let us consider an 8-bit colour system. This allows you to display 256 colours on the screen. This number of colours is usually more than adequate for decorative purposes (for distinguishing between different windows, for example) and for highlighting particular items within applications. But with applications such as graphics design, CAD, CAE and imaging, which need a great many distinct colours (for example, in photographic images), you start to get problems. First, 256 colours may just not be enough; if so your only option is to buy more capable hardware, such as a 24-bit system. Second, while no individual application might need more than 256 colours, there may be a clash between applications, about which particular set of colours they want. For example, you might have two mapping application on your screen; one needs 200 colours, mostly greens and browns, for a terrestrial map, while the other needs 100 shades of blue for a marine map. The total number of colours needed is 300, which is more than the hardware can display.

To get over this problem, X provides multiple software colormaps, and each window can have its own colormap associated with it; by installing one of these into the hardware colormap you easily change from one set of colours to another. It is the window manager's responsibility to install the appropriate software colormap into the hardware as you move from one application to another. So when you are working in an application, the colours showing are correct for that application, although they may not be correct for other applications on the screen. In a situation like this, as you move from one application, the screen colours change rapidly. This `technicolor' fiashing can be disconcerting at first, but it is the only way to use more colours than the hardware can handle.

Another very common problem is that the window manager, which is often one of the first clients to start up, has used quite a few of the available colours. (This is especially noticeable on small colour systems, such as PCs with standard VGA adapters offering only 4-bit colour, that is, only 16 distinct colours at a time.) To avoid this type of problem, programmers are encouraged to share colormaps, within and between applications. By minimizing the number of different colormaps you reduce the fiashing technicolor effects, and also maximize the number of windows simultaneously showing their correct colours. (The easiest practical way to do this is not to use any special colormaps, but to restrict the number of different colours you choose to use, and to specify the colours you do use with colour names rather than RGB or other specifications, which helps ensur you really are using the same colours throughout. You can also configure your window manager and restrict it to use only two colours.)

The Xcms colour management system helps here too. It lets you specify clearly the colours you are using. This avoids a common problem with the RGB system, which is that you may specify several colours that have different RGB values (thus xrquiring several slots in the colormap) but which look the same and are effectively one colour as far as the user is concerned.

3.7 Pictures and images --- bitmaps, cursors and pixmaps

Applications can create bitmap images, to be used as icons, as pictures or graphic images, or as patterns for window backgrounds, borders, etc.

We have already seen two of the principal types of graphical objects supported by X --- text fonts and 2-D graphics. In this module we look at a third type of object --- pictorial images. A picture is not described by geometrical formulas as lines or arcs can be; instead it is represented as an array of pixels with the colour of each pixel specified explicitly. As a result, picture representations consume a lot of memory.

The simplest form of picture is a bitmap. Each pixel is represented by a single bit: if the bit is zero, the pixel shows in the background colour, otherwise the bit is one and the pixel shows in the foreground colour.

Bitmaps are used for small pictorial images, such as window manager icons, or pictorial labels for pushbuttons. They are also widely used as a very general way of specifying a shape: the shape consists of all the 1-bits in the bitmap. This technique is used for cursors (Module 3.7.2). It is also used to restrict the area affected by a drawing operation; a bitmap is used to specify a clipping region, such that only pixels in the specified region are affected. (The bitmap effectively provides a stencil through which the drawing operation is performed, as shown in Figure A.) This technique is how X provides non-rectangular windows on scneen - windows are fundamentally rectangular, but can have an associated clipping bitmap, so the shape actually displayed is that specified by the bitmap. Note that in a bitmap any set of bits can be set on; in particular, the bits do not have to be contiguous but can be disjoint, and therefore by applying a disjoint clipping bitmap to a window, it can be displayed as several disjoint pieces. This is how the MIT program xeyes shows as two separate eyes (Figure B).

X provides a simple external representation for bitmaps. They are described as C-program fragments, as shown in Figure C. They can be incorporated into the source code of C programs, and there are also Xlib utility functions to write out internal bitmaps to a file in this format, to read them in from a file, and to convert them to X's internal representation within the server. Most X bitmap manipulation tools use this mechanism, so it is both portable and flexible. For example, the MIT bitmap editor bitmap is shown in Figure D editing the bitmap file of Figure C.

A text-picture representation of a bitmap can also be created with a normal text editor, as shown in Figure E, and converted to standard bitmap form using the MIT program atObm (`ASCII to bitmap').

In the next module we look at pixmaps, which are X's general format for representing images. In the module after next, we look at bitmaps used as cursors.

Figure A. Bitmap used to specify a clipping region for a graphics operation.

Figure B. The xeyes windows shows as two disjoint pieces.

Figure C. A bitmap is represented externally as a fragment of C code.

Figure D. The MIT bitmap editor.

Figure E. Text-picture representation of a bitmap.

3.7.1 Pixmaps - X's most general picture format

X represents pictorial images internally in the server as `pixmaps', which are rectangular offscreen arrays of pixels. You can draw graphics into,pixmaps just like you draw into windows. Client applications support conversion of X images to and from other standard image file formats.

So far we have only been dealing with bitmaps, where each pixel in the picture is represented by a single bit. X's most general internal representation of pictures is the pixmap. Pixmaps are similar to bitmaps, but each pixel in the pixmap is represented by a fixed number of bits. In fact an X bitmap is held internally as a pixmap with one bit per pixel. The number of bits used is called the depth of the pixmap, and this is analogous to the depth of a colormap in Module 3.6.3.

Pixmaps are often used within programs for tiling window backgrounds, borders, etc., that is, as (small) images which are repeated until the specified area is eompletely patterned.

Pixmaps (and bitmaps) reside in the server, and clients create them by issuing requests to the server. When first created, a pixmap is just a rectangular area of pixels; you must then transfer whatever image you require into the pixmap using the usual X graphics requests as described below.

As we mentioned in Module 3.3.2 pixmaps are drawables, that is, you can perform all graphics and drawing operations into a pixmap instead of into a window. This means you can maintain and manipulate an image offscreen, in a pixmap, and later transfer it rapidly to a window when you need to. This technique is used in complex graphics applications, where the cost of redrawing a window in response to an expose event can be very high, perhaps involving the reprocessing of a display list of thousands of elements. To get over this, when you originally draw the graphics into a window, you also draw the same graphics into a pixmap. Then, when the window has to be redrawn after an expose event, you block-transfer a copy of what is in the pixmap into the window - a relatively fast operation requiring no recalculation on the part of the application.

However, a problem with this technique (and any other use of pixmaps) is that offscreen memory in the server may be very limited. By assuming that a lot of it is available you can reduce the portability of your application; your program may fail to run correctly or at a usable speed on servers which cannot support such large pixmaps.

Other image formats, and applications

What we have outlined here are the standard X facilities for handling images and pictures. The server does provide these basic mechanisms, but to satisfy the requirements of imaging, of CAD, and of comprehensive end-user graphics systems much more is required. This extra functionality is provided by application programs, which convert to and from standard image file formats such as GIFF, TIFF, raster files, etc., so the X server itself does not need to support foreign file formats directly. For example, the contrib program xloadimage lets you display many of these different types of graphics files.

The X Imaging Extension which is under development by the MIT X Consortium (see Module 15.1) will expand the functionality X provides to support imaging applications.

3.7.2 Cursors

The server has its own default set of cursors. Applications can also create their own custom cursors trom bitmaps.

X systems normally have a mouse or similar pointing device, and as the mouse is moved so a pointer is moved on the screen. The pointer takes a shape called a cursor, and each window within an application can have its own cursor shape associated with it.

Internally, a cursor consists of a bitmap, that is, a pixmap of depth one, specifying the shape of the cursor and what colour is to be used to display it. It also has defined a hot spot - a single pixel which determines where exactly the cursor `is'. (Without this, if you had a cursor consisting of 16 x 16 pixels, say, on the boundary of two windows, how could the system know which window the cursor is really in?) A cursor also has another bitmap known as a mask, which has its own separate display colour. Why is this needed? Assume that the cursor's display colour is red; when the cursor is in an area which is also red, the cursor would be invisible. So, typically, the mask is defined to extend slightly beyond the basic cursor shape on all edges, and displayed in a contrasting colour, so that the cursor is now visible no matter what the colour of the underlying area is. A cursor bitmap and mask are shown in Figure A.

X provides a standard set of cursors in the special font named Cursor; Figure B shows this displayed by the MIT font display program xfd. Applications can also create their own custom cursors from a bitmap, using a special X request for this purpose.

Figure A. A cursor bitmap, showing hotspot, and cursor mask.

Figure B. cursor - the MIT font of standard cursors

3.8 Extras you may want or need: server extensions

Specialized servers incorporating features such as PostScript, or live video windows or other special facilities are available. These features are extensions to the basic system and are provided using a special extensions mechanism. New functionality is incorporated into new releases of standard servers using this mechanism rather than directly, to give compatibility with earlier releases and to ensure consistent operation on servers from different vendors.

When they developed X, its designers realized that for various reasons it was missing certain functionality which would have to be added later on. Either the technology had not yet matured enough (as with 3-D graphics) or the functionality was not crucial and could therefore wait (as with non-rectangular windows) or it was just outside the scope of the X development at the time. Moreover, any successful system will to some extent have to grow dynamically over time, requiring the addition of features which were not anticipated at the beginning. Because of this, X has a well-defined mechanism for adding extended functionality to the server. (There is no difficulty extending client functionality - you just add it in with extra code and libraries.) Also, it is likely that some extensions will be so popular that they will become almost standard in time so they must be well integrated into the system. (This has already happened with the MIT SHAPE extension, introduced with Release 4, which allows windows to be non-rectangular.)

The X extension mechanism is merely a clean way for a server vendor to add something new to a server, and to be able to make this enhanced functionality available in a standard and controlled way on servers from many different suppliers. For example, if you write a program that uses the Display PostScript extension on a server from DEC, that program should also work when displaying to another server with its own Display PostScript extension on a Hewlett-Packard workstation. The mechanism is not intended as a means for end-users to add enhancements to their servers themselves. How an extension affects the client programs is covered in the next module.

Common extensions

The following extensions are part of the standard MIT release:

Other common extensions are:

An example - the Display PostScript extension

For many applications, X's graphical and font capabilities are perfectly adequate. However, the limitations in its text handling (inability to rotate fonts and, except in Release 5, to scale them) and lack of coordinate transformation mean that for sophisticated typesetting and imaging something more powerful is needed. Accordingly Adobe have developed the Display PostScript system (DPS), to provide within X the same facilities which are available with PostScript printers.

The advantages of the system are resolution-independent imaging. Sizes (of text, lines, etc.) are specified in real-world units such as inches or centimetres, and the system transforms these into the correct number of pixels according to the number of dots-perinch of your screen. You also get high-quality scalable and rotatable fonts and, most importantly for many users, the fact that what is shown on screen can be printed exactly on a PostScript printer.

DPS is representative of commercial extensions. It is not part of every server, but is widely useful, and has been incorporated by vendors into servers for many different platforms. It is produced by a commercial software company and is not a minor or local change to the server by an end-user organization for its own use, but provides considerable functionality on top of the standard X system.

3.8.1 How extensions affect applications

An extension adds extra functions to a server. It is an extension to the X protocol, and defines new primitive functions which the server supports. Therefore there must be extra requests for the clients to invoke this functionality, and usually extra subroutine libraries containing functions to issue these extra requests.

(If technical details aren 't so relevant to you, you can skip over this module. )

Just as clients don't build standard X protocol requests themselves, but use Xlib and other libraries to provide a higher-level interface, so clients making use of extensions will use special extension-specific libraries. As shown in Figure A, there are extra components in both client and server.

To ensure that an extension works consistently across different servers, there must be some way for the client using the extension to identify that the requests it issues relate to the particular extension. We could reserve special request opcodes for each registered extension (and MIT does maintain a register of special names, etc., so this is feasible). However, there is a limited range of opcodes available, so a different scheme is used, based on the name of the extension. Each extension is allocated a unique name by MIT. (Figure B lists the names of the extensions registered with MIT when Release 5 was issued.) When an application first wants to use a given extension, it asks the server if it supports an extension of that name. If so, the server returns the major opcode for the extension. This is the special code which is included in requests sent to the server to identify what type of request this is, and the application inserts this in each request it issues for functions provided by the extension.

One extension is likely to require many different request types, that is, there are many different functions that the extension supplies, and the program must be able to specify which one is required. Rather than allocate a major opcode to each request, all requests for the one extension use the same major opcode, but use another minor opcode to distinguish between them. Effectively, the major output routes the nequest to the extension-specific portion of the server, and the minor opcode is used by that part of the server to decide which particular action it should perform. Figure C lists the minor opcodes used by the MIT SHAPE extension (which lets you use non-rectangular windows).

As we mentioned above, the application queries the server to see if the extension is present. This is essential, so that applications can decide what action to take if the extension is absent, rather than just failing. The action taken depends on the application, and why the extension was being used. If the application was using the SHAPE extension just to make pushbuttons look nicer by rounding the corners, the sensible course is just to use rectangular buttons if the extension is absent. However, if the application is a PostScript document previewer, and the PostScript extension is absent, the only useful action the application can do is to print an informative error message, and terminate.

Applications can also ask for a list of all extensions supported by the server, rather than explicitly querying in turn if each extension the application knows about is supported. This is how the MIT program xdpyinfo lists the extensions available on a particular display.

Figure A. Extra components of an X system with an extension.

Figure B. Extension names.

Figure C. Minor opcode definitions for the MIT shape extension

Summary

In this chapter you have seen what the role of the server is, and how the rest of the system is hardware independent because only the server interacts with the display hardware. We looked in detail at how the server is implemented and how it responds to requests from clients - to create and manipulate windows, and to draw text, graphics and pictorial images. We also looked at some of the more sophisticated mechanisms for handling fonts, which are especially useful when you are managing large X networks, and for the colour management required by advanced applications. Finally, we saw that the extensions mechanism allows extra functionality to be added to systems, so that X can grow organically but in a controlled way nonetheless.

So far we have not dealt with user input. We leave that to the next chapter, which examines what passes between client and server. We shall see that input from the user is processed initially by the server, but that it passes most of the work of input processing over to the client.