Recreational Bugs

At the San Diego Usenix in January 1989 I presented Visualizing X11 Clients, a paper written by David Lemke and myself. In email conversation about his Pie Menus: A 30 Year Retrospective, Don Hopkins unearthed the script for the talk I gave, which I posted to the "xpert@athena.mit.edu" mail list. To record the script for posterity, a slightly edited version is below the fold.

Don also unearthed A Window Manager for Bitmapped Displays and Unix, the paper James Gosling and I wrote describing the Andrew window manager for the Alvey Workshop at Cosener's House, Abingdon (29^th April to 1^st May 1985) (DOI). The entire workshop proceedings were subsequently published as Methodology of Window Management, and are online here. The Andrew window manager tiled the screen with windows because, as the quote at the head of the paper said:

You will get a better Gorilla effect if you use as big a piece of paper as possible. Kunihiko Kasahara, Creative Origami.

In retrospect, this wasn't a great idea.

Introduction

Good morning. I'm Sgt. Rosenthal of the Softies, the Software Police. Even though its early, you'd better listen up and listen good, because what I'm going to tell you this morning can keep some of you out of a lot of trouble.

Officer Lemke and I work in the X Division, and we've been out there on the net making undercover buys of X games programs. When we test them back in the labs, we almost always find recreational bugs.

The Zero-Tolerance Bugs Policy

I'm here today to talk to you about the Quayle administration's new ``zero-tolerance'' policy on bugs. Nancy Reagan's "Just Say No" to bugs program was all very well, but it didn't stop the wave of bugs that engulfed the nation during her husband's administration. The new administration has decided to get tough on bugs, and that's what the zero-tolerance policy is all about.

Surveys tell us that many of you have pretty tolerant attitudes to recreational bugs. I'm here to change that. There are three main reasons why the policy focusses on recreational bugs:

• In most of the cases we see, programs with recreational bugs go on to develop more serious hard bugs.
• The sharing of software involved in recreational bug usage can lead to the transmission of viruses, worms and other parasites.
• And, of course, the bottom line is that bug-free X software isn't just a good idea, its the law.

Penalties for Bug Possession

Under the zero-tolerance policy the penalties for even a small recreational bug can include:

• confiscation of resources,
• X protocol errors,
• termination of your program,
• visions of bizarre colors on your screen,
• and above all making the designers of X unhappy.

You do not want to make the X designers unhappy. If they get too unhappy with the way you're mis-using their creation, they may just get together and build you a whole new industry-standard window system. Look how much work it was last time they did this. Think what they've learnt since then.

The Neighborhood Bug-Watch Program

Everyone deserves a bug-free environment, but achieving it isn't easy. The scourge of bugs has affected every stratum of society. People you trust implicitly, role-models like the folks at MIT and elsewhere who work long hours into the night selflessly donating their software to the poor, even these people may be into recreational bugs. And you may not find out until its too late. When we in the Softies stop you out there on the net, it won't do you any good to say ``This code isn't mine, I'm just holding it for a friend''.

Types of Bug to Look Out For

X bugs you should be looking for can be divided into:

• The narcotics that leave programs catatonic. A warning sign to look for is a program that starts ignoring Expose events. You've heard plaintive complaints from people on xpert that they draw and draw and nothing is visible; they are suffering from narcotics abuse.
• The stimulants that drive programs into a frenzy. A warning sign is a program that tries to argue with authority figures like window managers.
• The steroids that cause programs (and especially servers) to swell up and get huge.
• The hallucinogens that cause swirling visions in bizarre colors.

This morning I'm going to review the hallucinogens and their symptoms, and we need to start with some basic physiology.

Basic X Physiology

When your X client draws, it does so in a Drawable, a rectangle of memory locations containing numbers called pixel values. The drawing operations like CopyArea, PolyLine, PutImage and PolyText8 all change these pixel values.

Some of these Drawables may be windows, and some of the windows may be mapped. Every few milliseconds, the screen you are looking at is refreshed by reading the pixels of the visible parts of the the mapped windows, and using these values to index into a colormap. The colormap cell indexed by the pixel value contains a red, a green and a blue value that is fed to the corresponding gun of the screen.

The details of this process can differ between different types of hardware. The differences are exposed; X gives no guarantee of portability and you have to be aware of these differences:

• Is there one gun (monochrome) or three (color)?
• How many entries in a colormap?
• Is there one colormap, or one colormap per gun?
• How many colormaps are simultaneously accessible?
• Can a client write the colormap entries?
• How many bits in each colormap entry for each gun?

All these differences are collected together into a concept called a Visual, and the possible Visuals divided into six classes:

• StaticGray. The pixel value indexes a predefined, read-only colormap. For each colormap cell, the red, green and blue values are the same, producing a gray image.
• StaticColor. The pixel values indexes a predefined, read-only colormap. The red, green and blue values for each cell are server-dependent.
• TrueColor. The pixel value is divided into sub-fields for red, green and blue. Each sub-field separately indexes the appropriate primary of a predefined, readonly colormap. The red, green and blue values for each cell are server-dependent, and are selected to provide a nearly linear increasing ramp.
• GrayScale. The pixel value indexes a colormap which the client can alter, subject to the restriction that the red, green and blue values of each cell must always be the same, producing a gray image.
• PseudoColor. The pixel value indexes a colormap which the client can alter. The red, green and blue values of each cell can be selected arbitrarily.
• DirectColor. The pixel value is divided into sub-fields for red, green and blue. Each sub-field separately indexes the appropriate primary of a colormap that the client can alter.

When you implement an X server for an individual workstation configuration, you have to decide which Visuals you want to make available to your clients. Sometimes, the choice is obvious. For example, a normal monochrome workstation like a Sun 3/50 would naturally export a single depth 1 StaticGray Visual. A simple color workstation like a Sun 3/160C with an 8-bit-per-pixel frame buffer and a single 256-entry colormap would naturally export a single depth 8 PseudoColor Visual.

As hardware gets more complex, the choice may be to export several Visuals for a single screen. For example, the Sun 3/110C in effect has a 10-bit deep frame buffer. 8 of the bits index a single 256-entry colormap, one of the bits acts as a simple monochrome framebuffer, and one of the bits selects whether the 8-bit color or the 1-bit monochrome image is visible at this pixel. In this case, the normal choice would be to export two Visuals, a depth 8 PseudoColor Visual and a depth 1 StaticGray Visual. Even more complex hardware may need to export many Visuals to allow clients full access to its capabilities.

If you're a sneaky server implementor, you may even export Visuals that you don't strictly have. For example, both the MIT server for the DEC QDSS color display and the X11/NeWS server for Sun color hardware export both PseudoColor and StaticColor Visuals even thought they both really have only a PseudoColor display. The StaticColor Visuals use otherwise unused entries in the hardware colormap, allowing simple color applications to achieve better colormap sharing than they would on a single PseudoColor Visual.

What this means to you as an X client programmer is that you are likely to be faced with a choice of Visuals, and that as hardware evolves to give you bigger and better thrills, this choice is likely to expand greatly. You have to choose a suitable Visual for your client, and unless you choose with great care, you will be increasingly at risk for hallucinogenic bugs.

Hallucinogenic Bugs and Their Symptoms

Hallucinogenic bugs achieve their effects by interfering with the Visual mechanism. Here's a list of the common hallucinogens and their symptoms:

• A program can assume that the default Visual is the only available Visual.
  

  Suppose you had a color application that made this assumption and a Sun 3/110C. The server implementors might have chosen to make the depth-1 monochrome Visual the default, in which case the application would fail even though the hardware actually supported color.

• A program can assume that all Visuals with more than two Colormap cells are color.
  

  Suppose you had a color application that made this assumption and a Sun 3/160GS (grey-scale). It might chose a foreground and a background color which mapped to the same shade of grey, and the output would be invisible.

• A program can assume that all Visuals with more than two Colormap cells have writable Colormaps.
  

  This is a very common assumption. Clients making it fail with an Access error when they try to write a read-only colormap cell.

• A program can assume that Colormaps (and especially the default Colormap) are infinitely large, so that attempts to allocate private cells in them will always succeed.
  

  Clients that make this assumption will sometimes fail with Alloc errors, and sometimes succeed, depending on the number of cells they ask for and the number of cells that other clients have left available in the colormap.

• A program can assume that all Colormaps work with all Visuals.
  

  Clients that install Colormaps in windows other than the one they were created for may get a Match error if the Visuals don't correspond.

Traditional Values Keep Your Colors Sober

At this point, you're probably saying ``this isn't a big deal, all my friends do bugs like this''. You're wrong; the only clients that can ignore the question of Visuals are those that use the BlackPixel() and WhitePixel() macros to paint a black and white image in the default Visual. All other clients must pay some attention to the details of the Visual(s) they are using, if they want their output to appear in sober, everyday hues.

When you choose a color, you really choose an RGB triple. For example, you say ``I'd like this text to come out blue'', and what you really mean is that you'd like the image of the text when its refreshed on to the screen to have the RGB triple [001]. Working backwards:

• this means that the colormap cell used in the refresh process must have the RGB triple [001] in it,
• and this means that the pixel value in the window must have indexed to a cell in the colormap with the RGB triple [001] in it,
• and this means that the foreground pixel value in the GC that you used to draw the text must have been one that would index to a cell with the RGB triple [001] in it,
• and this means that the foreground pixel value you supplied to a CreateGC or ChangeGC call that got you the GC that you used to draw the text must have been one that would index to a cell with the RGB triple [001] in it,

OK, enough of the "for the lack of a nail the horseshoe was lost" stuff. You need to have some way to convert your RGB value into a pixel value that will map back into the RGB value.

Where Do Pixel Values Come From?

I expect you've overheard whispered conversations among your friends about where pixel values come from and how you can get one. I'm here to dispel the myths, and assure you that there are just three legal ways you can get a pixel value:

• You can give the server a colormap and an RGB value, or a text name for an RGB value, and ask it to give you back a pixel value that will index to a read-only cell in that colormap that has the closest available match for that RGB value.
• You can ask the server to reserve you a private, writable cell in a Colormap. If this succeeds, the server will give you a pixel value for your private cell, and you can set whatever RGB value you like.
• Or there are various ways in which you can compute a pixel value from an RGB triple by predicting the values in the colormap:
• If the server has a TrueColor Visual it will provide linear ramps.
• The so-called ``Standard Colormaps'' also provide linear ramps.
• The connection handshake process tells you BlackPixel() and WhitePixel() for the default Visual.

Be warned that values you obtain any other way are illegal, and may be bugs.

Which Method Is Right For You?

• For beginners, and the simplest clients, using BlackPixel() and WhitePixel() in the default Visual is best.
• Using read-only colormap cells and letting the server do the conversion will work on any Visual and maximises sharing with other clients. It is the technique of choice unless:
• you have a lot of colors,
• the exact representation of colors is of primary importance to you,
• or you want to play colormap tricks.
• Clients that want to display an RGB image with many colors should use the prediction technique in one of the Standard Colormaps, as soon as the experts can agree on how to make the technique really work.
• Colormap tricks aren't actually illegal, but they're risky and we advise against them. If you want to play these tricks you should make sure beforehand that you have a dynamic Visual, and create a private Colormap so as not to disturb your neighbors.

Are You Safe if You Use a Toolkit?

I expect you'll hear people saying ``I don't have to worry about this kind of bug, I use a Toolkit''. Unfortunately, this is just another of the myths about bugs. Not that we don't advise people to use Toolkits; used in accordance with the manufacturer's instructions a Toolkit can keep you safe from many common bugs and save you a great deal of time and trouble. But the sad fact is that Toolkits don't protect against hallucinogenic bugs, and this hasn't had the publicity it deserves - the Intrinsics manual doesn't mention Visuals.

How Do Widgets Get Their Visuals?

At present, Visuals are hereditary. Widgets inherit their Visuals from their parents. The window for a Widget is created when the Widget is realized, and the Visual has to be bound to the window then. At the root of every application's tree of Widgets is a Shell Widget; the Intrinsics define this to inherit its Visual from its parent (ie. the root window), so it will have the default Visual. All the Widgets on the X11R3 tape have realize procedures that inherit their Visuals from their parents, so that by induction all Widgets have the default Visual.

If you're creating a Widget that others may use, it is your responsibility to ensure that it gets a suitable Visual. Simply trusting your parent to do this for you is not a suitable way to discharge this responsibility

How Can You Protect Your Widgets From Hallucinogens?

You can do this by equipping them with a suitable realize procedure. Based on your knowledge of the color requirements of the Widget you are defining, you should choose one of the legal methods for obtaining pixel values. Then, you should write a realize procedure that:

• Ranks the Visuals in preference order, and finds the smallest best Visual that will do the job. Code to do a similar task is in the paper.
• Obtains a suitable Colormap for the conversion method selected, by:
  • Using one of the Colormap description properties on the root window, if you're using the prediction method.
  • Choosing the default Colormap, if that will do the job.
  • Creating a private Colormap using the selected Visual and the root window of the screen.
• Creates a window in the selected Visual, and supplies the selected Colormap as one of its attributes.
• Sets the window and the colormap into the core attributes of the Widget.

Remember, educating your Widgets about Visuals right from the start is the key to keeping them bug-free for life.

Don't Let Your Widgets Accept Colormaps From Strangers

You're probably asking ``If colormaps are attributes of Widgets, why shouldn't I let them be Resources that can be changed by the user?'' The problem is that the bright shiny colormap that the nice user-man is offering your Widget may be a hallucinogenic bug. It might not belong to the same Visual as your Widget's window, and when the trusting little Widget tries to set the window's colormap attribute, wham! it gets a Match error.

Unfortunately, there is no way to test a colormap beforehand to see if it is compatible with the window whose attribute you're trying to set. Given a colormap ID, the protocol doesn't provide a way to find the Visual it was created for. So, its better to let your Widgets choose their own colormaps, and not to provide any way for the user to override their choice.

If you have to accept a colormap from someone else, and you don't know the Visual, you can create your own colormap for the Visual you want, read all the entries from the strange map, and create corresponding entries in your own map. There are problems with this approach:

• The entries have to be read back to the client and then shipped back to the server, because there is no way of copying entries between colormaps of different Visuals.
• There is no way of discovering which cells in the source colormap are sharable and which private. This means that the create-and-copy destroys the sharability of the entries.
• The whole idea of creating a new colormap works against the sharing of resources that was probably the reason for trying to accept a strange colormap in the first place.

How You Can Get Your Programs Off Bugs

• Understand the X display and Visual mechanism.
• Understand the legal methods of converting RGB triples to pixel values.
• Chose a method and a Visual that match your application.
• Don't let someone else choose your Visual for you.
• Don't accept colormaps from strangers.

What To Do If You Find Bugs In Your Program

• Isolate the program immediately.
• Do not pass the program on to others.
• Examine the program's assumptions carefully.
• Call for help from the xperts.