If I had to pick three words to describe my experience with BattleField 3, it would have to be; annoyance, disappointment & frustration. Some might argue that annoyance and frustration have a large overlap, but I thought it important to properly cover the extent of my bile.

Not only is the game extremely short –I finished it within a little more than 6 hours– it is also extremely boring. The story is ludicrously unsophisticated and focuses on stolen nuclear bombs to be detonated by terrorists in Paris and New York. Really Dice? Stolen fucking nuclear fucking bombs? How very 1980 of you. The story is presented as a sequence of flash-back missions of the main character, Sergeant Blackburn, who is being drilled by civilians (?) about having shot his CO and his potential involvement with terrorist attacks. Every mission is basically the same; lots of people firing back and forth, follow the blue diamond and follow the instructions on the screen. Press A when you’re told to. Press B when you’re told to. Press RT when you’re told to. Thrilling.

There’s almost no lead-in, no time to get comfortable with the controls or the weaponry. There’s also very little freedom. Wander but a little from the preset mission path and the mission fails, at which point you’ll be awarded with the loading screen and you get to try again. And again. And again. And again, until you figured out the correct sequence of buttons to press to get you through the mission.

The graphics and physics of BattleField 3 are fantastic. Too bad there’s no time whatsoever to enjoy them as the game-play and the story are amongst the worst I’ve ever seen in any game.

1. Ascension. I used to think that Ascension was to celebrate the 1969 moon landing. The error is somewhat less obvious in Dutch, as “Hemelvaart” —as it is known there— translates to “Heaven/Sky Flight”.
2. Kissing. When actors kiss in movies, it surely must be some sort of visual trick because only people who are in love could possibly want to kiss each other and the odds that two professional actors randomly thrown together are in fact in love is diminishingly small.
3. Adulthood. The best thing about being an adult is not —as I used to think— being allowed to do adult stuff. Turns out adult stuff is major league boring and frustrating. The best thing about being an adult is that you can do all the kids stuff you want and nobody is going to tell you to stop.
4. Lasers. The only mechanism by which I could imagine lasers to work was for them to be funnels made of reflective material. I knew there was a mirror involved, just got the geometry wrong, after all, if it works for water, why can’t it work for photons? These days I do understand how the electron states are perturbed by passing photons, though I still have major problems understanding the basic principles of holograms.

Most problems have multiple local optima, sometimes these optima represent very similar fitnesses and sometimes they represent very different fitnesses. A good solver should not allow itself to get stuck in a low-quality optimum.

Let’s consider the problem of fitting a circle to a collection of points in the 2D plane so that the distance between the circle and the points is minimized. There are good algorithms available for finding such a circle so we don’t have to rely on a generic solver such as Simulated Annealing, but it is a good example because it’s both easy to set up and see whether or not the answer we’re given is correct.

Our problem is three-dimensional while we’re limited to the 2D plane because the least amount of variables we need to define any circle equals three; x, y and radius. Every unique combination of {x,y,R} results in a unique circle, and every possible circle can be represented by picking the appropriate {x,y,R}. So the question basically becomes; “which values for x, y and R result in the best fitting circle?”

The Grasshopper file that represents both this problem and the fitness function looks as follows:

We have a collection of target points stored in the red point parameter. Our three variables are represented by the three sliders. The green group contains the two components that create a circle from the three variables and the blue group contains the components that measure the distance (i.e. fitness) from the points to the circle. As you can see I’m actually measuring the square of the distance rather than distance itself. Most fitting operations minimize the square of the distances as that’s a more useful mathematical concept. I explain why this is needed in the Define Fitness and Fitness Pressure blog posts.

The title of this post is “Multiple Local Optima” and it might surprise you to learn that the fitness landscape associated with this problem does indeed have multiple optima. There’s the obvious fitness peak at the {x,y,R} state which represents the best fit, but there’s an additional peak hiding along the edges of the landscape. To see why this is so have a look at the following two cases:

The green curve represents a circle which is obviously far too large to be a good fit. We’d expect a somewhat smaller circle to have a higher fitness, but unfortunately it doesn’t. Or rather, it doesn’t up to a certain size. The much larger red circle is actually more “fit” by our definition than the green one because even though it is farther away from the points in the upper left, it is closer to the points in the lower right. And due to the square root in our fitness definition, a somewhat smaller large distance is to be preferred over a somewhat larger small distance. When we actually plot the real fitness landscape this double peak characteristic becomes visible:

I had to collapse the x and y variables into a single axis called ‘Position’ because it’s difficult to display a four-dimensional landscape on a two-dimensional computer screen, but otherwise this graphic is synonymous with the real fitness landscape. In the lower left hand corner you can see a high peak that corresponds with the ideal circle. However towards the upper right corner there is a gradual slope. This slope never gets anywhere near as high as the green peak, but it does continue to get higher and higher the further you go (in a fashion similar to the ArcTangent function).

When we plot the path of steepest ascent for every point in this landscape and assign a colour to the point based on which peak we found, we end up with two regions; green and red. One leads to the ideal solution, the other leads to the wrong solution. These two regions are called ‘Basins of Attraction’ and every peak has a basin around it by definition*. The larger the basin of attraction the easier it becomes to ‘find’ a peak. Ideally we’d like our solvers to be able to find peaks according to their height, not by the size of their respective basins. So when we ask the Simulated Annealing solver to tackle this problem, we expect to end up at the green peak more often than the red one.

Download the file and find out!

* Actually this is not true for some Fractal landscapes, but then fractal fitness landscapes are near impossible to solve anyway.

In metallurgy, annealing is the process of controlled heating and cooling of metal to achieve certain material properties. At first, the metal is heated up to melting point so it can be cast or formed. At an atomic level, heat is nothing more than particle velocity. The particles (atoms & molecules alike) in a hot substrate move faster than the same particles in a cold substrate. At some point the velocity of two particles will be so high that they cannot succeed in forming a persistent bond between them. When this happens the substrate loses internal structure and turns liquid.

Atoms in liquid metal

Similarly, when a substrate is liquid but starts to cool down, there will come a point where the atoms can form lasting bonds and the internal structure of the substrate is resurrected, turning the liquid into a solid. These two processes are of course colloquially known as “melting” and “freezing”. However it’s not quite as simple as all this. In 1866 James Clerk Maxwell formulated the equations that described the distribution of particle velocities in a gas of constant macro temperature. For example, when you have a litre of helium gas at room-temperature it contains roughly 3 × 10²² atoms and the most probable speed of these atoms is a little over a 1000 meters per second. However there will be a lot of very slow atoms as well as a few much faster ones. This same phenomenon holds for liquids too, although the velocity distributions are not as well defined nor do they cover quite so large a range.

The upshot of all this is that when a substrate is allowed to cool, some atoms will cross the liquid|solid threshold before others. In other words, a substrate doesn’t freeze everywhere at once, small clumps of relatively slow atoms group together and form the freezing ‘seeds’. Especially when we’re talking about metals this is important because when metal atoms freeze, they like to form a regular lattice, or crystal.

Crystal seeds in semi-liquid metal

These small islands of glued together atoms grow over time as more and more slow atoms attach themselves to the seeds, allowing the micro-crystals to expand. When the cooling takes a long time, atoms are allowed to find the optimal (minimal energy) distribution. If on the other hand the cooling is very quick —for example by dumping the hot metal into water— there is no time to form large crystals and the substrate becomes amorphous.

Regular atomic lattice

So how does this help us find solutions in Galapagos? To be sure, the analogy is fairly tenuous but the simple fact remains that treating all the parameters in a problem as an atomic thermodynamic system allows us to find relatively good answers relatively quickly. Basically, with Simulated Annealing, we seek to crystallize the parameters into the lowest energy state.

A typical annealing run consists of a number of successive jumps in the problem phase-space, where the amplitude of each jump and it’s legitimacy are affected by the temperature of the system. When the temperature of the system is high, large jumps are allowed and there is a significant likelihood that a worse answer is adopted despite being a setback. The graph below shows a typical annealing track.

A schematic annealing track

Along the horizontal axis all possible states of the problem are collected. If the problem phase-space is one-dimensional (i.e. only a single Grasshopper slider), then this is an exact representation, but since this is a schematic representation the graph x-axis can represent any number of parameter dimensions. The vertical axis represents the fitness of each distinct state, so the thick black curve represents the entire fitness landscape. This particular landscape has 4 local optima with varying degrees of quality.

The first annealing jump must start at a random location because nothing is known about the landscape so no informed decision can be made. In this particular example, the annealing track starts along the left edge of the landscape at location (1). At this point in time the temperature is very high, so large jumps across phase-space are allowed. A jump from (1) to (2) would indeed qualify as large as it spans almost the entire phase-space. Since (2) is higher than (1), we automatically accept the new solution. Now we need to jump again, but there is less energy available as the entire system is cooling down. So the jump from (2) to (3) is most likely going to be shorter than the jump from (1) to (2). The fitness at (3) happens to be lower than (2), so the second jump actually represents a worse answer. However, since the temperature is still relatively high, we’re sometimes (where “sometimes” is in accordance with thermodynamic stochastics) forced to accept a worse case. This is not as stupid as it sounds though, as this characteristic makes the solver less prone to getting ‘stuck’ to a local optimum.

As time goes on, the temperature of the system drops and smaller and smaller jumps are possible. Also, at low temperatures the chance that a worse case is adopted over a better one becomes insignificant. Eventually the temperature has dropped far enough for the entire system to be frozen, at which point the best answer from the run is cached and a second run is started, once again at high temperature.

Because of the cooling schedule, it means that every annealing run only has a limited number of steps and therefore terminates in a limited time-span. Simulated Evolution does not have this characteristic; it will happily churn away for all eternity without ever looking back. This fundamental difference between the two algorithms makes Simulated Annealing a good candidate for finding many promising optima, while Simulated Evolution is much better at refining a single solution.

I’m not going to reproduce the recipe here as it has already been documented to the point of perfection by Joy the Baker and here’s an alternate recipe by Zoom Yummy.

These pictures were taken three days ago, since then summer has disappeared and has been replaced swiftly and without error by snow.

First, a bit of history. You might be surprised to learn that programming pre-dates computers by about a century and a half. The two are so synonymous these days that it’s hard to imagine a computer without programs or a program without a computer. But machines have been controlled using honest-to-goodness code from the early 1800′s (think of automated looms and pianolas). Most commonly the program was stored on punch cards that were fed into a machine which executed the instructions. But even a hand-cranked music box is an example of programming.

With the advent of the modern (read: electronic) computer in the 1940′s, programming as we understand it today makes its first appearance. Basically a program is a series of instructions. Some instructions pertain to reading or writing bits in memory, others are concerned with computational steps such as addition of two numbers. The main reason computers became so popular is that they are unwaveringly consistent in the application of their programs. You can always trust a computer to do as it’s told*.

Originally computer programs were stored on punch cards using machine code, which was quickly superseded by Assembler type languages. The last 2~3 decades have witnessed an amazing evolution in terms of programming languages and programming platforms. Languages have become both more human-centric and more versatile. And programming platforms like MFC, Cocoa, .NET or Google Apps provide a plethora of useful functions that allow programmers to create new applications in a matter of minutes.

Programs these days are often distributed, meaning the instructions are given to the end-user, whose computer will then execute them. Of course there are exceptions; a lot of programs run server-side, meaning they are executed elsewhere and only the output of the program is pumped to the end-user machine, but in my business the bulk of programs run client-side. There has been an on-going debate about ownership and patents and rights and how they pertain to software and while that discussion is both interesting and frustrating, it is not the focus of this post.

When one writes a piece of software one typically owns copyright on the source code**, just as one would own copyright on any other piece of text. When someone else uses your source code to make a different application they are guilty of copyright infringement, essentially theft. Though I find ‘plagiarizing’ a better parallel. This is not legal, but depending on the code stolen it can be very difficult to detect, let alone prove. This is the first legal problem I want to highlight.

It is also usually up to you whether or not you want to sell your software***. If you choose to sell your software, only people who have paid you money are allowed to use it. If someone is using your software without payment or explicit permission, they are breaking the law and are liable to prosecution. This is also often referred to as theft and it is the second legal problem pertinent to the topic at hand.

Both issues outlined above have the potential to reduce income for the original developer, which is why sentiments run so high over this. A good case can be made that illegal copies have positive as well as negative consequences for the author, but that, too, is not the focus of this post. It is not difficult to see that access to source code is the single most valuable asset to anyone looking to infringe copyright or circumvent license agreements. When one can read source code, algorithms can be extracted and license protection can be punctured.

Up until recently practically all large scale applications were distributed to users as machine code instructions. Source code was written by the developer on her computer, then turned into a collection of assembly-level calls by her compiler. The result is a set of instructions that can be executed very efficiently by the client machine since it is stripped from all human-centric content. Basically a compiler is a translator that converts code written by humans into instructions readable by machines. Compilers often use tricks to increase performance or reduce memory usage, so instead of just removing all superfluous content they also modify the logic of source code. Compiling is quite difficult and those who write compilers are often lionized by their fellow programmers. The opposite of a compiler is a decompiler, which takes machine instructions and translates it back into a human-readable language. This process is even more difficult as a lot of the ‘squishy’ stuff needs to be put back in.

I said “up until recently” because when you write .NET code this is not what happens. .NET programs are no longer fully compiled by the developer and then distributed to users. Rather, the compilation process has been broken up into two parts, one to be performed by the developer machine and the other by the client machine. Text-based, human-readable source code is converted to a language called IL (Intermediate Language). This process validates the code (which is a time consuming process) and turns it into a form which is more easily parsable by computers. You can think of it as binary source instead of textual source if you want. It is this binary code that is given to users and only when they run the program does it get fully compiled to machine code. There are a lot of really big benefits to this approach, but there’s also a down-side; it makes it much easier to decompile the program into high-quality source. This worries a fair few developers for reasons that are almost always, in my opinion, misguided.

There is a process called obfuscation which changes compiled (or semi-compiled in the case of .NET) code and in so doing makes it harder for a decompiler to make sense of the instructions. This is a very effective defense against the casual ‘code thief’ as it can fool most commercial decompilers. Although I agree that there do exist valid reasons in favour of obfuscating code, these reasons are few and far between.

My main objection is an idealogical one. It seems to me obfuscation is often used because “the competition is catching up”. Competition is always catching up, that’s its job. You won’t defeat the competition by flailing around and yelling “Mine!”. The only way to get on top is to write better code than the competition and the only way to stay on top is to keep writing better code than the competition (yes, I know there are marketing and legal departments involved as well, I concede the world is not quite as neat as portrayed here). Time and effort spend securing code can no longer be spend improving code. This applies to obfuscation, but also to elaborate licensing mechanisms such as on-line validation and (shiver) dongles.

Another objection is that obfuscation removes any possibility of useful exploration. “Useful” to legit users and authors that is. It can allow power-users to track down bugs and come up with work-arounds without the need for the official developer to fix, re-compile and re-release. It helps plug-in developers to see what’s going on when they rely on your code. It helps other programmers to see what’s going on when you run into problems and need help. These are real benefits and you might be sacrificing them in favour of imaginary ones when you decide to obfuscate.

Source: xkcd comics

* With the exception of hard-ware failures such as corrupt storage of course.
** I am not a lawyer and this is not legal advice.
*** Again, I am not a lawyer. If you make legal decisions based on this post I will sue you.

We can simplify this by choosing sensible dimensions for both the boundary and the winning region. Let’s say the boundary is a square with an edge length of one units, while the winning region is a circle with a radius of half a unit. This means the boundary has an area of 1 units², while the winning region has an area of pi × 0.5²  ≈ 0.785 units². Thus the chances of making a winning pick are roughly 80% in this simplified case.

But what are our chances if we’re supposed to pick a single point in three-dimensional space? Instead of dividing areas, we must now divide volumes. And instead of a square with a circle, we’re now dealing with a cube and a sphere.

Now, the volume of a unit cube is always 1.0, no matter how high the dimensionality. This is of course why it is a sensible value. The volume of a sphere in three dimensions is defined as (4/3) × pi × r³ which for a radius of 0.5 roughly equals 0.524 units³. So the volume of a sphere is much smaller compared to the volume of the containing cube than is the area of a circle compared to the area of the containing square.  In other words, there’s a lot more left-over space. What happens if we go even higher than 3 dimensions?

This:

Here you see for the first 12 spatial dimensions the ratio of volume of the (hyper)sphere* as a percentage of the volume of the (hyper)cube. With every additional dimension, the ratio is nearly halved. The tilted numbers next to the graph represent the volume of the hyper-sphere rounded to three decimal places and it doesn’t take many dimensions for the hyper-sphere to become utterly insignificant compared to the hyper-cube. This is probably somewhat counter-intuitive as the cube is still a tight fit around the sphere, but then things tend to get counter-intuitive with gay alacrity when more than 3 dimensions are involved.

The purpose of this post is not to tell you that you should pick a lower-dimensional reality from your opponents when playing Pin the Tail on the Donkey. Rather, Pin the Tail on the Donkey is an excellent analogy for computational random search algorithms. You don’t have to blindfold and spin computers, being ignorant and disoriented is their default state. It is actually removing the blindfold that typically requires an awful lot of difficult programming.

When an Evolutionary Solver (such as Galapagos) starts, it has absolutely nothing to go on. Eventually new generations are created from previous generations using a combination of breeding and selection and mutation algorithms, but the first generation has to be magicked into existence out of thin air. A common strategy is to completely randomize the individuals in the first generation, basically pinning tails onto the hyper-donkey. Since the dimensionality of the search-space equals the number of variables, there is a vanishing —indeed, rapidly vanishing— chance that any individual in Generation 1 is going to get lucky.

Thankfully only the first generation suffers from this problem, as subsequent generations use a far more directed (as opposed to random) search, but it is important that you allow for sufficient individuals in Generation 1, so that the chances of getting immediately stuck in a local optimum are small. This is why Galapagos has an Initial Boost factor setting, which multiplies the number of individuals in Generation 1.

* The formulas for hyper-sphere volumes are:

R²  → π · r²
R³  → (4/3) · π · r³
R⁴  → (1/2) · π² · r⁴
R⁵  → (8/15) · π² · r⁵
R⁶  → (1/6) · π³ · r⁶
R⁷  → (16/105) · π³ · r⁷
R⁸  → (32/945) · π⁴ · r⁸
R⁹  → (1/24) · π⁴ · r⁹
R¹⁰  → (1/120) · π⁵ · r¹⁰
R¹¹  → (64/10395) · π⁵ · r¹¹
R¹²  → (1/720) · π⁶ · r¹²

source: John Berkey

These are of course all examples of Science Fiction made for the silver screen, print has produced far more worthwhile SF to date. I can count the number of good SF movies I’ve seen without risking double digits, while the number of good SF authors alone runs in the dozens. So what makes good Science Fiction? Well, since the term is composed of two words, let’s tackle them in order.

Good science. We know an awful lot about an awful lot four centuries after the start of the Scientific Revolution. Sure, we also know that what we know is only a very small part of all the things we could possibly know and we know that some of the things we know are wrong or at least partially incomplete. The science part of any SF story does not need to adhere to what we currently know, I’m not arguing that at all. But being blatantly at odds with what we know today requires at the very least a decent explanation and/or justification. Faster-Than-Light travel for example is a common theme in many SF sagas, and it is a perfectly valid fictional concept. Though I have to admit that the best movies and books (in my humble opinion) do not break this tenet of current scientific orthodoxy. It is not beyond imagining that one day we’ll figure out that it is possible to beat those darn photons.

A lovely example of being blatantly at odds with what we know is the common idea in earlier SF works that a gas lighter than Hydrogen could be used to solve all the lifting and travel needs of humanity. One author (who shall remain unnamed, I’m not on a vendetta here) even went as far as to suggest that the higher the pressure of this magical gas, the more lift it generated!

Greg Bear is an example of an author who is exceptionally good at the hard science part of SF. The science in his work tends to be novel and unexpected, yet almost never unbelievable. In Eon he describes a group of humans who have left Earth in search of another planet. They travel there inside a hollowed out asteroid where they’ve build cities and landscapes against the rotating inner walls. Since it takes such a long time to get to their destination (generations, centuries) their own grasp of technology advances as they travel, enabling them eventually to construct a secondary universe inside the asteroid. Take that magical gas! There’s a lot more to the story of course, but you’ll have to read it to find out, and I highly recommend you do. Moving Mars and Blood Music are other examples of Greg Bear books that shatter all of today’s scientific understanding but do so in a highly believable fashion.

Good fiction. By this I don’t mean good story telling. Obviously that is an important factor in any book or movie in any genre. By ‘good fiction’ I mean that the created reality is internally consistent. It does not need to be realistic, just believable. The society we live in today is profoundly affected by scientific and technological advances. We are not just medieval people with cell-phones and fire-arms. The way we spend our days, the way we see the world and even ourselves is in large part due to the shifting scientific perspective. So when one introduces new science into an imaginary world, one must also take the indirect effects into account. This is one of my main complaints with Star Trek; for all this cool new technology (warp drives, transporters, universal translators) the humans (and even the aliens!) are basically the same as we are today.

Isaac Asimov was a master at making believable societies. The science employed in his stories was rarely explained in great detail, but the effects of that science on societies at large were always brilliant. The same goes for many Arthur C. Clarke stories.

source: 2001, A Space Odyssey

I mention all this because I just watched The American Astronaut by Cory McAbee. If you haven’t seen it, I’m not surprised. It’s particularly difficult to get a hold of. As far as I know there are no DVDs on sale for Region 2. The American Astronaut is a fantastically clever and well made movie and it’s the first time I’ve been left in confusion as to whether I consider this Science Fiction or not. The story itself is wonderfully ludicrous: Samuel Curtis —an inter-planetary trucker and roughneck— is on a mission to get rich. In order to achieve this he must return to Earth with the mortal remains of Johnny R, which are currently being held hostage on the planet Venus —the planet of beautiful women— until a suitable replacement breeding male is delivered. This new boy can be found on Jupiter —a mining planet inhabited by men only— where he is currently employed as a morale booster since he’s the only one who has ever seen a women’s breast. The man in charge of Jupiter will trade the boy for a ‘Real Live Girl’, who is apparently still in the process of being cloned as we never see her, but only the suitcase in which she grows. Through it all, Curtis is hunted by Professor Hess, who wants to kill him. Professor Hess is a psychopathic murderer who kills for no reason. But the catch is he only kills for no reason. If he’s got a reason to kill you, he won’t.

The movie is littered with musical pieces (the director and author is a professional musician) that only add to the bizarre atmosphere. It also contains the world longest bad joke, which is surprisingly funny because the audience consistently manages to laugh at the inappropriate times.

In summary; ten years ago, a crack unit of creative artists made a surrealistic sci-fi/western/musical movie. If you want to watch something that pushes the boundaries of weird, if no-one else can help, and if you can find it, maybe you can watch… The American Astronaut.

From a physics point of view colour is usually approached from the photon-wave-length direction. Every photon has a wave-length and some of these wave-lengths translate into visible light. Lasers tend to emit light of only a single wavelength (mono chromatic), but most natural light sources emit a continuous range of wavelengths. It’s easy enough to predict the result when mixing two light sources of known wavelength and intensity, but there’s a difference between the harsh numeric wavelength approach and the soft human biological approach. Just because you know the exact mixture of wavelengths in a certain light beam doesn’t mean you know how that colour will be experienced by living creatures. The rods and cones in our eyes do not measure all wavelengths equally well. The following graph shows how, on average, human cones respond to photons of specific wave-lengths:

The three curves each represent one of the types of cones humans typically have. They’re usually referred to as Blue, Green and Red cones, though it’s clear from the graph that it’s actually closer to Blue, Yellow and Red. When photons with a wavelength of 400~500 fall on our retinas, it’s mostly the blue cones that fire, for larger wavelengths it’s a ratio of Green and Red cones. Anything over 700 is ‘too red’ for us to see (infrared), as is everything under 400 (ultra-violet). Note that a larger wave-length equals less energy, hence ‘infra’ for red and ‘ultra’ for violet instead of the other way around.

Then there’s the natural variation in the population at large, Rhodopsin after all is subject to evolutionary forces, so we expect this graph to differ from person to person. Also a lot of people are colour blind and there are reports of women who have a fourth kind of cone, making them tetrachromats.

But physics (specifically optics) tells us that colour can also be the result of processes other than photon-pigment interaction. Structural colours for example arise due to quantum effects when light is reflected from thin layers. We see this effect in a lot of bird-feathers, which makes them appear somewhat metallic. Then there’s good old fashioned dispersion due to the index of refraction, as first properly investigated by Newton. All in all biology has found seven ways to generate colour, all of which are treated in Andrew Parker’s excellent book ‘Seven Deadly Colours‘, a worthwhile read.

(source: Hipgnosis Design)

But by far the most complex facet of colours is the cultural/linguistic aspect. How people experience and recognize colour is in large parts a function of their own personal history. Not only does colour perception differ between cultures, it also differs from time to time (which is of course not that surprising, British culture today is not the same as British culture a century ago). Pink for example wasn’t always a colour. Sometime in the 14th century the verb to pink changed meaning and is today exclusively used to identify photons of a specific wave-length and extrovert singer-songwriters. There is no equivalent word in Dutch or German or French or Slavic languages for pink, this particular shade is called ‘roze’ or ‘rose’ or some other variation. (Finnish does have ‘pinkki’, but it’s a loan word. Anyone who’s ever been to Finland will tell you that particular hue is not present anywhere in the country, which explains why they had to borrow it from elsewhere).

Whether or not you’ll be able to distinguish Indigo as a colour distinct from Purple depends on large parts on whether you’re familiar with Indigo. It is not a function of our cones or nerves, but rather our memory. The popular web-comic author xkcd performed a large scale colour survey where people were asked to name a random colour. The results of which he graciously published for anyone to play with.

Which brings us finally to the relationship between wavelength and word, which is what this blog post is about. Most modern computer screens are able to display 16,777,216 unique colours (256 variations for red, green & blue each), but we are only able to distinguish a few as truly distinct colours. This means there is a very limited amount of colour names needed to identify any combination of red, green & blue. Or, put another way, it is very easy to write an algorithm that matches a name to any given colour. All you need is a database of a few dozen to maybe a few hundred colour|name combinations and a way to measure distances between two colours. xkcd provided such a map of the fully saturated faces of the RGB colour cube with the regions that fall within a specific colour name:

(source: xkcd)

But what about going the other way? What if we want to write an algorithm that takes a textual description of a colour and returns an RGB value? Now we’re in trouble. Sure, we can just invert the aforementioned algorithm and whenever people happen to type a colour name like ‘Blue’ or ‘Pink’ or ‘Brown’ we can return a representative hue of that particular colour. But whenever we’re confronted with something like ‘Darkish Teal’ or ‘Pinkish Yellow’ or ‘Boring Maroon’ there’s nothing we can do. If we’re serious about writing an algorithm that can ‘reverse-engineer’ colours from colour names, we first need a large database of colour names. There are several official palettes that define a collection of named colours such as the X11 table, or the Pantone Space. Then there’s a large database of human-centric colour names as collected by the xkcd Colour Survey. When we put it all together we get the large image below. A lot of the entries are duplicates though since this table was created using several sources. Then there’s a large abundance of colours that we would ideally like to be able to generate directly with an algorithm such as ‘Dark Green’ or ‘Yellow Green’. When these unnecessary and redundant entries are filtered out, we end up with the thin images on the right:

(click for large versions)

If we map the reduced palette to an RGB cube, we get the following distribution:

As is already visible, the cube does not have uniform density. This is not terribly surprising, as I can think of a lot of names for whitish colours, but only a few for purplish colours. What I did find surprising is the exact shape of the density distribution. We can remove the most isolated colours from the set (those whose distance to the nearest other colour in the set is largest) and thus highlight the boundary between high and low density areas. The following two images are two different views of the entire set with the 50% most isolated colours removed:

As these images show, the bulk of colour names cluster around the black-white diagonal. We basically have lots of words for colours that are very greyish, but few words for colours that are saturated. A possible explanation for this would be that we experience colours in an imaginary context of nearby colours. After all, if you start to modify Barbie Pink, it takes a lot of modification to really get away from that specific colour, whereas even a little change to a very faint pink will already push it to a different hue. But this is pure conjecture, I have no evidence for this hypothesis.

Next up, how to parse compound colour names and end up with a decent result.

Imagine we want to divide a certain floor area into a fixed number of equally large spaces. Not equally shaped, merely equal surface area. So if the total floor area equals 100 m² and we want to create two rooms, they each need to be 50 m². Of course there is an infinite number of ways in which we could divide these spaces and end up with a perfect solution, but even if the system we’re solving only allows for one perfect solution (or maybe even none!), the ideal Fitness Function would still look the same.

There are a number of approaches that spring immediately to mind, each with its own benefits and drawbacks:

1. Compute the difference between the individual sub-space areas.
2. Compute the difference between each sub-space and the equal portion of the total area.

If we want both sub-spaces to have the same size, we can simply minimize the difference in area. This is an extremely simple Fitness Function that always converges on the correct answer and which always ends up at zero when the solution is ideal. The function and the associated fitness graph look like:

Abs(A – B)

The problem with this approach though is that it doesn’t scale very well to a more complex problem. What if we wanted to insert 3 sub-spaces into the master space? Then the fitness function will have to be expanded to:

Abs(A – B) + Abs(A – C) + Abs(B – C)

And the complexity of this function grows exponentially with the number of sub-spaces. For 5 subspaces we’re already looking at:

Abs(A – B) + Abs(A – C) + Abs(A – D) + Abs(A – E) +

Abs(B – C) + Abs(B – D) + Abs(B – E) +

Abs(C – D) + Abs(C – E) +

Abs(D – E)

which is clearly getting out of hand. So what about option #2? What if we were to compute the difference between each sub-space area and the desired area? We now have a function which grows in a linear fashion. In fact we can write it as a simple sequence:

Abs(A – A_D) + Abs(B – A_D) + … + Abs(Z – A_D)

Where A_D is the Desired Area, basically the area of the master space divided by the number of sub-spaces. At first glance this function appears to behave just as well as the first one, but there’s a problem with selection pressure. The pressure of  fitness graph is an indication of how strongly it pushes the Solver into a specific direction. The higher the pressure, the quicker a solver will run across the fitness landscape. A very high pressure is not necessarily a good thing as it tends to reduce the diversity of the gene-pool. All individuals are channelled into a narrow path from here to the summit of the fitness gradient. A very low pressure can also be problematic as it allows the population to spread out, and the random drift starts to counteract the progress of the solver. A pressure of zero is definitely very bad as the Solver doesn’t know in what direction to progress. The Fitness Function as written above, has zero pressure under certain circumstances. For example, imagine we have three rooms, each of which has an ideal area of 30 m². However, we are at a stage in the solution where this ideal is very far off. Room C is far too big (50 m²) whereas Rooms A & B are both too small (20 m² each). If we insert these values into the Fitness Function, we end up with:

Abs(20 – 30) + Abs(20 – 30) + Abs(50 – 30) = 40

Now the Solver mutates or recombines the genes and it comes up with a different answer, namely:

Abs(25 – 30) + Abs(15 – 30) + Abs(50 – 30) = 40

Even though the result is different, the fitness is the same. Equal fitness means equal worth means the Solver doesn’t know where to go from here. We need to decide which of these two cases better suits our goals and adjust the Fitness Function to reflect this set of preferences. Let’s assume for the time being that we prefer the former case, as it better represents our ultimate goal of identically sized rooms. How can we convert this preference into maths?

The easiest solution is to add penalty clauses, just like in the previous post. But instead of dampening the Fitness factors with a square root, we need to amplify them. The larger the departure from the ideal area, the more severe the penalty should be. We can amplify the components of the Fitness Function by squaring them, giving us:

Abs(A – A_D)² + Abs(B - A_D)² + … + Abs(Z - A_D)²

If we now insert the areas of the two cases we get different results:

Abs(20 – 30)² + Abs(20 – 30)² + Abs(50 – 30)² = 600

Abs(25 – 30)² + Abs(15 – 30)² + Abs(50 – 30)² = 650

with the former case giving a lower fitness value, which is exactly what we want since we’re trying to minimize this value.