Heuristics or Algorithms: Confused?

The Editor of the New Scientist (Vol. 3176, 5 May 2018, Letters, p54) opined in response to Adrian Bowyer ‘swish to distinguish between ‘heuristics’ and ‘algorithms’ in AI that:

This distinction is no longer widely made by practitioners of the craft, and we have to follow language as it is used, even when it loses precision.

Sadly, I have to accept that AI folk tend to consistently fail to respect a widely held distinction, but it seems odd that their failure has led to an obligation on the New Scientist – which has a much broader readership than just AI folk. I would agree that in addressing audiences that include significant sectors that fail to make some distinction, we need to be aware of the fact, but if the distinction is relevant – as Bowyer argues, surely we should explain it.

According to the freedictionary:

Heuristic: adj 1. Of or relating to a usually speculative formulation serving as a guide in the investigation or solution of a problem.

Algorithm: n: A finite set of unambiguous instructions that, given some set of initial conditions, can be performed in a prescribed sequence to achieve a certain goal and that has a recognizable set of end conditions.

It even also this quote:

heuristic: of or relating to or using a general formulation that serves to guide investigation  algorithmic – of or relating to or having the characteristics of an algorithm.

But perhaps this is not clear?

AI practitioners routinely apply algorithms as heuristics in the same way that a bridge designer may routinely use a computer program. We might reasonably regard a bridge-designing app as good if it correctly implements best practice in  bridge-building, but this is not to say that a bridge designed using it would necessarily be safe, particularly if it is has significant novelties (as in London’s wobbly bridge).

Thus any app (or other process) has two sides: as an algorithm and as a heuristic. As an algorithm we ask if it meets its concrete goals. As a heuristic we ask if it solves a real-world problem. Thus a process for identifying some kind of undesirable would be regarded as good algorithmically if it conformed to our idea of the undesirables, but may still be poor heuristically. In particular, good AI would seem to depend on someone understand at least the factors involved in the problem. This may not always be the case, no matter how ‘mathematically sophisticated’ the algorithms involved.

Perhaps you could improve on this attempted explanation?

Dave Marsay

Mathematician with an interest in avoiding 'bad' reasoning.

2 Responses to Heuristics or Algorithms: Confused?

1. davetaylor1 says:

Wikipedia says:

“Al-Khwarizmi was the most widely read mathematician in Europe in the late Middle Ages, primarily through another of his books, the Algebra. In late medieval Latin, algorismus, English ‘algorism’, the corruption of his name, simply meant the “decimal number system”.

I worked with the team who brought the word ‘algorithm’ into software engineering via the programming languages Algol60 and later Algol68. As against the ‘freedictionary’ definition, they intended it for a small and unambiguous set of instructions which can be REPEATEDLY used to produce a result, as in the construction of a decimal number by shifts enabling repeated use of the same numerals. This in the context of constructing complex programs algorithmically (and thus systematically, intelligibly and checkably) as far as possible in terms of nested algorithmic loops.

• Dave Marsay says:

In the freedictionary definition the ‘goal’ could be ‘to bamboozle people’, but I agree with you (and your account) that it would be better to reserve the term for situations where it can be proven that the algorithm necessarily meets the goal, which has an unambiguous technical (‘logical’) specification.

Thus a piece of code may be an algorithm when used correctly, a heuristic when misused. An important distinction, similar to Keynes’ distinction between proper science and pseudo-science. Maybe we should call things that look like algorithms but which appear not to be ‘pseudo-algorithms’?

This site uses Akismet to reduce spam. Learn how your comment data is processed.