So far, we were mostly concerned with the functional aspects of Objective Caml. Now, when interacting with the real world, it is in a certain sense inevitable to deal with state changes, input and output, and similar issues. We will also use the opportunity to do a bit of cleanup work and talk about other features that turn out to be very useful for real world applications, such as exceptions and exception handling, named and optional arguments, and more. Indeed, we by now know - but actually for less than ten years - that there indeed is a way to formalize all these "dirty" aspects of programming in a purely functional way even though the principle that same expressions have same values and may be substituted at will seems to be violated badly. Haskell is the prime example for a language where everything, including I/O, has to be done in a purely functional way.
But as we are not so much interested in on-going programming language research, but rather in getting some real work done, we do not have to care about such lofty thoughts. Presumably everybody in the audience has had some prior exposure to programming in one of the widespread languages other than Haskell which more or less all "get it quite wrong". So, we will not spend any time discussing the model of a register machine with random access memory here, but take for granted that that is known.
There are various forms of iteration. In its purest form, it appears in the guise of inductive mathematical definitions. Let us look at one such definition which we have seen in the last lesson (at least, something very very similar to it): taking the length of a list:
|The naive recursive form of a list length function|
This may seem to be the obvious way to define list length. However, there is a slight problem, which can be seen here:
|Recursion depth limits|
Of course, it is quite ridiculous that such a simple function as taking a list's length should cease to work when the list gets too long! Let me first demonstrate that indeed, there are other ways how to define a list length function that does not suffer from this problem, and discuss what's going on afterwards:
|list length function: an improved implementation|
There are various ways how to understand the difference between these implementations. On the purely conceptual level, it is interesting to compare what the first and second variants do, step by step.
Let us consider
my_list_len_v1 first. In every step
(except the last), we pass on the work of resolving a sub-problem to
another function (which is just again the function
my_list_len_v1, but that does not matter). What is
important now is that once we know the result of that sub-problem,
this has to be post-processed - by adding one - before we can return
our result. This means that, in every recursive call, we have to
remember that we will have to come back to this context after the
recursion finished, and do some more work. Note that this is not how
one normally would execute a task like counting the elements of a list
by hand. One just does not think along the lines of: "the length
of this list is one more than... oh, there is another list, and the
length of that is one more than... (...) and the length of the empty
list is zero; plus one, that is one, plus one, that is two, plus one,
that is three, plus one makes four." That is about as silly for
the machine as it is for us.
How to do it better, then? One just models the process one would
execute in one's head. When I go through a list to count its entries,
I remember at every step how many elements I counted so far, and as
soon as I reach the end of the list, I already know the answer. This
is just how
my_list_len_v2 works. Note that, unlike with
the first definition, the length of the intermediate expressions this
reduces to (which are effectively represented on the call stack of the
machine) does not grow as we proceed, and in particular does not
depend on the length of the list.
What is the trick, schematically? We use an extra "accumulator
argument" in which we pass along the result of processing the list so
far. As we usually do not want to make this extra accumulator argument
visible to the outside, we will often employ a local recursive
function to do all the work. Note that this is a very general
strategy. In the particular case of lists, quite many of the
situations where one would like to process elements one by one can be
classified as belonging to a set of frequently occurring patterns, and
there are pre-defined functions to deal with these. It is very
tempting to just give a definition of one of them which is called
reduce in many languages, but goes by the name of
List.fold_left in OCaml. The idea is to take a list, a
start value, and a function, go through the list element by element,
and in every step use the function to combine the value obtained so
far with the element at hand.
|Definition and applications of |
One conventionally calls such a repetitive sequence of operations
an "iterative process". Here, we use recursive functions to model
iterative processes, and the compiler is clever enough to turn this
into purely iterative code. From the perspective of a compiler writer,
one may say that we call a function by putting its arguments on some
call stack, and jumping to the code that computes the function's value
from those arguments on the stack. Now, one would normally want the
called function to return back to where it was called, and virtually
all processors today provide distinct "
jump to subroutine" machine instructions, where the
latter records the return address on the call stack. It is
important to realize, however, that this return address is nothing
else but an extra argument to the function that is called. What
is this argument? It tells the called function whom it should call
with its computed value. So, by putting our return address onto
the stack, we actually pass something like a function argument that
tells the called function how to continue after its work is done. This
holds for every programming language that knows about subroutine
calls. Generally, these go-on functions are called
continuations. Note that in languages such as C, a
continuation (call it a return address if you want) behaves like a
code pointer, but there would be no ANSI-compliant way to obtain, pass
around, or play with that kind of function pointers, with the
exception of the
This continuation point of view is very powerful. Returning a value from a function just becomes calling the continuation of that function with this value. This gives a nicely symmetric description where calling with a value and returning a value becomes just the same: a "goto with arguments". This also sheds a new light on a very basic observation: in many programming languages, such as C, functions can have zero, one, or more arguments, but they can only return zero or one value. Now, the point is: if we call a function whose result is just our result as well, we do not have to put a new continuation on the call stack, as it is perfectly fine if the function which we call passes on its result to our own continuation. In other words: just forget about calls and returns being nicely nested. There is also jumps and returns to places from where we were not called.
Sometimes, it is useful to partially transform a certain function to continuation-passing style by replacing the returning of a value with an explicit call to a function that represents the continuation. Things get much more interesting, however, if the programming language allows us to get a handle on "that which is to be done afterwards" in the form of an explicit continuation that can be stored away, passed around, and called, even multiple times. This allows one to implement quite crazy high wizardry which we unfortunately have no opportunity to discuss here. Making continuations explicit is one of the fundamental ideas underlying the Scheme programming language, a dialect of Lisp. Some other languages can do this as well, such as Smalltalk, and there is even a variant of the python interpreter that has advanced continuation support, see href="http://www.stackless.com
A more pedestrian point of view on that matter is that actually,
there is a simple but nice optimization: whenever the last thing
that has to be done inside a function before it returns a result is to
compute the value of another function (which is to be returned), that
inner function can be called with a simple
jump to subroutine, so that the return from the
inner function will be the return from the outer function as well.
All modern compilers are supposed to perform that particular "tail call elimination" optimization where appropriate. It is very interesting to take a closer look at this using a LISP system:
|List length in Common LISP (CMUCL)|
Note that quite many all Common LISP compilers are combined
compiler/interpreter systems, which means that one can compile LISP
code directly to machine code even from the command prompt. Note that
we cannot do this with OCaml. The set of instructions marked in red is
the inner loop where the counting happens. Note that this is about as
optimized as machine code can be. (There would be more to say why we
add four in the iteration, and not one, to increase the variable
now by one. Remember that OCaml's integers are a bit
shorter than native machine integers? The same holds for LISP, and
this is related to this issue here.)
So, there is really actually never a need for something like a
for loop in virtually any language. The ability to call
functions recursively and a properly implemented compiler are good
enough. Most languages nevertheless provide this and similar control
statements. In some of them, they are just macros that expand to such
"tail recursive" definitions. If there are extra iteration constructs
besides what we get through recursion alone, one should quite in
general use the approach that allows one to express the underlying
idea in the clearer way. Tail recursiveness generally is a good thing,
but one should be extra careful with lazy functional languages, such
as Haskell, where things behave quite different anyway due to the
wildly different execution model.
One of the key ideas of functional programming is to express more directly what a program is supposed to do, rather than how that what the program is supposed to do may be decomposed into a sequence of instructions to be carried out one by one. Therefore, when one learns to work with a functional programming language, it usually is not a good idea to dive into extensively using imperative features too early and carry over the bad habits from more primitive imperative languages which never were designed with that form of code abstraction in mind which the functional style allows.
On the other hand, many programs - even quite complex ones - usually contain a small core where execution time really counts. If the corresponding manipulations involve large but simple data structures, such as arrays, it often is a good idea not to create a new one in every step, but to destructively modify an existing one if possible. Even in those situations where it may not matter for the complexity of the algorithm, having to allocate a new array may be a bit wasteful, especially if this means that we once again have to transport a lot of data into the processor's data cache when we start to work on it.
So, even with the niceties of a functional language around, we may prefer not to do everything in a purely functional style. But how, then, do we modify values, and more importantly, if we have done something, how do we do something else afterwards?
One might claim that in a certain sense, we already know a bit about this. Presumably, nobody would be surprised by this particular function definition:
|Measuring execution times of functions|
In a certain sense, this provides kind of an answer to "how do I do
one thing after another". If one approached this question naively, one
may wonder whether the compiler may detect that the definition of
end_time seemingly does not interfere with the definition
result, and so execution order may be re-arranged to
the fastest possible form. Actually, OCaml guarantees that this is not
done: in a sequence of
let definitions, the first one is
evaluated before the second one, etc.
We may use this e.g. in the following way in order to do one thing after another:
|A sequence of printing instructions|
What if a function which is called purely for side effect returns
something else but
()? We may either use
let _ =
... instead, or alternatively use the
ignore: 'a -> unit" function.
There are other ways to do the same. Sometimes, the previous form
is the most elegant one, sometimes it is another one. OCaml provides
slightly different ways to notate a bare sequence of instructions;
which one to use is mostly a matter of taste. For clarity, I strongly
recommend the following alternative style should the
form be inconvenient:
|Another sequence of printing instructions|
All the entries in such a
begin ... end
semicolon-separated sequence should have type
except the last one, which will be the value of the entire
sequence. The semicolon after the last value is optional. Actually,
ignore function is much more useful here than in a
Aside from these, there are some explicit iteration functions, such as:
|Array iteration with index counting|
Now that we have seen how to do something destructive and then proceed with other work, how do we update array entries? This presumably will be the most frequent destructive operation we perform.
There are various approaches to this. A widely supported philosophy is that all access to modifiable state should run through functions (or methods, for that matter) for the sake of extensibility and modifiability of a program. Some C++ programmers know this very well. So do the Perl programmers, who introduced the concept of "magic": for any variable, one may register functions which are to be called when the contents of that variable are being read or updated, hence "slipping in" custom code in situations where something that actually required custom code originally was designed in a too simple minded fashion as just a variable access.
For the followers of the "everything is a function" school, there
is the "
Array.get" and "
functions. For those who prefer special syntax and brevity, there is
array.(n)" and left-arrow assignment:
|Getting and setting Array elements|
For strings, this is very similar, only that one uses
|Getting and setting String characters|
One may be tempted to abuse strings, which implement 8-bit arrays,
as an alternative to specialized vectors of small data, such as
boolean vectors, or vectors of numbers smaller than 10. In most such
situations, however, one should be better off using the
Bigarray module which provides specialized functions for
potentially large numerical data arrays of all sorts of integer and
floatingpoint numbers. These are intended to be especially useful in
conjunction with C and FORTRAN code, which means in particular that
garbage collection will never move around the data portion of a
In order to use
Bigarray, one must either first build a
custom interpreter that knows about the
ocamlmktop, or use an interpreter directive
to load it dynamically, or - when using the compiler - additionally
bigarray.cmxa among the compiler
Bigarray provides generic arrays as well as
specialized ones for rank 1,2, and 3. One interesting aspect is that
the data types in the
Bigarray may differ from OCaml's
native data types. For example, one may want to use this module to
read an array of single precision floatingpoint numbers generated by
some C code, but OCaml only knows about double precision. This is
resolved through some type mapping: although only single precision
values are stored in the array, what one reads and writes is double
precision, which is converted up from, or down to, single precision as
Record entries normally are not modifiable. Modifiability can be specified on a per-field basis when defining the record type.
|Mutable record entries|
While we can modify the contents of some containers, we cannot modify variables. Unlike in many other languages, variables are just names for values, and not containers where to store values. Of course, it sometimes would be nice if such a concept like "container for an integer which I can update" were available nevertheless. One may consider using a polymorphic record for this. Actually, OCaml already provides something quite like this: references.
|Polymorphic records as containers|
It may be very tempting to just use lots of references to mimic
some imperative coding style with OCaml. This is possible, but quite
often, there are better ways to achieve the same result. In this
particular example, one should prefer using
Always remember: never use imperative features if there is a better functional way to achieve the same. Actually, never use X when it can be done better using Y, whatever X and Y.
A very useful data structure is the hash, also known as "dictionary" in some languages. One may see this as a generalization of arrays: arrays map small numbers - indices - to values. Hashes implement a mapping of more general keys to values. Both provide modifiable associations that may be updated.
As hashes internally use some pseudo-randomness, only statistical statements may be made about their performance. On average, element lookup takes O(1) time, element insertion amortized O(log N) time, and deletion also O(1) time, as hashes will not shrink again if we take out elements. If, however, one hashes data from an external source, such as URLs from a webserver log, one must consider the question whether the external source may be modified in such a way that it targets worst-case behaviour of the hash. This effectively degrades hashes to associative lists, with average O(N) lookup and O(N) insertion. There have been attacks based on such techniques in the past, even on the Linux kernel. One solution is to use more robust balanced tree data structures which may be a bit slower, but can give worst case guarantees. (The Perl solution, by the way, is to randomize the internal hash function at every program start, effectively giving nondeterministic and irreproducible behaviour whenever there is any implicit dependency on the order of hash keys in a program. Inherent irreproducibility in a language will eventually turn out not to be a good idea, but only if enough people notice the problems.)
Hash keys may be almost arbitrary values. This is the major difference to Perl, where they are restricted to being strings. Python hackers may wonder whether only immutable types (that is, tuples and immutable records) are allowed as keys: this is not the case: one may just as well use strings or arrays as hash keys, but modifying a value that is used as a key in a hash will have quite bad effects on the hash, so one never should do this. Lisp hackers may be disappointed to hear that OCaml does not come with different hashes which can test keys not only for equality, but also for same-ness. So, it makes little sense trying to use a function as a hash key.
|Hash table example|
Note that there may be more than one value associated to a given
Hashtbl.add will return the last binding made, which
effectively hides the others. If new entries are to be made in a way
that overwrites previous bindings, one should use
Hashtbl.replace. This may seem a bit mysterious, unless
one knows how hash tables usually are implemented. There is much more
to be said about hash tables, this lecture is just supposed to
introduce the underlying idea. For details on hash table functions,
one should consult the
Hashtbl module documentation.
OCaml also comes with syntactic constructs for iteration that
for" and "
while do". One has to be
careful here, however, as
for has funny ideas about index
for range includes both the lower and the upper
limit, one is perhaps well advised to adopt the habit of
always using the idiom
for var=lower to upper-1 do".
Just like many modern programming languages, OCaml comes with means
to implement exceptional non-local execution flow. As in other
languages, this is intended to be used to report exceptional
situations that occur deep in the code to the first caller in the call
hierarchy that considers himself responsible to handle this. We
already have used exceptions, but only in their simplest form:
Failure exceptions which pass on a string and percolate
all the way up to the toplevel. There are a few other pre-defined
exceptions that are used by many library functions, and we may be well
advised to use some of them in our own code as well to match the
behaviour of the libraries. One of them is
which is raised e.g. by List.find.
|Handling and raising exceptions|
As exceptions unwind the call hierarchy, the installation of an exception handler will prevent tail call elimination. This can be seen in the following examples. The first of these two implementations of a function reading a file as a list of line strings will have problems with long files.
|Exceptions and tail call elimination|
A nice and very useful extension provided by OCaml but usually not by other members of the ML family is named and optional function arguments. These are especially useful if there is no clear order of specialization in the arguments of a function, or if it is easier to remember an argument by name rather than by position.
|Named and Optional Arguments|
Sometimes, we may want to have optional arguments without providing
a default. This is especially true if the default would nail down a
particular type. If we give an optional argument without providing a
default, OCaml will have to tell our code whether a value has been
provided or not, and what it is. To do this, it uses the "
option" polymorphic variant type. This is a very close relative
'a list" and may be considered as being equivalent to
a list of at most one value.
If we define an argument as optional, but do not provide a value,
then the corresponding value passed into our function will either be
None if the function was called without that argument, or
Some x, if the value
x was given for this
|Optional named arguments without defaults|
One final remark concerns the use of complex patterns. Sometimes it happens that one wants to extract not only some data deeply hidden within some complicated nested structure, but at the same time structures from outer levels. This may be for efficiency reasons, typically avoiding to re-construct the very same tuple that also occurred in the matched structure, or because one really needs the outer and inner structures at the same time.
What is slightly annoying about these as-patterns is that OCaml just does it the other way round than any other functional language that has them:
|As-Patterns in Haskell and SML|
With this remark, let us conclude our round trip through some of the slightly more esoteric corners. One presumably could give a demanding half-year course on functional programming without ever having to touch any single one of these issues, except perhaps tail call removal. Nevertheless, they turn out to be very useful to know, especially if one wants to tackle problems from the real world, and not just some pre-selected exercise questions. Speaking of exercises, we once again provide plenty of them:
Define your own tail-recursive variant of
List.map. Is the definition in the standard library tail
ocamlbrowser to check.) Can you use it on
very long lists?
Define a function that counts how often it has been called so far.
Define a "timing" function which will take as arguments a
f and a value
x to be fed into that
function, and optionally a number of evaluations of
to perform (to optionally slow things down, defaulting to 1), which
will return the value of
f x. In addition to returning
that value, it should print to standard output some information about
how much time the evaluation(s) took.
Define a "
gensym" function that takes a string as
argument and produces a new string which consists of the old one, a
dash, and a number appended to it counting how often it was called
with that particular string. That is, calling it three times with the
"xyz" should subsequently produce
Define a function that prints all those entries in a float
array in a nice and readable tabular form, index and value, which are
of absolute magnitude larger than some
epsilon, which may
be passed as optional argument, but comes with a reasonable default.
Define a function that removes all the entries from a hash table which satisfy a given property. Note: it is not at all a good idea to modify a hash table while iterating through it!
OCaml does not provide a function that can be used to
efficiently return an arbitrary entry from a hash table. This can,
however, sometimes be very useful. Hint: use
Hashtbl.iter and an exception.
Write a function that counts how many entries of a hash table
satisfy a given property. Hint: use
Hashtbl.iter and a
Define a function that maps an array to another array where
duplicate values have been removed. Note: you should be able to do this
with an expected run time of
O(N_total log N_left).
Define a variable that contains a reference whose type is polymorphic. Is it possible to break the type system via assigning to a polymorphic reference? (Note: this has been an important problem for some time, see e.g. this paper for background information.)
Write an OCaml program that can be compiled to a standalone binary which reads line after line from standard input, and prints every line it has read to standard output, unless that line was already encountered earlier.
Write a function which takes an array as argument, and destructively modifies this array in such a way that in the end, it will be perfectly shuffled. Hint: there is a quite obvious good algorithm to do this, known as "Fisher-Yates shuffle".
The imperative "
for ... do ... done" construct can
be modeled by the following conceptually equivalent piece of code:
Find a similar definition which is equivalent to
while ... do" that also uses tail recursion
to implement iteration.
Occasionally, one would like to iterate through all the indices of
a higher-rank array. Implement a "
construct along the lines of the previous exercise which works as
in the following example:
multifor [|5;2;3|] f will call
f [|0;0;1|], then
[|0;1;1|], and so on up to
f [|4;1;2|]. In the
f will have been called on all the possible indices
of a 5*2*3 rank-3 array, in lexicographically ascending order.
This should work with an arbitrary number of indices, so that one could as well loop through the multi-indices of the entries of a 10*10*10*10 array.
The Java programming language knows a "
try [code] finally
[cleanup-code]" statement. (Actually, this is part of
try/catch/finally). This will call
such a way that regardless of how execution flow leaves this piece of
program code, either via returning a value, or via an exception, the
[cleanup-code] will be evaluated as the call stack is
unwound. In Common LISP, the equivalent construct is called
UNWIND-PROTECT. Implement an OCaml function that mimics
this behaviour as closely as possible, taking two function arguments
similar to the previous exercises.
There is much more to be known about OCaml than what could be
covered in this crash course so far. We did not speak in detail about
modules, compilation, tools such as
debugger, the C interface, and much more. Nevertheless, this
introduction should have given enough background to (1) attack some
real interesting problems using OCaml, and (2) provide a solid basis
for reading and understanding the documentation. The last lecture is
an addendum covering highly technical details most people will not
want to care about anyway.