Why I'm Interested In Haskell

No Starch Press asked me to write a review of the new Haskell book, Learn You a Haskell for Great Good!. I started to write a section about myself and my view of Haskell for context, and realized that it really needed to be its own post as it grew to a length where it was self-indulgent to make it part of the review. But it fits as its own post nicely.

(Disclaimer: By the way, I'm going to be painting in broad strokes here. Everywhere you see a nit to pick I already mentally deleted a two page discussion of the nuances. Before getting up in arms, ask yourself whether that really would have helped the point I was trying to make.)

Computer science has two branches, the practical and the academic. Every major language in current use has a philosophy that comes from the practical side. Even Lisp, which may get to wave around the term "lambda calculus", only used it as inspiration for a practical language; car and cdr are assembler instructions, for goodness' sake! There's not much more practical than that. This is neither surprising nor bad. These languages either date from an era where megahertz and kilobytes ruled the day, or are only a small number of language generations removed from that. (Language generations are much longer than hardware generations.) "Not being practical" translated pretty directly to "not going to work".

But now we work with gigahertz and terabytes, and we've had decades of experience to refine our practical languages. And refined our languages have become, to the point where it doesn't seem like there's a whole lot of language work left to do anymore. Almost every language community I know is no longer innovating. They're shuffling around the same basic primitives (objects! methods! execution stacks! state updates!) and slowly optimizing for slightly different cases on the same basic operations, libraries, and control flows. At the time that I write this, if you look at the top 20 languages on Tiobe index, you get:

Java, C, C++, C#, PHP, Python, (Visual) Basic, Objective-C, Perl, Javascript, Ruby, Assembly, Delphi, Go, Lisp, Lua, Ada, Pascal, NXT-G, Scheme

Dropping the languages that in my opinions are no longer developing as languages (even if libraries and communities are vibrant), and dropping NXT-G on the grounds I think the Tiobe methodology burped to produce that result, you get:

C#, PHP, Python, Perl, Javascript, Ruby, Go, Lisp, Lua

Broadly speaking, it's not hard to see that they are all pretty much closing in on the same basic optima. Call it CLispScript. CLispScript is:

• A mutable-state language
• Mixes duck typing for convenience with optional manifest typing for efficiency, with the definition of "type" converging more or less on Java/C# types
• Believes "Everything is an object", with composition favored by convention over inheritance, but inheritance still technically supported
• Believes "Everything is a reference"
• Runs on top of a VM with a JIT compiler that has an internal scheduler for microthreads with minimal-by-convention shared state
• Uses Algol-filtered-through-C++-filtered-through-"Ack! C++!" syntax
• Offers the powerful approaches for manipulating collections as a whole borrowed from early functional languages like Lisp (map, closures, various syntax sugars like list comprehensions), suitably modified to work on more than just lists
• Easily interfaced with C libraries due to significant low-level philosophical compatibility (even if CLispScript is much more advanced)
• Does not have full-on Lisp macros, which are just a bit too powerful, but builds many of the classic Lisp macro use cases into the language, including powerful metaprogramming
• Embraces a philosophy of empowering the programmer to a significant degree and not putting more than minimal constraints on them

Reminder: Disclaimer. I know not all of those languages meet every bullet point... actually, that's part of my point... but the pattern isn't that hard to see if you take a moment to try to see it instead of trying to nitpick it apart. Also, I'm describing where we actually seem to be going, not where I think we "should" be going. (Sure, Lisp's syntax is perfect in every way and anyone who doesn't agree is mentally deficient ahem... but it's also clearly the Algol-descended syntax that is winning.)

CLispScript is the culmination of the practical side working on its own. It isn't quite here yet, but ECMAScript/Javascript is getting pretty close in the latest spec work, not that that is the only candidate or that there will be only one winner. There are differences between the branches of CLispScript but with each passing year switching between them feels more like a dialect change than a language change.

What about the academic side?

Academics want the ability to make mathematically rigorous statements about programming languages. "Proof of correctness" is an important subset, but not the only goal. From these efforts, the academics have generated a lot of good criticisms about the current state of programming, independent of the question of whether they have good solutions. Mutable state is a problem, even before you try to share it between threads, where it becomes an exponentially bigger problem. Weak composition abilities are a problem. Weak type systems are a problem, where "weak type" here means a type that does not carry much information about what it labels, which describes all CLispScript dialects. Lack of referential transparency is a problem. "All types automatically get NULL added to them" is a problem. Lack of compiler verification of interesting pre- and post-conditions is a problem. Goto-based spaghetti code and global variables are a problem, though academia has respectively won in practice and won in principle with the practicals on those points.

If you carefully think about all those criticisms and a number of others that can be made, they all turn out to be the same criticism repeated over and over again: Your imperative program's state space is too large and too complicated.

Your program's state space is represented by all the possible values of memory values and anything else your program needs to reconstruct its current state. The "shape" of that state space is the things we can say about it, like whether it crashes at some reachable point and what things actually change where. And as academic as talking about "state spaces of your program" may sound, there is a direct correspondence between the size and shape of the state space and the comprehensibility of your program. It directly maps to how much "stuff" you have to keep track of, and it directly maps to the "space" that bugs can hide in, which are basically bits of the state space that you either failed to understand or failed to correctly prevent from being enterable. And of course the true comprehensibility of a program is directly connected to how quickly you can write it, modify it, or fix it.

Mutable state is a problem, because you never have any guarantees about the contents of your variables. Any function call (even implicit ones like operator overloadings) may read from and or change any local variable, may call any mutation methods reachable from those variables, anything reachable from there transitively, read/write any global variable, any every file on the hard drive, anything on the entire Internet, all fair game. In theory you might think you can narrow it down, but in practice this can strike in serious ways. In fact I think it's important to understand the contents of that link not as an isolated incident but instead as a profound truth about CLispScript, and merely one particularly egregious element of a very large and well-populated pattern.

Global variables increase the state size of all functions in the program, in that any of them can have unconstrained effects. The unruliness of our state spaces makes it hard to compose any two things together. (I think this is the essential reason OO has not been able to fulfill its earlier promises of easy composition. Recall that even what we have today, "composition favored over inheritance", came after a solid couple of decades of refinement of OO, and that's still not very good, just better than "traditional" inheritance OO.) Weak type systems that hardly prevent anything mean that anything can happen from anywhere at any time and the presence of the types hardly slows state space growth at all. The ability to implicitly NULL any datatype or any subcomponent of a data type adds a sharp little spike to every data type that may poke anything that touches it, unto death. All of these things join together exponentially in theory, and while we can partially constrain that growth in practice, complexity growth is still unpleasantly quick in practice.

This correspondence between the size and "shape" of your state space and the various academic concerns is the critical observation that allows you to bridge the gap between the practical world and the academic world. Using this observation, we can rewrite CLispScript's last bullet point to:

• CLispScript is a language that aggressively expands its state space as quickly as possible by making sure the programmer is at all points empowered to do anything they may possibly need anywhere.

In direct contrast to this, academics have focused on creating languages that reduce the state space of your program in various ways. Because of this correspondence, it can result in programs that are much easier to understand and reason about, which is the academic goal, and having easier to understand programs brings practical benefits as well.

One of the basic tools is a strong type system. In C, a function declared as int add(int, int) may still format your hard drive when called, or less contrived, unexpectedly increment a secret global value that has some weird effect somewhere else. In a pure functional language, if a function has a type signature Int -> Int -> Int (the closest equivalent to the C given above), you know that the only thing that function will do is something with the two integers passed in, and nothing else. This function does not exponentially explode your state space by potentially pulling in arbitrary global values or opening a network connection and doing weird things. The only thing it can see in all the universe is those two ints, and the only thing it can do is somehow produce one more. Building a large program out of compositions of these little pieces can produce a program with a much smaller state space than a traditional mutation-based program, and this is where the promise of better comprehensibility for both humans and sufficiently smart compilers is born.

Broadly speaking, academic languages have been very successful at constraining the state space of a program by constraining the programmer. In fact, in some sense this isn't even that hard; the natural sarcastic retort of "Of course the programming language that permits nothing can't have any bugs" is profoundly true, and touches on the greatest challenge of programming in this style.... a challenge academia has so far not met very well. Speaking broadly, academia has failed to enable programmers to actually, you know, do things while working within the constraints. The two solutions have been to give in and allow mutation, and it takes a surprisingly small hole in the dike to let the ocean in, or to punt on practicality/popularity and work towards some other goal.... or technically a third choice: "Be in denial about the practical problems the pure language has and wait for people to flock to the better mousetrap." This has not proved a viable strategy to date. Those of use who have an interest in doing things with computers, and perhaps even being paid to do things with computers, have collectively ignored these languages. The problem these languages have had is that what parts of the state space they permitted their programs to reach either entirely failed to contain a solution to some real-world problem, or the path to get there was so convoluted and fragile that is was not practical way for a mere mortal programmer to get to the solution, or for the resulting program to be able to survive being changed over time. It is of little use to a software engineer if you can create a Perfect Solution if I can't add features to it ever again without a complete rewrite.

Enter Haskell. I see Haskell as a collaboration between academics and some of the more academically-inclined practicals to attempt to create a truly useful language whose state space encompasses a wide set of useful things while retaining as many of the state-space-reducing properties of their beautiful theories as possible. It is a unique-to-my-knowledge blend of the practical and the academic.

Consequently, the Haskell community is the only community I know that is doing new things in the field of language and API design. The Haskell community is not just shuffling around objects and global state and adding a dash of syntactic sugar and converging on another dialect of CLispScript, they are breaking genuinely new ground. As far as I know they are the first language to take being functional completely seriously and not just provide convenient escape hatches back to imperative-land when they can't figure out how to do something purely functionally, and still strive to be a practical solution to real programming problems, and therefore they are the first sizable community actually trying to build real, useful systems out of the reduced-state-space tools.

If Haskell is consequently a bit harder to learn than some other languages... well, yeah! The popularity of CLispScript has made people forget how challenging a new language can be. An expert 1990 C programmer who knows nothing else could have their mind blown by Lisp. An expert Perl programmer is much more likely to say "Huh, interesting, but I can live without it." Switching between two modern CLispScript dialects is hardly worth calling it "a new language". Haskell is actually a new language, and if you haven't even been eased into it by Erlang or ML, expect to have to unlearn some things you didn't even realize you learned.

There are some downsides to that. Since Haskell is on new ground, the community can make some missteps. No matter how traditional, "a string is a linked list of Unicode code points" was an error. The fixes for that are still coming in, but coming in they are. At this point I was going to point you at the libraries I was missing a few months ago when I went to do my blog conversion project, but they have since appeared in Hackage (the community package repository). There's a lot of experimentation going on in the web development community to work out the proper interfaces and abstractions, because "hey, let's just redo Django/Rails/Struts" isn't an appealing option (there are many ways those designs are CLispScript at their very core) and there's much less experience to guide everyone. Prelude, the standard namespace of functions immediately available to you without importing, has several crufty aspects that you will notice after a couple of months. There's some other issues that I could name but would make less sense if you don't know enough to even know what the issues are (such as the issues with the standard typeclasses), but the community is addressing them with due speed.

But the good things coming out make it worth watching. A few examples:

• Despite the apparent burial of Software Transactional Memory, in Haskell it works, and what's more, it works because of what Haskell is and does. The Haskell community's reaction to STM seems to be a "meh" as there are many other useful threading primitives and STM has proved less of a silver bullet than initially hoped, but in the world of Haskell that was determined with a working implementation, not because the implementation was too hard and had to be abandoned, having never worked quite right. The reasons that it only seems to work in Haskell are fascinating, and I find it unlikely this is the only idea that simply can not be done in something not like Haskell.
• BlazeHTML is an interesting project. The use of combinators to create HTML like that is not quite the same as any other language; yes, there are things with superficially similar APIs (I've written at least two myself, one each in Perl and Python) but the way that Haskell is doing it, and the resulting speed, is one of the few examples I've seen of the speed promises of Haskell manifesting. And what you can't see from the tutorial, especially if you don't know Haskell, is the exciting ways that BlazeHTML can be composed with other things in the language.
• There's also some interesting work in getting high performance threading going, for instance, see Threadscope. Some of this does overlap with Erlang and Erlang has the clear lead with multi-server distribution and generally field-tested code, but Haskell is catching up to Erlang-the-language quickly (still missing some primitives necessary to catch up to Erlang-with-OTP) and will eventually surpass it most ways, while still bringing the other Haskell things to the table.
• And I have not yet spent enough time with Haskell that I feel I'm really getting the full scope of the paradigm. (Arguably, this is true of everybody.... well, except Oleg.) Monoids and fingertrees fascinates me for what it accomplishes in how little code. The probability monad is interesting again not just as-is, but because of what it implies about the language. And so on. I am excited about possibly being so skilled in Haskell that I too can just whip this sort of stuff off, and even though these examples may seem academic I can see their practical promise. I've only done a bit but I'm already starting to miss some of this stuff when I program in Perl professionally.

I know the academic side of the Haskell community contributes plenty of "academic noise" to the conversation, but under noise that there's something novel and interesting happening in our field of programming, which for all the usual churn and burn of fads and buzzwords is actually pretty short on novel and interesting things left after 15 years of aggressive autodidacticism. I encourage anyone who thinks they might be interested in Haskell to just pass over the academic stuff if they wish. I assure you it is not necessary for the full experience.

Let me remind you of the title I gave this post: "Why I'm Interested In Haskell". Not why I "love" it or "use it for everything". I do not know to what extent Haskell can deliver on its promises over the next few years. I do not know whether we'll all be programming in Haskell in 10 years or if it will merely enter history as another language that influenced many but was adopted by few. I have my concerns about the nature of Haskell programs; a solution to a problem can not be simpler than the problem and I am still concerned that the reduced-state-space tools may still prove intractable for large systems, either by blocking access to solutions or resulting in a system too rigid to deal with design changes; overpowerful programming primitives may redeem themselves if they provide superior design flexibility. Or perhaps not, who knows? Only one way to find out. I do not consider myself a Haskell cheerleader or partisan, I still consider myself someone who needs to be "won over" even after dabbling for over a year now. I won't promise Nirvana or that it will solve your problems at all, let alone do so better than your current favorite CLispScript dialect.

But I will say that Haskell is worthy of interest. And that's still something.