Here's a small webapp I wrote yesterday. Since the app (as I envisoned it) was fairly simple, I figured I would invest some time to learn how to write web stuff in Haskell. I went with the
Happstack framework just because it's the most well known.
Sales pitch
I call the app "corpus sifter". It could use a nicer name. The code is BSD licensed and I'm willing to stick the Darcs repository up somewhere, maybe even upload it to hackage if anybody is interested, but I doubt it's useful enough to justify such actions.
The scenario behind this application is as follows:
- You have some pieces of structured data
- Using a search engine, you have retrieved a text file for each piece of the data
- You don't know if the data you retrieved really corresponds to the data; there may be some automatic ways of finding out, but you need to spend a little time scrutinising the data to find out
Solution? Well, you're stuck with manually sifting through the data, so you might as well make this manual process as quick and painless as possible. To this end, we present the data and text file side-by-side, and three buttons "good", "bad" and "unsure". Clicking on one of these buttons moves the pair to the corresponding "good/bad/unsure" bin from the new bin; it also presents you with another item to classify.
Two features of this approach is that is (a) fairly general and (b) simple/robust (hopefully).
generality
There is nothing in corpus sifter that is specific to the corpus in question. The data must be represented as a JSON object (which corpus sifter will render as an HTML table), and the text will just be systematically dumped out as raw text (these are wikipedia entries that I'm using, but you may have other things in mind).
simplicity/robustness?
Part of corpus sifter's design is to account for incremental corpus construction. When you begin building the corpus, you may not exactly know all the best ways of (i) sifting through it or (ii) searching for texts. So over time, you may want to revise your data. Corpus sifter supports this by adopting a very simple and stupid model of processing: the bins are represented as directories, and moving items from one bin to another consists of literally moving them on the file system.
If you change your mind about a bin (say, you hit a back button), corpus sifter will notice and move the data pair from the corresponding bin.
If over time you add more data to the corpus that you want to sift through, just dump it in the "new" directory.
In a future version of corpus sifter (maybe tomorrow?), I plan to add a feature that lets it notice that you have put something in the "new" bin which already has been classified. This would account for cases where you have new, hopefully better search results. What the tool would hypothetically do is to create some sort of backup directory and move the old data from its bin to this directory. This will be handy for cases where you incrementally refine your search methodology, leading to different results for files that you have already seen. For example, working with Wikipedia entries, I've discovered better and better ways of coping with disambiguation pages, but I need to find some graceful solution for merging these new results in with my old sifted corpus. I certainly wouldn't want to have two copies of my data lying around!
Implementation notes
You can count on me to extol the virtues of Haskell programming at any slightest opportunity. Today, the features I'm going to gush about are common among many functional programming languages (Haskell, ML-ish languages and Scala). These features are
algebraic data types and pattern matching.
How Haskell helped me here is to develop a simple and generic solution for rendering any JSON object as HTML. The code is as follows (renderer in 25 lines of Haskell, entire app in about 200 lines):
jsonToHtml JSNull = noHtml
jsonToHtml (JSBool b) = stringToHtml (show b)
jsonToHtml (JSRational float r) =
stringToHtml $ if denominator r == 1 then show (numerator r)
else show (fromRational r :: Float)
jsonToHtml (JSString js) =
case fromJSString js of
u@('h':'t':'t':'p':':':'/':'/':u') -> anchor (abbreviate u') ! [ href u ]
s -> stringToHtml s
where
abbreviate = stringToHtml . (++ "...") . take 13
jsonToHtml (JSArray xs) = unordList (map jsonToHtml xs)
jsonToHtml (JSObject o) = jTable . concatHtml . altMap dark light . map mkRow $ fromJSObject o
where
mkRow (k,v) = tr $ concatHtml [ td (stringToHtml k), td (jsonToHtml v) ]
dark = (! [ theclass "dark" ])
light = (! [ theclass "light" ])
jTable x = table x ! [ theclass "json" ]
-- | map on every other element
altMap _ _ [] = []
altMap f1 f2 [x] = [ f1 x ]
altMap f1 f2 (x1:x2:xs) = f1 x1 : f2 x2 : altMap f1 f2 xs
The basic idea is that JSON strings are escaped and turned into straight HTML, numbers are represented in decimal notation (where appropriate), lists are rendered as bullets (ul tag) and here's where it gets fun, objects (key-value pairs) as tables. The nice thing here for me is that I didn't have to go read a bunch of specs to know what I had to do. Since the author of the json library chose a nice simple sum type representation for JSON objects, I could simply write a function to pattern-match on each one of the cases individually...
These key is that algebraic types helped me to
think about the problem because they (i) divided the problem into cases which (ii) they made very readily apparent and (iii) which the compiler could then check that I have exhaustively explored all the cases [eg, did you consider boolean values in JSON? what about the null case?]. Furthermore, the recursion on the the types (JSON objects contain JSON objects) was very simple to deal with as just a straightforward recursive function.
Pattern matching and algebraic data types... this kind of stuff makes me really reluctant to give up statically-typed functional programming languages. Of course, there must be other ways to solve this with other languages' tools; if this would have been easier in some other language, give me a shout in the comments.
