deb packaging

I've been working on trying to make some of my work from last year easier to install.  So far I have a script to create a massive tarball (or copy everything to a root tree).  Now have some moderately tweaked default Debian package creator files with configuration files (like apache site) meant for a VM.  If I do this right, it should be possible for somebody to just

1. install Debian on a fresh VM
2. copy the deb file to the VM
3. use dpkg --info to list the package dependencies; install with apt-get
4. dpkp -i the-deb-file

I'd initially resisted doing this, opting the for the write-a-big-README file approach.  But really automation is the way to go.  Hopefully the readme will be useful as backup if something goes wrong.

Resulting deb file is going to be pretty massive.  Has GATE, RASP, Scala, FlexBuilder.... I suppose the right way would be to go down and make sepearate debs for each one of these dependencies.   But no.

the Scala rewrite

This week I've switched over to one of the projects that's been funding my time at Brighton, 3D-Coform. Took a while to dust off my old work from last summer, but basically slowly starting to get into a groove.

One of my goals is to improve the packaging of some complex pipeline I'd built. It involves Flash (not my choice!), PHP (I'm innocent!), Java (for GATE), Scala and Haskell. The Scala and Haskell parts are my fault, mostly a refusal to go back to writing Java. What's funny about the pipeline though is that it's essentially Java, with a thin bit of Scala which is really just calling this one dinky component written in Haskell. Making people install a Haskell compiler just for that dinky component is stupid. In my defence, writing said dinky component in Haskell was probably the right thing to do, in other words, use the tools you know best to get the job done and out the door. But now that we have a little bit of time to put into packaging it up nicely, I'm trying to replace that little dinky with of Haskell with Scala. This way all the backend bits run on JVM and we have simpler dependencies.

It's also a good way for me to learn the language a little bit better.  I guess I've reached a stopping point for the week.  I've reimplemented the shallow part of my code which defines a bunch of data types, parses from XML and pretty prints the output.  If you want to take a look, the Haskell and Scala code are not identical (the XML parsing may be a bit better in Scala version, due to reimplementation?) but I think they are roughly comparable.

The deep part is the bit that does graph traversals, basically simplifying RMRS semantics into subject-object-predicate triples.  That bit I'll have to try and implement next week.

Things I've learned so far:

• always use case classes, never use new: I was wondering why for some classes, like Option, I could just call the constructor (eg. Some), but for others I had to write new Foo(...).  Turns out that it was just because I wasn't declaring my classes to be case classes.  Using case classes means I get to avoid all the confusing stuff that mutability gives you, for example, having to worry about the distinction between equality and identity.
• I may not be using traits right, but I think I like Haskell type classes better simply because with type classes, I can separate the instance declaration/definition from the class definition
• subtypes make for easy heterogeneous lists: in Haskell I'd wrap the things I want in some existential type E and then make a list of E.  In Scala, I make my types all extend the trait E.  For some reason, I find the latter a bit "easier", either more memorable/clear or just more convenient
• to get methods associated with classes rather than objects, just create a singleton object (that extends the desired trait)
• Scala lets you pattern match XML: it's kind of neat that you can interleave XML in your code and do things like pattern matching on it.

I get the impression that you basically have two kinds of Haskeller.  The first is the kind that uses Haskell because they can take on a lot of abstraction and use it to do amazing things, the smart Haskeller.  I think I fall more into the dumb category of Haskeller who tends to use it because I find the consequences of subtyping (Liskov substitution) and mutability to be too confusing.  And maybe there are also smart-dumb Haskellers... the kind who fall into both camps simultaneously.

3DC on hold, preparing talk

Have pretty much handed off my 3D-Coform work for the next month or so. Now preparing a talk on some work I did before. Will be visiting Orleans next week. Talking to undergrad computer science students about open source, and also to the NLP group about our corpus extraction work. Spent yesterday catching up with my email, and making a bit more progress on the open source talk and starting to think about the corpus extraction talk.

virtual machine working

Have finally gotten things a bit into gear with the virtual machine stuff. Two things that were tripping me up and also a new trick:

• I'd forgotten about the Debian udev persistent rules feature that made it tricky to set up networking when I cloned my virtual machines. The VirtualBox issue tracker describes a solution which basically tells it to ignore virtual network cards from VirtualBox.
• I'd forgotten to select a "bridging" networking adapter
• VBoxHeadless --startvm vmname (avoids some extra windows, once you have a working ssh server and can log into your box)

Once I'd gotten over those silly technical hurdles, the rest of it was relatively smooth sailing. The basic workflow is something like

1. Create stock VM, installing a fairly barebones Debian on it (I used a business card network install ISO and deselected all the tasks). This is probably going too far, but I wanted to minimise assumptions so that my INSTALL instructions are more likely to work on somebody else's machine.
2. Clone VM: test INSTALL instructions on VM. They'll probably fall flat on their face, so lots of README/INSTALL tweaking to make sure you get the details right
3. (If you can be bothered): throw away Clone and start #2 again -- should be much more straightforward

The idea is that by doing something like this, you'll given your INSTALL instructions a test drive, which gives you an idea about hidden assumptions and what not. Granted, you being the test driver is not ideal, but it's better than nothing...

Now this doesn't mean all is done and dusted. Unfortunately, I'm only up to halfway in the installation process for the Duct Tape Monster. Stuck on a random component that seems to be either slow or stuck. EDIT: not sure what went wrong, was stuck on reading the output. Some sort of deadlock, I guess. "Fixed" by using exactly the same version of the GATE framework as on my development machine.

virtual machine boredom

Yesterday, I was sluggishly trying to set up a virtual machine for our infrastructure. I found myself not really paying much attention to the task, taking each progress bar as an opportunity to wander off and do nothing useful. Should have been probably concentrating on the analysis, or maybe even doing Darcs stuff instead.

More of the same today, hopefully a bit more disciplined

louvre stuff sucked down

OK, so I managed to finish yesterday tagsoup adventure. Ran into some bits where in retrieving pages from the louvre site, I was getting funny characters. This was due to using String as the internal representation of the data but without actual proper Unicode code points in the Chars (I think). I struggled for a bit trying to get the HTTP library to use ByteString instead (it accepts both), but gave up on some type inference problem [:-( !]. Instead I just told the IO stuff to save the files as binary, ie. just treat the Chars as padded bytes rather than trying to encode them. Anyway, I now have a sort of parallel corpus of 275 texts in English, 303 in French (so some missing English translations). English texts have been processed with the automatic extractor.

Storing this thing is a bit awkward since I don't know the best practices. You have lots of tiny decisions to make like:

• how to store things? just files on the filesystem or through some mechanism like a database engine
• if the former (pro: Eric knows what he's doing), then how do you structure the directories/files? I have {fr,en}{,-orig,-processed} So note the choices there: relatively flat structure (no directory hierarchy), and files "repeated" across the directories. The same file at various stages of processing might be fr-orig/foo.html, fr/foo.txt, fr-processed/foo.txt.debug. Maybe this could have all gone into the same directory fr/.
• how do you name the files? I have my files named with a unique identifier from the Louvre, so I get things like fr/work-10134198673225779 and en/work-10134198673225779. The unique identifier is easy to derive (grab it from the URL parameters) and it allows you represent that a text has different translations (ie. it's the same name in fr/en). But it's also totally opaque to me as a human. Not good

So I guess there's three basic tracks I want to cover next:

1. Packaging everything up: updating the README and virtual machine to cover the latest infrastructure (I may be going away at the end of October, so I want this done early)
2. Analysing the output (this time focusing on the Louvre stuff and probably forgetting about Sussex Sculptures)... which I keep putting off because it's a Thinky task rather than a Do-ey one.
3. Improve the infrastructure (this sort of depends on #2). For example, the current RMRS based extractor is really dumb: it just assumes that any two grandchildren of a verb node are its arguments in a relation, without actually paying attention to which one should be the subject and which one should be the object. RMRS actually tells us enough to distinguish between the two, but I just wanted something quick and dirty which works. I could also (somewhat reluctantly) think about rewriting the extractor in Scala. This would minimise the number of languages being used in the infrastructure, allow for easier plugging into GATE (no spawning processes, just invoke a method), and also give me a chance to learn more Scala.

soupe aux balises

Managed to get some simple processing of the Louvre HTML with TagSoup. I love the TagSoup library because it gives the same kind of whipupitude you might expect from Perl, the ability to just throw together to do something that works by eyeball. You can use it as a sort of tag-aware grep, for example, taking a potentially malformed page and extracting only the interesting text:

The Chiusi funerary monuments

This plaque from an Etruscan funerary monument was discovered in 1843. It entered the Louvre's collections in 1863 with the purchase by Napoleon III of the Marquis Campana's antiquities collection. It comes from the region of Chiusi in Umbria, where it was created around 490-480 BC. At that time, the Chiusi workshops were particularly famous for their funerary monuments - stone columns that were used to indicate the placement of tombs or to hold urns. This plaque is characteristic of this production. The bas-relief is sculpted in a local stone that, when quarried, gives off a very disagreeable odor caused by the infiltration of sulfurous water. For this reason it is called "fetid limestone".

A funeral ceremony
...

The TagSoup model should remind you a bit of SAX. You get a list of objects like TagOpen "a" [("href","http://...)..] which you can trundle through with your basic Haskell stdlib higher order list functions. The kind of code you get is not just terse, but simple, very very simple. Just a pipeline of function compositions. Here's the pipeline I used (read it right-to-left, mathy style like f o g)

louvreText = intersperse (TagText "\n\n")
           . map trimText
           . filter isRealText
           . takeWhile (~/= "<div id=bibliographie>")
           . drop 1
           . dropWhile (\x -> x ~/= "Description" && x ~/= "Notice")

The two part (with Description) chops off any bits of the documentation before a bit of HTML text that either says "Description" or "Notice", the second part chops off that little bit of text, then we grab anything up to the bibliographie div, and finally we get rid of any boring text (boring here is defined as whitespace only -- isRealText is a small helper function I wrote), get rid of the whitespace surrounding the HTML text we do have, and then insert double newlines in between each paragraph.

There... so much for my Haskell propaganda. It's not so much about building short programs -- code golf is stupid -- but simple ones. Simple here meaning few moving parts, absence of bureaucracy, homogeneity (ie. just a pipeline). Understanding what a higher order function is may be not so simple at first, but once you're happy with that, you get to knock out little programs like the above without having to think so much.

Today, I intend to continue my TagSoup adventure, grabbing links to all 303 texts and fetching the corresponding documents. Then I'll have built up a little mini corpus which I can do some more interesting text processing on. I'm afraid I'm still doing this to procrastinate on the main job of analysing the output on the Sussex Sculptures texts... probably one of my personal failings, a tendency to prefer tasks that involve Doing rather than Thinking :-/

tag soup ahead

Mostly spent yesterday writing a Darcs email.

Otherwise, I took a bit of a look at the SketchEngine (mainly with an interest in using it as a corpus collection tool.

Also, I found the bit of the Louvre website where you can find the one text example I tend to lean on. So lots more samples ahead. Unfortunately, their content is not valid XHTML (says the W3C validator), so I'll have to use a TagSoup approach to processing it. Maybe I should mention this to the site admin if they have contact details.

I think I'll spend today processing some Louvre text with TagSoup and otherwise thinking about corpus collection and storage. I'm probably just doing this to put off doing the kind of analysis I've been promising since last week...

lundi

Not much to report as I wrote a Friday wrap-up. This week, I think I'll be tackling that analysis of automatic annotator outputs. I think I just have to grab some texts and look at them first, no point trying to work out what I'm looking for yet

debug output

Trying the approach of moving the Monday morning blog post to Friday evening (maybe with a shorter post on Monday) to maximise downward slopiness.

I didn't do any actual analysis because I got caught up with trying to tweak the output so that it was more useful for me.

The context is that I use XSLT to convert the GATE sausage machine output into something that AnnoMAD can read. I also have parallel scripts that convert the same sausage machine output to something friendlier for human debuggers. Today I wanted to tweak it, but then decided I was bothered by the duplication between the scripts (which makes maintainability hard). So I ended up spending my time refactoring and tuning stylesheets. The new approach is to translate everything into an intermediary pivot format and then convert from the pivot. This means that any knowledge-intensive stuff can be funnelled into the translate-to-pivot step.

Things I learned from the process:

• PXSL supports declaring element text on same line as element
• XSLT (?) has a normalize-space() function you can call in selectors
• XSLT include to merge stylesheets, import for merge + default override
• XPath: you can flatten by selecting on descendant

Here's what the debug output looks like on one of the Sussex Sculptures:

Long Man of [Wilmington]crm:E53.Place 


The figure either holds lances, staves or spears or the uprights possibly represent portals.
-- represent(upright,portal) --
-- hold(figure,lance) -- The uprights are almost symmetrical, giving the illusion of an unfinished rectangular frame.
-- give(illusion,frame) -- Both legs are now turned to the East but the feet were originally wider apart and both facing outwards as seen in the [1874]crm:E50.Date 
 photo. The figure is naked and featureless but there is a possibility that it was originally clothed – early drawings indicate this and also place facial features and make the head helmet shaped. Interestingly, the figure’s proportions only appear correct from the air. There is elongation to counteract foreshortening (unlike Cerne or the Uffington horse).
== Annotations ==
crm:E53.Place   Wilmington
crm:E50.Date    1874
== Relations ==

hold(figure,lance)
represent(upright,portal)
give(illusion,frame)

monster running

Yesterday felt productive, although perhaps in a semi-mindless sort of way. I was trying to figure out why the automatic annotation tool just spat back these argh go crazy type errors whereas my test script appeared to be working.

Turns out it was several issues including me being wrong about the test script working, and also a permissions problem. I'm sure there's another issue in there as well.

No more clicky clicky

I guess what was satisfying about yesterday was a feeling of overcoming mindlessness during the debugging process. I was being awfully slow-minded about it, but finally managed to get my shit together and start teasing the problem apart.

Here was this PHP script being called from the Flash UI, and the way I was feeding stuff to the PHP middle layer was by loading the flash script, loading files in the GUI and clicking on "annotate".

Silly. I ended up writing a small Haskell script to replace the Flash UI for testing:

import Network.URL
import Network.HTTP
import System.SimpleArgs

main = do
 html <- readFile =<< getArgs
 let req = postRequest . exportURL $ mkUrl 42 html
 print req
 rsp  <- simpleHTTP req
 print rsp
 rspTxt <- getResponseBody rsp
 putStrLn "=== Response body ==="
 putStrLn rspTxt

mkUrl i html =
 URL (Absolute (Host (HTTP False) "localhost" Nothing)) "HST/auto_annotate_text.php"
   [("id",show i), ("html_text",html)]
 where
  host = "localhost"

eNothing to sneeze at, just evidence of starting to learn a bit more about HTTP and CGI (ugh, do the parameters really all get encoded in the query string?), very slowly crawl out of my pathetic ignorance. It's like telneting to a web server and writing HTTP headers by hand, the kind of thing you ought to do to get a "feel" for what's underneath all the magic. Anyway, being able to just mash the up arrow on my terminal rather reloading the Flash UI and going through all these stupid mouse actions certainly improved my productivity.

Other little bits of getting my shit together: dumping intermediary outputs in a single temporary directory rather than a bazillion different temporary files (temp-9861/{foo,bar} rather than /foo-38x8, /tmp/bar-9zcg); redirecting output/stderr to a tmp file (because I wouldn't be able to see it from the web, duh); refactoring my scripts so there were fewer code paths to debug; replacing an intermediate shell script with just another spawn and pipe. Just tiny little things you do step back from the general hazy mindedness.

Internal analysis

Now comes the potentially tricky part. I've run the monster on all the sculptures in the Sussex Sculptures database. I have some stylesheets (in PXSL, of course), converting the output into something human readable and debuggable. Now I need to wrap my head around the problem of analysing the output to see how we're doing. Tricky... don't know where to start.

Basically, we're looking to answer two questions:

1. Sorta precision-y: How good are the annotations/relations that we find? (what to fix, etc)
2. Sorta recall-y: What annotations/relations are missing?

I guess I'll go with some sort of iterative approach. Think up a kind of prototype analysis, randomly select two texts. Try to analyse them. Re-think analysis, pick another two texts, and repeat until I've got a good sense of knowing what to look for. Then pick 10 texts and analyse them properly.

the monster as a diagram

After just a bit of procrastination (and struggling with Inkscape), I managed to create a diagram showing the current state of the duct tape monster. The purple bits are components that I wrote (AnnoMAD has a tiny bit of purple to indicate some modifications)

I derive some childish amusement from detailing the languages that went into this: Flex (Flash) for AnnoMAD (soon to be C++ and QT?), PHP for the annotate layer (sigh), Bash for the wrapper, Java for GATE, Scala for the RASP/extractor wrapper, Common Lisp for RASP and Haskell for the RMRS extractor. Why the languages? Well, I guess it goes to show that there's a certain amount of diversity in the domain -- different people know different languages -- but that doesn't necessarily stop us from trying to smoosh together our various bits of jigsaw puzzle (GATE in particular, as much as I resent the Java, being a fairly nice example of trying to work out a universal smooshing framework).

I keep hearing that actually Haskell on JVM has been done before. But I wonder what it would take for it to be practically useful. For example, I tend to depend on lots of packages on Hackage. Are the various Haskell on JVM approaches going to reach a state where I can continue to make use of them (perhaps they'd be installed as jar files?)

Today's work is to try and figure out why AnnoMAD is not happily talking to the monster. The shell script tests seem to work, but going the clicky-clicky route leads to a frowny face Eric. To do this, I'd like to refactor some of my shell scripts: I have a test version and a production version of the script, which really ought to be sharing more code so that I can be more confident that something that works in test will work in production.

If I manage to sort this out early enough, then I'd like to run the monster on 10 documents from the Sussex sculptures database... see what we get back

bright future, eh?

Yesterday I attended a university-local conference for Early Career Researchers, basically a sort of helping hand for folks like me who are in the relatively early stages of their academic careers, stuff like writing successful grant proposals and what not. One of the nice things about this university, a sort of attitude of nurturing and investing in their staff.

I think it was useful, although I must admit I tuned out a lot of the morning session because I was too busy scribbling notes on the computer game idea I got, called Early Career Researcher (what else?).

This week I've got two things on my plate:

1. Informal presentation, taking stock of where we are in 3D-Coform, basically presenting the current duct tape monster (presumably eschewing all the actual bits-of-string details), just showing over all where all the pieces fit together. This is basically a variant of the Write-To-Think exercise. Present-To-Think. Use the mini presentation to get a sense for how to go forward
2. Do an internal mini evaluation of the current annotation/relations extractor. Run it on a handful of texts and do an analysis of what it finds.

I've got these two tasks because I complained about not really knowing how to tackle the problems of linking relations to the ontology, nor to the annotations they're trying to relate. Very grateful to have something concrete to gain traction with!

RASP plus extractor now talking to GATE

At last! I should probably write the what-I-did-Friday blog post at the end of the day rather than Monday morning because now I have no idea what I worked on.

But I do know that with learning a bit of the GATE API and a bit of Scala, I was finally able to send the output from my RMRS graph-based relations extractor to GATE in the form of sentence level annotations. As input GATE gives us as input a set of annotations over a body of text (basically set of attribute-value pairs, plus left/right offsets). We grab all the input annotations marked "Sentence", sort them by offsets and moosh them down into RASP input. Off this goes to RASP and then to my relations extractor and comes back is a list of list of relations. So my plugin just does a very simple thing of pairing each input Sentence annotation with the corresponding list of relations, creating for each relation a new GATE "Relation" annotation with the same offsets as its sentence. Child's play if you know what you're doing, a week of hacking since I didn't.

Implications of having the output as GATE annotations? Well, now maybe it'll be easier to think about how on earth I'm going to align the NP annotations with the Relation ones. Also I get some nice little bonuses like being able to use whatever tokenisers or whatever GATE works with. We could even think about doing some kind of domain-specific-knowledge tricks like tagging all output from a certain Gazeteer as noun phrases for input to RASP...

RASP plugin now with actual parsing

Didn't manage to accomplish what I'd set out to yesterday: turns out I was wrong about the Digital Pebble plugin for RASP. I think it must have been emitting RASP tagged input for an earlier version of the parser. Took me a while to understand that the parser was simply not accepting the input (or rather, it was accepting it, but the input was causing it to crash). The fix turned out to be easy, just tweak the Java code so that it emits something slightly different.

Also, my brief foray into YAML was a bit of a failure. Turns out that for what I wanted to do, the JYaml parser had bug where it was confused by sequences of sequences of mappings like

- - a:b
  - c:d

Phooey. Oh well, at least in trying to work out what was going on, I got a taste of YAML and the kinds of features it supports.

I think today I'll actually have to try out what I said I would do yesterday. Only now I'm starting from actually being able to talk to the relations extractor. Guess I'll be switching to JSON

scala bits listening to Haskell bits but not understanding

Got the relations extractor to output YAML instead of the simple notation for triples pred(s,o). This is really a silly little job which you don't need something like YAML for, but like I said, it was mostly just a way of getting a concrete taste of the language and seeing what libraries are available. Right now I'm using data-object-yaml because it's convenient (high-level enough). I wonder why there isn't some sort of typeclass for it like the json library has; could maybe just make one myself (multiparameter?) but doesn't matter. I have all of two functions. One handy feature is data-object-yaml is that it shares an internal representation with data-object-json, so I can just treat them as two ways of rendering the same data.

The Scala wrapper now knows how to call the relations extractor and print its results back. I didn't really know what I was doing in terms of spawning a process, so I just imitated the original Java code using Threads and Runtime.getRuntime().exec(foo). I kind of wanted to play with Scala actors instead of threads, but I guess now is not the time.

Today's agenda may be a bit harder: I need to get the stuff from the relations extractor parsed and transformed into GATE annotations. I'll probably use a Java library called JYaml to do this. One bit of trickiness will be zipping the list of list of relations with the list of sentences. The main hard part will be learning enough of the GATE API and overall model for things.

more java/scala

Copied over the Java module and changed some of the private methods to public as I mentioned yesterday. Today's plan is to write my YAML-ised variant of the relations extractor -- shouldn't take more than an hour -- and set up a standalone pipeline that calls RASP, pipes output to said variant. If I get that done, then I can start forcing myself to learn enough Scala to make a system call and read/write output.

Resistance to learning. What am I, 40?

plugin plugged in

My little bit of progress yesterday was to successfully create a trivial GATE plugin in Scala. All it did was to subclass the RASP plugin from Digital Pebble, so when you ran it, it just output the same thing as the DP one. The one hiccup I encountered was some kind of Object Not Found error message, which basically meant that I needed to tell my plugin to include the scala-library.jar file, after which everything was OK.

So that's pretty cool, now with Scala I get to have a slightly less painful way to do Java stuff. Maybe it would be nice to have this working directly in Haskell, but it's better than nothing and learning a new language is probably not so bad for me anyway.

Today, my hope is to extend the plugin to do something a little different, maybe not exactly the relations extraction I have in mind. Even just appending "hello" to all annotations that the DP plugin would output would be a start.

One of the obstacles I've encountered is that functionality I need to make my plugin work is marked private by the DP plugin. So to grab at this functionality, I either need to:

• make a copy of the source code and just change the private to public
• re-implement the functionality in Scala

I think I'm going with the first option. This is the sort of thing that makes me feel like I have no idea how to do Object Oriented Programming. I vaguely understand why the practice of hiding functionality in modules is important (you want a nice clean API), but from a code re-use standpoint, it really gets in the way, and sometimes I feel like it's not always clear when it's right to hide and when it's right to expose functionality. I would tend to think that if you program in a functional style -- no mutation -- it'd be a lot safer to expose a lot more code, so you'd have a lot more liberty and can make the decision to hide stuff only where it really is beneficial from an API-design standpoint. Is people doing copy and paste programming (which I'm about to do) is a greater evil than the confusion from insufficiently black boxen?

the plan continues

Posting more thoughts now because I'm away for a 4-day weekend (AngloHaskell) and don't want to forget.

Right, so I've written that Scala hello world I wanted, and have also done a stupid transliteration of the GATE how-to-write-a-plugin Java template into Scala (I say transliteration in that I'm not so much translating as shuffling Java-sounding-things into Scala-sounding-things without really knowing what I'm doing). So, without any claim that this is right, here is writing a GATE plugin in Scala.

The next step is to actually try and do something useful with this. Basically, I think what I want to produce is something like this:

So you may notice the not just the completely gratuitous use of Scala but also of the YAML format as the output of the relations extractor and the input of the extractor plugin. (I suspect there aren't any mature YAML parsers for Scala and I'll just have to use a Java one instead). I think you can say two things about this: on the one hand, it's completely idiotic: the mission here isn't to go learn some new programming language or file metaformat..., it's to extract relations.

On the other hand, maybe it's not 100% foolish. OK, my tendency to neophilia and wanting to do things in the Right or Clean way can lead to a certain paralysis or time wasted just dicking around. But it provides me with skills that come in handy later, in completely different contexts. Programming in Scala and YAML, making presentations in Prezi may not be immediately useful, but they could pay off in the future. You don't always know what you're going to need to know when. It's good to have options. Anyway, that's the old investment argument.

A slight twist to that is that, it's actually kinda hard to make myself sit down and learn something just for the sake of learning it; I really need some kind of context, some kind of reason to do the learning. And so this is a semi-bad reason -- I don't really need Scala or YAML for the job -- but it is a reason.

Another argument is that the task at hand involves lots of boring stupid bits, unpacking things from one format and re-packing them in another, plugging one tool into another, etc, and you know I feel about boring stupid bits. Giving myself something to play with at least makes it easier for me to make progress because I can spend less time resisting the boring/stupid and more time exploiting the boring/stupid for play. There needs to be a lot of room for just playing with shit, know what I mean?

battle plan

Anyway, I've taken a look at the GATE plugins source, so what I think I'll try to do is something like this:

1. run pre-existing RASP plugin from GATE gui on a "john loves mary" example -- does this thing even work?
2. subclass the DigitalPebble RASP plugin (provided it works), or if it doesn't just do copy+paste programming
3. play with JYAML for YAML processing.

rasp now connected to annotator

Yesterday (after much procrastination), I did the very easy work of connecting my RMRS graph based relations extractor to the annotation tool. Easy means it was all mechanical nonsense:

• packaging up my library to be cabal installable (should stick it on patch-tag)
• copying a Haskell script to a different directory
• adapting it to output XML
• dealing with silly things like shell scripts and paths

Silly stuff that I could get done in like one hour if I could just sit down and focus on it. Instead, I had to stay late, sigh.

Anyway, it's done. So I now have relations extraction using an approach I'm a bit more comfortable with. I should really compare it with how something like MultiPAX works.

What I'd like to work on now is the second half of what I talked about yesterday: turning my relations extractor into a GATE component. I think the way I want to do it is:

1. Write hello world in Scala - I know it's silly for me to be learning a whole new language just to do some stupid Java scripting, but (A) I still find Java somewhat abhorrent even thought I'm not nearly a good enough programmer to be hating on Java as much as I do [I probably don't have good reasons, just dumb Eric ones like "gah, it's so verbose" and "I don't know how to deal with objects reasonable"and (B) learning new things tends to pay off in the long run
2. Find the links to the GATE API
3. Write a variant of the Haskell program that returns relations in a some sort of easy to parse line-based mini-language
4. Write a Scala program that makes a system call to the RASP shell script and which then feeds the output to my Haskell program. If I was more clever, I'd work out how to do this directly in Haskell :-)