caaaaaatch up

No report for the past few days because I've been either away, ill, or plain catching up.

Got to wrap my head around those RMRS formulas a bit better, maybe take a look at the RASP code to see what these ing-{a,b} things are

rmrs via graphviz via fgl

I pretty much finished my RMRS visualiser yesterday, first transforming the XML files into a more readable text notation and then sucking the internal data structure into a graph representation using Martin Erwig's Functional Graph Library. A nice thing about using the fgl is that I can then just plug the stuff straight into Ivan Lazar Miljenovic's graphviz bindings. Nary lifting a finger (1.5 days of hacking), I've got both an ADT, a graph representation and a visualisation of an RMRS semantics coming out of a robust parser. Yay Haskell? :-)

Here's an example of such a semantic formula. The sentence being represented is According to the inscription engraved on the belly, this oinochoe was part of a wine service belonging to an Etruscan named Karkana.


I'm not sure how surprised I should be that the graph is not very connected. It could be that I'm just reverse-engineering (while scrolling through/squinting at the MRS paper) RMRS wrong. Or this could just be what you can expect out of a robust parser on real data.

the graphviz tip

By the way, digging through Google and the graphviz documentation, I found a useful trick for compacting your not-very-connected graphs: ccomps -x | dot | gvpack | neato -n2 (ccomps is a graphviz tool that breaks your graph into its connected components; dot just lays them out; I think gvpack somehow squishes everything together and neato I guess must place each of the subgraphs.

obligatory Haskell snippet

Also, to give an idea what the whole converter looks like, this is a sketch of mostly type signatures an a hint of code (this will be BSD3 or ISC licensed or some sort of permissive thing, but I'm not sure if it's worth putting it online yet):

rmrsXmlToGraphviz :: Xml.Element -> IO ()
rmrsXmlToGraphviz e =
 case getFormula e of
   Just e  -> do let g = toGraph e
                 writeFile "/tmp/foo.dot" (printIt $ graphToDot formatter g)
   Nothing -> fail "Not an XML doc"

data Formula = Formula Label [EP] [RArg] [Ing] [HConstraint]
-- plus another 40 lines fleshing out ADTs for EP, Arg, etc

getFormula :: Xml.Element -> Formula
-- about 60 lines of typeclass instances

toGraph :: Formula -> G.Gr EP RArg
-- about  20 lines converting my Formula ADT into an FGL graph

present :: Present a => String
-- a Show-like class, about another 50 lines of typeclass intstances

formatter :: GraphvizParams EP RArg () EP
formatter = nonClusteredParams { fmtNode = toLabel . snd
                               , fmtEdge = toLabel . thd3 }
 where
  toLabel = GV.Label . StrLabel . present

I've hidden away all the details about going from XML to my internal types to FGL. I mainly wanted to show how painless getting from FGL to graphviz was, basically a three-liner (although writing that 3-liner sure took a while as I tried to work my way through the API!)

So what next?

The whole point of Write First is to help you get your brain in gear, right? Now I'm not so sure what to do. I guess I ought to tweak this process a little bit so that I'm producing one-graph-per-sentence on a whole text file. Then I can sort of take a tour of the RMRS outputs for my whole example file and give myself a bit of an idea how I would go about doing relations extraction. Now that I've got a Haskell ADT and an explicit graph representation of the semantics, maybe this sort of thing won't be so hard.

PS:While I'm grateful for the free service/storage/etc, I must say that Blogger is a real pain in the butt with its eating your formatting on verbatim (well, almost verbatim) data. If I weren't so lazy, I guess I'd move to one of these Markdown-based blog engines where I'm not writing pseudo-HTML with interpreted line breaks but some proper text format.

bit of rmrs

Aside from a short meeting yesterday morning, I was mainly working on understanding the RMRS output from RASP.

I started work on a small Haskell script to convert the XML into something easier for me to look at. It doesn't get very far, but it does convert this angly blob:

<ep cfrom='-1' cto='-1'><realpred lemma='according' pos='p'/><label vid='1'/><var sort='e' vid='2'/></ep>
<ep cfrom='-1' cto='-1'><realpred lemma='to' pos='p'/><label vid='3'/><var sort='e' vid='4'/></ep>
<ep cfrom='-1' cto='-1'><realpred lemma='the' pos='q'/><label vid='5'/><var sort='x' vid='6'/></ep>
<ep cfrom='-1' cto='-1'><gpred>prpstn_m_rel</gpred><label vid='34'/><var sort='h' vid='35'/></ep>
<ep cfrom='-1' cto='-1'><realpred lemma='inscription' pos='n'/><label vid='7'/><var sort='x' vid='6'/></ep>
<ep cfrom='-1' cto='-1'><realpred lemma='engrave' pos='v'/><label vid='9'/><var sort='e' vid='10'/></ep>

into the much easier

l1:according(e2)
l3:to(e4)
l5:the(x6)
l34:_prpstn_m_rel_(h35)
l7:inscription(x6)
l9:engrave(e10)

Doing this kind of dumb translation has been somewhat useful because it helped to do (re)discover (R)MRS in a somewhat piecemeal fashion. It's about tiny details, right? For example, I was initially representing labels as data Label = Label String Int until I discovered on closer inspection than unlike variables, you only ever get "l" as a prefix for labels so I could just use data Label = Label Int (could be a newtype instead). It may sound like a triviality, but somehow getting to grips with these kinds of little details helps me to solidify the more important ideas.

review and millimeter forward

Didn't get all that much done yesterday. Now that I'm transitioning to a once-per-day email check, it makes sense for me to consolidate my email infrastructure a bit more. So I spent some time fiddling with getmail to check both my work and my gmail account (feeding everything a single procmail script). Also modified my mailer config to get rid of all the stuff that makes checking mail convenient. Finally, I did my weekly review of pending tasks and things I'm waiting for, something I'd neglected for over a month now. So lots of cobwebs to dust off.

With that out of the day, I proceeded to try and write that slightly-beyond-hello-world JAPE rule... but somehow couldn't really bring myself to it. I was just about to write a rule that would match X "entered" Y, but then I noticed that this is something that MultiPaX already picks up from MiniPar, so I thought that maybe it'd be worthwhile to write the little bit of code needed to transform MultiPaX output into what the UI wants. It's one of those cases where you get something that looks like it's doing what you want, but on just ever so slightly closer inspection, is pretty useless. But put more positively, this means that I've finally gotten a working interface between a relations extractor, the Duct Tape Monster and the clicky clicky GUI.

Another tiny bit of progress I made was to stare at RASP's RMRS output a little bit more and wonder to myself what I'd need to do to understand that XML. I need to refresh myself a bit on MRS, nothing too unfamiliar. I also need to understand what's behind the (R)obust. But maybe most of all, I just need to write some scripts. I think I tried installing the LKB (Delph-In) to see if there were any handy tools out there... but couldn't find anything. Maybe I'd better just write my own code.

Meanwhile, I was thinking that one nice thing about this JAPE language thing is that since I can pick up arbitrary annotations, I could actually maybe write some rules that go behind a proper parser. So you get a parser to pick out nice things like NPs, and you write rules like NP "entered" NP... Using some fairly low tech stuff to extract useful bits out of something higher tech and more powerful.

JAPE hello world

On Saturday, I managed to do two dinky little things:

1. Locate an hierarchical table of all the entities and relations in the CIDOC Conceptual Reference Model. It's just a 5 page table near the front of the PDF, the kind of thing I needed to get a birds eye view of the ontology and also a listing of entities I should be caring about.
2. Write Hello World in JAPE:

   Phase:firstpass // no idea what this does
   Input:Token
   Options:control = appelt
   
   Rule:JohnLoveMary
   Priority:20
   (
   { Token.string == "John" }
   { Token.string == "loves" }
   { Token.string == "Mary" }
   ): label
   -->
   :label.Relation = { subject="j", object="m", predicate="luuurve"
                    , rule = "JohnLoveMary" }

   This sort of thing should have been a cakewalk, but I getting tripped up over stupid stuff. For example, for the longest time, I couldn't figure out why my transducer was doing absolutely nothing. Then I figured out that in the clicky-clicky interface (as much as I'd hate to admit it, GUIs are realy nice for getting started when you have no clue what's going on) you have to re-initialise the transducer component if you make any changes to the file. Stupid stuff like that is why I have to stop worrying so much about what I'm trying to do -- relations extraction -- and start thinking more in terms of hello world.
   
   For what it's worth, this is very slightly more than Hello World, because it contains the ideas of (A) matching on a sequence of tokens instead of just one (B) dumping features into the output (like subject, object, predicate).

Next up

After this tiny little bit of progress, I want to combine the two strands; I just want to find one useful relation (as in using the CIDOC CRM) that I can extract using this sort of Hello World esque rule. Having done that, I'd like to plug it into the Duct Tape Monster, getting out of it a terribly useful automatic relations extractor that knows how to extract exactly one relation... and only just...

I might also like to go back to this RASP RMRS stuff, because, well semantic graphs are a lot more interesting than triplets.

PS: This is the second of my write-first exercises! I'm proud to say that this is the first time since thesis-writing days that the first thing I did in the morning was not to check my email

more MultiPaX and GATE stuff

On Friday, I made a bit more progress playing with MultiPaX for extracting relations out of text. I've managed to save my MultiPaX pipeline into a gapp file, and after going through a little bit of debugging (*) and API spelunking, run a MiniPar-to-MultiPAX without the graphical interface. Also learned that the XML output module seems to do something reasonably sane with the annotated relations, so at least I think I have a chance of turning the output XML into something I can use.

The only problem is the relations themselves. The output does not seem to have been very useful. It's all verb(subject, object) triplets, which is fine except that the subject and objects I get tend not to be very interesting. I've also managed to plug RASP into MultiPaX, but the output doesn't really seem to be any closer to what I'm looking for either. This is no fault of the authors, mind you; they probably never promised that they could do my job for me ;-)

So I guess I have three possible routes to go down:

1. I plan to learn how to write the hello world using the JAPE transducer package from GATE. JAPE is actually kind of powerful in that on the LHS, you can match tokens on features that other tools in the pipeline have marked up, and also because on the RHS you can use arbitrary Java, so well. Anyway, the hope is that I can at least write a stupid JAPE rule that extracts a triplet out of the sentence "John loves Mary". That stupid little bit would be progress for me.
2. I could think of turning the MultiPaX output triplets into the format I want, maybe mapping a subset verbs into relations from the ontology, but with the subjects and objects given, I'm not so inclined to do so.
3. I could go play more with RMRS output from RASP (which I think MultiPaX does not use, probably just the parse tree or something). I'd read somewhere that you can see an MRS as just being a sort of graph, which made perfect no-duh sense when I saw it, but which kind of makes me go "huh?" with a little distance. So I'd like to be able to reconstruct such a graph structure from an RMRS formula. Find or write a script that can turn an RMRS XML file into graphviz output (or at least the textual format). This sort of exploration would give me a better idea what I really can do with RASP.

(*) Sigh, Java and null pointer exceptions... I think the humble option type could make a huge difference in every day programming. Get all these Java hackers moving on to Scala already. That's the trouble with programming. It's so easy to make something that looks like it works, but then when your thing breaks because of a null pointer exception through some trivial unexpected use of your code, Somebody Else has to clean up the mess. At least the offending code was open source.

habitudes

This has been a busy week... unfortunately so much in terms of getting work-work done. Was coping with a hard disk failure on my laptop (saved by Time Capsule) and catching up with some Darcs stuff. It's been a strange cycle lately of not getting much done during the week and then feeling guilty and so coming in to work on the weekends... and then not getting much done during the week again.

Email diet

In the past couple of weeks, I've also gotten it into my head that I need to go on an email diet. I find myself spending just writing email on the time and this kind of needs to stop. What I'm trying to implement is a new policy of checking email only once a day (and being very very public about it in the hope that this will keep me honest).

This is my thinking on why checking email only once a day is a good idea:

1. It reduces overhead. Sometimes a lot of email triage just consists of deleting the message because it's some sort of automated noise or whatever. I reckon that such have clearly equivalent value whether you see them within the hour or within the day. We can say that the cost of checking your mail is C * N, where C is the task-switching cost (the huh-what? factor) and N is the number of times you do a check. Let's drive N down to 1.
2. It avoids me using email as a sort of self-distraction tool. OK bored now, email! now! now! now! I have a mental "ring fence" of websites that for some mysterious reason I have been able to enforce not looking at during work hours (facebook, reddit). Now I've added Quora to the list and am starting to shift email to it after my daily check. I suspect that part of the mysterious power of the ring fence is that it contains a very small number of obviously Bad (pathological-behaviour-inducing) sites. If I grew the list too much then I may find the discipline in not visiting these sites breaking down. Anyway, email goes in the ring fence.
3. It helps me get a better feel for how much time I spend doing email a day. If I'm always doing it in short little spurts here and there, I may not feel the actual cost. But if I sweep it all into one big pile, then it becomes crystal clear.
4. It avoids the slippery slope of compromise. Some of my friends think this is impractical (I hope they're wrong!) and then I should compromise by only limiting myself to once every few hours or something. But no, no, it doesn't work that way for slightly chaotic people like me. It has to be all or nothing because if I give myself an inch...
5. It lets problems take care of themselves. Sometimes you don't need to intervene, so you can just efficiently archive the whole thread and be done with it. In the case of public mailing lists (particularly, darcs-users which I feel a certain responsibility for), this gives a chance for other people to reply to messages first. I still have a bad habit of thinking I have to reply to everything, which is clearly insane.
   
   Interestingly, this bad habit is compounded by a Getting Things Done / Inbox Zero mentality where I kind of want to make sure everything that comes through my inbox has been tracked/processed/dealt with. The problem is that in the case of public mailing lists, it's not always clear who has the responsibility of answering what messages. If everybody leaves Somebody Else to do it, then does that mean messages go unanswered? Maybe there's a technical solution, some way of tracking messages like tickets. Or maybe the answer is more social, that we need to have clearer lines of responsibility, like X is responsible for answering user queries.
6. It makes me less of an information polluter. I tend to be very very wordy (how strange, I think a past version of myself was too terse for people). If Somebody Else responds to an email first, then hopefully they'll be more to the point. Also, if I only have one chance to get it right and if I know that there will be a 24 hour delay before my next chance to emit, perhaps this will make me more careful with my choice of words. Finally, doing the email in one big batch will hopefully also tamp down on my desire to weigh in on stuff, simply from the email fatigue.
7. Tricky one - It will hopefully train people I correspond with NOT to treat email as a form of instant communication. It's fine if you want to be a slave to your email; just don't drag me into it. You want an instant reply? Tough shit, you're not getting it. I'm not sure if this one will really work out and I'm slightly afraid that I may not be able to win this battle (which would ruin the diet). For now, I've put my phone number in my signature saying that this is what people should use if they want to contact me. I'd also put my jabber address, but I'm afraid of people confusing it with email. Sigh, I wish jabber addresses looked different.

We'll have to see how long this one lasts. I hear that out there in the Real World, some people only check their email like once or twice a week, which sounds pretty unbelievable to me. Note that I've been cheating a lot already, and in fact only yesterday was I really able to be very serious about the idea (although I did break down and peek around 18:00).

Write first

A new-old habit which occurred to me this morning is that I should bring back the write-first habit from the thesis-writing days, only this time making it a koweynlg posting. or something.

The idea is that (A) it's really sad that the very first thing I do in the morning is email... seems that something else should be on my mind instead (B) having email be a one-time dose-controlled thing means that I can just it as a reward for some other activity, so I thought maybe what I should implement is Write First...

I should start the day by writing what I've been working on (write-to-think) and what I want to do. This should help me get started a bit easier because I find that my work day tends to involve a lot of initial huh-what-am-I-trying-to-do-again? Hopefully there will also be some benefit to making this the first thing I think about when I get here and not some email thing. And then once I've got my little bit of work written down and sort of know what I need to be doing today, I can reward myself my with The Email Check.

Is any of this stuff going to last? We'll have to see. Getting Things Done has lasted, so maybe some of this other stuff will as well...

minipar, pax and GATE

Was trying to play with this JAPE transducer module that GATE supplies. While Googling a bit, I stumbled on a 3rd party GATE component called PAX that seems to do a bit of what I want, extracting something that looks a bit more like relations between entities.

A little bit of yak shaving involved (plugging MiniPar into GATE means realising that the plugin ships its own minipar binary, which is dynamically linked against an old libstdc++ but also supplies the source, which you have to recompile, blah de blah)... and well, it seems to work, but nothing particularly useful.

But dang, minipar is fast and it doesn't have this startup overhead that RASP does. Haven't quite worked out how to plug PAX into RASP yet (I can't tell if I need a custom script or something), but it's possible in principle.

Got to admit, this idea of providing a universal framework into which you can plug all sorts of 3rd party components that talk to each other is really attractive. Still not sure how well it really works in practice... getting the impression that it's mostly just components within the same family that share enough assumptions that combining them actually leads to something useful (say, POS tag sets). But maybe I'm wrong. Maybe any Haskell NLP components I make should come with little bits of Java (or Scala) that would let me plug into GATE...

extracting stuff from databases

Just a tiny bit of work today combining my practically non-existent SQL-fu with some shell scripts.

Have systematically dumped all columns from all tables in a Louvre database that had an "en" field (for English, albeit with lots of the "English" actually being en francais). Since I have no idea what I'm doing, I'm trying to be slightly more methodical by writing what I want to express as a bash script first, instead of plugging away at the command line. I keep all these little scriplets in a subdirectory in the event that I'll need to reproduce the procedure. The scripts are really just stupid things like "select en from foo where en is not null", plus a little bit of sed.

The sort of things that take a few minutes to do if you could just sort of set your mind to it without going "bleurgh, big database..."

This sort of thing should have been done a couple of weeks ago. Just took me that long to get my act a little bit together.

Anyway, now I have some simple lists of things I can use as a lexicon.

One thing that I'd like to know how to do with mysql is make queries like "select all the tables from foo where there is a column of type text". Not too important; I just show tables; then do a for loop showing columns on all tables and then grep. A little shell scripting can go far.

Update: I'm not sure my what my blockage is wrt databases. Writing these queries just seems like playing with some list comprehensions... I just remember walking out of that undergrad class 10 years ago with the conviction that I was really bad at this stuff (one of those things where it was all too easy to write something which seems to make sense but actually delivers totally the wrong result)

bit of progress

OK! Bit more progress this weekend. Once I managed to identify a progression of tasks, each of which with just the right granularity for me to get traction I was able to tackle them one at a time.

Traction 1

I really have to train myself to see what is the simplest stupidest possible task I could work on that would move me in the right direction. So for example in the context of parameterising the information extraction system from domain-specific data, I really should not have been worrying too much about databases and thesauri. That was not the right angle of attack, just led to a sort of inability to latch on. The way I got traction was to boil the task down into something very very simple.

• Forget the web interface
• Forget automation, the interaction with duct tape monster
• Forget generality
• Forget the notion of data from a 3rd party source

Just taking the GATE graphical interface and an example text, can I write a custom gazetteer that would find locations in my text? Where do I get my list of custom locations from? Well, duh, the text itself. It has the word "Cumae" in it, so let's see if I can get GATE to recognise "Cumae" as a location. Dead simple. Took me a tiny bit of RTFM, done. More progress in five minutes than I'd made in two weeks. It's all about applying a set of successful habits, such as boiling tasks down into the simplest possible thing that would still be novel for me.

Traction 2

Now that I know how to pick up something with my custom gazetteer, the only thing I needed to do was to plug in into the duct tape monster. But since I'd gotten the basic GATE concepts down from doing the simplest possible thing, it was just a matter of wading through API and example documents. Actually, I didn't really want to wade through the API. I just remembered from perusing the docs that it was possible in principle to build embedded GATE applications that were defined in some sort of XML file (gapp). So I found an example on the wiki of loading a GATE pipeline from a gapp file (like how you'd build a GUI in something like Glade), and struggled for a little bit until I got further.

The struggle was that my pipeline seemed to be producing boring old default ANNIE output.

• One trick there was to try disabling the standard gazetteer in my pipeline... only to find that I was annotating standard things. That trick lead me to understand that I'd foolishly copy+pasted bricolaged my code without knowing what I was doing. Basically while I did successfully load the custom pipeline, I was still loading the standard one after that, basically doing annotation in two passes and clobbering my output. So here, I guess the trick is: Strip out standard components and see what's left
• A second trick which helped me was to plug in the path for the standard ANNIE gazetteer resource files into my custom gazetteer. This allowed me to see the custom gazetteer actually doing something reasonable and incrementally adapt it to my custom stuff. In other words, Work backwards from a known working state

tricks for the distractable

So that's three tricks right there, general techniques for gaining traction if you have as hard a time with focus as I do sometimes:

1. Find the absolute stupidest possible task that could still be considered progress, ie. that teaches you something you didn't already know.
2. Strip out standard components and see what's left
3. Work backwards from a known working state..

Not rocket science, just a matter of the right habits.

Now I guess the next step for me is to try and extract stuff from my database, see if I can use it to write some customised lists for the gazetteer. Still fairly stupid in that we're not doing any kind of tricks with the data, just using it for a list of unknown words... but that's the idea. It's still progress. Just focus on making little incremental bits of progress. I should know this by now...

come on, focus...

Part of achieving traction is being able to break down the task into smaller pieces, small enough that you can bring your mind back to the task at the at hand, small enough that your natural obsessiveness takes over and compensates for the extreme distractability.

So... on the agenda is to tackle parameterising the annotator with domain specific resources. I think it goes a little like this:

1. learn to customise gazetteer in GATE GUI (with vim)
2. learn to embed customised gazetteer in standalone annie
3. populate customised gazetteer from DB

Since these are granular enough, I think they'd better go into hsgtd

insufficient progress

Having immense trouble concentrating these past couple of weeks for some reason. I have two core jobs on my hands right now:

• Parametrise the automatic annotator with domain-specific information
• Extract relations

The kinds of difficulties are sort of Eric-internal-cognitive more than anything else. Part of the time I was grappling with the notion of a thesaurus and probably just having trouble concretely visualising how I'd use one to help us pin words down to a spot in the ontology like crm:E53.Place. So that's one mental failing: inability to make progress without concrete examples to guide my thought process... I guess I should just recognise this tendency and be very very efficient about turning things into concrete examples.

Another issue is sort of an attention-control/focus issue. I get the impression that if I could just train myself to focus on a problem for more than 5 seconds I'd be able to make the sort of tiny breakthroughs I need to connect two thoughts together.

Hard to say what I mean... for example, working on this database stuff, right? What tends to happen to me is that I make very slow progress just by not being experienced with databases, just not knowing the syntax for things like "show me all the tables in this database" (SHOW TABLES in MySQL), or "show me all the items in this table" (SELECT * FROM foo)... simple things, right? Very quick to look up. But somehow, I have this sort of tendency to wander off when I encounter one of these barriers, lose my place and then have to start all over (what was I doing again?). If I'd just focus for a couple more seconds, I'd have that reference documentation up, just the few extra seconds more of thinking about it... Gah. These kinds of things shouldn't be blocking.

It's a question of traction: once I find a hook, once I find that sort of place where one thought leads to another thought and the next one in a sort of smooth succession down the road to completion, I can get things done... But often I find myself in a sort of muddle where 2 thoughts fail to connect because I just don't give them the time. It sounds so simple: if wandering off every 3 seconds prevents you from completing a thought, don't wander off (I don't mean physically walking off, which I seem to do a bit less than I used to), but maybe something like checking my mail instead. I guess I need to train myself or something. Something helpful was to whip out the old Pomodoro timer and force myself to work for 10 minutes, then up to 15, then up again. See if I get further tomorrow.

Anyway... the kind of thinking I've been failing to do is that which consists in breaking the task down into smaller parts. Here's my progress on that front

domain specific information

• extract useful information, even a lexicon from current catalogues
• teach ANNIE to use such resources (eg. plug in a custom gazetteer)
• blocking point (the kind that makes me wander off: not knowing what are good examples of possible annotations our ontology supports [answer, get CIDOC CRM reference])

detecting relations

• extend duct tape monster to support relations (now done, with dummy relations) - DONE
• check for ANNIE relations support - DONE (yes, you can write JAPE grammars that return triples)
• learn how to write JAPE grammar to extract an extremely simple relation
• learn how to plug in my custom JAPE grammar into the duct tape monster
• determine good relations to extract in sample (look it up in ontology)
• learn how to get relations from RASP [progress: I now know how to turn sentences into MRSes (I think)... seems like dangerous knowledge for a surface realisation guy...]