read wider

Three things I constantly forget to do:

1. read
2. review my action/waiting lists
3. check my calendar

I get so caught up in trying to put things together that I never get around to these feeding activities.  I guess there are three basic kinds of reading that I'm aware of:

• Depth - this is the sort of unspooling-the-ball-of-yarn reading I did in the first part of the PhD, a lot of "OK, I want to learn about X... wait I don't understand X, maybe I need to read background Y and Z... oh wait, I don't understand those either... and so forth"
• Breadth - somebody had to teach me this (thanks B!): you need to build up a sort of awareness, not only about what's going on right now but just more generally how the whole field fits together.  You're not reading to understand things here but (A) to know that they exist (B) to understand what things are connected to what other things.  Benefits to this are not being stuck in your sub-sub-sub-sub-specialisation (being a bit more open to cross-fertilisation) and also having an easier time when you're reading for depth (I find that it takes me several passes to understand something and having some reading-for-breadth awareness that X is a software for Y makes it easier to understand X)
• Hell of it - this is the sort of reading I do on No Screen Time Sunday.  Things like Language and Society undergrad sociolinguistics textbook I unearthed during a trip home, or Evidence Based Teaching (although I don't lecture), Designing for the Digital Age, just whatever strikes your fancy.  It's kind of depth-ish because you're going in book-deep, but not really trying to unspool any balls of yarn.  Just letting the author take you on a journey.  It's kind of breadth-ish because you want to cover ground that's outside your discipline.  But unlike the first two it has a kind of purity of being not-so-goal-directed.

So I guess I used to do reading for depth, never much reading for breadth.  Now I tend to do a lot more reading for the hell of it, but I've totally neglected the first two.  Sitting here waiting for my program to crunch through the first 10% of Europarl, I just realised that I have a chance to do a little reading for breadth.  You know, look at the ACL anthology.  Read the titles, read the abstract, flip through the paper and get that glimpse of awareness.  This guy is connected to that guy.  This algorithm builds off that one. 

Kind of like twitter for research.

dunning 1993 - accurate methods for Statistics of Surprise and Coincidence

Call me childish. The provocativeness of this introduction makes me very motivated to read this 1993 paper by Dunning, Accurate Methods for the Statistics of Surprise and Coincidence

There has been a recent trend back towards the statistical analysis of text. This trend has resulted in a number of researchers doing good work in information retrieval and natural language processing in general. Unfortunately much of their work has been characterized by a cavalier approach to the statistical issues raised by the results.

I still aspire to understanding all this statistical mumbo jumbo one day, not because I like the idea (au contraire) because I felt it would irresponsible for me to say I don't like it without knowing more about it. Djame has suggested that I bootstrap from what I do know, asking myself questions like "how do you probabalise CKY?"

In the meantime, I have taken a small detour by giving up on learning the statistical NLP stuff and just focusing on trying to learn some basic statistics. That stuff, I actually like in a sense, because instead of being voodoo/mumbo-jumbo/black-box thinking, it's actually quite "respectful" in an intellectual sense. There are principles at work, the quest for truth (if you'll excuse my cringey self-important language), basically how can you make sure you're not being misled by your data?

Of course, I never got very far in that either, no time or brains. Sigh... hopefully I'll be able to understand this Dunning paper. It looks promising

exps and weights

exp foo

I often complain about the terse nature of math notation (mathematicians know what they're doing so the terseness is probably a feature, cf Haskell code).

Another source of difficulty is just plain ignorance on my part. A little bit of notation I google-learned today:

exp(foo) means e^^foo

That wasn't so hard, was it?

WASP weights

For generation,

As for the lexical weights (P_w)...


Huh?

Oh!

Lexical weight formula and example from Koehn 2003, Statistical phrase-based translation. Can't have one without the other... need formula to understand example and example to understand formula. Notice their very helpful use of whitespace to make the example easier to parse visually!

But wait, wait! What does it mean to take the lexical weight of an SCFG rule? What do we do with the non-terminals? Answer in source code, more later!

Update 16:51Ok, just stared at this source code for half an hour. Get the impression that we're also treating the parse tree fragments as lexical items (which I understand we do on the GIZA++ level, but does it really make sense to do it on the param estimation level? Would it make any sense at all to do this for the words?) The other thing is that since I don't understand the code, I'll just have to assume that we ignore the non-terminals and go with any terminal symbols in the production, which would seem relatively sane. Sigh, Java...

various notes

implementation progress

I now have a small Haskell wrapper around GIZA++. It only outputs word alignments. I've also extended my Chiang 2005 implementation with one constraint, that

If there are multiple initial phrase pairs containing the same set of alignments, only the smallest is kept.

This is one of those devil-in-detail moments: define small? define same set of alignments? I went with size of source phrase + size of target phrase, and went with considering the powerset of alignments for a given pp.

I'd be happy to release it (provided that University policy allows me to do so) soonish in the hopes that other people can improve it.

lambda SCFG

Ok, now I think I understand this better. Djame, you're right. C'est pas complique. I just like to confuse myself for nothing.

Structured questions to help me understand Wong and Mooney's λ-SCFGs:

1. Where do functions and their arguments come from?
   
   • functions come from the expansion of a non-terminal symbol. The RHS2 of each SCFG rule is a lambda expression (λx₁..λx_n.β). (note that n can be zero, I think)
   • the arguments come from a requirement in the formalism that "Each non-terminal A_j in β is followed by a list of k_j arguments (k_j can be 0)." In other words, the arguments come from the top down
2. When does function application take place?
   • at the end of SCFG derivation (nothing special). Presumably the lambda expression associated with the start symbol of your grammar must not have any lambda operators
   • there is no "outermost" lambda term and therefore no "outermost" argument (i.e. corresponding to the start symbol)... You traverse the expression produced by derivation and reduce any lambda terms you see inside of that expression:
     

     whatisthe(x1,
     (λa1.smallest(x2,(
     (λb1.state(b1))(a1),
     (λb1.λb2.area(b1,b2))(a1,x2)
     )))(x1)
     )
     

   • My first instinct reacting to the above is "can you do that?"... but then I guess, "why not?"
     
3. Why is it possible to have free variables in λ-SCFG lambda terms?
   • I'm still a bit shaky on this one.
     
   • Consider the use of smallest below. How do we guarantee that if we try to parse a sentence like "which state has either the smallest river or the smallest mountain" that we generate a new instance of the size variable (x₂) for each one?
   • Perhaps one answer seems to be that λ-SCFG requires that you systematically rename all the free variables in a lambda term before doing function application. (Well, I guess in case the name conflicts with a variable name either free or bound in the argument is enough). Yeah... I guess that's enough to guarantee that we're generating a new instance of each free variable per use. I think this is why I was confusedly trying to say that they were like let..in.
     
   • Hah, I had circled x₂ but of course x₁ is also free above

limits 1 (inversion and rotation)

I like to map myself out, try to understand my shortcomings (and should I be so lucky, strengths). Here are two related thing I've noticed about myself:

1. I have a very hard time with things that have polarity, for lack of better term. True vs False, Plus vs Minus, Source vs Target, Then vs Else, Left vs Right. I cannot cope well with inversions or inversions of inversions. I'm always getting things backwards. This is why things like subsumption are a nightmare for me (X subsumes Y, or X is subsumed by Y). I get the idea, (I think), but putting potentially inversible things together and I'm bound to screw something up. Unfortunately, you can't type-check your way around a boolean-inversion problem.
2. I have a hard time dealing with rotation. I think this actually the same problem as #1. I cannot mentally rotate things (or ideas!) and reason about the results. When I look at a map, it takes me hours to reason my way through things. I have to physically turn the map around and still it takes me a long time to work out where I am and where to go.

If I have a clear picture of this, then I can hopefully use my strengths (few as they may be) to offset my shortcomings. The danger of course is that I attach myself to a potentially incorrect picture of myself, or one which is no longer true with the present circumstances. The other danger is taking universal things (maybe everybody sucks at inversion and rotation) and thinking them to be specifically true of myself. But I think for the case of #1 and #2, I am probably worse at it than my peers.

P.S. none of this explains my confusion of the lambda SCFGs, probably another kettle o poisson.

wasp and node merging

One of many things I don't really understand about this Wong and Mooney SCFG stuff. So there's two cases where acquiring a lexicon (set of SCFG rules) can fail for a production p:

1. none of the descendants of p in the MR parse tree are linked to any words
2. the NL string associated with p covers a word w linked to a production p' that is not a descendant of p in the MR parse tree

the complicated case is easy

The second case (complicated as it sounds) is actually quite clear. If you have

• semantics: left(penaltyarea(our))
• English: our left penalty area

You've got a problem, because in effect you're trying to extract an SCFG out of these two non-isomorphic trees:

• left(our-penalty-area)
• our(left-penalty-area)

the simple case...

On the other hand, the first case is actually not so clear to me. The idea is that if you've got a something like

• semantics: should-stay-in(goalarea(our),goalie(our))
• English: the goalie should stay in our goal area

lexical acquisition may fail because the "our" in our goalie was not realised... but here's what I don't get: wouldn't we just align "the" with "our" and be done with that? I guess maybe what happens is that you don't see enough instances of "the == our" for the alignment to work.

Hmm.

something unrelated

I keep forgetting how the rule extraction works. I always vaguely remember that it's simple and straightforward, but then I can't conjure up a sensible recipe from memory.

So here it is again:

• traverse the MR parse bottom-up

That's it... start from the leaves and work your way up doing the obvious thing. Wong talks about starting from the rear end of the linearised MR parse, which is the same thing (flatten a tree traversal and see what I mean)

eric can't read

Or can't think or whatever. After me going around in circles trying to grasp at the essence of the Chiang ACL paper, AB tells me about the new CL paper which supersedes the ACL one. And yes! Nice and concise sentence saying what I've been dimly trying to get at:

Second, in order to obtain rules from the phrases, we look for phrases that contain other phrases and replace the subphrases with nonterminal symbols.

So much electronic ink spent and I could have boiled it down to just that. Silly me

wasp notes

So some questions for myself about training WASP:

• why aren't the results better?
• why are our rules so big?
• why can't we train on entire forecasts

And I think I'm starting to have a glimmer of a clue, re-reading Yuk Wah Wong's thesis. Sometimes things don't sink in until you actually work with them a lot and this another one of those situations.

The reason that rules get big, as far as I can tell is that WASP merges them in order to preserve parse tree isomorphism. So merging is bad in WASP-land; it's a necessary evil. WASP also does some fancy link-cutting stuff to avoid merging.

The reason we don't like merging is, as Wong points out, that it leads to overfitting.

So one thing that may be useful for me to do is to hack WASP up (for just find the right knob) so that it tells me how much merging it's doing? Or maybe I can just tell by looking at my rules. One idea is that you might be able to tweak the grammars to avoid merging. Or somehow inspect the learning process. Hmm

poking at hiero 2005

I'm still reading that Chiang 2005 paper, A Hierarchical Phrase-Based Model for Statistical Machine Translation... this probably being my 6th pass? Yeah, I guess it would probably be more efficient for me to read more papers instead of trying to read the same paper in depth. Just can't let go though if I've got the feeling that I haven't "gotten" something yet and still have something trying to get to the bottom of.

local reorderings and such

Anyway, I highlighted this bit of text, bullet points mine:

Phrases, which can be any substring and not necessarily phrases in any syntactic theory, allow these models to learn

• local reorderings,
• translations of short idioms, or
• insertions and deletions that are sensitive to local context

I sure wish I had nice examples of each one of those bullet points

Xes all around

I've had a couple of minor ah-hah moments reading this paper. One of my favourites is still this very very simple idea:

Note that we have used only a single nonterminal symbol X instead of assigning syntactic categories to phrases.

And maybe that doesn't strike you as much. Me, I liked it. It means you're looking at these grammars without any sort of mental baggage or pre-conceived notions of what they're supposed to do. You just look at them formally and say "well, what can we use these for?"... oh ok "translating between phrases". But I guess if you start from the MT notion of phrase as just "sequence of words", then this idea is obvious. But for me who's starting with all my preconceived notions of grammars, who is not starting from MT, I still expect my grammars to differentiate between syntactic categories. Which is where the ah-hah comes in. I had never bothered to ask myself "what if we didn't do this thing?". I don't have any ideas of my own yet and may never will. But for the moment, that ah-hah of reading is where all the satisfaction lies (so why do I resist reading so much? because it's hard, that's why)

building rules

So here is the second ah-hah I'm still trying to find in this paper. In my first 5 readings of the paper, I somehow glossed over the (crucial) process of building the SCFG. The thing is that I was carrying around my notions of "building-up" so I just naturally and stupidly read the process like that. You accumulate smaller things into bigger things.

But it doesn't really work that way here, does it? Crappy math notation and de-greekification mine):

if r = X -> (g,a) is a rule and (f_i^j,e_i'^j') is an initial phrase pair such that g = g1 f_i^j g2 and a = a1 f_i'^j' a2 then

X -> (g1 X_k g2, a1 X_k a2)

is a rule where k is an index not used in r

The point here is that you don't "build up" new rules as a naive Eric might expect. You tear them down. You see a bunch of initial phrase pairs, and then you refactor. I guess that's my second ah-hah moment for this paper. If you see a rule that has bits you can factor out, then generate a new rule with the common bits excised and replaced by a new X symbol.

But there's still something I don't feel that I grasp about this yet... some implication. Chiang talks about a rule (X 之一, one of X), which can be used to translate "yu Bei Han you bangjiao de shaoshu guojia 之一" into "one of the few countries that have diplomatic relations with North Korea". If I understand correctly though, you would never learn such a rule from such an example. Rather, you have to see a smaller example where an matching initial phrase pair from a corpus can be boiled out. You would learn the rule, for example from something like "benshu 之一" vs "one of the books" [sorry if I'm butchering the Chinese language].

That's how the recipe goes, right? You see the following initial phrase pairs:

1. benshu, the books
2. 之一, one of
3. benshu 之一, one of the books

Which you then trivially write out as the set of initial rules:

1. X -> (benshu, the books)
2. X -> (之一, one of)
3. X -> (benshu 之一, one of the books)

And then you start refactoring! You notice that one of the rules, notably #3 has a piece in common, notably the benshu, the books pair, so you create a new rule:

4. X -> (X_1 之一, one of X_1)

I suppose by the same mechanism you would also create this silly rule:

5. X -> (benshu X_1, X_1 the books)

And if you kept applying the mechanism, you would get the equally ridiculous:

6. X -> (X_1 X_2, X_2 X_1)

Except that the latter is filtered out by some extra processes that say that f_i^j must have length greater than 1 and that a rule must have at least one pair of aligned words.

Anyway, so we get the rule we want, which is fine and we also get some other rules we presumably don't want which is also fine... but I wonder how often we miss something just because there isn't a short example to learn from? Maybe this doesn't happen in practice because there are always enough short examples, or -- this is why I still feel like I'm missing a Key Insight -- maybe what happens is that you tend to learn them inductively, by biting off a small piece (which reduces them a bit), and telescoping down. Only I don't have a very good handle on how this telescoping process would work, or if this vision of telescoping actually corresponds to anything (I just envision a rule being able to cover successively larger phrases once you start replacing bits with X'es)... oh fuzzy-headed erician thinking that never gets us anywhere.

Anyway, you look at the examples of rules that it learns and you can see that it works, and that it learns things that a standard flat phrased-based translation approach wouldn't like (在 X 下, under X)... the point here being that you can use this to translate phrases of arbitrary length as long as they start with a 在 and end with a 下. I guess I'm just confused whether you (a) only learn them from short examples or (b) somehow have a way of progressively cranking through long examples until you learn things from them too, say by virtue of repeated refactoring of initial phrase pairs.

crappy math notation

After playing around with charmap to type Unicode characters (cumbersome), and wasting an hour installing greasemonkey + latexforblogger + mathtex, I'm a bit frustrated about my inability to type the occasional mathy thing in blogger. The latexforblogger thing is cute, but I really don't want to produce a png for every stupid formula. Need to learn some more about MathML some day maybe. Meanwhile, I'm going the route of (1) de-greekification (2) simplifying of puncutation and (3) subscript deletion when using numerical subscripts. If I had some handy greasemonkey scripts or whatever for typing in common Unicode characters, we might be able to talk...

more scfgs

Have not been working much lately (focusing on darcs a bit, but trying to steer things back into work-work a bit more)

No this blog isn't really back yet, but I guess I could use it jot random stuff down.

Been rereading Chiang 2005 : A Hierarchical Phrase-Based Model for Statistical Machine Translation, and trying to make sure I understood it better. Also want to revisit the WASP stuff and see what they say about training.

churn churn

Well, I haven't produced much this week... but I think I've tried hard enough that I can take the whole weekend off.

researcher dimensions

Idea I've been kicking around are key parameters for a researcher

personality

attitudes, tastes, temperament

ability

intelligence, creativity, aptitudes (classes of things you're likely to be good at)

knowledge

skills (e.g. coding), breadth (needed for cross fertilisation), depth (needed for follow-through), self-knowledge

methodology

reading,publishing,"reacting",talking with people

The basic point is that you can more easily control the latter two than the first two.

Put another way, personality and ability are irrelevant... sure maybe they will help to distinguish the grunts from the superstars. But hey, look, I don't want to be a superstar. I just want to figure out how to do my job, (a) to make sure I continue to have a [useful and non-boring] job and (b) to reassure myself that my keep is being earnt. Since I am merely trying to figure out how to be your standard run-of-the-mill research grunt, what I need to be focusing on are knowledge and methodology.

No amount of dual n-back is going to fix Eric-is-dumb, so don't worry about it. What you have to do is (i) make sure he knows as much different shit as humanly possible and knows it well (ii) change the way he does things. An example of poor methodology is sitting around writing blog posts, or sitting around worrying that you're doing it all wrong instead of doing it (even if all wrong) [to be fair, the "doing it all wrong" consists of sitting around a lot]

hinted caledonian

I've been working on my hinted Caledonian MR grammar today. It occurred to me just this minute that I don't actually need logical variables here (so maybe I could go play with WASP 1.0 for a while). There may be some other domain where they come in handy, though. Making a little bit of progress

for monday

finish off CRU port of -light grammar. Make an aligned corpus. Try training WASP 1.0 on light grammar and see what happens.

parsing model / language model

Another entry for the Handbook of Simple Answers: what's the difference between a parsing model and a language model?

Lifted straight from p 415 of M+S (*)

parsing model

P(t|s,G) where SUM_t(P(t|s,G)=1

language model

P(t|G) where SUM_t(P(t|G)) = 1

The parsing model allows you to discriminate between different parses for a given sentence. The language model tells you only what the likelihood of any given tree is in the whole grammar. Conversion from language model to parsing model is easy [divide joint prob by prob of sentence], but not the other way around.

(*) This is a case of not keep enough things consistent across comparisons. In M+S they drop the (|G) for the language model [why, just to keep things looking nicer?], but that makes it hard for the dumb reader to understand the relationship between parsing and language model. Maybe there is a reason to do this, but it seems like this kind of inconsistent simplification is needlessly confusing.

M+S

Finished up the parsing chapter (ch12.2 up), although I don't think I absorbed anything from it [again, me being thick]. My reading of it was along the lines of blah blah blah model blah blah blah blah dependency blah blah blah Collins [woof!]. So, state of the art a decade ago was Collins 1997, dependency based parser with bits of linguistics creeping in, under 90% precision and recall... I wonder how the parsing community has been doing since. Note the

Qualitative breakthroughs may well require semantically richer lexical resources and probabilistic models

Seems also like the question of how to evaluate parsers should be poked at some more. Dependency grammar also looks like a worthwhile thing to explore (huh, is what that the recent flurry is about? here's me living under my rock)... Oh and the bigger picture of why we want to do parsing in the first place, presumably so that we can build meaning representations.

λ-SCFG

Argh! I'm confused again.

Another thing is that they claim they really are doing the lambda calculus, and not just a custom lambda-inspired variable binding mechanism... which throws up all my discomfirts about renaming free variables. Ok, fine, whatever, we just gotta trust people; besides I don't know nothing about lambdas [I'm just a user]

The confusion comes from actually trying to write a MR grammar. Need to remember that we don't use the lambdas here; it's just used for propagating logical variables correctly in the synchronous stuff. Just not sure about where the lambdas come from though. Also tend to mix up the actual tree I'm trying to build, with the derivation tree of the tree.

Oh, and here's one kind of thing I can get easily confused by:

answer(x1,
(λx1.smallest(x2,(
     (λx1.state(x1))(x1),
     (λx1.λx2.area(x1,x2))(x1,x2)
)))(x1)
)

Let's rewrite this so that somebody like me can understand it

answer(x1,
(λa1.smallest(x2,(
     (λb1.state(b1))(a1),
     (λb1.λb2.area(b1,b2))(a1,x2)
)))(x1)
)

Yeah I can understand deliberately shadowing that x1 to communicate the intention (it's the same x1 in the MR), but that just trips up people like me. Back to the original question. I understand that the MR is lambda-less... but how do we decide to attach lambdas (and arguments) to non-terminals when we're building up the SCFG? The answer is right there in the thesis (and highlighted even). A variable must be bound if it appears both inside the rule and outside of it. This way we can pass it down from the outside... make sure we get really the same variable. In that light, the only "free" variables are local ones, i.e., in the sense that given the current MR parse tree, you will never see that variable again.

... but when you actually try to use the SCFG, you may combine that rule with other rules that use the same variable name. Uh-oh. I guess then this is where you have this free-variable renaming thing. Because the rule might be used to construct whole new parse trees, and you really do have the possibility of wanting to substitute one of your bound variables with something which just so happens to be the same as your free variable.

I shouldn't be thinking of those free variables as globals, but as local lets. I wonder if there is a word for this not-really-free-variables. In my new hazy understanding of it, the lambda terms in an SCFG rule really are closures after all (if you think of those free variables like let..in). Seems like you'd want to introduce a new binding thingy, ι (iota) just to keep such variables unique. You don't do anything with ιs except alpha-convert when the need arises. At the end of the day, you just drop any ιs in your expression. But then again, I guess this kind of machinery really just amounts to their "you must rename your free variables" rule, so why bother? Only reason I'm resisting it is instinct, a spluttering protest, "but, but, you can't do that!" that only arises because you've never seen it done.

The core problem is that I don't actually know about lambda calculus. I've seen it used for natural language semantics, so I have a non-rigorous mechanical view of things (and now the alligator eats this family, and the egg hatches), and I've done some functional programming (which is why I talk in terms of globals and lets). But I don't actually know this stuff. Anyway, it is purely an angels on pinheads discussions. Not practical. Doesn't help me use the stuff.

fear in a high chair

maybe I just suck at this

I'm doing this all wrong. I keep telling myself that I'm doing this all wrong. I feel like I should doing more stuff, picking a direction and running with it. Instead, I just try to read things (which I don't understand, which just makes me fall asleep), or I spend a lot of time doodling, slash pacing the hall. Doodling and "thinking" is fine, as long as you got something to think with. All fire and no fuel.

It's not that I'm not trying. I'm just doing it all wrong. There is a difference.

At least I'm starting to put this stuff down in computer [in mindmap form; harness the chaos], so that I go around in circles less (less likely to forget what I just came across two days ago).

Furthermore, the Eric-is-dumb problem is not the point. Whatever my limitations are, I should be learning to work around them. If your brain is a lemon, make lemonaaaaaaaaaaade. It's all methodology, custom methodology. Yep.

underspecification for NLG

It occurred to me that I should go see what other people how other people use this notion of underspecification in NLG. Found a couple of papers, one on incremental generation (it uses this thing called Constraint Language for Lambda Structures), and then another thing by some folks in Italy that says something about mixing template based and NLG-based generation in a more principled fashion. I couldn't really read the articles very well, brain too foggy, rate of information absorption too low.

M+S

learning more about parsing (ch12) 12.1 is nice in a way, short bursts of info-nuggets. Was interested to learn about the PARSEVAL metric (which reminds me of some objections we had to n-gram based evaluation metrics; could something like this be applied? watch for easiness-to-fool though... or is that PTB related?). Grateful to be nudged into not thinking that PCFGs are the only way to add probabilities to a CFG (for example, p left-corner grammars [good thing I made the effort to understand that left-corner parsing stuff a couple of years ago]). Had a brief taste of dep parsing, very brief.

rtg

When searching for the above, I accidentally stumbled on some very fresh seminar slides (23 may 2008) on Regular tree grammars as an underspecification formalism [A. Koller]. I wasn't really paying attention (again more of this inability to absorb information quickly), but I sort of get this idea of eliminating redundant/identical unpackings from a packed structure (i.e. before you actually do the unpacking)... and saw some impressive looking performance estimates (e.g. something which should take billions of years would only take 75 seconds). Sounds like a pretty cool idea, another variant of, eliminating combinatorial blow-up via exact [non-lossy] methods by getting rid of the stupid stuff. [which is makes the polarities stuff cool; thanks, G!]. Or maybe I didn't actually understand the slides. But it may be good to look into this. Downloaded a Graehl and Knight paper called Training Tree Transducers which looks pretty incomprehensible to me (in the Eric-is-dumb sense)

underspecification bis

Still banging my head against the wall about this one. What the heck can I say about this? It occurred to me maybe that I am thinking about it completely the wrong way (no shit). Instead of saying "how do we introduce a notion of underspecification into our semantic inputs [which would work with the SCFG stuff] so as to permit saying things in more than one way", I should instead be working backwards, saying "what is the absolute minimum that goes into a message, and then how would we embellish it to guide the generator?"

For the trains time table, and just the simple examples we've seen, it looks like we can't say anything less than:

leave(train_1, city_1, times(t1,t2,t3))

Oh oops. I've implicitly aggregated the times (what if you want to print multiple slices, say tell about different cities). Would it be more something like

leave(train_1,city_1,t1), leave(train_1,city_1,t2), leave(train_1,city_1,t3)

? Hmm, ok, confusing myself. Maybe we should just make the assumption that pre-aggregated units are inviolable and that is that. In other words, if the input specification lumps something together, you can't split it up! (On the other hand, you could lump together other stuff if you wanted)
Idea then would be that you could add hints to bits of the message with extra information to guide the generator:

leave:[lex=depart](train_1:[ref=pronoun],city_1,times(t1,t2,t3))

Still not sure what to do about multi-sentential generation. We should really just have a philosophy of keeping things simple and stupid [because we don't understand all the literature out there]. Assume discourse is a tree, the leaves of which correspond to individual messages. Discourse relations are binary. No textual order implied [learned from context]. Still have no idea about what an appropriate inventory of relations should be, so I propose the make-it-up-as-we-go-along strategy (frantically flipping through related work to see what exists). Discourse relations can also be annotated in the same fashion with guiding information.

It's conceivable that this [the hints idea] could be made to work with the SCFG stuff. In its lexical acquisition phase, we would pick up the link between hints and specific connectives. Would also perform acquisition over un-hinted inputs, paired with same texts.

I'm also worried about the referring expressions. If we've got some entity to refer to, and it has some attributes (say, the big red onion), how do we make repeated references to it without having to duplicate all the attributes [baroque solution: go ahead and duplicate the attributes]. To be clear, we do not necessarily have to think about generating distinguishing descriptions [that would require more fancy tech], but we do want to be able to refer to it more than one way. Well, actually, what do we want really? We could just be happy with being able to discriminate between pronoun, common-noun phrase (the onion), and proper-name. We could treat the fully specified thing (the big red onion) the same as a proper name. We could maybe have a separate lookup table that maps from, say common-noun-phrase(x1) to "the onion" and pronoun(x1) to "it". We could then say that it's not the job of the probabilistic component to learn these. Just plug in templates like _COMMON-NOUN(x1)_ and let them be filled in with post-processing. All we do then is learn the distinction between types of expressions.

I have no idea what I'm saying. No justification for this, no prior research, no specific questions which lead me to think that this makes any sort of sense. Just making this shit up. It sort of sounds like an idea [likely a rehash of something I saw somewhere else]. Question is: how can I run with this? Could I write a MR grammar for a hinted input?

less fear

pcfg

Today M+S taught me about PCFGs. There really isn't much to it, is there? So each Foo -> ... gets a probability. Universe is the set of things that Foo could expand to. You want to

• figure out the probability of a sentence given a PCFG? - that's your inside and outside probabilities (like your forward and backwards)
• find the most likely parse for a sentence - do something Viterbi like
• assign weights to the rules - do EM, just like like HMM stuff you saw in second to last chapter

Oh believe me, I still don't understand all the math (sorry), and I'm still too much of a chickenshit to sit down and crank out a toy implementation as an exercise [there's your wheat and your chaff right there... wheat rolls up her sleeves, does this easy thing, and learns something from it]... But I do sorta see how the pieces fit together. I don't really understand how to transpose these ideas into what people do with other formalisms, but maybe that will come later. ch 12 looks like it might be kind of a fun read.

Also, I need to get around to understanding what these log-linear models are that everyone keeps talking about. Just get an idea so it doesn't phase me when I see the word. Barrel down that checklist.

trains

Tried putting the trains grammar together. First observation is that unless I'm willing to shift to a variable-free formalism, I'm going to need WASP 2.0, but then I'm WF for some tech support. Second observation is that we can't use the LHS to pass down logical variables (although, there is the lambda-SCFG stuff which we need to pay a little closer attention to). This isn't CRU; shift your thinking, curb your dogs. It all boils down to what we're trying to do with this grammar. Simple, it's just a grammar that tells you what all the possible MR strings are, right? Well, do we them to be fixed strings (with an order), or should we try to be flat-flat (literals can come in any order). I guess flat-flat is easier for users. But then again, fixed order does not imply any need for linguistic knowledge, just of the MR grammar. I'm going flat-flat for now because it's easier to fit into my text editor window, and writing the type annotations otherwise would be a pain in the butt. But it's not a good reason.

building applied natural language generation systems

Easy read. Noticed it from the Prodigy case for support (my Nth reading). I've seen this bit before, but I feel like I need a reminder

six tasks

1. content determination :: Data -> Set Message
   
2. discourse planning :: Set Message -> [Message]
   
3. sentence aggregation :: [Message] -> [[Message]]
   
4. lexicalisation
   
5. referring expression generation
   
6. linguistic realisation

a typical architecture

1. text planner [cd, d-plan, sent-agg]
2. sentence planner [lex, reg]
3. realisation

No, it's not parole de $DIEU but it's a pretty good overview, especially for somebody who has been realisation-realisation-realisation from 2003-2007. And of course, my type signatures are grossly oversimplifying things (for example, discourse planning probably throws in some extra information so that you know what's holding the list together), but for a dumb programmer like me, it helps.

Odd. It seems to be sticking better, this time, don't feel as lost. A lot of this is just repeating the blue book (as far as I can tell), but somehow I find that I "feel" this more, whereas before it would be just sorta rote. Guess it's just a feature of Erician Learning: frequent repetition from different sources (and angles) or else it don't stick (or make sense). Either that or the particular article I'm reading has just the right mix of high-level and comprehensiveness (good survey). Or maybe it's at the right time after me trying to wade through all this discourse-related literature, and then think about the semantic underspecification stuff. Or maybe I'm wrong and tomorrow I'll just be discouraged again.

Anyway, now the Caledonian example makes more sense. Now when I see a rule that expands into leave/departs, I get the idea that it's a about lexicalisation being modeled in the representational space. Kinda wish I noticed this paper way way earlier. It would have helped me understand what this project that I've been working on for the past 6 months is about.

occupational health

Had a workstation assessment from the occupational health folks. Good she came by. She thinks it's ok that I continue the vertical keyboard (since I like it). Spent an hour and a half looking at what I do. She got me to scoot my keyboard closer to me and raise my chair way up so that my arms were in a more natural position, raise my monitor (now got three monitor stands stacked below) and get a foot rest. Basically, either the keyboard goes down (impossible, since the desk is fixed), or I go up. She's also going to recommend that I get a vertical mouse.

Yeah, it seems obvious now that she mentions it. Really is much more natural to have this keyboard closer and my arms at its height. Only problem now is that if I raise everything up, working on paper becomes trickier (because desk is too far). So I have to lower my chair to deal with paper, and then raise it again to deal with computer. Not fluid. That, or find some stuff to effectively raise my desk, maybe big stacks of stuff to write on. Also, with the chair at a good height, my legs get squished by the desk. Sigh.

But yeah, it's good to have an actual person -- a pro -- do this, because she can help you figure out what all the constraints are and how they fit together. Double-good that the pro actually sees your situation so that she can tailor the advice to your needs.

constraints

It suddenly occurred to me that I am quite interested in constraints. Not necessarily in doing constraint programming, but in the idea that different generation tasks interact with each other in a non-pipeline fashion. How do we make systems that can deal with that? Or maybe I'm not as interested as I thought. Dunno. would be fun to build a truly non-directional generation. A bit like what ILG is doing with her rows and columns, but somehow using a more traditional (from the SE standpoint) notion of modules. Sure, sure, blackboard architecture's been done before and doesn't give so nice results in practice (yet). And yes, there is the SPUD stuff, which I have always liked. But maybe we can do more. Or less! Maybe a more modest version of this idea. Argh, nothing new in this little brain, but it's a good note to Future Eric.

Maybe Future Eric wants to work on this. Or maybe Future Eric wants to work on incremental generation. Or maybe Future Eric decides he really does suck at research after all [inner research is nowhere to be found], and wants to shift to teaching [which he might actually suck at too], or research-hacking [which would only be fun if working for the right person, i.e. with the right mix of Vision and realism], or (yuck) the Industry.

But this is weird. Why have I suddenly stopped feeling scared of the future, or not being good enough, or just Not Getting It? How long will it last? And can I turn it into something useful?

today

I haven't actually done anything today. I sat down and tried to think about underspecification. But then I got confused about why we want this in the first place. Then I tried reading our case of support, which lead me to the applied NLG paper, which suddenly made me feel calm. But I didn't actually do anything today. [hey, maybe we should use the WASP stuff for just realisation, and go back to trying to model all the hard stuff with CRU]

But stick a happy face tag anyway, because it's been a while.

tomorrow

what do we do tomorrow? More reading? More "thinking" about underspecification?

and then they fight

Sorta decently behaved today. Doing a decent job respecting my darcs-ban (although maybe I should see if I can get an MSR internship to go hack on it for 3 months... wonder if I could convince them to let me do that).

Still not producing very much though.

playing with WASP

Getting a bit of tech support on this. Guess we want the 2.0 version (which does generation and knows about lambdas [the wasp ones, not lambda lambdas].

bit more M+S

Breezed through ch10 of M+S (i.e. not really paying attention, just trying to pick up key words). Did not so much read, but daydream as the words passed by, thinking of possible tagging strategies.

One thing I'm wondering is if it would be any use to train a sort of meta-tagger than can classify words according to [I tend to get this one right] and [I tend to make mistakes on this one]. You could imagine that if the meta-tagger was fairly accurate, it could then do something like change, with a high probability, the tag it would pick for things that it tends to get wrong.

Another vague thought was that you can have a bunch of one-tag taggers, you could imagine taggers that are either very good at picking out NNs (or maybe a bit over-zealous about picking out NNs). They either would just tag everything else as X, or maybe do proper tagging, but somehow more sloppily. Then you make them fight. "Well, I'm a VBG tagger myself. I think that's a JJ right there [I don't actually know what these tags mean], but mister NN says he knows an NN when he sees one, so, like, whatever, man"

Sorry, nothing precise enough to be useful, and probably been done anyway.

wasp chart generation

I think I now have a better idea how the chart generation component of WASP works. Before, I vaguely understood that somewhere lurking beneath it was the standard Kay chart generation idea (non-overlapping bit vectors). But what these bit vectors corresponded to was a bit trickier. As usual, the answer is actually pretty simple.

1. Each bit corresponds to a production in the original MR grammar (I guess, note here that the MR part of the SCFG does not generate the same language as the original MR grammar). You have to (?) parse the MR using the original MR grammar.
2. Your initial chart items are the SCFG rules with a bit vector corresponding to the parts of the parse using the original MR grammar

flat semantics?

I wonder if there would be any help to switch to a genuine flat semantics all the way (this stuff here is already pretty flat, I guess, with a little stuff done to assure the commutativity and associativity of conjunction). Would it buy us anything here? I keep thinking that maybe if you're doing phrase-based alignment on your texts and MRs, it would be a bit easier (or maybe not, maybe using MR productions is actually better). Also, another issue which I would have to struggle to wrap my head around is what you do about passing variables around.

trains

Slowly advancing on the train grammar. Realised, for one thing that the grammar we use needs to be a whole lot simpler than what the CRU paper provides. We don't need three logical forms for train -- caledonian_express will do just fine -- nor do we need two and orderings. But somehow there is a nagging thought that the MR grammar at the same time needs to be more general. For example, it ought to generate all permutations of literals, perhaps. Wonder if it's too flat, in the sense that it would make it non-obvious how to pass logical variables down.

Anyway. Found a little trick for alignment. I just repeat rules to add 'padding' so that if there is an english rule with 2 variants, and the equivalent MR one only has one, I copy and paste the MR one (make a twin). Wrote a little Haskell program to turn the two files into a corpus (using the Galois lightweight xml package).

This is where the hour of daydreaming kicked in.

underspecification

Trying to work out a good way to do underspecification for generation. First I'm trying to figure out what underspecification is for (in NLG)... what are we really after? This will be the 5th time I've read this tech report, trying to squeeze something out of it, refresh my crumbling memory. On my fifth reread, I notice these things:

• generation space != representation space (in AB language)
• underspecification seems to basically mean 1 representation = many possible realisations (and in that light, the input we gave to GenI for its multiple paraphrases can be thought of as an underspecified input wrt certain syntactic decisions)

Another thought is that transposing the CRU idea of just going up the syntax tree won't do the same thing, because the CRU grammar is meant to represent a generation space (or was it the other), where as the MRL grammar really is just representing the err... representational space. Or maybe I'm not using the right words. Or maybe I'm just babbling.

What are we really after?

Part of the problem is that I'm having trouble shifting my way of thinking. From the old Eric who basically focused on realisation, thought of it as a box providing an input and an output, to the CRU Eric who's trying to see this all as one space, single decision tree (despite my reservations), to the SCFG-based Eric. Yeah, it's all the same thing, but all this repackaging is confusing me. Mental habits that worked for one Eric have to be suppressed. Things don't mean the same thing.

What are we really after? Thinking about this just confuses me. Need more concrete.

downhill slope

Tomorrow, I should try to write my caledonian2 grammar as a wasp mrl-grammar, even if I have no idea if it makes sense as an input for such a small domain (it doesn't). Then I shall see what happens if we train on the corpus and try to parse some simple train strings.

still not working (but it is a sunday)

I thought I would come into the office this morning to make up for Friday's misdeeds. Have not been very good (although I have banned myself from contributing to darcs, with David's help by asking him to automatically reject any of patches for the next two weeks).

It's just hard for me to channel myself into one thing, hard to get started, muster the mental resources to get something done. Just sit around slack-jawed.

M+S

Well, I read a little bit of M+S, finishing the chapter on HMMs. Kinda flew over the Viterbi algorithm (the combination of better getting the forward/backward algorithm, and having seen this before made me less motivated to go through it). Also skimmed over the hill climbing Baum-Welch thing (because I saw a brief glimpse of EM in a Nesson and Shieber tech report). Fine. I think I get the picture, which isn't saying much because I'll just forget it again in a month.

Noted the practical aspects of implementing HMMs (because you're multiplying a bunch of < 1s, numbers will become very very small and then underflow [I guess, if you look at the wikipedia page on IEEE 754 floating point) you can take that to mean that 11 bits just aren't enough of an exponent... I wonder if it would be useful to have a super-low-precision floating point representation, an anti-IEEE with 11 bits for the fraction, and 52 bits for the exponent). They suggest to use logs. I didn't really understand the bit about the scaling co-efficients.

Also, I started the chapter on tagging (what the hell), but then I got bored and closed the book. I guess I should skip to the PCFG chapter, read that one and ch12 on parsing. Then look in Jurafsky and Martin. Not entirely sure what I'm trying to accomplish by doing this. So you want to learn about probabilistic parsing? Why? Does it particularly interest you? Are you just trying to "catch up" (where people have already long moved on)? Are you just trying to make it easier to understand the things you have to for work? Must bet the latter.

TIGs, TAGs and TSGs

Yesterday on the beach, in a cafe, on the couch at home, read a bit of Shieber and Nesson on synchronous TIGs and TAGs. I have lost all my intuitions about why we want TAGs. Yeah, I can recite the phrases "long distance dependencies" and "tight linguistic packages" (that comes from my thesis, just erician for "linguistically meaningful units"). I can even tell you about lexicalising context free grammars. But I've just completely lost the "ah-hah", the "getting" it. How can you un-get something? This is so embarrassing.

Word of advice to any PhD students who feel like I did: do not believe that you will stop feeling dumb afterwards. It only gets worse.

Sigh.

hierarchical phrase-based model for SMT

Reading David Chiang's ACL paper on SCFG based statistical MT. I really like the general idea here. SCFG for MT's been done before (Yamada and Knight, 2001). What's cute about the Chiang approach is that he doesn't try to go for the linguistically motivated grammar. Just formally, ok, fine, we need some notion of syntax. But lets learn our grammar from bi-text, not from annotations. Maybe I'm noticing the wrong things here, but I find it really really amusing that he just uses two categories, S and X, mostly the latter. We're just learning how to group things hierarchically [and order them], s'all. Oh, I guess that should be an obvious consequence of the we're-not-doing-linguistics bit, but it was just one of those minor 'ah-hah!' moments. Oh nevermind.

Conclusion says they want to look into adding in syntactic categories (more syntax). Still, it was useful for me to have de-blockage about the categories being necessary

playing with CRU and WASP

Just being so slow here. So now I've got an English grammar for the trains stuff, and a MR grammar. But (a) I haven't given any thought into what a reasonable MR grammar would be for trains -- still re-using the one that generates MRS-like expressions -- and (b) I haven't tried to solve the practical issue of how to align texts produced by both grammars if they have different ambiguities [uh, not the word I'm looking for... more than one way to say the same thing]. For example, the English grammar has two rules for "leaves" [leaves/departs], but the MR only has one [why would you need two?]. So rather than deal with that, I just commented out one of the rules. Likewise, the MR grammar has two rules for AND, for which the English grammar has only one. So I just commented out one of the rules. This way we have identical texts that we can just zip. (b) is just a practical problem, not a theoretical one. Like, aw man, do I have to write code myself to enumerate all the possible sentence pairs produced by an SCFG? [damnit, why have I become such a wuss? was I always like this? I think so, come to think of it]

Having a hard time trying to get myself to think about [1] what would be a sane MR for the trains grammar, simple domain as it is [2] what would be a reasonable notion of underspecification in the input? Why do we want underspecification again? Oh yeah, so that the guy giving you the input doesn't need to know too much stuff. On Friday, I wrote down something on the whiteboard which I thought was a useful insight, but now I don't really know what it means anymore. I said:

we only to generate from underspecified representations... which should be possible

statistical mumbo-jumbo

I say this not as an insult, but as a reference to the fact that I don't actually know what all this stuff means yet. Just note the phrase 'we depart from the traditional noisy-channel approach and use a more general log-linear model' (ok, I heard about the noisy-channel idea in 2001 and tell you about English having been garbled into French...). Right, so this sentence is a clue. It tells you that whatever the noisy-channel approach, and log-linear models are, they are sort of in opposition.

WASP λ-SCFG

I'm re-reading the λ-SCFG stuff (because it didn't stick the first two times around)... and I keep tripping over this bit:

If any of the arguments x_i appear in β as a free variable (i.e. not bound by any λ), then those free variables in β must be renamed before function application takes place

I guess that must be right, but it sure makes me feel uncomfortable... Err, maybe the trick is to realise that this is NOT λ-calculus, but just a variable-binding mechanism which is λ-calculus inspired. That's all there is to it, self. Nothing to be confused about. The rules are exactly as Wong makes them up.

So, ok, Eric, try not to get confused.

1. λ-SCFG rules look like this: <α,λx1..xN.β>
2. The non-terminals must be followed by some number of variables [arguments]
3. When we expand a non-terminal, it must be expanded into a λ-function of the same arity
4. [not in the paper, but should be clear] Once you've expanded a non-terminal, you can do function application.

That's all. There are no inner-λs, okay? The only λs are on the outside of each MR part of the rule. Non-terminals are typed (by arity). Type is declared in the SCFG rule (implicitly by the number of arguments you associate with the non-terminal).

That wasn't so hard. We'll leave it at that. Maybe it'll stick this time. Tomorrow, we figure out how this meshes with the statistical stuff.

Well, maybe I did so some work this weekend, after all. Guess I'll go to gardening now, then go home.

really really bad!

The story this week has been : wake up early and get to work before 9, sometimes before 8 (great!)... and then go hack on darcs (no! bad!)

I spent most of today preparing the ChangeLog for the upcoming darcs 2.0.1, which was really not what I set out to do for the day.

Otherwise, I did spend some time reading M+S

markov

Look, this is something I have seen a couple of times, (maybe 3 or 4), but maybe now I think I grasp it well enough that it will stick.

1. Markov Model: the past does not exist; only the present moment matters (probability of transitioning to state N+1 given state N, doesn't matter what states N-1 , etc are)
2. Hidden Markov Model: you don't have a way of telling for sure what state you are in

Actually pretty simple now that you "get" it. Sigh. Crazy soft drink machine: it would be a visible Markov Model if the machine had a little readout that said "I am now in Cola-Preferring mode", or if it as a rule, consistently spat out cola when it was in that mode. It's a Hidden MM because the drink you get doesn't actually tell you what state it's in. Just tendencies.

I think maybe the confusing thing is that people talk about "hidden states"... when what I think they really mean is that "you know what the states are and you can give them names; you just don't know which one you're in at the moment". I guess a priori one assumes that all states are connected to all states (although you may eventually learn that some of those connections have zero or little probability).

I'm a slow learner. I get side-tracked by things like that. But I hope that as the years roll by, I eventually figure out all the parameters behind Erician learning, and optimise things so that I pick things up quicker (and understand them more deeply, etc)

Also have the intuitions for the forward-algorithm, I think. Motivation being to find the probability of a sequence of observations. Crazy soda machine example again, worked through this with them. You could just add up all probabilities for all paths, but then you'd be running through each possible path one by one. Slow. Forward-algorithm is dynamic programming. Fine. Backward algorithm, my eyes sort of glazed over at first, but then you remember that "the past does not exist; only the present moment matters". If forward algorithm is about summing up all the previous states I could have come from, the backward algorithm is about summing up all the states I could go to. Talking this out in my own private language. The alphas and betas are the same as P(O)s... just incremental (up to time t, state i).

could be better...

Had a good 7 hours last night. Think that made a difference. Finished reading the Wong thesis. Was easier to understand with not as much falling asleep. Reading just knocks me out in general. Brain dis-enagages.

So: synchronous CFG acquired from an unambiguous source CFG using word-alignment techniques with special tricks (use production rules, not tokens; do double-alignment, prune, re-add). Notion of lambda-SCFG (still a bit annoyed about this free variable thing). Lambda-SCFG for generation just uses standard chart generation instead of LR parsing of the LF. The overlap-bits come from the productions involved in the parse (using only the unambiguous grammar). I didn't actually read the mathy bits. Sorta ignored them. Going for the bits I can understand.

Working much more slowly through M+S. Also taking an attitude of not trying to understand the math. Just take it as given. Try to see the big picture, ideas. Why we want a null hypothesis, what we use the t and chi^2 test for (and not worry about what makes them tick).

Was slightly optimistic this afternoon, as though I was starting to figure out what I was doing, starting to learn how to learn, what to look for. I had the feeling all of a sudden that not-getting something no longer frightened me, that I could learn to manage and wield my ignorance, figure out how to figure out where to shine the flashlight.

But that burst of optimism vanished. Gah. So little time. So little brain. Got to use both wisely. Basic brain trick is that you've got to understand stuff in order to understand stuff. Compound-interest effect. So how do we use it? Quantity-not-quality is something to apply sometimes, just vacuum stuff up at a superficial level. Make connections, see how bits fit together without really deeply understanding the bits. But that doesn't always work. Sometimes you don't understand what the bits are until you've really worked through them. You can't always play the forest game. The trees are there for a reason.

Gah... so little time...

grouchy and tired

Well, had some trouble sleeping last night, so I'm pretty tired today. Spent too much time in the garden (going crazy with shears). Also worked a bit too much on darcs (trying to track down a bug). Intention was to stay late to make up for lost time, but just falling asleep and not understanding things.

Sigh.

Just so grumpy. I'm taking it out on the thesis I'm reading "ARRRRGH, I DON'T UNDERSTAND, AND IT'S ALL YOUR FAULT" (heh).

Finished chs 3 and 4 of M+S. Now on on the meaty stuff. Uh-oh.

back in brighton! (sleepy day)

Alright, back from holiday [was in Liverpool the last 4 days]. Hmm, maybe I should also count the three days before that being bad into my holiday quota.

Now, can we behave a little better?

Let's see: got in at 11 (after gardening). Did some darcs stuff until lunch, and a little bit more after. Tidied the heck out of my desk. Chatted with ILG about work stuff [so that's good].

Started reading Yuk Wah (John) Wong's PhD thesis.

Not doing much.

I guess the answer is no, then.

Oh, and I've been reading some Wong/Mooney papers on the way up to Liverpool and back. So maybe that makes up a little bit for badness.

only slightly less misbehaving;

Sigh. Really should not be left to my own devices, huh?

Did a bit of gardening in the morning, and then some more darcs stuff. Long lunch, chatty. Sleepy in the afternoon.

Read some M&S, tried some simple exercises. Decided to give up on their intro to information theory. Moved on the next chapter. Figure I can pick up more as time goes on. Kept... falling... asleep. Need to be going to be earlier.

Random impulse to go implement that generic chart parsing library. It's going to be crap and inefficient, but very very simple, just something you can whip things up with. Probably won't go anywhere, though:

class Chart c where
 getAgendaItem  :: c a -> Maybe a
 getChartFor    :: c a -> a -> [a]
 putAgenda      :: a -> c a -> c a
 putChart       :: a -> c a -> c a
 getChart       :: c a -> [a]
 empty          :: c a

type ChartRule a = a   -- ^ item from agenda
                -> [a] -- ^ item from chart
                -> [a] -- ^ new items

-- XXX THIS IS WRONG
runChart :: [ChartRule a] -> Chart c => c a -> [a]
runChart rules chart =
 case getAgendaItem chart of
  Nothing -> getChart chart
  Just ai ->
    let cis = getChartFor chart ai
        nis = concatMap (\r -> r ai cis) rules
        chart2  = --- XXX no! NO! NO!
                  foldr putChart chart nis
    in runChart rules chart2

Oh shoot, that's not right. New items don't go straight to the chart, what am I smoking? This is where the dispatch filter stuff I was talking about comes in, I guess.

another naughty day

Boy, I've sure got my mind stuck on darcs these past couple of days. Been very very very bad, then stayed late to do some work, and was bad some more. Refactored some code, did a little triage. I don't feel particularly crummy, but I'm slapping a frowny on today to remind myself what a bad Eric I've been lately.

book

Tried to read a little bit more of Manning + Schutze, but I kept falling asleep, which as you know, is what I do when I don't understand. This is a bit discouraging. We're only at the mathematical background part, and not the actual meat.

talk

Listened to a very nice talk by Carlos Gómez Rodríguez, fitting a bunch of dependency parsing algorithms into this parsing schemata framework (what's the difference between that and the parsing-as-deduction stuff? it looks superficially the same at least...). Very clear, in my naive opinion, but probably because I'm being too superficial to notice any potential problems. The talk really worked for me because (a) i'm naturally inclined to like this stuff [let's grab all this shit and talk about it in a clean, uniform manner] and (b) the explanations were implicitly visual and lego-like. You saw what bits fit into what fits. Also gave me a nice slide on constituency parsing vs dependency parsing which would have made me really really happy at ACL 2006 when I was like "huh? what's this thing everybody's talking about?". Oh well.

Ok. Better behaved tomorrow. I have a clearer idea what I'm supposed to do.