re-stuck (12::5531/26::11810)

Uh-oh... I think I'm stuck again on the ISOFT stuff. Was trying to write up the comparison stuff, ack this stuff is hard to kill off and somehow get the feeling that I understand this not quite as well as I thought... first you think you get it, then you think you don't know shit... Can't properly formulate the question, it's something about decomposing the input specification, something about network traversals, tree-shaped decomposition of the input, relationship to preselect operations, does it matter that input is shaped as tree? Why am I trying to write about this? Basic idea is how you might adapt GenI to this kind of input, but maybe not worth going in that direction. Argh... point is that we have tree identifiers, they have traversals; can you slap traversals onto what we do? Not entirely clear... at first I thought, well yeah, no duh should be easy and now got a mental block. I hate how easily I get stuck. Wonder if I'm worrying about something completely useless.

statistical stuff

Plowed through langkilde1998gec article and friends, langkilde2000fbs, knight1995tlm, langkilde1998pvn. Probably not reading as carefully as I should, just trying to get reacquainted with the stat generation stuff, figure out what I want to say since I'm stuck on the isoft stuff. Find that it's helpful to read EVERYTHING that a group publish on the same subject, repetition helps, redundancy, slightly different angle. One stupid thing I used to think was that if I had two articles by the same people talking about a system, there wasn't any point in reading the earlier one (stupid, I know) because it's now out of date. Now I realise that (1) repetition helps (2) since the earlier one is trying to introduce some whole new idea, it sometimes takes greater pains to make itself understood. In the new version, the authors are like "eh, they know what I'm talking about". Anyway, knight1995tlm was a very nice read, very friendly. Helped me understand langkilde1998gec better.

a short lab afternoon (12::5125/25::11404)

I'm still not done with this chapter! But I'm writing up the stuff about ISOFT (systemic TAG). Have more or less described the system, although haven't talked much about how it works. Tomorrow will the harder part, maybe trying to compare it with GenI.

I guess so long as I'm behind schedule, it's going to be lab weekends for me. I've never been able to pull off lab weekends, partly laziness, partly superstition that people need down-time to be productive. But what I'm thinking is that with a little bit of hobby time on Saturday morning and regular exercise (at the gym three days a week, just like a RadioHead song); I should be able to pull off the 7/7 thing. Won't be missing much anyway. All I ever do on the weekend is procrastinate on my hobbies (how the hell do you procrastinate on hobbies?) Just got to watch out that it doesn't get counterproductive, monitor progress watch for frazzledness. But this is what I get after slacking off for two months, waah wah wah, can't back into rhythm, boo-hoo... well look at that... ya happy now eric?

yang1992iag

Spent today finishing off my reading of yang1992iag (chapters 4 and up). I think I understand the fr-root mechanism better... have a more solid idea how the whole thing works. Might be worth noting the following sentences:

• John opened the door
• It was John who opened the door
• Was it John who opened the door?
• The door was opened by John?

Quoth the thesis: "different syntactic variations of a common predicate argument structure. Although the predicate argument structure of these sentences is the same, the focus (or theme) and illoctionary force is different in each case"

Another interesting note is that they seem to say that they can't handle the case where there is more than one TAG tree possible for a given region. I wonder what they do, then? Backtrack? Or just pick one?

Behind schedule, arrrrrrrrrrgh... There's a paper deadline for 16 march, there's this chapter I wanted to have finished a couple of days ago, the next chapter for end of february... stuff I wanted to implement.

Meh.

more kpml/fuf (11::4799/25::11078)

Think I've written enough of that I can mentally move on.
Still missed by deadline, though.

tired (9::3791/23::10070)

Just a general fatigue, worn down feeling. Not being as productive as I'd like, not really goofing off a lot, just being slow. Going nowhere. Slow. Molasses. Something bothering me. Wrote some general SFG stuff. At least now I have a rough description of SFG in general... but I'm stuck on something, it's all about the reversibility thing; don't really know what I have to say about it, don't know what the right thing to say about it is. I just don't like saying that what we have is inherently reversible as opposed to the work in that direction, because I don't have the justification for it. Maybe somebody's that's been in this field and been working on this stuff for a while knows what they're saying, but me talking about bidirectionalness, talking about reversibility is just empty. Stuckedness... I don't know what I'm saying well enough to be writing this chapter. Feel like I've written myself into a hole, cranked out a couple of paragraphs I thought made sense, but then realised that maybe I'm just being vague.

Writing up the state-of-the-art and the comparison-with-related-approaches is the hardest part because you have to say something reasonably correct about something you don't really understand well. You can't make shit up because, you know, you're all conscientious and stuff, and you can't learn things fast enough, deep enough to be satisfied that you are justified in what you are saying. It's a form of paralysis... I wonder how other people do it? Do they just not worry about convincing themselves about every single thing they say? I make myself sound more rigourous in my thinking than I really am. But that is one of the things that slows me down, the whole "waaaaaaitasec, I don't believe that". Just not smart enough to figure out how to cut through this kind of stuff, how to learn exactly what it is I need to learn to find out what I have to say.

Oh, and commented out one paragraph in the chapter intro that was making me very unhappy. Something about lfe.

writing surge (9::3591/23::9870)

Finally moving forward on the comparisons stuff. Spent most of the day digging through the surge manual and the example grammars (gr3/ir3 and gr4/ir4 are particularly instructive). One basic thing I was just not getting was how that input structure was being fed through the network. I remember that it was recursive unification, unify the top level with the grammar, grab the constituents and unify those... but I didn't fully get the details. Now I know that

• the sub-constituents can also come from the grammar, not just from the input spec
• if you don't explicitly say "these are sub-constituents", then fuf will just look for anything that's got a cat attribute

Another thing I learned about this morning/early afternoon was how patterns are used to control word order. It's neat, patterns just part of the fuf language, just another constraint like the features. Know also that they are a source of non-determinism; linearisation is non-deterministic. All still kind of hazy, but I see a lot better what the moving parts are, and somehow that's very comforting, maybe just because I'm a coder and I can't understand things unless I actually look at its gears and cogs.

Managed to write a bit; don't know if I really know what I'm saying. I just know now a little bit what it means to encode an SFG in FUF...

surge spelunking, systemic parsing

I am not ready to write up the comparison stuff. Something I'm still not fundamentally getting.

random notes on SURGE

Downloaded the FUF and SURGE source code today. Ever since freshman Scheme, I've had a sort of soft spot for LISPs. I don't see why everyone hates the parentheses so much. Things I had to learn to help me read this code:

• 'foo means quote(foo) which means "don't evaluate foo"; haven't quite worked out where the quoting stops, though, but this is enough for me to feel my way through the code.
• ` is just a funny ' that's apparantly handy to have in macros. Since I'm only trying to get a rough idea what's going on, I can live with that lie
• (FUF){foo bar baz} is a macro for paths. This is an absolute path, which means "go up to the root, then down to foo, then bar, then baz". ^ means cd .. ; you can have as many ^ as you want ; you can also write ^3 to mean cd .. three times

Was useful to see how the grammar was laid out. Neat to understand that the grammar is the realiser. There is no realiser. Just unify, remember? At least, that's how I understand it.

I wonder how SURGE handles a sentence like "Who does John believe the tall man saw?" from yang1992ffs (says we need splicing, hey that's exactly what TAG is good at). That was the question that prompted me to download. Still haven't figured out yet.

I'm also still not so sure how you represent a system network in FUF (guess I ought to read kay1985fpu if I can get my hands on it). One thing I understand is that choices are just alts. So

   |- x
-n-|
   |- y

is just something like (alt n x y).

The network below would just be something like (alt n x (alt y a b))

   |- x
-n-|      |- a
   |- y --|
          |- b

Yeah, ok, it sounds like a stupid little syntactic detail... but somehow, understanding this makes a difference. Understanding that it boils down to just a bit of mostly-understandable code helps.

Another thing that helps is knowing what the entry point is. The interesting file here seems to be gr-modular.l (see (def-grammar gr...). Another thing is that you look at the the articles and you see that all the input FD start with cat "clause", so you look into clause.l and start reading.

Feel like I'm just stupidly dancing around the real deal, but vaguely optimistic that enough of this will make sense, make enough sense.

I think this might count as a PhD student lifehack. If you're a coderly student, more programmer than researcher, you might find it easier to understand an article if you can get your hands on the source code. Something about code... just so much more concrete, greppable, spelunkable... somehow, knowing that the grammar is made of 7K lines of code helps, gives me sort of an idea what's going on. (FWIW, kpml-english is about 10K lines... though not necc comparable... just ballpark [then again, for all i know it could be like trying to compare Java code with Haskell])

Oh and winograd1983lcp talks about unmarked choices. Right, so that's where the defaults come from; just an SFG thing.

parsing with SFG?

One mission I've been sent on and from which I am only now starting to return was to find out what work has been done with SFL parsing. I never got very far with that before because I was always either looking in the wrong place, or using the wrong keywords, or otherwise moving on to do other stuff. Now that I'm trying to write up the comparisons part of the paraphrase selection chapter. I've always taken a bit of distance from the idea that SFG isn't very usable for parsing. Here's what I've dug up on the matter.

Note the convention ~"foo"~ to mean that I'm not quoting, just paraphrasing and probably doing a bad job of it.

kasper1988eps

(An Experimental Parser of Systemic Grammars) Basic attitude is bidirectional SFG can be done. You just got sit down and make sure your grammar works for parsing as well.

~"Implement your SFG in FUF, slap on a phrase structure grammar, et voilà."~ Parsing doesn't seem to be the problem, efficient parsing does, and efficiency problems might just be a problem of the grammar not yet being tuned for both parsing and generation. Note also "most modifications that we have made to enable parsing involve eliminating uninteded ambiguities or disjunctive alternatives in the grammar that require an inordinate amount of time to resolve".

Chart parser, modification to PATR-II. Works reasonably well, several minutes to parse complex sentences, but that's on computers from ca late 80... so that's probably pretty good compared to today.

Around 60 phrase-structure rules tacked on it. Kinda reminds me of the yang1991iag stuff where you tack on a system network to a TAG grammar. Oh and odonnell994saa says that bateman1992??? does something similar, but using an HPSG grammar instead. Parsing over a context-free backbone as a pre-processing step.

kay1985pfu

(Parsing in Functional Unification Grammar) I haven't read this, but kasper1988eps above says ~"it don't work on realistic grammars"~ (think 2^100 disjuncts if you reduce the grammar to disjunctive normal form)

odonnell1994saa

(Sentence analysis and generation). Hmm! It seems we should have paid a little more attention to WAG. PhD thesis on bidirectional use of SFG. Notable comments, ~"we tried to use NIGEL, but it was too big, so we wrote our own smaller grammar"~

odonnell2005uam

(The UAM Systemic Parser). Follow-up work. If I understand correctly, it's ~"I'm older, wiser and more pragmatic about it. Let's reduce functional complexity in the formalism"~ [keyword unifunctionality], "only the Mood structure is accounted for"

article name, article name

I never like putting article names in my blog posts, because I'm always afraid somebody's going to google for them and find my blog instead (and figure out what an idiot I am). It's the same reason I've recently been putting initials for all my friends' names. Don't want koweynlg to fall into the search engine cross-fire. But I guess life's like that, and part of growing up in the Internet is coming to accept that somebody's going to bound to think you're an idiot one day, and that's O.K.

misc stuff

freewriting

Hey, I think I'm liking the freewriting thing. This time I actually wrote about work related stuff, although in the same mental dump format, don't think, just purge. And what came out of it is a bunch of questions on yang1991ffs. Questions are good in that they are a whole lot more precise than my general HUH? Questions are a recipe; they say, if you can answer me, maybe you'll understand how it all fits. Another reason I'm happy with it is that I actually found the way to explain what had confused me about SFG all this time. (This might sound similar to yesterday).

The confusion behind SFG is that there are two kinds of parallelism: parallel networks, and parallel traversals. What's nice about this is that it also accounts for my many other confusions. You can make a table showing

parallel networks	parallel traversals
hierarchy of choice points	hierarchy of regions
conjuction/disjunction	conflation

The other thing that helps me is to try and translate things into some language I know better:

 type NodeName = String

 data Region
  { tagTree :: Tree N
  , mappingTable   :: Map Ikkiwubble NodeName
  , functionalRoot :: NodeName
  , conflationSet  :: [[Ikkiwubble]] -- ^ self explanatory
  , expansionTable :: Map Ikkiwubble Region
  , unrealisedFeatures :: [(String,String)] -- ^ not sure about this
  , subRegions     :: [Region]
  }

(Oh yeah, ikkiwubble instead of ixiwubble; it's just easier to say)

I guess my main issue now is figuring out exactly what happens when you hit a preselect operator. How do you know what part of the input specification the ikkiwubble (function), say agent corresponds to? One guess is that if you look at the input specification as a recursive feature structure, then an ikkiwubble simply corresponds to one of the features (really simple).

I was also initially confused about the idea that there is an hierarchy of regions, but then I found an analogy which made it all make sense. You know how you're writing a program, and you call fork() to spawn a new process? The new process is the child of the parent process, right? There ya go, instant hierarchy. Yes, you can have an hierarchy of parallel traversals. Reconciling the idea of parallel with hierarchical isn't hard after all.

Oh well... I still don't get it yet. I dunno, maybe I shouldn't be going too deeply into this. It's just annoying because now you actually have enough meat to work with; there is enough material for you to actually understand what's going on. Hmm... actually, maybe I should just wait for his PhD thesis to arrive. Maybe that will lay it out all nice and slow.

category theory

Attended a short course on category theory. There's two more sessions to go. I dunno... I'm just here as a random Haskell programmer, trying to see what this could do to make me a better programmer. So we've got a reeeeaaaally abstract thing that generalises over a bunch of less abstract stuff. But being a non-math guy, I don't even know what these less abstract things are, so is there a point? But maybe this would give me sort of a top-down view on things... maybe it'd be like speaking Esperanto as your mother tongue?

hunting for overgeneration

Had a discussion with C on hunting for overgeneration. Yeah, the programming and the fun let's-hash-some-ideas-out-on-the-whiteboard thing; these would be the high points. Reading and writing, basically the low ones. However, understanding something that you've been reading on and off for two years and FINALLY getting it, that's a mega high point.

yang1991ffs

Alright, I basically spent the whole day trying to read yang1991ffs. It was tough in the beginning because I kept wanting to something else, anything but read, but after a few hours of just being stupid I finally managed to coax myself into it. This article is a bit longer than the other two I've read in this series, which is good because they can afford to put a bit more meat on their technical bone. Indeed, if ever there was a text that would help me to "get" SFG, this might be the one. My head is still kinda swimming from trying to understand what's going on, but here we go:

SFG in my terms

The thing that makes SFG confusing for newbie eric is that there different kinds of "decomposition". I have a hard time telling them apart, or even knowing if they are genuinely different. As I understand it now, there are three kinds: the network hierarchy, conjunction/disjunction of networks, decomposition of the input specification, and rank-wise decomposition (?!). Oh wait, the last two are the same thing.

The other thing is that SFG throws around the word "function" alot -- this is just me being stupid -- whereas for a functional programmer like me, that word triggers all sorts of irrelevant associations. (Although AR seems to say something about representing Functions in the SFG sense with functions in the FP sense). So let's say we don't like the word function; let's call it something else like "purpose" or "ixiwubble". Ayup, SIL, Systemic Ixiwubblistic Linguistics.

decomposition of input specification

So here's the new thing I learned about today: rank-wise decomposition of the input specification. Ditch the rank-wise, for now; don't really know what it means. The big deal here is that when you're traversing the network, you sometimes hit a certain trigger (PRESELECT) which causes part of the input specification to be cut out and realised completely independently of what you're actually doing. In programmer terms, the thing spawns a new thread and now you got two network traversals going on in parallel. The big secret is that the new thread consists of re-entering the network! It's a recursive call: traverse net foo(bar) = traverse net bar ++ bunch of other crap

So what? Well, it means my mental model of SFG has been all wrong. I somehow thought there was ONE network traversal but different things going on in parallel, right? I somehow imagine that the whole input was just there as a resource, that gets consulted whenever one piece of network wants to know something. But nope, it's not like that. That sounds soooo similar, but it's not the same thing. What we got here is decomposition of the input specification, and a whole separate traversal. In fact, you start over! (Actually, the PRESELECT operator can make it so that you start from a specific point in the network, but in order to get what's going, it really helps to just think of this being a whole new traversal starting out in a separate thread).

SFG vs TAG

This paper explains why SFG wants TAG (and why TAG wants SFG) in terms that stupid programmer/engineer eric can understand, sort of. It's really like trying to clean up a computer program: you've got a big messy SFG grammar that's trying to do too many things. It's trying to represent how ixiwubbles (functions) relate to each other, and it's trying to say enough things about syntax so that you don't get output like "Who Mary John love did?". A better way to do it is strip all that syntax crap out of your SFG and just talk ixiwubbles, ixiwubbles and only ixiwubbles.

pick this up later

Aww man I'm too tired for this. I'm just going to go home and talk about this in my free-writing exercise tomorrow. WYD 15 min/day suggests that I try to get into a habit of (1) writing every day (2) writing first. Like, literally first. The desired outcome is that from now on, the first thing I do whenever I get to the lab is write 1000 words. Like really first: remove your jacket, sit down, write 1000 words. The reward for which is that I get to check my email.

For now we focus on regularity, not on content or on quality. 1000 words of anything, as long as it's the very first thing I do. What should hopefully happen is that after a while, it just gets easier to associate go-to-lab with write-1000-words (ding ding ding! salivate...). Once we've established this habit, we start increasing the roughage: slowly turn up the dial on the structure and focus (i.e. write about work stuff, instead of whatever random crap just passes through my head). But the important thing is crank ze words out.

I realise that koweynlg means that I already spend lots of time 'writing' (especially writing crap). The two key differences are that (1) this one is first thing in the morning (2) this is eventually going to be my thesis-writing time.

elhadad1997fcl continued

more notes on elhadad1997fcl

so what does a lexical chooser do?

The first question is what a lexical chooser should take as input. Given the notes from before, the obvious no-duh answer would be "well, the output of the content planner"... which would be "structures from the application domain selected during content planning, possibly augmented with discourse relations". You also want stuff about discourse and speaker intent, but that can all come from the planner.

Ok, so what does it output. Again the no-duh answer, "the input to the syntactic realiser" (golf clap), and this would be something more like... oops. They don't say explictly. I guess it means "go pick up a SURGE [or your favourite syntactic realiser] manual". Point is that the lexical chooser picks one paraphrase based on the input, and feeds that to your realiser.

I feel a bit uncertain about this, but I'll suppose that this does not neccesarily preclude minor syntactic variations. The lexical chooser would choose between

• The 6 AI assignments require programming
• AI has 6 assignments which involve programming

Since choice basically is "realise this relation with the verb 'require', and that one with the noun 'assignments'"; but not really any of the closed-class words (the function words), and no word on how they're supposed to be combined. It would be syntactic realiser which would then select between

• The 6 AI assignements require programming
• Programming is required by the 6 AI assignements

I think the SURGE manual says this anyway. Naturally, this decision can be made with the help of input passed through from the lexical chooser.

Anyhoo...

The 3rd piece of the puzzle is how we get from input to output. Really, what does a lexical chooser do? There are two tasks to perform, associated with decisions to make

• phrase planning (syntagmatic decisions) - from conceptual net to tree of thematic roles
• lexicalisation proper - (paradigmatic decisions) for each piece of the thematic role tree, choosing among the alternate lexicalisations for that role. The tricky part to this is that you can't make these choices independently of each other; one choice engender constraints on other choices, constraints back-propagate, all that nasty business.

The paper describes this as a two phase process, each with floating constraints. I wonder if there are cases where phase 2 constraints can force you backtrack up to phase 1 and start over, though... or if I'm just not understanding this right.

phrase planning

Skimming over this. Sorry, not what we're working on right now. Would be nice for general culture though.

merged realisations

I'm not worrying too much about merged realisations either. Again, good for general culture, but for the moment, not my problem. Something I can imagine is shunted off to "some other system" in the sky (like, this system).

cross-ranking realisations

This might be more interesting. The question is how do you map thematic structure onto syntactic structure, especially when some things can be mapped on different ranks. Same thing can be expressed as a verb, adjective, conjunction, etc [the word "but" can mean so much...]. This was a bit tricky for me to read because I didn't really know what an argumentative evaluation was, just treat it as some constraint you have to meet.

I was quite confused by the notion of expressing constraints in a unification-based formalism. I can understand that you get unification failures from two different ground values, but how do you express something like "this feature must be expressed"? From reading, I get the impression that it's because FUF has special mechanisms, noted by two keywords. One keyword is GIVEN, which basically means "this feature is expressed". If you look at Figure 16 on page 221, you can see two values for {process ao}: GIVEN and some other feature structure. So, that keyword is not a value per se, it's just some extra annotation you add into the feature. Another such keyword, by the way, is ANY, which tells FUF that you want to postpone unification on that feature until the very end. I guess not understanding that there were these extra mechanisms that tripped me up.

The feeling I get is this: you've got a constraint that you're trying to apply somewhere whilst doing your lexicalisation. Lexicalisation is top-down breadth first: for the current node, pick a word/category, instantiate, now flesh out your thematic kids. If I understand correctly, you also have these constraints that you're trying to express SOMEWHERE (they're cross rank), and in order to do that, you need to make provisions for the current constraints not to be realised at this level, but somewhere down the road. Why this sucks is that you could be happily doing your lexicalisation search, drilling all the way down to bottom, procrastinating like hell on one constraint or the other, until you realise at the very end... whoops. Can't meet that constraint. One of my previous lexicalisations was bad because it ultimately does not allow me to fulfill all my constraints.

So, backtracking is a problem.

One thing I'm shaky on is this distinction they seem to make between deterministic decisions and non-deterministic ones. How can you just "map to the first slot found in the constituent tree that can account for it?" Aren't just inherently searching anyway?

Anway, backtracking is a problem; got a big search problem on your hands. To reduce backtracking, they introduce a mechanism which smells a bit like a cut (only the opposite; it's not 'stop backtracking beyond this point' but 'leap up to this point'). The idea is the grammar writer can annonate alternations as being able to affect one constraint or another. Whenever we hit the problem that oops, that constraint wasn't realised, we aren't going to dick around backtracking all around our search tree. We look at the missing constraint, and then we jump up to the choice point that our grammar writer said was important. Something like that, right? Cut but inside-out.

where this all fits in

I guess that was a false alarm. Hard to articulate my worry really; just a sudden feeling of panic, a feeling that I suddenly didn't know how all the pieces fit together. My dividing of the world into surface realisation (what I'm doing), and everything else (what other modules are doing before they talk to me) felt supremely inadequate, or somehow that once all the important lexical choice decisions were made; eh... all this polarity business, not so interesting. It's mostly a matter of trying to see where all the pieces fit in, what my job is, so to speak.

elhadad1999scp

Flipping back to the syntactic realiser, just trying to figure out how it uses its input. Looked through the tables in the appendix and tried to mentally extract some value out of them. Table 1 and 2 maps you from process to list o participants. Table 3 goes from mood to (dunno) realisation stuff. Three kinds of input you can provide:

• ideational - via process, participants or lex-roles (if no lex-roles, the system looks up participants on the basis of process?)
• interpersonal - via the mood feature
• textual - via the focus feature

Is that the idea? If you want to run throug these systems, you specify values for these features? And then the rest happens by magic... uh... unification, I mean?

elhadad1997fcl

Slowly making my way through a re-read of elhadad1997fcl, mostly so our discussion tomorrow can be semi-productive, and also because I want to have a good idea how GenI's lexical selection process differs from other systems and/or what kind of input I can expect us to have.

Am getting increasing self-conscious about our inability to handle recursive feature structures with reentrancy. I guess you could say we've got reentrancy, but no recursion, but then that doesn't get very hard. Problem is that it makes it a bit tricky to just pop from one system to another. I wish somebody would write an FS library for Haskell. Maybe me, one day, when I decide I have the time to (1) learn how to do it (2) code the sucker up (3) test it (purrrfect for QuickCheck). It would be a nice library, useful for anybody who wants to do NLP.

But the reading is going slow. It doesn't help that I'm coming in to work so late (lunchtime) and leaving so early (dinnertime), but I haven't flipped into discipline mode. Sleep bi-polar. Either I'm being really good and doing 22h00/05h45; or I'm sort of straggling off to bed around 03h00 and getting in after noon. The other problem is that reading can be boring. Well, not so much that; it just requires you to Purposely Concentrate, you know flip a switch and Bing! you're concentrating. And that was never something I was good at. It's hard to resist the temptation to just wander off and go check or mail, or see how the Haskell wikibook is doing, or just do anything other than sit down and concentrate. What a pity, because in the medium term, reading is extremely rewarding, you know that great feeling you get when you actually understand something and see how it fits in with all the other stuff? Yeah, I'm not even talking about reading being good for you (faut se nourrir scientifiquement); just in terms of sheer pleasure, reading can be loads of fun... but only in the medium/long term. You have to fight yourself over the short-term i'd-rather-be-doing-something-else hump.

the notes

Lexical choice is all about associating concepts to words. Not an easy job. First of all, you're dealing with a many-to-many situation; one concept could map to several words and vice versa. Choice is also subject to constraints from many sources: syntax, semantics, the lexicon [e.g. collocations], the domain [words have different meanings in different fields], pragmatics. Having a pipeline architecture for your lexical chooser isn't a a very good idea either, because the constrains are multi-directional. Syntactic constraints could affect semantic ones, etc. Another thing which makes this job tricky is that some constraints can float - appear at different levels in linguistic structure. Note that generation algorithms only handle the easier "structural" constraints, which are consistently expressed on a given linguistic rank.

where do you do lexical choice?

The article argues that lexical choice should be an independent module between content specification and syntactic realisation. Those of us working with reversible grammar generators tend to do it as sort of "by-product" of realisation. So the SFG people (although they handle their semantic constraints during content specification; e.g. their realiser input already has the semantic stuff figured out -- but wait don't we as well?). Arguments why this is not the best way to do things are

1. limited range of possible constraints -- they doubt that pragmatic constraints can be taken into account for example
2. hard or impossible to deal with floating constraints (need "global knowledge of interaction between semantic units").

There's also a paragraph after this that shows why just doing it as part of content planning is a bad idea (combinatorially expensive? need to read more carefully).

so what does a lexical chooser do?

Eh... we're supposed to go have drinks or something. Guess I'll have to pick this up tomorrow.

more reading

Wading through elhadad1999scp and now elhadad1997fcl. Again, both are quite written, but it's just a lot of material to absorb in a couple of days. Reading is hard for me, not quite as hard as writing, but you get the same trouble forcing yourself to concentrate, and not wander off to check your mail or something. Feeling kind of woozy. Got three sources of confusion here:

1. Specifying processes in terms of transitivity or "lexically" with subcat features -- how are they different really? is one finer grained than the other, for example? What are the implications?
2. Role of a lexical chooser vs. a syntactic realiser - seems like GenI is trying to do some of the lexical chooser's job, but not all of it. We seem to be what elhadad1997fcl describe as what traditional reversible-grammar based generators do; have a lexical chooser component that's part of the syntactic realiser. Our lex chooser seems pretty poor wrt to theirs; we allow for less paraphrastic power, less non-determinism; but is it ok for us to say "well, that's another module's job"? What kind of syntactic constraints is the lex chooser supposed to handle. What's the division of labour between the lex chooser and the syntactic realiser? It's all in the papers, so I probably just need to reread and digest for a bit
3. Sub(sub)grammar hiearchy vs. systems - these are supposed to be orthogonal layers of organisation. How? Certainly not redundant information?

Je suis analphabet.

mostly reading

Mostly just reading and thinking about elhadad1999scp, intro to SURGE. Nicely written. I say that because I go through the thing without going HUH? And writing something that people can do that with (without treating them like idiots or just being a vapid fool) is fricking hard. I'm going slowly though because I still haven't figured out how to tune out distractions or not just wander off and do something less boring, like check my mail.

transitive lambda

Other than that, a friend was asking me why transitive verbs get such complicated lambda terms, something we had seeen in class a while back. This is the kind of thing you learn once, go "ahah! neat!" and then forget. So, it's always fun to sort of reconstruct it. The way I like to think about is with types. Here's the low-down in Haskell notation.

john walks

walks :: E -> T
walks = \x -> _WALKS(x)

john :: (E -> T) -> T
john = \v -> v _JOHN

Now if we do functional application, everything works (uh, being sloppy with my notation here):

john $ walks :: T -- yes! (btw, result is _WALKS(_JOHN)

john loves mary with a stupid transitive verb

mary :: (E -> T) -> T
mary = \v -> v _MARY

stupid_loves :: E -> E -> T
stupid_loves = \o s -> _LOVES(s,o)

mary $ stupid_loves :: TYPE ERROR!
-- mary was expecting (E -> T), but got E -> E -> T
stupid_loves $ mary :: TYPE_ERROR!
-- stupid_loves was expecting E, but got (E -> T) -> T

If you want to get a feel for why this blows up, try something like "john loves every_woman", where

every_woman :: (E -> T) -> T
every_woman = \v -> _EXISTS(x, _AND( _WOMAN(x)
                                   , v(x))

If we magically removed type-checking, and managed to apply (every_woman $ stupid_loves) you'd get something weird and useless like

_EXISTS(x, _AND( _WOMAN(x)
               , \s -> _LOVES(s,x))

Try it the other way 'round too. Just as ugly.

john loves mary intelligently

smart_loves :: ((E -> T) -> T) -> E -> T

-- yuck... maybe...
type NP = (E -> T) -> T
smart_loves :: NP -> E -> T
smart_loves = \npd s -> npd $ \o -> _LOVES(s,o)

mary$smart_loves :: E -> T -- yay! just like an intransitive verb!
john$mary$smart_loves :: T -- double yay!

more reading

Same old warning. This is just my (mis-)reading of the article. Opinions here are neither mine nor those of the authors. Rather, they are my understanding of the authors' arguments.

mccoy1990uta

why TAG for SFG

The paper begins by motivating TAG as the structural basis for systemic grammars (uh, did my sentence make sense?). SFG is great; it gives us a semantics-oriented view of language (function over form) rather than syntax-oriented one (form over function). The problem with SFG is that not enough attention is paid to the syntactic nitty gritty.

Sentences are built by accretion: when your functionally motivate something piece of language, you thrown it out into the pool and glom it on to whatever structure you are trying to build. This works rather poorly when you get fancy sentences, for example, with discontinuities. Consider "It was Mary that John thought Bill liked", for example. Said sentence can be thought to be made of bits like "Bill liked Mary" and "John thought...". SFG does not provide a mechanism for that Mary to be extracted or for that "John thought" to be "spliced" into the sentence, so what you get is the situation:

possible - John thought it was Mary that Bill liked
impossible - It was Mary that John thought that Bill liked

There is a mention in this paper of some independence criterion, some principle of grammar design that I haven't completely absorbed yet. It goes something like: "realization of different independent functions do not influence each other". I guess the point is that you should be able to pick out the "It was Mary that Bill liked" and the "John thought" without caring about the relationship between the two components... (for example, "John thought Bill liked Mary")
The reason why TAG is a good choice is because it gives us exactly the splicing operations we need to combine language fragments together. And because we have the right tools (adjunctions), we can design our grammars the right way, the clean way, the respecting-the-independence-criterion way. It's a choice between (1) getting it wrong (2) using dirty hacks, or (3) the best option, splicing.

It's not all about the adjunction, either I guess. The whole extended domain of locality thing is pretty big, having self-contained trees that express basically everything you need to know about a structure. But then again, having that only makes sense when you got the adjunction. Or something.

how S-TAG works

You got a lexicon, tree groups (tree families?), and trees. A key quote from the paper might be

Rather than building up structure only when we have enough evidence to know that it is correct (as has previously been done), our operation can be characterized as deciding between syntactic possibilities that are consistent with what is known at the given point in the network

Argh, now too tired to post anything substantial. Basic idea seems to be (1) pick your lexical item (2) pick a tree group/family (3) pick a tree out of it.

other than that

More discussing and thinking about the selection stuff. Think hard and all that.

reading day

Let's see here: upgraded the toolbox chez Claire, observed some small bugs, more or less got things running.

Now focusing on learning about input specifications, with a bit of hand holding because I said I was scared and didn't know how to do this kind of stuff well. We're going about this in a highly controlled fashion, first focus on the TAG-based systems. So I compiled a list of TAG generators and their basic characteristics (a whole bunch of ??? because I didn't really grasp the details very well). I then zeroed in on two SFG-related systems, Mumble (mcdonald1985tgf) and the mccoy1992uta stuff. I really appreciate the hand-holding here. After two years of this, I still haven't figured out how to use the literature effectively. This is a really good chance to practice.

Here are my notes for today. Keep in mind that these are Dumb-Eric-Doesn't-Know-How-To-Read notes, so you shouldn't try to use them for any of your work. It'll probably set you back more than anything else.

mcdonald1985tgf

This is about MUMBLE, what seems to be a surface realiser. In they paper, they say that they are mainly talking about "(3) realizing the specified information and rhetorical intent as a grammatical text".

mumbling about sfg

It turns out that this stuff is not so much SFG-based, but SFG-inspired. The article claims that they use the same information as you would find in a systemic grammar, just "bundled differently". They say that in a systemic grammar, you've got five different things being encoded

• dependencies among structural features - some choices constrain others: e.g. main clauses must express tense/modality, but subordinate clauses must not
• usage criteria - uh... ok
• coordinated structural alternatives - e.g. active vs. passive
• efficient ordering of choices - no choices made unless required, no choices made before its dependencies are settled
• typology of surface structure - huh? i do like the "almost by accident" phrasing though

realisation specification

Here's an idea of what the linguistic component (i.e. mumble) should be receiving as input: a text plan, aka a "realization specification" (see Figure 1, page 97). The input has three kinds of symbols:

1. particular features of the utterance - (:event-require-certification-as-to-source)
2. content items of the specification (main-event ...whole bunch of stuff...) - big structure
3. information units from the knowledge base ().

The paper also talks about recursive planning. So, wait, I guess their realiser is actually a combination sentence planner-realiser? What MUMBLE is not doing, is "(2) planning what information content and rethorical force will best meet those goals given the conext" [uhh... text planning?], but I guess they've got a good healthy dose of sentence planning in there. Fine. Tactical generation. Or something. I just don't have a good idea how all the pieces fit together yet.

thee guts

I really don't get this part. Seems to be talking about how they map functional bits (SFG inspired, remember) into TAG trees. Whenever there are "alternate choices" to be made, such choices are grouped into "realization classes" (Figure 3, page 98).
Realisation classes map "desired characteristics" [e.g. mentioned-elsewhere(agent)] on to "choices" (raise-VERB-into-PROP). One seemingly important thing they point out is that where as SFG requires you to make the selection of linguistic features incrementally (uh... efficient ordering of choices), their system lets you do it simultaneously. No networks; just pick alternatives from each one of the classes you need to choose from.

Somewhere in the system, there is an association between each "choice" into TAG trees. I don't really get how this maps exactly, but there is some association between the choices and the actual TAG grammar. The paragraph between Figures 4 and 5 confused me. What constraint? What parentheses? Huh? It's probably just me not reading things right, though. Another I didn't understand too well was this business of attachment points. Attachment points are sub-components of the "choices". Once you make a choice, you get its attachment points, and that tells you where you can insert specific information units. Seems like choices tell you what trees to select, and attachments tell you how to put them together. Or something like that. Ugh... will need to read this again to get a better idea.

mccoy1992fag

I kinda liked this paper, although I didn't read it deeply enough to achieve grokkage. The main feature of this paper is its clarity. You don't so much go "huh?" as "ok, I think if I sat down read more slowly, I would actually get it". Paper seems to go like this:

1. generating from logical forms is nice and all, but problem is you don't know which paraphrase is the right one
2. mumble 86 goes beyond logical forms
3. here's how Mumble 86 works
4. here's how we're (systemic tag) are different
5. here's how we work

mumble according to McCoyEtAl

The input is an LSpec (Linguistic Specification), basically a set of logical forms. Mumble uses a dictionary mechanism to associate LSpec bits to elementary trees (oh, that's all? just look stuff up in a table and spit out the corresponding elementary tree?). It's just recursive descent. You pick a tree for the initial sub-piece; you traverse the tree in a set order; whenever you hit an attachment point ("a palce where new information could be added into the evolving surface structure - what, a subst or open adjunction site?"), you grab look up the corresponding bit of LSpec, grab its elementary tree (dictionary, remember?) and plug it in (adjoin, substitute). The paper has some complaints to make about Mumble:

1. no distinction between decisions made for functional vs. syntactic reasons (makes it hard to hack/maintain the system)
2. syntax driven - as opposed to... function-driven?
3. strictly left-to-right processing makes it "problematic for generating some connectives"

how these guys are different

1. this system is much more based on SFL, as in they actually use system networks
2. not syntax driven, but "follows a functional decomposition called for by the systemic grammar"
3. clean separation between syntactic and functional bits
4. no constraints on LF (mumble makes some assumptions... nothing I deeply understand yet); they can read the same stuff an SFG generator would
5. can do standard SHDGA

how it works

The system can be broken down into two phases, a descent process, which traverses the SF network and picks out the elementary tres we want, and a ascent process which combines them together (SHDGA?). I'm not sure I understand the stuff about regions though. Every region is associated with one tree. When traversing the systemic network for one region, you can sometimes spawn sub-regions. In every region, you go through the same flow (I really liked Figure 7 of this paper... it gave me a good feel of things for not very much headache):

1. traverse a functional network - this gives you a set of features
2. traverse a TAG network (using said features as input) - this gives you an elementary tree

Traversing these networks can spawn subregions (which go through the same flow), but it seems the input to these subregions comes from the parent traversal: the functional features collected, and the elementary tree selected.

I suspect the ascent process is pretty run-of-the-mill (the fact of which is actually quite exciting, imho). Two keywords I need to reread the paper for are "mapping table" and "fr-root". Was skimming, so I didn't really understand what those are for. The ascent process, that's all I got out of it.

hmmm...

That's odd. I have the vague feeling that I was relatively productive today (though still not getting to work early enough), but I don't really remember what I did. Figured out how to do http authentication with a darcs repository. Started reading some SFL stuff. Gave Claire and Yannick a hand with some missing lexical entries. Fixed my KPML parser. Talked a lot with a Masters student, giving the traditional how-to-be-a-PhD-student (as if I knew what I was talking about).

sfgtools

I'm still not back in my work-rhythm -- amazing amount of laziness and inertia to overcome -- but hey at least I'm coding. Let's see, I spent today upgrading software on my machine, preparing the latest GenI release (might as well), talking to some guy from the MI who might be taking over custody of the loriawiki (yay!), getting lost in a talk about minimalist grammar, and hacking up some SFG tools. I've got a little extractor script (Haskell, Parsec) that prints out all the statements, and now I'm working on a little script to group them into trees.

Composing functions together is such a fun and fast way to program. All those little Perl scripts I used to write, half an hour here, half an hour there become something like one minute here, one minute there.

interact $ dropWhile null . map dropParens . sort . nub . lines

Voilà. It's like those Linspire guys say: "when we code in Haskell, we go home on time"