the Scala rewrite

This week I've switched over to one of the projects that's been funding my time at Brighton, 3D-Coform. Took a while to dust off my old work from last summer, but basically slowly starting to get into a groove.

One of my goals is to improve the packaging of some complex pipeline I'd built. It involves Flash (not my choice!), PHP (I'm innocent!), Java (for GATE), Scala and Haskell. The Scala and Haskell parts are my fault, mostly a refusal to go back to writing Java. What's funny about the pipeline though is that it's essentially Java, with a thin bit of Scala which is really just calling this one dinky component written in Haskell. Making people install a Haskell compiler just for that dinky component is stupid. In my defence, writing said dinky component in Haskell was probably the right thing to do, in other words, use the tools you know best to get the job done and out the door. But now that we have a little bit of time to put into packaging it up nicely, I'm trying to replace that little dinky with of Haskell with Scala. This way all the backend bits run on JVM and we have simpler dependencies.

It's also a good way for me to learn the language a little bit better.  I guess I've reached a stopping point for the week.  I've reimplemented the shallow part of my code which defines a bunch of data types, parses from XML and pretty prints the output.  If you want to take a look, the Haskell and Scala code are not identical (the XML parsing may be a bit better in Scala version, due to reimplementation?) but I think they are roughly comparable.

The deep part is the bit that does graph traversals, basically simplifying RMRS semantics into subject-object-predicate triples.  That bit I'll have to try and implement next week.

Things I've learned so far:

• always use case classes, never use new: I was wondering why for some classes, like Option, I could just call the constructor (eg. Some), but for others I had to write new Foo(...).  Turns out that it was just because I wasn't declaring my classes to be case classes.  Using case classes means I get to avoid all the confusing stuff that mutability gives you, for example, having to worry about the distinction between equality and identity.
• I may not be using traits right, but I think I like Haskell type classes better simply because with type classes, I can separate the instance declaration/definition from the class definition
• subtypes make for easy heterogeneous lists: in Haskell I'd wrap the things I want in some existential type E and then make a list of E.  In Scala, I make my types all extend the trait E.  For some reason, I find the latter a bit "easier", either more memorable/clear or just more convenient
• to get methods associated with classes rather than objects, just create a singleton object (that extends the desired trait)
• Scala lets you pattern match XML: it's kind of neat that you can interleave XML in your code and do things like pattern matching on it.

I get the impression that you basically have two kinds of Haskeller.  The first is the kind that uses Haskell because they can take on a lot of abstraction and use it to do amazing things, the smart Haskeller.  I think I fall more into the dumb category of Haskeller who tends to use it because I find the consequences of subtyping (Liskov substitution) and mutability to be too confusing.  And maybe there are also smart-dumb Haskellers... the kind who fall into both camps simultaneously.

RASP plugin now with actual parsing

Didn't manage to accomplish what I'd set out to yesterday: turns out I was wrong about the Digital Pebble plugin for RASP. I think it must have been emitting RASP tagged input for an earlier version of the parser. Took me a while to understand that the parser was simply not accepting the input (or rather, it was accepting it, but the input was causing it to crash). The fix turned out to be easy, just tweak the Java code so that it emits something slightly different.

Also, my brief foray into YAML was a bit of a failure. Turns out that for what I wanted to do, the JYaml parser had bug where it was confused by sequences of sequences of mappings like

- - a:b
  - c:d

Phooey. Oh well, at least in trying to work out what was going on, I got a taste of YAML and the kinds of features it supports.

I think today I'll actually have to try out what I said I would do yesterday. Only now I'm starting from actually being able to talk to the relations extractor. Guess I'll be switching to JSON

scala bits listening to Haskell bits but not understanding

Got the relations extractor to output YAML instead of the simple notation for triples pred(s,o). This is really a silly little job which you don't need something like YAML for, but like I said, it was mostly just a way of getting a concrete taste of the language and seeing what libraries are available. Right now I'm using data-object-yaml because it's convenient (high-level enough). I wonder why there isn't some sort of typeclass for it like the json library has; could maybe just make one myself (multiparameter?) but doesn't matter. I have all of two functions. One handy feature is data-object-yaml is that it shares an internal representation with data-object-json, so I can just treat them as two ways of rendering the same data.

The Scala wrapper now knows how to call the relations extractor and print its results back. I didn't really know what I was doing in terms of spawning a process, so I just imitated the original Java code using Threads and Runtime.getRuntime().exec(foo). I kind of wanted to play with Scala actors instead of threads, but I guess now is not the time.

Today's agenda may be a bit harder: I need to get the stuff from the relations extractor parsed and transformed into GATE annotations. I'll probably use a Java library called JYaml to do this. One bit of trickiness will be zipping the list of list of relations with the list of sentences. The main hard part will be learning enough of the GATE API and overall model for things.

more java/scala

Copied over the Java module and changed some of the private methods to public as I mentioned yesterday. Today's plan is to write my YAML-ised variant of the relations extractor -- shouldn't take more than an hour -- and set up a standalone pipeline that calls RASP, pipes output to said variant. If I get that done, then I can start forcing myself to learn enough Scala to make a system call and read/write output.

Resistance to learning. What am I, 40?

plugin plugged in

My little bit of progress yesterday was to successfully create a trivial GATE plugin in Scala. All it did was to subclass the RASP plugin from Digital Pebble, so when you ran it, it just output the same thing as the DP one. The one hiccup I encountered was some kind of Object Not Found error message, which basically meant that I needed to tell my plugin to include the scala-library.jar file, after which everything was OK.

So that's pretty cool, now with Scala I get to have a slightly less painful way to do Java stuff. Maybe it would be nice to have this working directly in Haskell, but it's better than nothing and learning a new language is probably not so bad for me anyway.

Today, my hope is to extend the plugin to do something a little different, maybe not exactly the relations extraction I have in mind. Even just appending "hello" to all annotations that the DP plugin would output would be a start.

One of the obstacles I've encountered is that functionality I need to make my plugin work is marked private by the DP plugin. So to grab at this functionality, I either need to:

• make a copy of the source code and just change the private to public
• re-implement the functionality in Scala

I think I'm going with the first option. This is the sort of thing that makes me feel like I have no idea how to do Object Oriented Programming. I vaguely understand why the practice of hiding functionality in modules is important (you want a nice clean API), but from a code re-use standpoint, it really gets in the way, and sometimes I feel like it's not always clear when it's right to hide and when it's right to expose functionality. I would tend to think that if you program in a functional style -- no mutation -- it'd be a lot safer to expose a lot more code, so you'd have a lot more liberty and can make the decision to hide stuff only where it really is beneficial from an API-design standpoint. Is people doing copy and paste programming (which I'm about to do) is a greater evil than the confusion from insufficiently black boxen?