My first contribution to the project is an extensive set of JUnit 4 test cases that exercise OCL constraints specified for objects in the  model.  Most of the tests are fairly simple, basically setting up configurations where validation is expected to fail or it is expected to pass, and verifying the expected outcome.  To my great satisfaction I was able to catch dozens of real errors, mostly things like typos in the OCL, but also errors in which OCL statements were improperly processed during code generation and ended up mangled in the result (a surprise).  The tests even tripped across errors in OCL processing  itself when particular invalid objects were erroneously validated as correct (another surprise).

The real story, however, was the design challenge needed to create the tests.  We don’t usually think of test cases as being particularly hard to create, and they usually aren’t, but in this case I had a particular problem.  It turns out that the HL7 standard has a few interesting attributes, one of which is that it is possible to define new types of “documents” by referencing an arbitrary collection of parent “templates.”  The practical result of this is unconstrained multiple inheritance.  This is something that the EMF can handle, but it makes testing a potential nightmare.

The main problem is that the classes to be tested could and did have somewhat arbitrary collections of things to be validated, but because Java doesn’t support multiple inheritance, it isn’t possible to simply mimic the EMF supported multiple inheritance hierarchy to create the matching collection of tests.  To complicate things, many of the tests are very similar, often only varying on just the particular field participating in validation and the exact OCL expression.  These two attributes combined to create the potential for massive redundancy in the test cases.

What to do?

The strategy came up with was to create a collection of independent test case “templates” that could be customized using EMF reflection to test the exact combination of field and validation.  These templates were implemented as abstract (static inner) classes.  Anonymous instances of these template test case classes were then created as appropriate to put the right set of tests together to match what was inherited in the EMF generated code of the class under test.  These anonymous instances were initialized with details like the name of a feature to customize the template.  The collection of anonymous test case definitions were then grouped together for a particular class being tested by defining them in a static initializer of an array defined in the JUnit 4 test case.  That class then only had one “test” which had the simple job of running through the collection of anonymous test templates, triggering each one along the way to do its thing.

The end result worked nicely, arbitrary tests could be combined to adapt to the whims of document design allowed by the HL7 standard, and they could be specified in a simple and consistent pattern.  A potential explosion in test redundancy was avoided by using the Template design pattern and exploiting the capabilities of the Eclipse Modeling Framework to do reflective accesses.  The only downside of this approach seems to be that the count of tests performed by JUnit seriously under reports the actual number of tests performed as each JUnit test can actually trigger the execution of dozens of test templates.

Summary: This is a well executed and detailed explanation of how to develop modular Java systems and applications using OSGi and Equinox.  It uses the development of an easy to follow example application, called Toast, as the vehicle to explain OSGi theory and practice using Equninox.  One of the main themes of the book is the whys and hows of OSGi Declarative Services.  In general, it is a good book for OSGi beginners, but familiarity with Eclipse is a perquisite.  Advanced developers will find it to be a good resource and example of OSGi best practices.

I read the book in two passes, the first while I was on an extended business trip this Spring where it became my “on the road” late night reading.  The second pass was at my desk with my laptop and the book propped up on a book stand while I did the examples.  I like to read technical books this way as the first pass allows me to concentrate on understanding the material without being bogged down by the examples. It also gives me an overall view of the material that is quite useful when I do starting focusing on the example code.  The second pass is deliberately set at a slow pace so that I can type in the examples myself and linger over how they work, your mileage my vary.

I found that this book lent itself quite nicely to that style.  The writing was clear and well edited; you could tell that it had been through many revisions to get it to its current polished state.  The step-by-step instructions for the examples was at the right level for me, with enough detail to get things to work without being overly long. For instance, I find that I quickly get the concept of the example, but then I like to be given the details of what to name things and exactly what else to do, so that I can concentrate on the example and keep things moving along.

Another bonus that comes with the book is an Eclipse plug-in that can be installed from the web which includes the source code of all of the examples organized by chapter.  It manifests itself as a special view in Eclipse that lists all of the example code.  One can use this view to populate the workspace with the example code from any chapter, or, to compare the current workspace contents to the book’s example.  I found this last feature to be a great help as there were several times where things were not working and I was stuck for a solution.  By simply comparing my manually entered version of the example code with the chapter’s reference version, I quickly found the small differences that were causing problems and was quickly on my way.  This ability is the next best thing to having the authors look over your shoulder and tell you what you did wrong.

I did run into a few issues with the book.  There were several times where the steps provided to produce the example code were not complete.  These were minor things like a missing dependency specification or in one case a default value produced by a wizard that needed to be explicitly set to something else.  Mostly, these were no problem to correct.

I have a few suggestions for a second edition.  It would be nice to have a better explanation or “tour” of the Equinox bundles and their contents; the book currently glosses over their organization.  Another addition would be some positioning or contrast between Equinox and other OSGi implementations.  This wouldn’t need to be long or extremely detailed, but it would be nice to know a bit more about the trade-offs between various implementations and why Equinox is such a good choice.

Basically, I wasn’t disappointed, this is a good book that delivers on what it promises.

But we can try.

The current focus of my self-education efforts is Git and I think I’m making some progress.  Or, at least, I’m at the point of understanding enough things that I can reflect back on my ignorance and look at the path I took to my current state.  Of course, I’m absolutely certain someone will read this posting and confirm that I’m deluding myself. Anyway, I’ve always been fascinated by the “Ah. Ha!” moment, which is the point where something just clicks and I suddenly just “get it,” whatever it is that “it” is.  I always wonder why someone didn’t just tell me the “Ah ha!”  stuff first.  I certainly got that feeling learning Git.  I’ll try to explain my version of the “Ah ha!” stuff below, maybe it will work for you.

Most explanations of Git that I’ve encountered begin with some true, but not very enlightening statements.

• “It’s distributed.”

• “Everyone has their own copy of the repository.”

• “It’s a graph.”

• “It is easy to branch, people do it all the time.”

These initial descriptions of Git are well meaning and actually do accurately summarize many of its main characteristics, but they seem to lead away from giving the student the “Ah. Ha!” moment they need, or at least they did for me.

To someone only familiar with CVS and Subversion, which is probably the case for most people trying to learn Git, these initial descriptions seem like rather undesirable characteristics for a revision control system, they may even sound like a recipe for complete chaos.  They certainly raised a bunch of immediate questions in my mind.  For instance, wouldn’t it be hard to coordinate and manage the different distributed instances?  Also, how could everyone have their own copy and why would that be a good idea, how would you synchronize all the different copies?  And, what about back-up?  What happens if you lose one of distributed copies of the  repository, wouldn’t you lose part of your code base?  And, finally, what’s with the branching thing anyway, that just sounds like an extra complication?

To figure this out I read “Pro Git,” by Scott Chacon, which I thought was pretty good, and other material I could find online.  The moment things became clear, though, was when I realized the significance of the role played by the SHA-1 cryptographic check sum that Git computes for each set of changes committed to a copy of the repository.

So, here’s where I had my “Ah. Ha!” moment.  Centralized repository systems like CVS and Subversion produce change set identifiers, version or revision numbers, that are relative to, and only valid within the context of, a single centralized repository instance.  And, the relationships between change sets are implicitly expressed by the value of the identifier (i.e., revision 999 comes “after” revision 998).  Here it is:  In Git, the use of the SHA-1  cryptographic check sum allows these two characteristics to be different. The check sum plays the role of an absolute, not relative, identifier.  In essence, it is the absolute address of a change set in the virtual address space of change sets, as defined by the SHA-1 computation.

Git it?!

Ah!: The use of the SHA-1 cryptographic check sum allows Git to produce
unique revision numbers without the need to consult an external
(centralized) authority.  This means that there is no need to be
in communication with anything to commit.  The computation of the
check sum can be performed independently, offline, but because all
implementations of Git perform exactly the same computation, this
is not a problem.  Independently committed change sets can come
together at any time in the future with no clashes.

Ha!: Git explicitly links change sets by storing the check sum of the
previous change set (i.e., the “parent” or “parents”) in the current change
set, and using this value in the computation of the current check
sum.  This produces a unique, explicit, chain of linked change sets that can
be followed back to the original state of the system.  This means
that arbitrary sequences of change sets, created by many
independent programmers, can be committed to their own instances
of a Git repository and then later, if desired, shared and
integrated with other related Git repositories by finding common
ancestors.

Now we’re talking “Shrimp dip.”

With an understanding of these two points, the idea that Git is “distributed” starts to make some sense.  There could be two instances of a Git repository on two different computers, and two different programmers could commit to their respective different repository instances without consulting each other.  Later, the change sets from the different commits could be exchanged between the two repository instances and would be available to both programmers and could be merged as appropriate, or not.  And, of course, there’s nothing special about having just two instances, there could be an arbitrary number of instances “distributed” across many different computers.  I must say that this isn’t what I thought when I first read “distributed,” instead, I envisioned a repository that stored different portions of its contents in a, mostly, non-redundant manner on many different machines.  Now, I understand it means that it stores mostly redundant copies of its contents on different machines, with the differences being propagated, occasionally, as necessary.

The idea that everyone would have their own copy of the repository also doesn’t seem so strange, it is  completely obvious, of course they would, how else would it work?  However, I have to admit that it took me a moment to realize that this flexibility doesn’t imply that all of the Git repository instances are equally “valid” with respect to creating the contents of an eventual system release.  There is still the need for a single Git repository instance to serve as a curated, backed up, collection point for contributions from copies (“clones” in Git-speak).

Finally, the ideas that Git repositories form a graph and that branching is easy and happens all the time,  become obvious.  The change sets form the nodes of a graph which are linked together using the check sums as pointers.  Branching is simply a function of how the different commits point to common ancestors.

Now, it’s time for some more dip.

I figured out that the problem was that the PATH wasn’t being set correctly when the Git Gui client was attempting to connect. That seemed simple enough to remedy so I fired up putty on my laptop, connected into the server and did an “echo $PATH” to see what the path looked like. Sure enough, it was missing the “Git/bin” and ”Git/libexec/git-core” directories, the later being the home of the missing “git-upload-pack.” I updated my “~/.bashrc” file to add the directories to the path and optimistically expected things to just work.

Yeah, right.

I used putty to log back into the server to examine the path and discovered to my surprise that my additions were not there. I eventually figured out that I should have been updating “~/.bash_profile” and not “~./bashrc“. I also fixed a typo in my addition. So I ended up with the following in my “~/.bash_profile“:

export PATH=$PATH:/cygdrive/c/Progra~1/Git/libexec/git-core:/cygdrive/c/Progra~1/Git/bin

And, sure enough, when I logged back in with putty and examined the path, they were on it, and “which git-upload-pack” returned exactly what it should. At last, I was done, everything should just work, I fired up the Git-Gui on my laptop, give it the correct URL’s to clone the test repository on the server…and got exactly the same error that it was unable to find “git-upload-pack“.

An hour, or so, of bumbling around lead me to comparing the path returned from:

ssh myid@mygitserver echo \$PATH

with that obtained from putty…they were different. It turns out that the ssh session started by the Git Gui used “~/.bashrc” while that started by putty used “~/.bash_profile.”

The real problem was that I had gotten my first PATH fix wrong when I used “~/.bashrc” and had assumed that the problem was that I had the wrong file rather than simply having a typo in the path. That error was masked later when I discovered that I could change putty’s PATH from “~/.bash_profile” and I just assumed that that was the correct file to use. Once I put the correct path into “~/.bashrc” “everything” started working.

Now, it’s time for my plunge into the wonderful world of Git, I expect everything to just work.

Yeah, right.

One practice that I’ve encountered, and have tried in practice, is to make it as easy as possible for a new person to join a software development project. This is a simple practice that can payoff big by getting new people to be productive as soon as possible. For open source projects, it’s pretty much a requirement for attracting other developers, because if it is hard to join a project, they’ll simply move on to something else (and you’ll never know). The surest way to do this is to codify the project’s tacit knowledge in an “immigration” document. This is a manual or wiki that you give to a new person that explains EVERYTHING about the project other than its design (and maybe a bit of that too). For instance, simple things like the names of the other developers, their contact information and the parts of the project that are their responsibility should be included. Also, a complete inventory of the tools in use, their versions, where to get them and how to install and configure them is incredibly useful to a newcomer. Administrative trivia is also essential; things like how to obtain an account, where the repositories are, the names of servers and where they’re located, and anything else that “everyone” just knows. A “starting out” check list is also a big help. This is step-by-step list of tasks for a new person to execute, the basic idea is to get them to be instantly productive by setting up their environment without any kind of drag on the rest of the team. The most important component of the immigration document, however, is an accurate, up-to-date explanation of the project’s build process. Why? Well, first, it means that the project actually has a build process! Second, it gives the broadest, least ambiguous, picture of the system being developed. The build process is also a reflection of a capabilities and professionalism of the organization that created it and it sets expectations for the project newcomer, and you want those expectations to be set to the max from day one.

This last point actually cuts both ways, if you’re a developer interviewing with a company and they get to the point where they say “Do you have any questions for us?” You should sit up and start digging for information about their build process. If it involves some guy running between two ancient PC’s with a couple of floppy disks, don’t expect to show up on the first day to a desk, a chair, a computer and the necessary accounts already set up; you shouldn’t expect an immigration document either. However, if the process is completely automatic, look knowingly impressed, throw them a compliment, and then ask to see a copy of their immigration document. If you get one, say “Wow! Where do I sign up?” If you get a blank stare, be nice and explain, interviewers love candidates that that can teach them something.