Berkeley Bits

July 14, 2006

Noisy Spring Bean Destruction

Filed under: Hard-Won Wisdom — jholtzman @ 4:45 pm

When spring’s DefaultListableBeanFactory (a superclass of a ClassPathApplicationContext, which is at the heart of Sakai’s ComponentManager implementation) fails to pre-instantiate all of it’s singleton beans, it immediately destroys all of them to ensure that they aren’t hanging around in a bad state. This makes sense, but in 1.2.8 the cleanup is implemented in a very quiet way. No logging, no stacktraces, nothing.

In order to debug why my component manager was failing to start up on my machine, I edited Spring’s DefaultListableBeanFactory so it would be a little noisier about *why* it was immediately destroying all of its singletons. If you find yourself in the uncomfortable position of debugging a component manager that is quietly failing, this will help you out.

The modified java file is available, as is the modified jar for those with better things to do than modify and build third-party jars.

Jar Hell with Maven? It can happen…

Filed under: Hard-Won Wisdom — jholtzman @ 4:35 pm

For the Fall 2006 bSpace deployment, we’re running a modified Sakai 2.1.2 along with an unreleased, bleeding-edge, post 2.2.0 version of the gradebook that’s been backported to work in 2.1.2. There were a lot of small changes to make to get this version of the gradebook to work in 2.1.2, including reverting to hibernate 2, updating the gradebook’s dependencies, and changing config files and import statements to point to the new (old) package names. But in the end, it was no big deal.

However, the name of the external gradebook service changed between Sakai 2.1.2 and 2.2. What was sakai-gradebook-service became sakai-gradebook-service-api. Notice the “api” appended to the end of the name.

Remember, other projects rely on the gradebook service to add assessments and scores into the gradebook.
So the name of the API changed… big deal. Well, it turns out that when a client of the gradebook service can’t locate its dependency locally in the current source tree, we get a build failure. However, that didn’t happen. What we did get was happy developer deployments to our local machines and happy deployments (mostly) to the integration box. But once it was time to deploy to QA, we got compilation errors.

That’s right, compilation errors. Not your typical NoClassDefFound classpath errors, but “symbol not found” errors.

After doing some digging, I realized that the sakai 2.1.2 jars had been published to the sakai maven repository. So even though there were gradebook service clients that couldn’t find the “sakai-gradebook-service” project locally, those clients could find the dependency out on the web. If the client code used a method that didn’t exist in the old jar, there would be a compilation failure. And this is what we saw on happen on QA.

So, I did two things to fix this. First, I updated the gradebook service clients in bspace to use the new API name so their build dependencies are satisfied locally. Second, I changed our jar version numbers to match our official bspace version, ‘bSpace-2.1.2_1_RC1′. This way, none of the bspace code will try to compile against published jars. They will always use the local code.

The moral of the story? If you’re making modifications to services, always use a custom version number for your jars.

July 6, 2006

jboss-cache and multicasting

Filed under: What I'm Up To — jholtzman @ 11:18 am

It turns out that the jboss-cache config file used in jforum must have a bind_addr set to the (non-loopback) IP address on the NIC. Because of our proxy / load balancing setup, we have routable IP addresses on the loopback, which confuses jboss-cache. It simply doesn’t know which IP address to bind with.

So, we now need to customize each jforum-cache-cluster.xml file with its own bind_addr setting.

It also turns out that the ‘multi’ in multicast is no joke. I was picking up multicast packets on my laptop in Dwinelle Hall that originated from the app servers across campus in the data center. According to Cisco’s documentation:

“The range of addresses from 239.0.0.0 through 239.255.255.255 contains limited scope addresses or administratively scoped addresses. These are defined by RFC 2365 to be constrained to a local group or organization. Routers are typically configured with filters to prevent multicast traffic in this address range from flowing outside an autonomous system (AS) or any user-defined domain.”

So, we need to ensure that we use these “limited scope addresses” to separate the cache traffic on the dev, qa, and prod clusters… and from the rest of the world. We don’t want a script kiddies sending jgroups messages via multicast to our apps.

Powered by WordPress