Berkeley Bits

September 25, 2008

REPLAY –> Opencast update

Filed under: What I'm Up To — Tags: opencast, replay — jholtzman @ 4:33 pm

Quick progress update…

I’ve managed to get the REPLAY manager webapp to run individual jobs, return the ID of the job, and then return the status of the job. This proof-of-concept (based on replay v0.4) just executes an indexing job on a pre-defined media bundle. More work will obviously be needed to make this fully functional.

This is no big deal, really, but it does mean that REPLAY can fulfill the “media processing manager” component, as described in the opencast draft architecture document.

Comments Off

September 14, 2008

Replay requires Perian

Filed under: Hard-Won Wisdom, What I'm Up To — jholtzman @ 9:05 am

I spent a lot of time last week trying to get replay to encode, OCR, and index some video files that were captured and encoded for webcast.berkeley.edu. It was productive to go through the source code, looking for the reason I couldn’t get anything to index. After a couple of hours help from Tobias, we tracked down the problem: there is an undocumented dependency on the Perian toolkit on OS X. Now that perian is installed, mp4s, dv files, and the like are properly encoded, OCR’d, and indexed.

Comments Off

September 9, 2008

Webcast integration with REPLAY

Filed under: Hard-Won Wisdom — jholtzman @ 12:08 pm

The Opencast project’s first deliverable will be a version 0.1 integration of UCB’s Webcast system and ETH Zurich’s REPLAY (possibly also including Bjoern’s ‘channel syndication’ work). The primary barrier to the integration is the mismatch between REPLAY’s statically defined workflows (defined as ant scripts) and Webcast’s task based finite state machine (stored in the DB and loaded into memory at startup).

Tobias and I have thrown some ideas around for overcoming this mismatch, including:

Colocate Webcast and REPLAY in the same JVM (or package the custom ant tasks and scripts in a jar, deployed to both systems), where the commands executed on state transitions call the ant tasks directly
Call REPLAY’s internal SOAP interfaces
Iterate through the “success” transitions in the state machine to generate a “directives” file for REPLAY to execute
Generate a REPLAY directives file from the front-end, avoiding the state machine for everything but status reporting
Colocate Webcast and REPLAY in the same JVM and call the REPLAY Java APIs from Webcast’s transition commands
Integrate with Webcast’s ical based task reporting. This would involve:

Adding discrete, atomic workflows to REPLAY
Deploy an adapter to REPLAY’s manager component to consume Webcast’s ical tasks and execute the relevant workflow
Update Webcast’s ical feed to include all relevant recording metadata so the adapter can re-create the bundle if necessary

I’d like to go with option #6, but I need to do some more digging in REPLAY to see if it’s possible to either reuse existing media bundles from past workflows or less desirably, whether we can create new bundles by reusing an existing media file.

Comments Off

July 7, 2008

Sakai gadgets, sdata, and DWR

Filed under: What I'm Up To — Tags: ajax, gadgets, sakai, sdata, web20 — jholtzman @ 6:33 pm

I spent today at the Sakai hackathon hosted by CARET. In addition to enjoying the sights and avoiding the pouring rain in Cambridge, I spent some time looking at the new gadgets+sdata techniques that the cambridge folks are using to build their dynamic (and flat!) portal. It’s great to see that the ajax “revolution” has made its way into sakai. Here are my notes on sdata, and how DWR may be an appropriate substitute for this tool in certain situations.

Sakai’s sdata tool is a webapp that can be configured to dispatch URLs to specific handlers. The documentation, as far as I can tell, is here. The web.xml file contains the configuration that maps URLs and query parameters (methods) to the Java handlers.

My first gadget experiment involved calling /sdata/mcp (which stands for my course and project sites, I believe) to begin implementing part of the UX improvement mockups. The call to /sdata/mcp returns a list of sites for which I’m currently a member, including interesting tidbits like the number of members in the site. It’s clear that this could be a useful and reusable “Javascript Service” for a number of UI gadgets.

However, once I started working on this particular UI, I realized that I needed data from a tagging service, and this isn’t provided via /sdata/mcp (or any other /sdata URLs at this time). I could imagine getting a set of requirements that required merging gradebook data with, say forum posts, along with site memberships, for some kind of forum grading overview widget. Who knows what those UI designers will come up with

The problem this poses is that there will be a constant push on the sdata tool to either incorporate new URL handlers as gadget designers and implementors think of new and exciting gadgets, or there will be a proliferation of ajax calls from clients trying to join data from a variety of services.

In these cases, I propose that we forego the use of sdata entirely, and utilize the DWR library instead. DWR provides some compelling advantages over sdata, along with some added costs. Here’s how I think DWR stacks up against sdata for sakai gadget development

Pros

Data can be shaped on the server side however the client wants it shaped, so a single XHR can provide everything the gadget needs to render itself.
Gadget developers can do all of their development in their gadget without need for modifying sdata, and dealing with the merge headaches that will inevitably come from a centralized service.
DWR is a popular library for exposing java classes as ajax services, so the pool of developers with DWR expertise will likely be larger than those with sdata expertise.

Cons

Each gadget (or collection of gadgets sharing a webapp) wishing to use DWR requires a dwr.xml configuration file. My gadget’s dwr.xml file is 9 lines, so I don’t think this is too onerous.
Gadgets wishing to reuse an existing ajax service either need to copy the service into their own dwr-enabled webapp, or they need to assume that the other webapp will be deployed at runtime. For this reason, it may make sense to centralize often-used ajax services in sdata. It’s all a matter of tradeoffs between reuse and flexibility.
Performance. DWR generates Javascript files on the fly via a servlet. This will presumably add more load to an application server than serving up a static .js file, but my guess is that this will be minimal. DWR 3, along with JAWR, provides a mechanism for serving cached and compressed .js files.

I’ve uploaded a zip file containing a preliminary sites gadget. It is nowhere near feature-complete, but it demonstrates both the sdata method (via /devwidgets/sites/sites.html) and the dwr method (…/sites_dwr.html). Please have a look and let me know (via email, IM, or via sakai-dev) what you think.

Comments Off

April 6, 2008

JSON-based UI Templating

Filed under: What I'm Up To — jholtzman @ 7:02 am

I think I’ve come up with an interesting approach to building a UI, and I’d like some honest feedback. I’m in that geeky “I just built the greatest thing in the world” state, and I’d benefit from a reality check.

Here’s how it works:

1) On the server side, you make your service calls and marshall the objects that the UI needs into JSON strings. Stick those strings into the request, and consume them in the JSP/templating language, or whatever. Nothing new here, other than the fact that you’re sending JSON strings rather than java objects.
2) In the JSP, you consume those JSON strings *once*, at the top of the section of the DOM. Here’s an example:

$(document).ready(function(){
var eventAsJson = (function () {
return eval(’${eventAsJson}’);
})();
…

Note that ${eventAsJson} is the only JSP-ism on the page, and this could be easily replaced with a real JSON string (provided by a developer). See where I’m going with this yet?

3) Build the dynamic parts of the page in JavaScript using jQuery. For example:

var eventTitle=eventAsJson.title;
$(’#title’).html(eventTitle);
…

<h3 id=”title”>The Event Title</h3>

4) What you have now is a 99% generic html template, and with a tiny bit of effort, you can do all of the UI development you like without ever touching the server side.

What do folks think? Feasible?

Comments Off

June 25, 2007

Embedding an XMPP Server in Sakai

Filed under: Hard-Won Wisdom — jholtzman @ 5:02 pm

The first problem is finding a suitable project to start with. The openfire server is very nice, but it’s GPL. See my previous post for more whining on this issue.

The next problem is clustering the XMPP server itself. If we embed the server as a spring bean, we’ll have multiple instances running in our cluster, and these instances will need to coordinate among themselves. Some servers have this capability built in, but with such limited choices already due to licensing, the clustering feature may shrink the number to zero.

Also, the XMPP protocol requires that users log in when connecting to a server. This is fine for external applications wishing to communicate with a sakai-hosted XMPP server, but it’s problematic for internal use because we (rightly!) have no access to a user’s password at runtime. One solution to this could be to pass the session ID as the password, and have the XMPP server’s directory service implementation check to see whether a sakai session with that ID is currently active for that user. It’s a hack, but at least it’s a place to start.

zithromax
buy zithromax
sildenafil citrate
sildenafil
generic sildenafil citrate
viagra sildenafil
buy sildenafil citrate
sildenafil viagra
levitra
buy levitra
cheap levitra
levitra online
buy levitra online
order levitra
amoxil
buy amoxil
amoxil online
accutane
buy accutane
accutane online
cialis
Generic Cialis
Buy Cialis
Buy Cialis Online
cialis online
Order Cialis
Cheap Cialis
rimonabant
buy rimonabant
acomplia rimonabant
rimonabant acomplia
cipro
cipro online
buy cipro
synthroid
buy synthroid
synthroid online
diflucan
buy diflucan
diflucan online
vardenafil
levitra vardenafil
generic vardenafil
vardenafil hcl
buy vardenafil
vardenafil online
prozac
order prozac
prozac online
buy prozac
viagra
buy viagra
buy viagra online
viagra online
order viagra
cheap viagra
generic viagra
doxycycline
buy doxycycline
tadalafil
buy tadalafil
cialis tadalafil
buy tadalafil cialis
cheap tadalafil
tadalafil citrate
fosamax
order fosamax
fosamax online
lasix
buy lasix
voltaren
buy voltaren
propecia
buy propecia
order propecia
cheap propecia
propecia online
order propecia online
buy propecia online
generic propecia
amoxicillin
buy amoxicillin
amoxicillin online
zoloft
buy zoloft
generic zoloft
soft tab viagra
viagra soft tabs
viagra soft tab
zyrtec
buy zyrtec
zyrtec online
glucophage
buy glucophage
clomid
buy clomid
nolvadex
buy nolvadex
finasteride
propecia finasteride
buy finasteride
cialis soft tab
cialis soft tabs
cialis soft
lexapro
lexapro online
buy lexapro
flagyl
flagyl online
acomplia
buy acomplia
buy acomplia online
acomplia online
zithromax
buy zithromax
sildenafil citrate
sildenafil
generic sildenafil citrate
viagra sildenafil
buy sildenafil citrate
sildenafil viagra
levitra
buy levitra
cheap levitra
levitra online
buy levitra online
order levitra
amoxil
buy amoxil
amoxil online
accutane
buy accutane
accutane online
cialis
Generic Cialis
Buy Cialis
Buy Cialis Online
cialis online
Order Cialis
Cheap Cialis
rimonabant
buy rimonabant
acomplia rimonabant
rimonabant acomplia
cipro
cipro online
buy cipro
synthroid
buy synthroid
synthroid online
diflucan
buy diflucan
diflucan online
vardenafil
levitra vardenafil
generic vardenafil
vardenafil hcl
buy vardenafil

Comments Off

XMPP Servers

Filed under: Hard-Won Wisdom — jholtzman @ 2:36 pm

I’ve been looking at the available ECL license compatible XMPP server options out there, and the pickings are unfortunately slim. A quick perusal of this comparison shows that there are just two non-GPL XMPP servers that we can use: jabberd14 and openim.

Upon checking out openim, I noticed that the svn revision number was 10. There haven’t been any commits since May 7, so this project is either incredibly mature and in no need of bug fixing or new features, or the project is dead. My guess is the latter.

jabberd14, unfortunately, isn’t a java project. It’s implemented in C/C++, which means that it’ll be tricky to add to Sakai’s build and to the component manager.

Onward…

Update: openim pegs my CPU without having any work to do… lovely.

Online RX Meds Drug Store for men and women Cialis Professional is a newly formulated and chemically improved prescription medicine Buy Viagra without prescription to treat erectile dysfunction

Comments Off

January 26, 2007

Transactions, Hibernate Sessions, and Spring initialization

Filed under: Hard-Won Wisdom — jholtzman @ 5:03 pm

Sakai’s CM service is very chatty. In particular, there are two places that make many small calls to CM. The first is Sakai’s CM-based GroupProvider. The second is a data loader to populate CM with a sample university SIS.

In both the GroupProvider and the data loader, I’ve been noticing some pretty poor performance. After reading a lot of spring source code and hacking in some extra logging, I realized why. If I call the CM service from outside of its transaction boundary, I get a new hibernate session for each CM call. When each method call uses a unique hibernate sessions, we get no L1 caching at all. And without a properly configured second level cache in Sakai, this means that there’s no object caching at all, and CM — due to the shape of the api — is just going to perform horribly.

Fortunately, there’s a way around performance bottleneck. If I make the CM calls from within one of the global transaction manager’s transactions, all of the hibernate activity will share the same session and will thus take advantage of hibernate’s L1 object caching.

Well, this worked pretty well for the GroupProvider. I noticed much less DB activity after I wrapped the GroupProvider impl in a transaction proxy. But what about the data loader? Well, the data loader does its thing during startup, with an init() called by spring after the bean has been instantiated and its dependencies set. That means that the data load occurs becore the current thread has a transaction bound to it. So even though the loader is wrapped in a transaction proxy, the init() isn’t transactional.

Of course, this didn’t deter me. I just need another spring bean to do the initialization, so it’s init() calls the dataloader’s init() through the transaction proxy! Brilliant, no?

No.

I believe the reason why this didn’t work is because the transaction proxies are set up with bean post processors, and my init() was run before the post-processing. Oh well, I guess having a slow sample data load (which is only for demo purposes) is better than having slow runtime performance.

Comments Off

July 14, 2006

Noisy Spring Bean Destruction

Filed under: Hard-Won Wisdom — jholtzman @ 4:45 pm

When spring’s DefaultListableBeanFactory (a superclass of a ClassPathApplicationContext, which is at the heart of Sakai’s ComponentManager implementation) fails to pre-instantiate all of it’s singleton beans, it immediately destroys all of them to ensure that they aren’t hanging around in a bad state. This makes sense, but in 1.2.8 the cleanup is implemented in a very quiet way. No logging, no stacktraces, nothing.

In order to debug why my component manager was failing to start up on my machine, I edited Spring’s DefaultListableBeanFactory so it would be a little noisier about *why* it was immediately destroying all of its singletons. If you find yourself in the uncomfortable position of debugging a component manager that is quietly failing, this will help you out.

The modified java file is available, as is the modified jar for those with better things to do than modify and build third-party jars.

Comments Off

Jar Hell with Maven? It can happen…

Filed under: Hard-Won Wisdom — jholtzman @ 4:35 pm

For the Fall 2006 bSpace deployment, we’re running a modified Sakai 2.1.2 along with an unreleased, bleeding-edge, post 2.2.0 version of the gradebook that’s been backported to work in 2.1.2. There were a lot of small changes to make to get this version of the gradebook to work in 2.1.2, including reverting to hibernate 2, updating the gradebook’s dependencies, and changing config files and import statements to point to the new (old) package names. But in the end, it was no big deal.

However, the name of the external gradebook service changed between Sakai 2.1.2 and 2.2. What was sakai-gradebook-service became sakai-gradebook-service-api. Notice the “api” appended to the end of the name.

Remember, other projects rely on the gradebook service to add assessments and scores into the gradebook.
So the name of the API changed… big deal. Well, it turns out that when a client of the gradebook service can’t locate its dependency locally in the current source tree, we get a build failure. However, that didn’t happen. What we did get was happy developer deployments to our local machines and happy deployments (mostly) to the integration box. But once it was time to deploy to QA, we got compilation errors.

That’s right, compilation errors. Not your typical NoClassDefFound classpath errors, but “symbol not found” errors.

After doing some digging, I realized that the sakai 2.1.2 jars had been published to the sakai maven repository. So even though there were gradebook service clients that couldn’t find the “sakai-gradebook-service” project locally, those clients could find the dependency out on the web. If the client code used a method that didn’t exist in the old jar, there would be a compilation failure. And this is what we saw on happen on QA.

So, I did two things to fix this. First, I updated the gradebook service clients in bspace to use the new API name so their build dependencies are satisfied locally. Second, I changed our jar version numbers to match our official bspace version, ‘bSpace-2.1.2_1_RC1′. This way, none of the bspace code will try to compile against published jars. They will always use the local code.

The moral of the story? If you’re making modifications to services, always use a custom version number for your jars.

Comments Off

« Newer Posts — Older Posts »