gitlive
is a new service leveraging some of the new (and some of the proven) technologies in ejabberd.
It’s a widget displaying commits from GitHub … live!
Usage is very simple, just head over to the service for detailed instructions and sample usage.
It is also deployed on the Tsung and ejabberd homepages (see the “Commits” section on the bottom right).
There are two main components:
- ejabberd with custom modules
- client code in javascript
The client component will attempt to connect using the most efficient protocol to ejabberd but will fallback on BOSH is necessary. It will then connect anonymously and subscribe to the PubSub projects nodes.
To be continued after the jump!
Transport protocol
Most efficient protocol?
The client will attempt to connect using the WebSocket transport to ejabberd. WebSockets are a bidirectional protocol that solve one of the problems HTTP can’t solve — by design. Many crutches have been built on HTTP to solve this problem. In the XMPP world it’s BOSH. But it’s not efficient on the server and on the client (think mobile applications).
WebSockets will solve all this. “Will solve” because even though we have implementations in the latest versions of Safari, Chrome, Firefox and Opera, it is still an IETF draft. Little progress has been done on the subject in the last year, and the latest iterations of the protocol are not implemented. For a good overview of the issues, Dave Cridland has you covered.
But still there is an installed base we can build on. So there, we have developped a native websocket server implementation for ejabberd. And for browsers not supporting websockets, we have Adobe Flash™.
Flash?
Yes, the irony isn’t lost on us either. A nice project available on GitHub, web-socket-js, implements the WebSocket API on the client and the protocol towards the server. So if you have Flash and no native WebSocket support, the gitlive widget will load the Flash file. It’s only 18Kb.
Back to BOSH
If all the previous techniques fail, the widget will attempt a BOSH connection. It’s many times slower than websockets and consumes more resources on client and server. But it’s very failsafe as it works everywhere vanilla HTTP works.
This combination of transport mechanisms make sure that gitlive notifications can be viewed with any browser, any time.
Front-end
The rest of the code is pretty much straightforward. The client code uses a combination of StropheJS and jQuery, and the commits are templated with mustache.js. Clients connect using anonymous authentication and subscribe to the project nodes.
It also uses the p1:rebind feature, a mechanism enabling fast rebinds to a previous connection.
You can actually customize the commits by overriding the CSS and mustache templates:
var GithubPush = {nodes:["cstar/erlang-oauth"],
location:"#commits",
css: "http://web.site/your.css",
template: "inline mustache template"};
This is default mustache template:
"<li class='gh_commit' id='{{id}}'>"
+ "<a class='gh_author' href='https://github.com/{mfoss}'>"
+ "<img src=' http://www.gravatar.com/avatar/{{avatarhash}}?s=70'/>{{name}}</a>"
+ "<div class='push'><h5><a href='{{repository_link}}'>{{repository_name}}</a></h5>"
+ "<h4><a href='{{link}}'>{{title}}</a></h4>"
+ "<abbr class='timeago' title='{{time}}'>{{time}}</abbr></div>"
+ "<pre>{{message}}</pre><a class='watermark' href='http://gitlive.com'>GitLive</a></li>"
Note: that would be nice if you’d leave a link back to http://gitlive.com/ with your custom templates.
The default CSS is available here.
Back-end
It is a very streamlined ejabberd 2.2. The only module activated is mod_pubsub. On the HTTP side, we have our custom websocket module, the http-bind module and the GitHub webhook.
After each push, GitHub will POST on the webhook a JSON payload with all the meta-information about the the commits just pushed. The GitHub webhook will create the pubsub node if necessary. (That’s why you need to Test Hook on the GitHub admin inteface). The JSON payload is converted to an Atom PubSub payload, and published on the node. ejabberd takes care of the rest, distributing the payloads to subscribers.
Bugs
There are still a couple of bugs that I hid under the carpet. But this time, and this time only, I am going to remove the carpet:
- BOSH connections are not reattached. The last item is not sent in that case.
- WebSocket connections need to resubscribe to the channels every time even after rebind, putting an unnecessary load on the storage backend.
Both bugs will be corrected soon, and workarounds are deployed to silence these bugs in the meantime.
Future plans
Though we plan to keep this product simple, but we will federate the gitlive server with the XMPP community. You will be able to get GitHub push notifications in real time in your favorite XMPP client. Provided it can parse Atom payloads. A little birdie told me that OneTeam will in a not-so-distant future.
This product is also the first of a series enabling websites to add realtime notifications, events, or … Javascript.