Scalable XMPP bots with erlang and exmpp, part III

In the first articles of this series, we learned how to use the exmpp API and how to use it to write a simple external XMPP component.

In this last installment we will complete our component, making it aware of the presence status of our users.

The source code for this article can be downloaded from web_status_v2.tar.gz.

User subscription revisited

If we want to know and keep track of the presence status of our users, we need to make our bot a participating entity in the presence flow of the XMPP network.
Immediately after a user successfully registers with us, our component should send a request, subscribing to the user’s presence information. If he accepts, his server will inform us of his current presence status and from now on it will broadcast to us any subsequent change. Perfect!

But the opposite is not true. As an external component, our bot resides in a kind of virtual domain (webstatus.testdomain) inside our main ejabberd server (testdomain). It is not a normal client, and ejabberd (or any other XMPP server) won’t manage our roster for us. In fact, ejabberd doesn’t even know that it exists, ejabberd’s only duty is to forward any packet directed to webstatus.testdomain to our component. So, we must manage our subscription state by ourselves in the component. This in practice means that our bot must be capable of handling subscription requests directed to it, and must broadcast its own presence information to users that have previously subscribed with it.

We start by adding a new field to the #user{} record, to keep track of the current subscription state.

-record(user, {	bjid :: binary() ,	conf :: #user_conf{} ,        subscription_state = none :: none | from | to | both ,   	state = [] :: [#user_state{}]}).

We are using a simplified set of possible subscription states, this is enough for the purpose of this example. For a complete reference, see RFC3921.

State Description
none no subscription
to we are subscribed to the user’s presence
from the user is subscribed to our presence
both mutual subscription

And then modify the function that handles registration, to subscribe to the user presence when a new user registers with the bot.

process_iq(Session, "set", ?NS_INBAND_REGISTER, IQ) ->    From = exmpp_jid:parse(exmpp_stanza:get_sender(IQ)),    UserConf = parse_form(exmpp_xml:get_element(exmpp_iq:get_payload(IQ), ?NS_DATA_FORMS, 'x')),    User = case web_status_db:get_user(exmpp_jid:prep_bare_to_binary(From)) of	{ok, ExistingUser} ->                ExistingUser#user{conf = UserConf};	not_found ->               #user{ bjid = exmpp_jid:prep_bare_to_binary(From),                      conf = UserConf,                      subscription_state = none,                      state = []}    end,

    ok = web_status_db:store_user(User),    send_packet(Session, exmpp_iq:result(IQ)),    case User#user.subscription_state of        none ->            %% this is only necessary if we aren't subscribed yet.            SubscriptionRequest = presence_subscribe(exmpp_jid:bare_to_binary(From)),            send_packet(Session, SubscriptionRequest);        _ ->            ok   end.

Remember that we were using the same code for both registration and preference changes. That is why we check if we are already subscribed with the user before sending the subscription request.

If the user accepts our subscription request, it must reply with a <presence type=”subscribed”> stanza. If he rejects us, it should send a “unsubscribed” response. We must handle these presence stanzas.

First, add a dispatch on presence stanzas (our first bot only handled IQ stanzas):

process_received_packet(Session,     #received_packet{packet_type = 'presence', type_attr=Type, raw_packet = Presence}) ->	process_presence(Session, Type, Presence);

Now handle the required cases, if the user accepts:

process_presence(_Session, "subscribed", Presence) ->    From = exmpp_jid:parse(exmpp_stanza:get_sender(Presence)),    case web_status_db:get_user(exmpp_jid:prep_bare_to_binary(From)) of	{ok, #user{subscription_state = State} = User} ->                NewSubsState = transition(State, subscribed),                web_status_db:store_user(User#user{subscription_state = NewSubsState});	not_found ->                ok	end;

If the user rejects our subscription request or cancels it, we remove it from our users DB:

process_presence(_Session, "unsubscribed", Presence) ->    web_status_db:delete_user(        exmpp_jid:prep_bare_to_binary(            exmpp_jid:parse(                exmpp_stanza:get_sender(Presence))));

transition/2 is a function that drives the underling finite state machine behind the subscription state logic. It takes the current subscription state and a received event as inputs, and gives as output the resulting state. Again, it is not complete (it doesn’t take into account subscription requests made but not yet confirmed, etc), but is complete enough for us:

transition(none, subscribe) -> 'from';transition(to, subscribe) -> 'both';transition(from, subscribe) -> 'from';transition(both, subscribe) -> 'both';transition(none, subscribed) -> 'to';transition(to, subscribed) -> 'to';transition(from, subscribed) -> 'both';transition(both, subscribed) -> 'both'.

The rest of the process_presence/3 handles are left as exercise to the reader ;) . The lazy reader might prefer to look at the provided sources.

React to presence changes

Now that our bot is successfully subscribed to the presence of our users, let’s start receiving them! Easy enough, we must considerer two cases:

process_presence(_Session, "available" , Presence) ->    From = exmpp_jid:parse(exmpp_stanza:get_sender(Presence)),    case web_status_db:get_user(exmpp_jid:prep_bare_to_binary(From)) of       {ok, #user{state = State} = User} ->           JID = exmpp_jid:prep_to_binary(From),           Status = get_status(Presence),           StatusMsg = get_status_msg(Presence),           NewState = [#user_state{jid = JID, status = Status, status_msg = StatusMsg} |                           lists:filter(fun(#user_state{jid = J}) -> J /= JID end, State)],	   ok = web_status_db:store_user(User#user{state = NewState});	not_found ->           ok

    end;
process_presence(_Session, "unavailable" , Presence) ->    From = exmpp_jid:parse(exmpp_stanza:get_sender(Presence)),    case web_status_db:get_user(exmpp_jid:prep_bare_to_binary(From)) of	{ok, #user{state = State} = User} ->            JID = exmpp_jid:prep_to_binary(From),            NewState = lists:filter(fun(#user_state{jid = J}) -> J /= JID end, State),            ok = web_status_db:store_user(User#user{state = NewState});        not_found ->            ok    end;

We didn’t talk much about the state field. It is a list of #user_state{} records because in XMPP a user can have several simultaneous connections to the network, all with the same username but with different resource identifiers. That means that at any given time a user can have many active sessions, each one with a different presence status. We keep information for all of them, and when the presence status on one changes, we update the information for it without changing the rest.

Web page

The last missing piece is just display in the user’s page the information that we have.

user_status_response(Req, #user{bjid = BJID, conf = Conf, state = State, posts = Posts} = _User) ->    StatusOutput = case State of          [] ->             "offline";           _ ->              lists:map(fun(#user_state{jid = JID, status = Status, status_msg = Msg}) ->                    presence_to_html(Conf, JID, Status, Msg)              end, State)    end,    Req:respond({200, [{"Content-Type", "text/html"}],        [<<"<html><head><title>">>, BJID,<<"</title></head><body>">>, StatusOutput,         <<"</body></html>">>]}).

Presence information is displayed according to user preference settings:

presence_to_html(#user_conf{show_status_msg = ShowMsg, show_full_jid = ShowJID}, JID, Status, Msg) ->    ["<li>", jid_to_html(ShowJID, JID), Status , status_msg_to_html(ShowMsg, Msg)].

jid_to_html(false, _) -> "";jid_to_html(true, JID) -> ["<i>", JID, "</i> :"].

status_msg_to_html(false, _) -> "";status_msg_to_html(true, undefined) -> "";status_msg_to_html(true, Msg) -> [" - ", Msg].

Bonus track

Every one is talking about XMPP and microblogging.  As here we don’t want to be less than anyone, lets add a (fairly naive) way to post messages to our web page via our XMPP client. Nothing more than that, posting messages to your own pages. Hey, what more do you want for approximately 20 additional lines of source code?

First, add a new “posts” data field to our user{} record:

-record(user, {   bjid :: binary() ,   conf :: #user_conf{} ,   subscription_state = none :: none | from | to | both ,   posts = [] :: [{Subject :: binary() | undefined, Body :: binary() | undefined}] ,   state = [] :: [#user_state{}]}).

Then, handle <message ..> stanzas. We keep only the last ?MAX_POSTS entries:

process_received_packet(Session, #received_packet{packet_type = 'message', raw_packet = Message}) ->    process_message(Session, Message).
process_message(_Session, Message) ->    From = exmpp_jid:parse(exmpp_stanza:get_sender(Message)),	case web_status_db:get_user(exmpp_jid:prep_bare_to_binary(From)) of        {ok, #user{posts = Posts} = User} ->            Item = {exmpp_message:get_subject(Message), exmpp_message:get_body(Message) },            NewPosts = case Posts of                           _ when length(Posts) < ?MAX_POSTS ->                                Posts ++ [Item];                           [_Older | Rest] ->                                Rest ++ [Item]                        end,           web_status_db:store_user(User#user{posts = NewPosts}),           exmpp_message:set_body(exmpp_stanza:reply_without_content(Message), <<"published">>);        not_found ->             ok    end.

And finally, display them on the user’s web page:

user_status_response(Req, #user{bjid = BJID, conf = Conf, state = State, posts = Posts} = _User) ->[... same than previosly .. ]    PostsOutput = lists:map(fun post_to_html/1, Posts),   Req:respond({200, [{"Content-Type", "text/html"}],        [<<"<html><head><title>">>, BJID,<<"</title></head><body>">>, StatusOutput, PostsOutput,         <<"</body></html>">>]}).

post_to_html({undefined, Text}) ->    post_to_html({<<"no title">>, Text});post_to_html({Subject, undefined}) ->    post_to_html({Subject, <<>>});post_to_html({Subject, Text}) ->    ["<h2>", Subject, "</h2> <p>", Text, "</p>"].

Done! Users can now send messages to the bot, and the bot will publish them:

And this is how it looks on your browser:

Final comments

During this article series, we introduced the exmpp API and showed how you can use it in a real world application. In part II, we have learnt the advantages that component bots have over client bots, and developed the basis of our web status component.

In this final part we showed how to manage presence subscriptions in a component bot, one of the few things that needs to be adapted between client bots (for which the server maintains a roster) and components bots (for which the server does not maintain anything).

There are things to not take too seriously though, our component has some simplifications here and there in sake of brevity. For example, it keeps posts and user’s preferences on the same table than user presence, which may not be the smartest design. Presence information change much often, and it doesn’t even need to be persisted; if the bot crashes it will receive them again when it reconnects.


Related jobs at ProcessOne


Go Backend Developer
Paris, France

Javascript / Typescript Frontend Developer
Paris, France

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.