Yammer Benchmarking Edition 1

First in a series of SWOOP Yammer Benchmarking video blogs. SWOOP has benchmarked some 36 Yammer installations to date. This first video blog shares some insights gained on the important measures that influence collaboration performance.

SLIDE 1

Hello there, my Name is Laurence Lock Lee, and I’m the Co-Founder and Chief Scientist at SWOOP Analytics.

If you are watching this you probably know what we do, but just in case you don’t, SWOOP is a social analytics dashboard that draws its raw data from enterprise social networking tools like Yammer and provides collaboration intelligence to its users, who can be anyone in the organisation.

Our plan is to provide an ongoing series of short video blogs specifically on our Yammer benchmarking insights, as we work with the data we collect. We will aim to use this format to keep you appraised of developments as they happen. We have also recently signed a joint research agreement with the Digital Disruption Research Group at the University of Sydney in Australia. So expect to see the results of this initiative covered in future editions.

The SWOOP privacy safeguards means its pure context free analysis, no organisational names, group names, individual names…we don’t collect them.

SLIDE 2

This is the “Relationships First” benchmarking framework we designed for our benchmarking. But we also measure traditional activity measures, which we tend not to favour as a collaboration performance measure…but more about that later. The 14 measures  help us characterise the organisations we benchmark by comparing them against the maximum, minimum and average scores of those in our sample set,  which currently sits at 36 organisations and growing rapidly. They represent organisations large and small from a full cross section of industries and geographies.

SLIDE 3

For those of you who have not been exposed to the SWOOP behavioural online personas, you will find a number of articles on our blog.

Because I will be referring to them it’s useful to know the connection patterns inferred by each of them. We don’t include the ‘Observer’ persona here as they are basically non-participants.

Starting with the Responder; Responders make connections through responding to other people’s posts or replies. This can be a simple ’like’, mention or notify..…and it often is, but sometimes it can be a full written reply.

In contrast the catalyst makes connections through people replying to their posts. A good catalyst can make many connections through a good post. Responders have to work a bit harder. They mostly only get one connection per interaction.

The Engager as you can see is able to mix their giving and receiving. This is a bit of an art, but important as engagers are often the real connectors in the community or group.

And what about the broadcaster? Well if your posts don’t attract any response, then we can’t identify any connections for you.

SLIDE 4

This is how we present our benchmarking results to the participants. You can see that we have the 14 dimensions normalized such that the ‘best in class’ results are scored at 100 points and the worst performance at zero. The orange points are the score for the organisation with lines connecting their scores to the average scores.

A few points to note are that we only count ‘active users’ being those that have had at least one activity in Yammer over the period we analyze, which is the most recent 6 months.

Some of the measures have asterisks (*) , which means that the score has been reversed for comparison purposes. For example, a high score for %Observers is actually a bad result, so this is reversed for comparison purposes.

Finally, not all of the measures are independent of each other, so it is possible to see recurring patterns amongst organisations. We can therefore tell a story of their journey to date, through seeing these patterns.  For example, a poor post/reply ratio indicates to us that the network is immature and therefore we would also expect a high % observers score.

SLIDE 5

One way of understanding which of the 14 measures are most important to monitor is to look at the relative variances for each measure across the full sample set. Where we see a large relative variance, we might assume that this is an area which provides most opportunity for improvement. In our sample to date it is the two-way connections measure which leads the way. I’ll go into a bit more detail on this later on. The % Direction measure relies solely on the use of the ‘notification’ type, which we know some organisations have asked users to avoid, as it’s really just like a cc in an email. So perhaps we can ignore this one to some extent. The Post/Reply measure is, we believe, an indicator of maturity. For a new network we would expect a higher proportion of posts to replies, as community leaders look to grow activity. However, over time we would expect that the ratio would move more toward favoring replies, as participants become more comfortable with online discussions.

It’s not surprising that this measure shows up as we do have quite a mix of organisations at different maturity stages in our sample to date. The area where we have seen less variance are the behavioural personas, perhaps with the exception of the %Broadcasters. This suggests that at least at the Enterprise level, organisations are behaving similarly.

SLIDE 6

This slide is a little more complex, but it is important if you are to gain an appreciation of some of the important relationship measures that SWOOP reports on.

Following this simple example:

Mr Catalyst here makes a post in Yammer. It attracts a response from Ms Responder and Mr Engager. These responses we call interactions, or activities. By undertaking an interaction, we have also created a connection for all three participants.

Now Mr Engager’s response was a written reply, that mentions Ms Responder, because that’s the sort of guy he is. Mr Catalyst responds in kind , so now you can see that Mr Catalyst and Mr Engager have created a two way connection.

And Ms Responder responds to Mr Engager’s mention with an appreciative like, thereby creating a two-way connection Between Mr Engager and Ms Responder. Mr Engager is now placed as a broker of the relationship between Mr Catalyst and Ms Responder. Mr Catalyst could create his own two-way connection with Ms Responder, but perhaps she just responded to Mr Catalyst with a like…leaving little opportunity for a return response.

So after this little flurry of activity each individual can reflect on connections made...as Mr Engager is doing here.

So in summary, An interaction is any activity on the platform. A connection is created by an interaction and of course strengthened by more interactions with that connection. Finally, we value two-way interactions as this is reciprocity, which we know leads to trust and more productive collaboration.

SLIDE 7

Finally I want to show you how the two-way connections scores varies amongst the 36  participants to date. Typically, we would look to build the largest and most cohesive Yammer network as possible, though we accept this might not always be the case. While the data shows that the top four cohesive networks were relatively small, there are also three organisations that have quite large networks with quite respectable two-way connections scores.

So there is definitely something to be learnt here between the participants.

SLIDE 8

So in summing up, as of September we have 36 participants in our benchmark and growing rapidly now. The two-way connections measure, which is arguably the most important predictor of collaborative performance, was also the most varied amongst the participants.

By looking at the patterns between the measures we can start to see emerging patterns. We hope to explore these patterns in more detail with our research partners in the coming year.

Finally, we show that network size should not be seen as a constraint to building a more cohesive network. We have reported previously that another common measure, network activity levels are also an unreliable measure for predicting collaboration performance.

SLIDE 9

In the next video blog we will be looking at Yammer groups in more detail. We are aware that for many organisations, it’s the Yammer groups that form the heart of the network, so it makes sense to take a deeper dive into looking at them.

Thank you for your attention and look forward to seeing you next time.