The One Community Metric to Rule Them All

[00:00:04] Announcer: You’re listening to Community Signal, the podcast for online community professionals. Sponsored by Open Social: community building for nonprofits. Tweet with @communitysignal as you listen. Here’s your host: Patrick O’Keefe.

[00:00:24] Patrick O’Keefe: Hello and thank you for listening to Community Signal. Bas van Leeuwen is the co-founder of an analytics tool and dashboard for online communities and he’s our guest on today’s show, where we’re talking about, you guessed it, analytics. If our program helps you as a community pro, please consider supporting it on Patreon at communitysignal.com/innercircle. Carol Benovic-Bradley, Luke Zimmer and Joseph Ranallo are a few of the community pros who have already done so.

Bas van Leeuwen has been working with and managing communities for more than six years. For almost as long, he has been frustrated with a lack of tools to see and track what’s actually happening within online communities. He ran countless manual reports, organized dozens of spreadsheets and still never felt like he could fully answer, “What’s the ROI of community activity X?” When Bas and Alle Veenstra started working together, they wanted to make tools to make community managers’ jobs easier. Bas knew what community managers needed to be effective, while Alle, a forensic data scientist, brings state-of-the-art data science methods, hence, the seed for community analytics was born.

Essentially, the duo built the analytics tool that Bas wished he had from the get-go. This tool takes the guesswork out of the analytics process. It frees up hours of your community manager’s time on manual data collecting, so they can focus on building and executing the community strategy.

Community Analytics is loved and used by well-known companies including Amsterdam Schiphol Airport, Immediate Media Co, Udemy and Nethserver. Bas, welcome to the program.

[00:01:42] Bas van Leeuwen: Hi, Patrick. Thank you for having me.

[00:01:45] Patrick O’Keefe: It is a pleasure to have you. Well, we’ve been talking fairly regularly [laughs], recently, not just about the podcast but probably more commonly about Community Analytics and the work you’re doing, as it’s something that when I took my new job at CommunityCo and I needed to find an analytics platform that would work with Facebook Groups, thought of you immediately and we’re using you [laughs] and because of that I’m throwing ideas your way fairly regularly now.

[00:02:11] Bas van Leeuwen: Yes, they’re very welcome. We’re doing well.

[00:02:14] Patrick O’Keefe: Awesome. I definitely want to talk about analytics, but before we get into that, you’re in the Netherlands. When I talk to people from outside of the US, I like to learn about what the community space means in their country. What’s the state of community management in the Netherlands?

[00:02:32] Bas van Leeuwen: It’s really all over the place if I’m being honest. There’s quite a few properly advanced community managers here in the Netherlands and then we have all the way to very advanced social media managers, all the way to people just getting started to interns managing a forum or Twitter or whatever. It’s all over the place. In general, I’d say that we’re doing quite okay here in the Netherlands. We have regular community meet-ups and all that, so pretty good.

[00:02:59] Patrick O’Keefe: When I talked to– I think it was Venessa Paech, who’s in Australia, her and Alison Michalk do the Swarm Conference in Australia and she was talking about how in Australia they have a lot of social media managers and a lot of the times, they say that’s community or they think that’s community. I think that one is something that occurs pretty regularly, around the world. Is that also true in the Netherlands?

[00:03:20] Bas van Leeuwen: Yes, of course. It is the logical mistake we make. In the end, to us, that’s a wide difference between community management, the way we know it, and social media management. To a layman, like you’re the person that interacts with all the people. They’re not wrong. I’m always glad if they knew community managers exist even if [laughs]– I’m happy if I don’t have to explain what a community manager does even if I do have to correct them. In general, it’s the logical mistake to make. I wouldn’t be annoyed or anything, it’s just as I said, logical.

[00:04:00] Patrick O’Keefe: There is an association of Dutch community managers, right?

[00:04:05] Bas van Leeuwen: That’s right. We have the CMNL, the Dutch community managers’ association, which was founded by Kirsten Wagenaar, who is really well-known here in the Netherlands, who recently started a proper community management agency, Bind. She basically handed the association over to a couple of people, which I was proud to be a member of. I was the secretary of the whole association, handling over legal bits. I left after two years, early this year and the association is still continuing, holding regular meet-ups, having their own little community for community managers here in the Netherlands, which always gets a chuckle.

[00:04:41] Patrick O’Keefe: About how big is the group?

[00:04:43] Bas van Leeuwen: There are about 30 paying members. I think about 200 people registered on the forum with another thousand or so on the LinkedIn group. That’s pretty big for a tiny country such as we are.

[00:04:57] Patrick O’Keefe: Yes, that’s a good size group. I always love to duck in and out. I can’t always because, unfortunately, I don’t speak more than one language, which is not a great thing. I wish I spoke more languages, but I like to duck into different groups in different countries, whether it’s a Dutch group or a German group. Just use the translation feature on Facebook, or use Google translate and just get a sense once in a while, of what they’re talking about.

Obviously, it’s the same stuff we’re all talking about, but I like to duck in there and see what people are saying. Even once in a while, I forget which group it was but I participated. It might have been a Polish group, but I participated in my English way because they allowed me to.

[laughter]

If it was only for their language because, that’s the guidelines, and I want to respect the guidelines if it’s only for a particular language.

I always love meeting community pros from other countries, so I appreciate the background. Before the show, you told me that at Backbase, where you are a community manager, you are in the process of, “trying to transform an inside first business into a business that takes sharing information seriously.” What’s an inside first business?

[00:06:03] Bas van Leeuwen: Basically Backbase has always been a company that– It’s a traditional software company. They develop their own software, then they bring it to the market, they sell it, they refine it. In the end, the interaction with the developers that have to use the software is always done near the end of that cycle. As a community manager, I’m trying to get that further and further and further earlier in that cycle, so that we can actually learn from our users, which is something that we tend to– Like just humans in general, we tend to forget this.

There’s always our natural instinct to go away into our little caves and tinker with something and then at the end, jump out of the cave and say, “Tada!” Then people go, it’s nice but not exactly what we want. I think community management has the absolute potential everywhere really to transform that way of doing business into more of a cooperation, the way you and I work for our Community Analytics. You bring ideas, we look at the ideas, we discard some of them, we help you implement others. That’s the way I see business happening more and more and more, compared to the more traditional way. For me, I have an open source software background, to come into a more traditional software business. It’s an interesting dynamic.

[00:07:23] Patrick O’Keefe: Yes, I was going to ask about that because its almost like, obviously you have a sense of what you’re going into, but you’ve only been there for five months. The idea of community or open source development, or involving the community of users and customers with software development is not a new one to you, but to go to this company where apparently that’s just not the case at all. I imagine it’s somewhat of a, I guess, a culture shock or just really different.

[00:07:47] Bas van Leeuwen: It’s a combination. The fun thing about Backbase is that its so rapidly growing, that all ideas are there. It’s not a monolithical idea of a company having a certain way of working. That said, the whole open source togetherness vibe to bring that to sales is always a really interesting thing. That’s even true in open source companies like sales. Generally, accepts the fact that they were open source, but they don’t necessarily like it or enjoy it, but that’s always a struggle. It’s an interesting struggle.

[00:08:20] Patrick O’Keefe: It’s the struggle that open source, obviously, here is the code [laughs]. You’re paying for support or whatever the model is, here is the code. Is that the main thing they don’t like, that they don’t have a secret private thing that they can sell as a proprietary secret?

[00:08:35] Bas van Leeuwen: That is the easiest thing to sell. You sell knowledge, which is really easy and usually really easy in the short term. Whereas in the long term, a lot of the more famous companies, they live because they made themselves the platform. They don’t care that other people can also use it and the forum that nearly every community manager here loves, Discourse, is actually open source.

If you want to pay them you have to let them host it. If you don’t that’s fine. They don’t mind, they don’t really care for you, they don’t accept donations whatsoever. Their business is making the best forum in the world, making sure that a lot of people use it. The few people that need hosting, that’s their business model. It works, they’re growing every year.

[00:09:22] Patrick O’Keefe: With this inside first business, with a business like that, it can be scary to share something in a community context because as you said before the show, what if its proven untrue, or it sets the wrong expectations for a customer? “Whereas the overall feeling of community is the way that people work together. If they trust each other, that feeling is nearly impossible to describe to someone who hasn’t experienced it.”

People who haven’t experienced sharing because of mutual trust, that’s tough. How do you get around that?

[00:09:52] Bas van Leeuwen: Baby steps.

[00:09:53] Patrick O’Keefe: [laughs]

[00:09:54] Bas van Leeuwen: The thing is you have to start serving an example. The way to do that in a company like this is to just get one or two advocates, preferably people that have a good reputation, and have them share stuff, and have them ask questions. Then people see it like, “Hey, asking questions is allowed, asking questions is appreciated. Hey, maybe I can help someone.” Then you get this little snowball going. Then you go to your next advocate, and you repeat your process, and you repeat your process until that culture has changed. Ironically that’s not something you can measure.

That’s something that you feel, you have to experience, you have to do, you have to push, you have to prod, you have to test, you have to make a lot of mistakes. Then, it either works or it doesn’t. I’m still hopeful that it will work.

[00:10:40] Patrick O’Keefe: I like that you have to engender the emotion before you can actually get to the cold hard heartless [laughs] data. People actually have to believe in it and feel it and enjoy it so that you can actually get to a point where yes, then you have something you can measure.

[00:10:52] Bas van Leeuwen: Of course, and measuring is vital, but it’s not the end goal. In the end, you always want to know if the thing you’re doing makes sense and measuring is one way, your gut feeling is another way, and they complement each other. You shouldn’t focus on one or the other. Without metrics, you tend to follow your own ideas.

They might not actually be true. It’s always good to have a objective measure of your own ideas and your own mental model and see if reality agrees with you. Whatever the metrics tell you, it’s probably a good direction if you want to keep on doing the same thing.

If you want to change, you probably need to listen to your gut a bit more than to the metrics. Also, when you’re just started in your community, you probably need to listen a bit more to your gut than to the metrics. Then if you have a community that’s two or three or four years old, and you just have reached a bit of an equilibrium or a continuously rising state of new members or something like that, then the metrics tell you whether you’re continuing on the same path. That’s the path you want to stay on.

[00:12:03] Patrick O’Keefe: I like that because to me it says that metrics are important, but if you’re a slave to the metrics then you’ll just– Eventually those metrics will go down [laughs]. You’ll just be watching them and watching them until they eventually go down. You need some level of creativity to actually come up with new ideas and to go back to your gut to have new ideas and do things to try and then use the metrics to decide what works and what doesn’t.

If you aren’t actually applying that sort of creative component to it, the art of it, so to speak, then you’re just kind of sticking to the same things. For the most part, communities go in waves and eventually, the same thing will trend downward, for the most part.

[00:12:39] Bas van Leeuwen: Yes, spot on. Also, this is not specific to community management. This is something they see in retail and whatsoever. You can see this– Like they saw 10 years ago, the trend of, “It’s going down guys. What do we need it to do?” Some people acted, creatively, some people failed to act and just reported quarterly results going down, down, down. The guys that acted on time, they’re now really big, and the guys that didn’t, they’re out of business. That’s not specific to community management.

[00:13:11] Patrick O’Keefe: No, you’re right, it’s not. It’s not at all. Let’s talk about measurement. Community Analytics, that’s community-analytics.com, your analytics tool, is platform agnostic. You can hook into hosted platforms. The reason that we work together is because I was looking for a platform to hook into Facebook Groups. That’s what we use it for at CommunityCo.

How frustrating are Facebook’s limitations?

[laughter]

Because that’s what a lot of our conversations seem to run into is, man, Facebook? The limitations of Facebook are frustrating. If you’re on your own platform– If I was on my own platform like Discourse or XenForo or anything really that I hosted myself, the limitations wouldn’t be there. Having to work on Facebook, where we’re dealing with their terms of use and how they allow you to use their API, and it just seems like it’s a constant battle of trying to identify new things but also, certainly respecting Facebook’s TOS.

[00:14:01] Bas van Leeuwen: Yes, I know it is. Ironically, in truly outside worlds, when you are trying to integrate with Facebook, they are really hawkish on privacy. That’s because they have a legal obligation to law to protect their user privacy. Even though they are the bigger data collector in the world, they still protect the privacy of their users for other people. The concept of privacy for them is really, really stringent. Basically, you’re almost not allowed to retain any of the data you get from the platform.

Whereas for example with Discourse, it’s your data. You can do with the data whatever you want. It is on you to comply with the law. Whereas for Facebook, they have to think of– In terms of service, that works everywhere in the world, whether it is in the US or in Tehran or in Russia or in Europe or in the islands or wherever. That leads to some very stringent rules that we have to abide by, which is frustrating but understandable. Sometimes it hinders us into providing you with what you want.

[00:15:03] Patrick O’Keefe: I’m not behind the scenes saying, “Bas, I want to get everyone’s social security number and I want to get it dumped into this Excel file so that I can just, unencrypted, have it up online there. I want all their home addresses and all the phone numbers and forget opting in.”

[00:15:18] Bas van Leeuwen: No, it’s things like, “Give me a list of the people that left my group last week.” Facebook just says, “No, once they left, you’re not allowed to know anything about them anymore.” It’s like they were never there.

[00:15:30] Patrick O’Keefe: Which you can understand. When you explained it that way, I was like, “Okay, I can understand that. I’d still like the round number of just saying 10 people left.” I don’t necessarily need to know all their names, but when you explain it that way I’m like, “Yes, I can see that. They’re taking their data with them, they can leave.” I understand that.

[00:15:47] Bas van Leeuwen: That’s also like something that in Europe at least, but probably not the entirety of the world. We’re getting the GDPR, the new regulations around privacy, the right to be forgotten. That is going to be interesting for community managers as well. At the moments notice, people will have the right to basically force you to delete their data and to force you to give them a list of everything you know about them. I think a lot of community managers aren’t prepared for that eventuality. I know I am not, but we’re slowly getting there I think.

[00:16:18] Patrick O’Keefe: We should probably do a show on that.

[laughter]

That’s a whole different thing like how you think about that. I haven’t even spent any time thinking about that partially being from the US and probably being blissfully ignorant honestly, but as you say there one thing, in community, that’s a pretty normal standard in online communities is to say that, “Okay, we get rid of your account, we can get rid of identifying characteristics, your name, avatar, your name, whatever.”

If you talked about this hockey team within this conversation where there was one hundred posts in this thread and everyone’s replying to each other, then if we delete your five posts out of that hundred post thread, then 50 other posts in that thread now are harmed and lack context, so we won’t delete the post, but we will delete the member. It sounds like this might [laughs] change that especially for certain communities operating in Europe.

[00:17:07] Bas van Leeuwen: Well yes, it’s going to affect everyone globally, basically. It is something that the EU says that all Europeans citizens or even all people currently residing in Europe have no matter where the forum is located. This is going to be either a very big thing or a very small thing because no one is going to care. The EU cookie consent law, we were all really scared and in the end, there’s a popup and that’s about it.

[00:17:33] Patrick O’Keefe: [laughs]

[00:17:35] Bas van Leeuwen: No one really knows how stringent they’re going to be. Personally, this is not legal advice, I would take a bit of a wait and see approach and just, “What’s your procedure for handling GDPR thing?” Well, they send an email and we’ll handle it when it comes.

Chances are that a hundred are going to send you an e-mail and then you’re in pain [laughs].

[00:17:50] Patrick O’Keefe: Wait and see mode right now [laughs]. Wait and see because it can either be really bad or as you said, it could be not that big a deal.

Let’s pause for a moment here and talk about our excellent sponsor, Open Social.

Open Social is the community platform for nonprofits looking to coordinate volunteers and bring together stakeholders at a grassroots level. Hosting over 1,000 active communities, with Greenpeace Greenwire and several United Nations projects leading the way, Open Social has been proven to increase engagement within your community by up to 600%! Open Social is a Drupal-based open source solution, offering two fully maintained and hosted subscriptions starting at just $110 a month as well as a free do-it-yourself version. Request a free trial today at getopensocial.com.

When I asked you about metrics that you’d like to discuss on the show, you mentioned a couple. The first being active members, which is a commonplace metric, one that a lot of people have been focusing on for a long time. What made you want to bring it up?

[00:18:51] Bas van Leeuwen: When we look at dozens of forums or even hundreds and we look at the one measurements metric that is basically useful for all of them, the number of people that have participated in that community in the past month, or in the past 28 days, is by far the most reliable indicator of future growth. That presumes a few things namely that more members equals a better community or that on average, the number of things they do is equal to what they did a year ago.

It’s a very rough metric, but if you would send me into the desert with one metric to rule them all, that would be the one. How many members do you have and do they keep coming back, are they active, do you manage to draw on more people? If that number goes down, there’s something wrong. The problem is it’s a bit of a lagging metric so it’s probably that something went wrong a couple months ago, but still, it’s a very, very strong indicator that something is amiss or if it keeps rising, it’s a very strong indicator that something is going well.

If it fluctuates a lot then I don’t know what’s going on, but you should have a look [laughs].

[00:20:05] Patrick O’Keefe: It’s interesting because there’s a lot of different metrics that tie into community health, but active members is consistently the most valuable. I think that whenever you’re looking to build a dashboard out for a community or to use a tool like CA, active members are the first metric you should put on that dashboard.

[00:20:23] Bas van Leeuwen: Yes. Definitely. It’s the one thing that everyone understands, it’s the one thing that works for nearly everyone. It’s a really good thing to have on your daily screen.

[00:20:34] Patrick O’Keefe: Someone’s coming to a company that, maybe they have a community, maybe they don’t, but they probably do, and they’ve never measured any really meaningful data, like they don’t have analytics. They have the numbers. They have some numbers. Let’s say their using CA, they have the numbers that are in a community analytics platform like that. If they’re just getting started and they need– I don’t know, some quick and deadly community wins, they want to show something.

[00:20:57] Bas van Leeuwen: Right.

[00:20:57] Patrick O’Keefe: We all know community is not a short-term game. People come to me and they say, “How long should I wait for community?” I say, “You’re asking the wrong the question [laughs]. Do you have a year because if you’ve only got a few months, go send some tweets.” That’s what I like to tell people because community is not a short-term game, but-

[00:21:12] Bas van Leeuwen: Yes. Go buy some ads or whatever.

[00:21:13] Patrick O’Keefe: Exactly. We don’t know everyone’s employer. To your point earlier, the people who are sometimes hiring the people or who might control the budget in a lot of cases, they’re not necessarily that community savvy and not always willing to be informed. The community person’s coming in, they need something to show in the next few months. Where do you think they should start? It probably depends on the business and how community gets into the bottom line, but where would you point people towards to get a couple of community wins within a few months? What should they be watching?

[00:21:42] Bas van Leeuwen: Ask your boss.

[00:21:43] Patrick O’Keefe: [laughs]

[00:21:45] Bas van Leeuwen: Seriously, he’s reporting in something and that’s a number. If he only cares about that number, make that number bigger, whatever the number is. It’s as easy as that. If you want to keep your job, if you only have two months to show that you can move things around, ask him what number he wants bigger. It’s as dumb as that.

[00:22:05] Patrick O’Keefe: That’s a good point. Figure out what the number is and then figure out how community supports it, and figure out as quickly as you can. There’s a lot of different ways that– I’m just thinking about the numbers. Obviously, there is revenue. They might have a revenue figure or they might have a number of subscribers in mind or licenses.

[00:22:23] Bas van Leeuwen: Or a number of posts. There’s so many different things that they could be reporting on.

[00:22:28] Patrick O’Keefe: Yes. If your boss cares about number of posts, you might have it easier than some others. But- [laughs]

[00:22:32] Bas van Leeuwen: Yes. That’s trivial. You can make them yourself probably, or you can hold a competition; who can write the most posts and you give it a $50 voucher. It’s not good for your community but-[laughs]

[00:22:43] Patrick O’Keefe: Yes, it’s true. It’s usually revenue but sometimes it’s not, you’ll never know. To your point, you have to ask [laughs]. You should ask what people want to know, not assume because maybe for some reason they just want more Google search results. I’ve heard that one. Maybe they care about time on the website, and you can certainly increase that, whatever the number is?

[00:23:02] Bas van Leeuwen: True. Whatever the number is, if it’s only one number, it’s so easy to increase that. You have 40 hours a week to fix that one number. You’re going to do it, no matter what.

[00:23:14] Patrick O’Keefe: I like the spirit. I like the optimism.

[00:23:16] Bas van Leeuwen: Then I will strongly advise, during the meantime, actually to take a look at what you want to achieve with that community and also start, while your boss is happy, with the big number, right?

[00:23:26] Patrick O’Keefe: Right. You’re satisfying this number that is most important to the boss. Whether or not it should be the most important is beyond the point. You have to make them happy to keep the work going. Then, in your free time, whatever that extra hour you can spare away when you’re not making that number bigger, you can be building something that, in the future, that you can point to and say, “Well, look at this other thing community did. Isn’t that amazing?”

[00:23:46] Bas van Leeuwen: Yes. “By the way, I also made revenue happen.” Or, “Hey, by the way, I also increased customer’s satisfaction.” Or, “Hey, by the way,” whatever. You’re hired for a job, that job usually in reports, is dumbed down into one or two numbers. This is basically true for nearly everyone. Focus on those two numbers first, but don’t forget that there’s also an actual job to do.

[00:24:08] Patrick O’Keefe: The other metric that you mentioned when I asked you what you’d like to discuss was, cohort analytics. Explain cohort analytics.

[00:24:15] Bas van Leeuwen: Cohort analytics. It’s a bit of a difficult concept to grasp the first time you hear about it, but in the end, it’s really easy. Basically, when we look at analytics, we usually dump everyone into one big heap and see what happens, right? That’s exactly what it is. With the previous metric where it’s like, how many active members always we starts by grouping people into, for example, every month, how many blah in January did I have? Or how many people that started in January did XYZ? Whereas with cohort analytics, you would actually go, “Okay. Let me compare the people that started four weeks ago with the people that started eight weeks ago.” Then you compare their numbers in their respective first week and you compare them to each other. That way, you can actually see whether the things you’re doing now are influencing people compared to the previous period. The way I’m explaining it now, I think, even I’m lost in the forest again. Maybe we should start over again.

[00:25:16] Patrick O’Keefe: [laughs] No, I think I get the concept of it. I’ve looked at them myself. The idea that it’s comparing people at similar life cycles. Like, this person that was a new member here, that this other person is a new member now, but that person did this in their first week or their first or their first three months; let’s compare them to this other person’s first three months and see if we’re making progress as far as how newer members are doing, or how older members doing.

[00:25:42] Bas van Leeuwen: Exactly.

[00:25:42]  Patrick O’Keefe: Here’s a group of our one-year-old members. This is the average contribution for that member group who has joined a year ago. A year from now, we can compare that other group to this new group and see if we’re actually making any headway with our experienced members or are they falling off. Are we losing the contributions from experienced members?

[00:26:01] Bas van Leeuwen: Yes, exactly. It’s comparing apples to apples in that, let’s say that you want to increase the freshness of the apple or the color of the apple or whatever. You would pick an apple, you would make a photograph and then this year, you will pick another apple, you must compare it to the photograph. You wouldn’t compare it to that apple because that apple now is a year old.

[00:26:19]  Patrick O’Keefe: [laughs] You don’t want to eat that one anymore.

[00:26:22] Bas van Leeuwen: No, you don’t. That way, you shorten this whole feedback loop that you have, where you try an initiative and you see if it works better. You want to see, can I make new members? Can I entitle a larger percentage of the new members to make their second post? Instead of just lumping it all together and seeing how many people make their second post, you actually look into percentage like, “Hey, of the people starting in January, 9% made a second post within a month, and of the people in March, that number climbed to 11%. We’re doing really well there. We’re really seeing improvement there.”

This way of looking at numbers really allows you to start to focus on quickly measuring like almost in this lean start-up way where you have an initiative, you try it for two weeks, you see if it has effects. Maybe a little, maybe a lot. If it’s a little, maybe try it a bit longer, if it’s a lot, let’s go for it and if it’s none, well then, probably you should stop doing that initiative and try a new idea.

[00:27:20]  Patrick O’Keefe: Yes. I mean, I think a simple way to look at Cohort Analytics, in this case, is just– We don’t need any more euphemisms, but it’s almost like a college class. Like the class of ’17 versus the class of ’18 versus the class of ’19 and what they did after they left school [laughs] in those first months, six months. Do they have a job yet? What’s their average salary? If you’re a college, you want to increase the employability rate of those grads. So that, let’s say in 2017, 50% of people found a job. That’s probably really optimistic, but let’s say 50% did [laughs]. Then next year, if you get to 60%, “Oh, 10% more of our grads got jobs this time.”

[00:27:57] Bas van Leeuwen: Exactly.

[00:27:57] Patrick O’Keefe: We’re doing something better.

[00:27:58] Bas van Leeuwen: Yes. Hopefully, you know what that something is.

[00:28:01]  Patrick O’Keefe: Hopefully you know what you changed, I guess is the answer. You know what you did different.

[00:28:05] Bas van Leeuwen: Exactly. That sometimes it’s just noise, or it’s an outside influence or your company just became more famous so people are coming into your community more. It’s always difficult. In the end, using the cohorts really sharpens your vision. It really helps you.

[00:28:21]  Patrick O’Keefe: When I hear that, I hear a boss saying, “Don’t rush to take too much credit.” [laughs] It could be an outside factor. It might not be related to you. You could have got some good press or maybe something happened that drove it upward and it’s not necessarily what you did in the community.

[00:28:37] Bas van Leeuwen: Yes. Maybe I’m being too humble. Just take the credit and go for it.

[00:28:40]  Patrick O’Keefe: No.

[00:28:41] Bas van Leeuwen: [laughs]

[00:28:41]  Patrick O’Keefe: I think you have to be honest about these things. When you look at cohort analytics, is it mostly a short-term thing? Like I mentioned a year out, but just tp kind of throw it out there. But is it mostly a short term, sort of measurement? Is most of the benefit that you’ve seen in it in comparing groups within their first relatively short period of time?

[00:28:57] Bas van Leeuwen: It depends. The specific use case I had in mind, yes, because you really use it to, for example, try a new initiative with new members. Then you want the period to be as short as possible, but still be relevant and significant. There’s just a few natural windows to look in, like their first week, their first month, but then, comparing year over year is always very useful because there’s a lot of seasonal influences in community. Depending on the community.

For example, one of our clients is a gardening community. They just wilt and die in winter. They go from 10,000 to 500 active members every year. If you only look at the members again in a monthly window. you’re going to stressed out for six months a year.

[00:29:40]  Patrick O’Keefe: [laughs] It’s just like the garden, 90% of it just dies [laughs].

[00:29:46] Bas van Leeuwen: Yes, but-

[00:29:47]  Patrick O’Keefe: That’s tough.

[00:29:47] Bas van Leeuwen: -we really see that happening and in their statistics. It’s interesting because they’ve existed for like, more than 10 years I think. Just every year, they die and they die and they die again, and then, they rise again. You really see– Especially in that community, you see it on January 1st, that people go like, “Okay, new year, new ideas, new garden. Let’s think–” The traffic starts again on January 1st, not on January 7th. The inflection point is always January 1st. It’s fantastic to see.

[00:30:15] Patrick O’Keefe: It’s a great point because communities are seasonal and some communities are very seasonal. You see that on a lot of difference basis like sports, for example. When the season starts versus when it’s the dead, off for the off-season. There’s an optimism at the start of the season. The same thing happens with exercise and fitness communities after the New Year. People are making their resolutions, they’re going to get back in shape. January 1st is often, again, the inflection point there too for people who are ready to get the information and get talking about fitness.

[00:30:44] Bas van Leeuwen: True, that’s the thing. You have to know your own community. You have to understand why things are happening when you see them happening. Metrics also help you to understand your community. If you see a seasonality like that, you can go, “Hey, why is that happening?” Then you’d go, “Right.” It doesn’t make sense to advertise in fall so you just halve your advertising budget or you double the number of advertisements you can place in spring.

[00:31:10]  Patrick O’Keefe: Looking forward, is there a particular metric that you are hoping to incorporate into CA? Is there a metric that excites you? Something you discovered recently, something on your mind. If there is, what is that metric?

[00:31:24] Bas van Leeuwen: Recently, we did some analysis of 200 communities together with another party that will write about it soon. We want to basically see if this is enough to get some machine learning going and maybe see if we can start automatically ranking communities and automatically finding the little points that should matter in the noise. I’m really excited to see if we can pull that one off. I’m not sure, but that would be really really cool. Automatically sending alerts saying, “Hey, there is this many people now active and it should be more or it should be less.” Or, “Hey, people in your industry are seeing an uptake. Why aren’t you seeing it? Or the other way around.”

That would be really cool, to really start working together with a community manager that uses our software and helping them, saying, “Hey, we’re seeing some spikes. Even though there is a bit of noise, we’re seeing the spikes. Did you do something special?” Then, they can go, “Yes, we started an advertising campaign.” Or, “No, nothing special.” Then they could dig in and it turns out that one of their members invited two other friends and they have their own group of friends and then everyone came in. That would be really really interesting to me. Apart from that, I haven’t found any global metric that really excites me at the moment. I’m sure that there are [laughs].

[00:32:40]  Patrick O’Keefe: Bas, thank you so much for taking some time with us today. I’ve really enjoyed talking with you.

[00:32:44] Bas van Leeuwen: Thank you so much for having me.

[00:32:46]  Patrick O’Keefe: We’ve been talking with Bas van Leeuwen, co-founder of Community Analytics at community-analytics.com. For the transcript from this episode plus highlights and links that we mentioned, please visit communitysignal.com. Community Signal is produced by Karn Broad. See you next week.