Campaigner

This is my final wrap-up post on CalFree in Chains. The campaign has been live for a little over a month now, and I thought it might be interesting to look at the metrics around its performance so far. I'm sure there's a little ego involved, but my main motivation is to provide data that might help other content creators who are evaluating whether to spend time making their own campaigns on this platform.

In this post I'll frequently refer back to my post-launch analysis of The Caldecott Caper. That game was launched just a few months after Shadowrun Hong Kong came out, back when there was more marketing for the official game but before it started going on sale. There are now more players who own Hong Kong (potential subscribers), but probably fewer folks actively playing it at any given time.

First of all, the top-line numbers:

38 days is a weird figure, that just happened to be the delay before my Caldecott post, and I wanted to use the same value here for an apples-to-apples comparison. I am shocked that the number of subscribers is so close to Caldecott's; that must be very encouraging to anyone considering releasing additional content. Regardless of how many people are playing Hong Kong, it looks like there's still an appetite for playing new UGC campaigns more than two years after the official game was released. If that's the case, there's a slight chance that the potential audience could even be larger in the future, as more budget gamers pick up Hong Kong and look for more content.

I had anticipated that CFiC would receive far fewer subscribers due to the lower profile of Hong Kong, and I was mostly curious whether it would be "slightly fewer" or "far fewer". It may have helped that Caldecott was well-received; I have more people watching me on Steam now than I did when I released Caldecott, and I've heard from a few people who specifically re-installed Hong Kong because they enjoyed Caldecott and wanted to try the new campaign (quite a compliment!).

The subscriber graph for CFiC looks similar to the original one for Caldecott:

Caldecott's graph was close to flat throughout, actually accelerating a little after the first three weeks and then dipping down a little. CFiC had slightly stronger uptakes in the first 2 weeks before beginning to level off. I suspect that this may indicate that my more-assertive "marketing" actually had some impact: I'd announced the game on Reddit and Steam several weeks before launch, as opposed to Caldecott which just dematerialized overnight, so some early adopters may have been primed to check it out early. That said, it doesn't seem to have made a huge difference in overall numbers, just slightly shifted around when various people discovered it.

Oh: it may also be worth noting that the numbers are fairly consistent across the two releases despite the change in "competition". Caldecott was released around the same time as the first wave of mods were arriving for Hong Kong, mostly balance and unofficial bugfix mods, and it only spent about a week in the featured mod slot (visible within the Steam client) before being shuffled off. The workshop is a bit quieter now, although with a couple of original campaigns releasing around the same time as mine. I think CFiC may have been in that featured slot for a bit longer, which I had assumed was the primary driver of traffic, but I don't have any metrics to support that.

And, finally, the deltas:

No huge surprises here. The biggest interest came early on, and has bounced around since then at a slightly lower level; from a quick eyeball, I'm gaining roughly 35 new subscribers a day, down from the roughly 80 daily over the first 2 weeks. As a reminder, the numbers at the end of these graphs always under-count the actual values.

And, just for fun, let's check in on how the earlier campaigns are doing!

It looks like The Caldecott Caper has picked up about 15,000 new subscribers in the approximately two years since we checked in last - not too bad! That's roughly eightfold growth since I stopped thinking about it. I'm really curious how this compares to commercial games; I tend to think that those titles, especially AAA entries, sell the most in the first few months, but Caldecott has chugged along slowly and steadily.

Speaking of which, give me those deltas:

As I'd predicted before, there was a big spike when the Hong Kong mini-campaign was released, which turned out to be by far my biggest one-day gain, as well as driving more subscriptions in the following weeks (presumably as returning players wrapped up that campaign and went looking for more content). There are a couple of other spikes in there that are mysteries to me: Jan 17 2017 and May 7 2017 also saw big surges in views and subscribers, but I can't think off hand what could have caused those; I wonder if I may have overlooked some Let's Plays or media snippets or something. Oh! Actually, now that I think about it, it's more likely that these were times when Hong Kong went on sale. I think I've seen upticks in comments on my campaigns in the weeks after a major discount, and it makes sense that those would correspond with those subscriber counts.

It doesn't look like my interminable blogging drove any traffic to Caldecott, which is fine and understandable: that wasn't a motivation in doing this. It does seem like the release of CFiC incentivized more people to download Caldecott, which is cool and interesting: far more people played Caldecott and then followed that with CFiC (which makes sense for all sorts of reasons), but it looks like at least a few people are going in the other direction, or hearing about CFiC and deciding to try Caldecott for the first time.

Caldecott has a long tail, but it's definitely tapering down. It looks like it's currently gaining around 25-30 subscribers per day; in the quiet time before CFiC was released, it was averaging around 15-20. If that continues around the lower end of the range, I might acquire around 7,000 additional subscribers per year going forward.

Finally, let's look back one last time at Antumbra Saga, where this all began.

Those are big numbers, and I don't think the later games will ever overtake it. Even though you could argue that The Caldecott Caper is objectively better (I'll note its 99% favorable rating, as opposed to Antumbra Saga's dismal 96%), I think that the larger Dragonfall install base and the earlier enthusiasm for mods will ensure it continues to rule the numbers. And it isn't just due to early spiking, either. Here's the lifetime graph:

As a reminder, Antumbra Saga is actually just a collection of existing mods for Shadowrun Returns, with some updates to run properly in the Dragonfall engine. Because I only had to update existing stuff, I released it pretty soon after Dragonfall Director's Cut came out, and caught a wave of people playing the official game. Even since then, though, the week-over-week growth on Antumbra is almost always bigger than on Caldecott. Here's a zoomed-in view of the past month:

Here, my average is around 30 new subs per day, which matches the highs of Caldecott.

So, what's the take-away? If your primary goal is to reach as many players as possible, you're probably better off modding Dragonfall instead of Hong Kong. It has more players, it has a lot of Workshop interest, and even older mods are doing better on there. (That said, it doesn't seem to be a huge difference, so you won't penalize yourself too badly if you'd prefer to take advantage of the superior Hong Kong engine and assets.)

And now, let's take a quick visit to the parallel universe of Nexus Mods:

The numbers for CFiC seem a bit better than the launch of Caldecott, but it doesn't really matter: I'm maybe averaging about 5 downloads a day for CFiC, versus around 4 a day at the launch of Caldecott. The values are so low that I hesitate to draw any conclusions from them, it's all just statistical noise.

The numbers get a bit more meaningful over longer periods of time; I can't yet analyze CFiC, but Caldecott now has enough data to be potentially useful. Here's a graph showing monthly downloads of Caldecott since it was released.

It's surprisingly consistent over time. As with Steam, the first peak came after the release of the official expansion Shadows of Hong Kong. I think the second peak may have come after I removed the "Not Safe for Work" tag; I'd initially set it based on a strict interpretation of the Nexus TOS, but after further reflection realized that it didn't really apply to the mod. Removing the tag allows it to appear in more places in the Nexus. Since then, I've been averaging roughly 3 downloads a day. Nothing huge, but Nexus users tend to be polite and supportive, so I haven't minded the effort of maintaining parallel versions.

And here are top-level stats for Caldecott:

Ever since my very first mods back in 2013, I've been seeing about 10x as much engagement in Steam as on Nexus, and that continues to be true in 2018. However, Nexus players are about twice as likely to endorse my campaign as Steam players are to rate it.

So! This will probably be my last-ever blog post on my Shadowrun stuff. It's been a really fun run! I've deeply enjoyed experiencing every stage of these mammoth projects, from pie-in-the-sky brainstorming to nitpicky bugfixing and data analysis. Seeing numbers like this is very encouraging: it's hard to evaluate "real" users that partially overlap across three (or six!) titles on two different portals, but my best guess is that somewhere in the neighborhood of 50,000 people have at least downloaded at least one of my campaigns. That isn't bad! If and when I move on to doing something else, it will be very encouraging to know that there's a group of folks out there who have enjoyed my work in the past, and at least a portion of them might follow me to future projects.