That's A Wrap

Part of a series on The Caldecott Caper.

Okay! This is my final wrap-up post on this crazy huge mod I started working on almost exactly one year ago. I was hugely inspired by Failbetter Games' shockingly transparent post-mortem, which went into enormous detail about the sales figures for Sunless Sea and how they compared to the company's own internal projections. Since this is a free mod, I don't have any similar financial data to share, but I thought it might be interesting to share the data I do have regarding downloads, subscribers, and so on.

First, timeline: the mod was initially published on Steam on November 29th, after the Thanksgiving holiday. As with previous mods of mine, I actively monitored the early adopter feedback and released quick fixes for early-reported bugs, initially averaging 1-2 updates per day. After the first week this slowed down, and I now average 1 week between updates. No new features have been released, and the latest updates focus more on quality-of-life issues and not so much on crashes or other significant problems.

Within Steam, I get access to a few nifty author tools, the most useful of which are Item Stats. Here are the stats as of January 6th, 38 days after the initial release.

There seems to be roughly a 50% conversion rate from visitors to subscribers. I think that's pretty good, although if the campaign cost money I'm sure the rate would be far lower. I don't have access to the analytics to know how many subscribers actually ever launch the campaign, or how many people who start the campaign finish it, but a little over 5% of subscribers will take the time to come back to this page later to vote on it. Again, that seems pretty good to me... I'm fairly certain that many more people complete the game than vote on it, partly because there isn't a good flow from within the game / Steam to this page, and also because people have better things to do with their time.

From a quick eyeball of the comments pages, I think roughly 100 individual players have provided some form of written feedback (bug report, review, or "attaboy!"). There seems to be close to a 1:1 correlation between providing written feedback and clicking "Like", which seems a little silly... but again, if someone is taking the effort to come back to this page, they may be sufficiently motivated to take the time to write a personal comment. (Which, I should add, are greatly appreciated!)

On a negative feedback perspective, I didn't get my first troll until about 3 weeks after the mod was released, which is by far my best track record so far. I think I've gotten better at dealing with trolls now, which is to say that I don't. I used to try to respond to every piece of feedback, positive or negative, but realized that doing so was profoundly unproductive. I have a better sense now for when someone has valid criticisms, and when they're just trying to get a rise out of people, and have found that ignoring the latter group works well. Along the same lines, while I continue to reply to individual bug reports, I now reply to general comments in batches, which allows me to more gracefully provide different levels of reaction to different posters.

Back to the data: my subscriber graph is shockingly consistent.

My previous mods were all quite a bit more jumpy. I think I'm now seeing a steady trend because we're in a fairly slow time for Shadowrun. I expect there will be a spike around the time that the Hong Kong mini-campaign is released, and maybe a smaller spike when the Commentary update goes out. Only super-fans are regularly checking the Workshop for new content, and I suspect that I'm mostly seeing a trickle of people who are looking for new content after beating Hong Kong or Dragonfall for the first time.

The deltas, of course, look more dynamic than the overall trend graph, though if you look at the Y axis it's still pretty consistently hovering around 60 new subscribers per day.

That initial spike at the beginning was when The Caldecott Caper first appeared in the featured Workshop slot in the Steam client, which is a huge driver of new subscribers. In Shadowrun Returns it would take me a week to claim this spot, but I think the much smaller mod volume for Hong Kong results in faster movement of mods. (Which works both ways, of course; I was booted off as soon as the next mod came along.) There's a very strong correlation between visitors and subscribers, as you would expect. The biggest dip came around Christmas Eve, when there are probably fewer people at their PCs. The drop-off at the end is misleading; I think that Steam's data trails actual results by about a day, but it includes the most recent day in the graph, so on any given day it will look like the current day is declining.

For comparison, here's the current lifetime graph for Antumbra Saga.

As you can see, this is much spikier, with the highest days an order of magnitude larger than the average. There's a good reason for each of those spikes: the release of the mod; the announcement of Hong Kong; the ending of the Hong Kong Kickstarter campaign; and the release of Hong Kong. You can also see a small but noticeable trend upward over the past month, which I attribute to new Hong Kong players who have encountered The Caldecott Caper and are going back to play the earlier campaign.

And, for the curious, here are the top-level stats for Antumbra Saga after the year-plus it's been on the Workshop.

Overall this seems to track pretty closely with what I'm seeing for Caldecott, although the Saga has a slightly higher conversion rate.

The total subscriber count for The Caldecott Caper at 38 days (2,203) is smaller than that of Antumbra Saga at the same age (3,507). This may be due to a smaller installed base for Hong Kong. However, subscriber growth for The Caldecott Caper is much more consistent than for the Saga. The Saga's mean acquisition is higher, while the Caper's median acquisition is higher.

In a separate universe, I'm also maintaining a DRM-free version of The Caldecott Caper on the NexusMods site. Antumbra Saga got about 10% as much traffic and 10% as many subscribers on Nexus as on Steam; but Nexus users tend to be highly engaged, and I've gotten some great bug reports and feedback from there, so it's been worth the extra pain of maintaining two separate distribution portals.

The very first Nexus release came out about a week after the Steam release; by this point I'd plugged the major bugs and it felt like things were stabilizing nicely. Here's the equivalent graph for Nexus:

Due to the much smaller sample size, the variations here don't seem significant, it's all just noise. I'm averaging about 3 downloads per day, with an all-time high of 8, and the distribution is essentially random. As with Steam, I expect to see spikes in the future around the time of major updates to Hong Kong, although historically Nexus has been more consistent than Steam (probably because there is no in-game portal to direct players here, so downloads are the result of deliberate searching on the part of players rather than the result of impulsive reactions to featured items in the workshop). I currently have just over 100 downloads on the Nexus, about 5% of my subscriber count on Steam.

I should probably mention that I've done basically no marketing at all for the campaign. Here's the sum total of everything I've done:

• Written these blog posts (this is an extremely low-traffic blog!)
• Tweeted a link to the download. (I have 34 followers, of whom perhaps 3 have ever heard of Shadowrun.)
• Posted a link to the campaign in the Shadowrun Returns subreddit. This is probably my most effective move yet, which isn't saying much.
• Started a thread on Good Old Games advertising the Nexus release. This has actually been a pretty good thread, although, again, the traffic on Nexus is tiny in comparison to Steam.
• Linked together Antumbra Saga and The Caldecott Caper, so anyone who stumbles across one will be able to easily find the other.
• Created a demo video for The Caldecott Caper and released it on YouTube (and then promptly neglected to promote it in any way).

And, that's it!

I periodically search for mentions of The Caldecott Caper on social media or the Internet as a whole. There are very few references, but the ones out there are encouraging; I've been especially pleased to see several women who responded to elements of the game, which is particularly gratifying to me. These sorts of things (personal tweets or tumblr posts or Facebook posts or whatever) are a great way to reach new players who wouldn't necessarily search through the Steam Workshop in order to find new campaigns, and also an excellent sign that people are personally responding in some way to the story I had to tell.

It's possible that one day there will be a Let's Play video or a mention by one of the major Shadowrun fan sites or something, which would probably bring a significant new wave of subscribers to my campaign. In the absence of that, though, it will probably continue chugging along for a while longer, driven by a combination of word-of-mouth, serendipitous discovery, and feeding players' appetites for new Shadowrun stories. There probably won't be anything else worth reporting on this front, but if something unusual or exciting does happen in the future, I'll let you know!