Daily newsletter testing
In which I keep you updated on the testing and implementation of my daily newsletter, Authoresque.
Hi friends,
This post continues the saga of trying to figure out how to make a sustainable lead generation machine for The Author Stack using ads. These are the previous articles I’ve written about this topic.
In the previous two articles, I explain why I started using ads again this year, and in the second I talk about how those initial tests ended in spectacularly terrible fashion.
From there, I began testing a daily newsletter in the style of Morning Brew, 1440, the Skimm, but for indie authors. I’ve been studying these newsletters for several months, and feel like I have a pretty decent template to start moving forward with a full test.
I wrote this article in real time as I was going through the tests, which might take a bit to get used to, but it gave me the more visceral reaction to the events transpiring. I thought that was important.
This article also has a paywall in the middle. I gave enough information so you can see what I’m doing and hopefully how to do it yourself, but not enough to see all the juicy bits about the success of failure of the subsequent tests.
Beehiiv catch up
I previously tested multiple options through Beehiiv with decent results. I tested two different options, and these were the stats for the winning test.
My main list has relatively similar results, so I did some simple math and decides that if these carried through, I should see roughly 4x the results, which would put me right around 15,000 opens and 300 unique clicks. You can see that the top ad for this set is the ad I ran, which meant I could expect roughly 200 clicks total and 80-100 total clicks, which would put my ad costs right around $300-$500, which would be a great result.
Convertkit test 1
If you’ve followed along the previous articles in this saga, you can probably guess this is not what happened. If you want to see the first test, here is a link to check it out.
I heard great things from people who reached out to tell me what they thought, but here are the results after the first day.
That’s 9,490 opens, well below my benchmarks. Remember, I was expecting closer to 15,000. To give you some context, this is what an email looks like after 48 hours on The Author Stack.
You can see that despite being sent to 8,000 fewer people, this email was viewed 15.1k times, or by 50% more people. Now, you might think that my Convertkit emails get worse open rates, but when I ran tests to each segment of my life, they performed mainly as I would expect.
As you can see, that’s about over 15,000 opens and over 350 clicks on a post that didn’t have many places to click. To go from these results to what happened on this test is an absolute disaster.
A 35% open rate is not very good in this game, but a 23% open rate is a awful. On top of this, the ad break only got 32 clicks, which means that instead of charging $300-$500, an ad break would cost $100-$200.
That takes the daily from the realm money generating to barely breaking even, especially if I bring in help.
So, what happened? I wasn’t sure. Maybe the headline was bad, though I don’t think it would have gotten a 50% reduction in open rate. Perhaps there were too many images, though I don’t think they had more than the initial test I ran on Beehiiv. Perhaps it was just too long to reach inboxes.
The only thing you can do when something like this happens is do some testing, so I decided to resend to unopens. Here are the results.
I tested a much hookier headline, and shorter, which didn’t seem to do anything to help deliverability. After a week, these were the results.
32% isn’t terrible, but when you send a daily you don’t really have a chance to resend to unopens.
If you’d like to play with Convertkit yourself, they have recently opened a free 10,000 subscriber plan, so you can get in there and see if it works for you.
Convertkit test 2
For the first test, I ran a 5x5, which is five articles and five bonus bits. For my next test, I tested a 3x3, with little or no imagery to see if that helps. I had such high hopes before this test as I had every indication this would go really, really well. Then, we ran it. You can read this email here.
The first of these emails went worse than the first one, even though I was running less graphics and fewer links by a wide margin. The resend went okay, but only compared to the last one.
Overall, after a couple of days, here’s how this one stands.
You might think this was normal, but I also run my Wannabe Press emails through Convertkit, and this is what our resend looked like over there.
That’s a 43% open rate, which way better than on Flodesk, and then with our next email we hit 44% without even a resend.
This second test went out to 41,000 people, and so after two sends I have to assume that a significant percentage of people just don’t want to read it, which is understandable. This is a whole new product and I’m basically asked other people to care about it from jump.
I’ve identified 12,000 people who haven’t opened or shown any interest. These include close to 5,000 people who haven’t opened an Author Stack email in a long time, and 7,000 people people who are only on my Wannabe Press email list, not my combined Wannabe Press/Author Stack list. So, there is no reason to believe that they would even care about what I have to say.
Additionally, holding a bunch of dead emails could seriously affect deliverability as well.
I created a cold email flow that should allow us to not deal with this once we have a good baseline. However, for these first tests I want to be more involved to manually see where we stand.
Next test we’ll isolate those people and remove them from the delivery segment. So, we’ll be sending to 28,000 people instead of 41,000, and see if we can get a 40%+ open rate from that. If I can get the same 12,000-15,000 opens then it should be possible.
Depending on how that goes, I’ll have to go back to my sponsorship team and tell them we actually only have 28,000 people on that list, not 41,000, which will probably force us to lower ad rates. I think what’s important to advertisers are opens, though, not total reach.
I have a lot more to share, but only for paid members. If you want to keep following my journey (and I have some juicy bits still to share) then I hope you’ll consider becoming a paid member, or at least start a free trial to access over 850 member-only posts, interviews, courses, and more.
The Author Stack open rate interlude
Keep reading with a 7-day free trial
Subscribe to The Author Stack to keep reading this post and get 7 days of free access to the full post archives.