Context
I received a “feature request” to have a sticky, floating table of content for each blog post, which is technically feasible. However, it comes with two trade-offs:
- I have to remove the sidebar because it takes too much space. Haivng a sidebar and a floating table of content would crush the main blog size by half.
- I have to spend around 20-30 more minutes of editorial time in order to create a floating table of content (due to the technical limitation of my theme). That will amount to around 25 hours of extra time spent on just creating tables of content each year. I’m not even sure it’s that important to all readers and can actually improve my North Star Metrics (average session duration).
I was wondering between keeping the sidebar, or removing the sidebar and replacing it with the sticky table of content.
Project goal
To help me validate the hypothesis:
Removing the sidebar will negatively impact the average page per session and thus will directly reduce the average session duration.
Project overview
I removed the sidebar during the period of 18th October to 6th November 2021. I call it the experiment period.
A blog post without a sidebar looks like this: a simple one-column piece of text
A blog post with a sidebar looks like this
During this time, I tried not to do anything special to increase the traffic to my site. However, there were two accidental (but lucky) spikes of visitors on the 22nd of October and the 28th of October. It was due to my subscribers actively sharing about my posts in a Facebook post’s comment.
Project results
Removing the sidebar does reduce both the average page per session and the average session duration.
In particular, compared to 20 days earlier, the experiment period observes a downturn of 15.23% in average page per session and 7.30% in average session duration.
To double-check, I compared the experiment period with 20 days in August (from 1 Aug to 20 Aug) and noticed an even more staggering impact :<
Aite, let’s give this one last shot. I compared the targeted metrics in the experiment period with the average of the whole year up until when the experiment period ends. Though the decline seems to be migitated a bit, it still solidifies fact that removing the sidebar is one but a fruitful move to my north star metric.
Chotto minute, how about the two spikes of traffic?
As mentioned, we observed a large influx of blog visitors on the 22nd and 28th of October. I was concerned that they might skew the measurements because the traffic came from a post which came from a group sharing puns or other types of funny, trendy statements. It’s very likely that the audiences of this group do not have interests in my blog’s topics, or whose behaviors are more about quick consumption rather than long-form content contemplation.
A quick look at the period-over-period comparison and I shall see a relatively similar average session duration for these two days and the days afterward.
So, false alarm! It’s time to bring the beloved sidebar back!