Model Development Diary #1: Which way to swing?

Re-evaluating some of the first principles in seat projections.

It don’t mean a thing if it ain’t got that swing — especially when it comes to seat projection models.

But which way should a model swing?

I’ll try to answer that question in this first instalment of this model development diary.

First, some background. I’ve been using and developing seat projection models ever since I launched ThreeHundredEight.com in 2008. My first models were quite rudimentary, but ahead of the 2011 election I developed a model the basic structure of which has powered all the seat projection models I’ve used ever since. While I’ve made adjustments over the years and re-designed the back-end of the model ahead of the last election to make it easier to use, the general theory of my seat projection models hasn’t much changed over the last 15 years.

I’ve now decided to start over from scratch.

The seat projection model behind the Poll Tracker performed well enough in the last election. It over-estimated the Liberals and under-estimated the Conservatives, but the results were within the confidence intervals and the model’s mistakes were for the most part due to its inability to account for sub-regional trends — trends that were hard to pick up on in the publicly available polling data.

But I didn’t like how the model operated. There were too many ad hoc adjustments I had to make on the fly and too many judgement calls. I didn’t like what the model was producing at the riding level in many cases. It did the job, but I didn’t like how it did the job. I wanted to feel more confident in what it was spitting out, especially when it came down to individual ridings.

What I did like, however, was running a seat projection model again, something I initially wanted to back away from when I first launched The Writ. While imperfect (and I’ll get into that), seat projection and poll aggregation models are useful tools for understanding where public opinion and the electoral map stands on any given day.

Using a model with a consistent set of rules makes it possible to assess the ups and downs of politics in a more objective, uniform way. It also allows us to translate national and regional polling numbers into reasonable estimates of electoral outcomes, an absolute necessity considering the vagaries of our first-past-the-post system. And, personally, I just find them fun. That’s why I got into this business in the first place.

I’d like to have a model running on The Writ in the future. (How it will be presented is a problem for another day — I’d like it to be good-looking and marginally interactive, which might be a challenge on Substack. I’ll cross that bridge when I get to it.)

But I don’t want it to be the model that I’ve used for the last 15 years. I’ve learned a lot over those 15 years and some of the rationales behind the model’s inner workings no longer make as much sense to me today as they did a decade ago — that is when I can remember those rationales to begin with.

So, I’m tearing it down to the studs and rebuilding. Of course, there are only so many ways to build a house. The final product will undoubtedly have a lot of similarities to my old model. But I’m looking forward to building it, piece-by-piece, with more confidence behind each of those pieces.

It’ll be a long process, but I suspect it’ll produce some interesting findings. Why not share them? These model development diaries will explain my thinking as the new model comes together.

First principles: uniform or proportional swing?

In starting over, I have to return to first principles. I want to have testing and data behind every decision for how to design the model, and the first place to start is with the basic theory behind seat projections: the swing.