*this post originally appeared in Computerworld*
Discovering and targeting micro-populations for politics and profit
As the election season rampages on, we categorize voters into broad demographics — soccer moms, NASCAR dads, blacks, whites, ALICEs, yuppies — in an attempt to understand and discuss this complex, churning electorate. In doing so we’re tapping into something fundamental about how we perceive the world: not as a sequence of singular individuals, but rather as a massive set of overlapping taxonomies that, taken together, comprise an impressively structured human experience.
Think of 20 Questions. With fewer than 20 yes/no queries on category membership we can often identify a single object amidst a staggering breadth of possibilities. We’ve grouped everything that we know to exist and the groupings themselves are the primary subject of our thoughts.
We can go the other direction as well — taking an object and placing it in its many groups. That’s a dog, that’s a mammal, that’s friendly, that’s living. That’s a book, that’s paper, that’s printed, that’s man-made.
Part of being an expert is being able to distinguish membership in more granular categories. Is this a Cabernet or a Pinot? Is that a Renoir or a Degas? With those fine grained distinctions an expert has a better grasp on how something might taste, look, or behave. So what if, as an election campaign, you could decompose the broad demographic of soccer moms down into hundreds of micropopulations and have strong expectations about how those populations will react to donation drives and get-out-the-vote campaigns? With clustering and classification, you can.
The goal of clustering is to find groups of similar entities within a dataset. For example, within a dataset containing the physical characteristics of dogs one could automatically find subpopulations that share similar traits (and that likely map well to breed boundaries). Clustering is an example of unsupervised machine learning, meaning that you do not know ahead of time what groups you are looking for — you want the algorithm to find those groups for you.
The supervised counterpart for clustering is called classification. With classification you know the groups that a large set of entities belong to, and you want to train an algorithm to classify unknown entities into the appropriate group. For example you might measure the physical traits from a new dog (one not in the original dataset) and then determine which group it likely belongs to.
Put into action, these techniques are powerful. They allow a campaign to take a basic demographic profile from a website interaction — say a voter that signs up to receive emails from a candidate — and map those to particular actions on behalf of the campaign.
Both Hillary Clinton and Donald Trump’s campaign websites have prominent calls to action that collect two pieces of information from supporters: zip code and email address. A campaign can use data from these forms — the email address and zip code themselves, time spent on the form, which browser and device type (mobile vs. desktop) the form was accessed through, IP address, etc. — to classify those who sign up into demographic groups. Those group memberships can then influence campaign actions. Does the campaign call, email, or text you when seeking your support? What are the default dollar amounts for donation prompts? Which issues are emphasized in campaign emails?
Once a donation campaign is complete, it creates a new piece of metadata about each supporter that was targeted. Did they donate or not? With this information in hand campaigns can go back to unsupervised learning and cluster within just that group of supporters that received the donation prompt. Are there any subgroups that were particularly likely to donate or not donate? What are their demographic characteristics? These insights can be instrumented back into the supervised classification models that do the targeting in the first place.
With machine learning, political groups can interact with their supporters at a level of automation and granularity that would have never before been possible. In the business world the opportunity for clustering- and classification-driven operational improvement is likewise immense. Election campaigns are essentially marketing campaigns, and the capability to micro-target and infer demographic membership from limited information is similarly impactful in that domain.
You might know that a product line is particularly popular in the 18-35 year-old demographic, but you want to further segment that group to understand popularity amongst natural subpopulations of the larger group. You might track user behavior on a website to profile for likely demographic membership, and then target product recommendations accordingly. With these tools you can quickly build up an expert taxonomic understanding of your customers, their preferences, and their behaviors.
Clustering and classification represent the opportunity to apply categorical reasoning to vast data contexts we would otherwise find overwhelming. Every voter and customer is unique, and broad demographic groupings do that diversity a disservice.
With fine-grained distinctions between micropopulations derived from machine learning, businesses and campaigns can interact with their constituents in an effective, data-driven manner.