Data Science for Business

02Identifying Informative Attributes

Finding the right variables to predict future outcomes is much like searching for a needle in a massive, ever-expanding digital haystack. When trying to solve a predictive business problem, we are usually looking for specific attributes—or features—that have a strong relationship with the target variable we want to predict. For instance, if a telecommunications company wants to predict which customers are likely to cancel their contracts a problem widely known as customer churn, they have hundreds of attributes to consider. They know the customer's age, location, monthly bill amount, data usage, number of calls to customer service, and the type of phone they use. Out of all these attributes, which ones actually contain valuable information about the customer's likelihood to leave? To answer this, we must introduce a core concept of data science: Information Gain. Information gain measures how much a given attribute reduces our uncertainty about the target variable. To understand this, we first need to grasp the concept of entropy, which is simply a measure of disorder, uncertainty, or impurity in a set of data. Let us break this down with a highly relatable example. Suppose you have a jar filled with 100 marbles. If 50 marbles are red and 50 are blue, the jar is completely mixed. The uncertainty—or entropy—is at its absolute maximum because if you reach in blindfolded, you have no idea what color you will pull out. On the other hand, if all 100 marbles are red, the entropy is zero. There is no uncertainty; you know exactly what you will get. In predictive modeling, we want to find attributes that split our mixed, high-entropy dataset into neat, low-entropy subsets. Returning to our telecommunications company, imagine our total customer base is a mixed jar: some customers will stay, and some will churn. We want to find a question we can ask that separates the loyal customers from the churning customers as cleanly as possible. What if we split the customers based on their favorite color? We might find that the group of customers who like blue has a 10% churn rate, and the group who likes red also has a 10% churn rate. Splitting the data by favorite color did not reduce our uncertainty at all. The entropy remains high. Therefore, "favorite color" provides zero information gain. It is a useless attribute for predicting churn. Now, what if we split the customers based on whether their contract expires in less than 30 days? We might discover that the group with expiring contracts has an 80% churn rate, while the group with long-term contracts has only a 2% churn rate. This single question has dramatically separated the churners from the non-churners. The resulting subsets are much purer than the original mixed group. Because this attribute dramatically reduced our uncertainty, we say it provides a massive amount of information gain. This elegant, logical concept is the fundamental engine behind one of the most popular and intuitive machine learning models used in business today: Decision Trees. Building a decision tree is essentially just having an algorithm play a highly efficient game of "Twenty Questions" with your data. When playing Twenty Questions, you do not start by asking, "Is the object a 2018 Honda Civic?" That is far too specific and unlikely to be true. Instead, you start with broad questions that split the possibilities in half, such as, "Is it an animal?" or "Is it bigger than a breadbox?" Decision tree algorithms do exactly the same thing using mathematical calculations. The algorithm scans all the available attributes in the dataset and calculates the information gain for every possible split. It finds the single attribute that provides the highest information gain and makes that the first branching point the root of the tree. Using our telecom example, the first split might be: "Is the contract expiring in less than 30 days?" The algorithm then takes the resulting subsets and repeats the process. For the customers with expiring contracts, it asks: out of the remaining attributes, which one gives us the most information gain now? It might find that "number of calls to customer service in the last week" is the next best predictor. It creates another branch. The tree continues to grow, splitting the data into increasingly pure segments until it reaches a point where further splits do not provide any meaningful information gain. The extraordinary beauty of a decision tree lies in its profound transparency. Unlike complex neural networks that operate as inscrutable black boxes, a decision tree provides a clear, highly readable roadmap of human logic. A business manager can look at a printed decision tree and trace the exact path that leads to a high probability of churn. "Ah, I see. If a customer's contract is expiring soon, AND they have called customer service more than three times this month, AND their monthly bill is over $100, they have a 92% chance of leaving." This transparency is invaluable for decision-making. It not only tells the business who is likely to churn but also offers deep clues as to why. Armed with this specific knowledge, the marketing team can craft highly targeted interventions. Instead of offering a generic 10% discount to everyone, they can proactively call the specific high-risk segment, apologize for the recent customer service issues, and offer a customized, premium retention package. By systematically identifying informative attributes, businesses can cut through the noise of big data and focus entirely on the signals that drive real-world profitability.

Continue reading with LeapAhead app

Full summary is waiting for you in the app

Continue reading