|
18 | 18 | "\n", |
19 | 19 | "This puts us in a difficult situation, because if we have unmeasured confounders, we have bias. One way to deal with this is with instrumental variables, like we've seen before. But coming up with good instruments it's no easy task and requires a lot of creativity. Here, let's look at an alternative that takes advantage of time or the temporal structure of data. \n", |
20 | 20 | "\n", |
21 | | - "The idea is to use **panel data**. Panel data is when we have **observations on the same individual for multiple periods of time**. Panel data formats are very common in the industry, where they keep records of customer behavior for the same customer and for multiple time periods. The reason we can leverage panel data is because we can compare the same unit before and after the treatment, and see how they behave with it. Before we dive in the math, let's see how this makes intuitive sense." |
| 21 | + "The idea is to use **panel data**. Panel data is when we have **observations on the same individual for multiple periods of time**. Panel data formats are very common in the industry, where they keep records of customer behavior for the same customer and for multiple time periods. The reason we can leverage panel data is because we can compare the same unit before and after the treatment and see how they behave with it. Before we dive in the math, let's see how this makes intuitive sense." |
22 | 22 | ] |
23 | 23 | }, |
24 | 24 | { |
|
49 | 49 | "cell_type": "markdown", |
50 | 50 | "metadata": {}, |
51 | 51 | "source": [ |
52 | | - "First, let's take a look at the causal graph that we have once we include multiple observations of the same unit across time. Suppose we have a situation where marriage at the first time causes income at the same time and subsequent marital status.. This is also true for times 2 and 3. Also, suppose that beauty is the same across all time periods (a bold statement, but reasonable if time is just a few years) and it causes both marriage and income." |
| 52 | + "First, let's take a look at the causal graph that we have once we include multiple observations of the same unit across time. Suppose we have a situation where marriage at the first time causes income at the same time and subsequent marital status. This is also true for times 2 and 3. Also, suppose that beauty is the same across all time periods (a bold statement, but reasonable if time is just a few years) and it causes both marriage and income." |
53 | 53 | ] |
54 | 54 | }, |
55 | 55 | { |
|
379 | 379 | "cell_type": "markdown", |
380 | 380 | "metadata": {}, |
381 | 381 | "source": [ |
382 | | - "Think about it. We don't know the attributes like beauty and intelligence, but we know that person is the same individual. So, we can create a dummy variable indicating that person and add that to a linear model. This is what we mean when we say we can control for the person itself. When estimating the effect of marriage on income with this person dummy in our model, what regression does is it finds the effect of marriage **while keeping the person variable fixed**. Adding this entity dummy is what we call a fixed effect model.\n", |
| 382 | + "Think about it. We can't measure attributes like beauty and intelligence, but we know that the person who has them is the same individual across time. So, we can create a dummy variable indicating that person and add that to a linear model. This is what we mean when we say we can control for the person itself: we are adding a variable (dummy in this case) that denotes that particular person. When estimating the effect of marriage on income with this person dummy in our model, regression finds the effect of marriage **while keeping the person variable fixed**. Adding this entity dummy is what we call a fixed effect model.\n", |
383 | 383 | "\n", |
384 | 384 | "\n", |
385 | 385 | "## Fixed Effects\n", |
|
510 | 510 | "y_{it} = \\beta X_{it} + \\gamma U_i + e_{it}\n", |
511 | 511 | "$\n", |
512 | 512 | "\n", |
513 | | - "where \\\\(y_{it}\\\\) is the outcome of individual \\\\(i\\\\) at time \\\\(t\\\\), \\\\(X_{it}\\\\) is the vector of variables for individual \\\\(i\\\\) at time \\\\(t\\\\), \\\\(U_i\\\\) is a set of unobservables for individual \\\\(i\\\\) that are unchanging through time, hence the lack of the time subscript, and \\\\(e_{it}\\\\) is the error term. For the education example, \\\\(y_{it}\\\\) is log wages, \\\\(X_{it}\\\\) are the observable variables that change in time, like marriage and experience and \\\\(U_i\\\\) are the variables that are not observed but constant for each individual, like beauty and intelligence. \n", |
| 513 | + "where \\\\(y_{it}\\\\) is the outcome of individual \\\\(i\\\\) at time \\\\(t\\\\), \\\\(X_{it}\\\\) is the vector of variables for individual \\\\(i\\\\) at time \\\\(t\\\\). \\\\(U_i\\\\) is a set of unobservables for individual \\\\(i\\\\). Notice that those unobservables are unchanging through time, hence the lack of the time subscript. Finally, \\\\(e_{it}\\\\) is the error term. For the education example, \\\\(y_{it}\\\\) is log wages, \\\\(X_{it}\\\\) are the observable variables that change in time, like marriage and experience and \\\\(U_i\\\\) are the variables that are not observed but constant for each individual, like beauty and intelligence. \n", |
514 | 514 | "\n", |
515 | 515 | "\n", |
516 | 516 | "Now, remember how I've said that using panel data with a fixed effect model is as simple as adding a dummy for the entities. It's true, but in practice, we don't actually do it. Imagine a dataset where we have 1 million customers. If we add one dummy for each of them, we would end up with 1 million columns, which is probably not a good idea. Instead, we use the trick of partitioning the linear regression into 2 separate models. We've seen this before, but now is a good time to recap it. Suppose you have a linear regression model with a set of features \\\\(X_1\\\\) and another set of features \\\\(X_2\\\\).\n", |
|
587 | 587 | "cell_type": "markdown", |
588 | 588 | "metadata": {}, |
589 | 589 | "source": [ |
590 | | - "Notice how this model is predicting the average income in 1980 to be 1.3935, in 1981 to be 1.5129 (1.3935+0.1194) and so on. Now, if we compute the average by year, we get the exact same result (add the intercept to the parameters of the other years to get the mean for the year)." |
| 590 | + "Notice how this model is predicting the average income in 1980 to be 1.3935, in 1981 to be 1.5129 (1.3935+0.1194) and so on. Now, if we compute the average by year, we get the exact same result. (Remember that the base year, 1980, is the intercept. So you have to add the intercept to the parameters of the other years to get the mean `lwage` for the year)." |
591 | 591 | ] |
592 | 592 | }, |
593 | 593 | { |
|
628 | 628 | "1. Create time-demeaned variables by subtracting the mean for the individual: \n", |
629 | 629 | "\\\\(\\ddot{Y}_{it} = Y_{it} - \\bar{Y}_i\\\\) \n", |
630 | 630 | "\\\\(\\ddot{X}_{it} = X_{it} - \\bar{X}_i\\\\)\n", |
631 | | - "2. Run regress \\\\(\\ddot{Y}_{it}\\\\) on \\\\(\\ddot{X}_{it}\\\\)\n", |
| 631 | + "2. Regress \\\\(\\ddot{Y}_{it}\\\\) on \\\\(\\ddot{X}_{it}\\\\)\n", |
632 | 632 | "\n", |
633 | | - "Notice that when we do so, the unobserved \\\\(U_i\\\\) vanishes, since \\\\(\\bar{U_i}=U_i\\\\):\n", |
| 633 | + "Notice that when we do so, the unobserved \\\\(U_i\\\\) vanishes. Since \\\\(U_i\\\\) is constant across time, we have that \\\\(\\bar{U_i}=U_i\\\\). If we have the following system of two equations\n", |
634 | 634 | "\n", |
635 | 635 | "\\begin{align}\n", |
636 | 636 | "Y_{it} & = \\beta X_{it} + \\gamma U_i + e_{it} \\\\\n", |
637 | | - "\\bar{Y}_{i} & = \\beta \\bar{X}_{it} + \\gamma U_i + \\bar{e}_{it} \\\\\n", |
| 637 | + "\\bar{Y}_{i} & = \\beta \\bar{X}_{it} + \\gamma \\bar{U}_i + \\bar{e}_{it} \\\\\n", |
| 638 | + "\\end{align}\n", |
| 639 | + "\n", |
| 640 | + "And we subtract one from the other, we get\n", |
| 641 | + "\n", |
| 642 | + "\\begin{align}\n", |
638 | 643 | "(Y_{it} - \\bar{Y}_{i}) & = (\\beta X_{it} - \\beta \\bar{X}_{it}) + (\\gamma U_i - \\gamma U_i) + (e_{it}-\\bar{e}_{it}) \\\\\n", |
639 | 644 | "(Y_{it} - \\bar{Y}_{i}) & = \\beta(X_{it} - \\bar{X}_{it}) + (e_{it}-\\bar{e}_{it}) \\\\\n", |
640 | 645 | "\\ddot{Y}_{it} & = \\beta \\ddot{X}_{it} + \\ddot{e}_{it} \\\\\n", |
641 | 646 | "\\end{align}\n", |
642 | 647 | "\n", |
643 | | - "To be honest, not only do the unobserved variables that are constant in time vanish. This happens to all the variables that are constant in time. For this reason, you can't include any variables that are constant across time, as they would be a linear combination of the dummy variables and the model wouldn't run. \n", |
| 648 | + "which wipes out all unobserved that are constant across time. To be honest, not only do the unobserved variables vanish. This happens to all the variables that are constant in time. For this reason, you can't include any variables that are constant across time, as they would be a linear combination of the dummy variables and the model wouldn't run. \n", |
644 | 649 | "\n", |
645 | 650 | "\n", |
646 | 651 | "\n", |
647 | | - "To check which variables are those, we can group our data by individual and get the sum of the standard deviations. If it is zero, it means the variable isn't changing for the same individual. " |
| 652 | + "To check which variables are those, we can group our data by individual and get the sum of the standard deviations. If it is zero, it means the variable isn't changing across time for any of the individuals. " |
648 | 653 | ] |
649 | 654 | }, |
650 | 655 | { |
|
682 | 687 | "cell_type": "markdown", |
683 | 688 | "metadata": {}, |
684 | 689 | "source": [ |
685 | | - "For our data, we need to remove entinicity dummies, `black` and `hisp`, since they are constant for the individual. Also, we need to remove education. We will also not use occupation, since this is probably mediating the effect of marriage on wage (it could be that single men are able to take more time demanding positions). \n", |
| 690 | + "For our data, we need to remove entinicity dummies, `black` and `hisp`, since they are constant for the individual. Also, we need to remove education. We will also not use occupation, since this is probably mediating the effect of marriage on wage (it could be that single men are able to take more time demanding positions). Having selected the features we will use, it's time to estimate this model.\n", |
686 | 691 | "\n", |
687 | | - "To run our fixed effect model, first, let's get our mean data." |
| 692 | + "To run our fixed effect model, first, let's get our mean data. We can achieve this by grouping everything by individuals and taking the mean." |
688 | 693 | ] |
689 | 694 | }, |
690 | 695 | { |
|
968 | 973 | "cell_type": "markdown", |
969 | 974 | "metadata": {}, |
970 | 975 | "source": [ |
971 | | - "If we were to believe that fixed effect eliminates the all omitted variable bias, this model is telling us that marriage increases a man's wage by 11%. This result is very significant. One detail here is that for fixed effect models, the standard errors need to be clustered. So, instead of doing all our estimation by hand (which is only nice for pedagogical reasons), we can use the library `linearmodels` and set the argument `cluster_entity` to True." |
| 976 | + "If we believe that fixed effect eliminates the all omitted variable bias, this model is telling us that marriage increases a man's wage by 11%. This result is very significant. One detail here is that for fixed effect models, the standard errors need to be clustered. So, instead of doing all our estimation by hand (which is only nice for pedagogical reasons), we can use the library `linearmodels` and set the argument `cluster_entity` to True." |
972 | 977 | ] |
973 | 978 | }, |
974 | 979 | { |
|
1079 | 1084 | "cell_type": "markdown", |
1080 | 1085 | "metadata": {}, |
1081 | 1086 | "source": [ |
1082 | | - "This model is saying that marriage increases the man's wage by 14%. A somewhat larger effect than the one we found with the fixed effect model. This suggests some omitted variable bias due to fixed individual factors, like intelligence and beauty not being added to the model. \n", |
| 1087 | + "This model is saying that marriage increases the man's wage by 14%. A somewhat larger effect than the one we found with the fixed effect model. This suggests some omitted variable bias due to fixed individual factors, like intelligence and beauty, not being added to the model.\n", |
1083 | 1088 | "\n", |
1084 | 1089 | "## Visualizing Fixed Effects\n", |
1085 | 1090 | "\n", |
1086 | | - "To expand our intuition about how fixed effect models work, let's diverge a little to another example. Suppose you work for a big tech company and you want to estimate the impact of a billboard marketing campaign on in-app purchase. When you look at data from the past, you see that the marketing department tends to spend more to place billboards on cities where the purchase level is lower. This makes sense right? They wouldn't need to do lots of marketing if sales were skyrocketing. If you run a regression model on this data, it looks like higher cost in marketing leads to less in-app purchase." |
| 1091 | + "To expand our intuition about how fixed effect models work, let's diverge a little to another example. Suppose you work for a big tech company and you want to estimate the impact of a billboard marketing campaign on in-app purchase. When you look at data from the past, you see that the marketing department tends to spend more to place billboards on cities where the purchase level is lower. This makes sense right? They wouldn't need to do lots of advertisement if sales were skyrocketing. If you run a regression model on this data, it looks like higher cost in marketing leads to less in-app purchase, but only because marketing investments is biased towards low spending regions. " |
1087 | 1092 | ] |
1088 | 1093 | }, |
1089 | 1094 | { |
|
1165 | 1170 | "cell_type": "markdown", |
1166 | 1171 | "metadata": {}, |
1167 | 1172 | "source": [ |
1168 | | - "Take a minute to appreciate the figure above and let it tell you what fixed effect is doing. Notice that fixed effect is fitting **one regression line per city**. Also notice that the lines are parallel. The slope of the line is the effect of marketing costs on in-app purchase. So the **fixed effect is assuming that the causal effect is constants across all entities**, which are cities in this case. This can be a weakness or an advantage, depending on how you see it. It is a weakness if you are interested in finding the causal effect per city. Since the FE model assumes this effect is constant across entities, you won't find any difference in the causal effect. However, if you want to find the overall impact of marketing on in-app purchase, the panel structure of the data is a very useful leverage that fixed effects can explore. \n", |
| 1173 | + "Take a minute to appreciate what the image above is telling you about what fixed effect is doing. Notice that fixed effect is fitting **one regression line per city**. Also notice that the lines are parallel. The slope of the line is the effect of marketing costs on in-app purchase. So the **fixed effect is assuming that the causal effect is constants across all entities**, which are cities in this case. This can be a weakness or an advantage, depending on how you see it. It is a weakness if you are interested in finding the causal effect per city. Since the FE model assumes this effect is constant across entities, you won't find any difference in the causal effect. However, if you want to find the overall impact of marketing on in-app purchase, the panel structure of the data is a very useful leverage that fixed effects can explore. \n", |
1169 | 1174 | "\n", |
1170 | 1175 | "## Time Effects\n", |
1171 | 1176 | "\n", |
1172 | | - "Just like we did a fixed effect for the individual level, we could design a fixed effect for the time level. If adding a dummy for each individual controls for fixed individual characteristics, adding a time dummy would control for variables that are fixed across time. One example of such a variable is inflation. Prices and salary tend to go up with time. If the wage marriage proportion also changes with time, we would have time as a confounder. To give a more concrete example, suppose that marriage is increasing with time. Since inflation also makes salary increase with time, some of the positive association we see between marriage and wage would be simply because both are increasing with time. To correct for that, we can add a dummy variable for each time period. In `linear models`, this is as simple as adding `TimeEffects` to our formula and setting the `cluster_time` to ture." |
| 1177 | + "Just like we did a fixed effect for the individual level, we could design a fixed effect for the time level. If adding a dummy for each individual controls for fixed individual characteristics, adding a time dummy would control for variables that are fixed across time. One example of such a variable is inflation. Prices and salary tend to go up with time. If the wage and marriage proportion also changes with time, we would have time as a confounder. To give a more concrete example, suppose that marriage is increasing with time. Since inflation also makes salary increase with time, some of the positive association we see between marriage and wage would be simply because both are increasing with time. To correct for that, we can add a dummy variable for each time period. In `linear models`, this is as simple as adding `TimeEffects` to our formula and setting the `cluster_time` to true." |
1173 | 1178 | ] |
1174 | 1179 | }, |
1175 | 1180 | { |
|
1232 | 1237 | "\n", |
1233 | 1238 | "Another less obvious case when fixed effect fails is when you have **reversed causality**. For instance, let's say that it isn't marriage that causes you to earn more. Is earning more that increases your chances of getting married. In this case, it will appear that they have a positive correlation but earnings come first. They would change in time and in the same direction, so fixed effects wouldn't be able to control for that. \n", |
1234 | 1239 | "\n", |
| 1240 | + "\n", |
1235 | 1241 | "## Key Ideas\n", |
1236 | 1242 | "\n", |
1237 | | - "Here, we saw how to use panel data, data where we have multiple measurements of the same individuals across multiple time periods. When that is the case, we can use a fixed effect model that controls for the entity, holding all individual, time constant attributes fixed. This is a powerful and very convincing way of controlling for confounding and it is as good as it gets with non random data. \n", |
| 1243 | + "Here, we saw how to use panel data, data where we have multiple measurements of the same individuals across multiple time periods. When that is the case, we can use a fixed effect model that controls for the entity, holding all individual, time constant attributes, fixed. This is a powerful and very convincing way of controlling for confounding and it is as good as it gets with non random data. \n", |
1238 | 1244 | "\n", |
1239 | 1245 | "Finally, we saw that FE is not a panacea. We understood two situations where it doesn't work: when we have reverse causality and when the unmeasured confounding is changing in time.\n", |
1240 | 1246 | "\n", |
| 1247 | + "\n", |
| 1248 | + "\n", |
1241 | 1249 | "## References\n", |
1242 | 1250 | "\n", |
1243 | 1251 | "I like to think of this entire book as a tribute to Joshua Angrist, Alberto Abadie and Christopher Walters for their amazing Econometrics class. Most of the ideas here are taken from their classes at the American Economic Association. Watching them is what is keeping me sane during this tough year of 2020.\n", |
|
1255 | 1263 | "\n", |
1256 | 1264 | "Finally, I'd also like to compliment Scott Cunningham and his brilliant work mingling Causal Inference and Rap quotes:\n", |
1257 | 1265 | "\n", |
1258 | | - "* [Causal Inference: The Mixtape](https://www.scunning.com/mixtape.html)" |
| 1266 | + "* [Causal Inference: The Mixtape](https://www.scunning.com/mixtape.html)\n", |
| 1267 | + "\n", |
| 1268 | + "" |
1259 | 1269 | ] |
1260 | 1270 | }, |
1261 | 1271 | { |
|
1282 | 1292 | "name": "python", |
1283 | 1293 | "nbconvert_exporter": "python", |
1284 | 1294 | "pygments_lexer": "ipython3", |
1285 | | - "version": "3.6.9" |
| 1295 | + "version": "3.8.5" |
1286 | 1296 | } |
1287 | 1297 | }, |
1288 | 1298 | "nbformat": 4, |
|
0 commit comments