Interaction Terms in 3D • regress3d

Introduction

This article examines the functions add_3d_surface() and add_marginals() for a linear regression surface that features an interaction term. The functionality of add_3d_surface() and add_marginals() is the same as in a linear regression model without an interaction term. However, the visuals are sufficiently different to merit a separate description.

Setup

The package regress3d is built on the syntax of plotly in R. Both libraries are called in the initial setup.

library(regress3d)
library(plotly)

Interaction terms

The equation for the predicted values from a linear regression with two $x$ variables and an interaction term is below.

$\begin{align*} \hat{y} =& \hat{\beta}_1 x_1 +\hat{\beta}_2 x_2 +\hat{\beta}_{12} x_1 x_2 + \hat{\beta}_0 \end{align*}$

An interaction term in a linear regression can be depicted as a curved regression surface with linear marginal effects. The marginal effect of $x_1$ ( $x_2$ ) is allowed to linearly change as $x_2$ ( $x_1$ ) changes. This captures heterogenous effects in a regression, where the effect of $x_1$ is different for different values of $x_2$ . Note that the change in the effect of $x_1$ as $x_2$ changes is restricted by the linearity of the interaction term.

Data

The variables in this example are county level demographic and voting measures from 2016. The data was chosen to emphasize the twist in the regression surface and highlight changes in the marginal effects. The explanatory variable college_2cat_num was chosen to be binary to make it easier to see how the marginal effect for one value of the binary variable is different from the marginal effect for the other value.

prcnt_GOP16: the percent of the county’s voters who voted for Trump in 2016,
prcnt_unemployed_log: the logged percent of the county that was unemployed in 2016, and
college_2cat_num: a binary measure of county college experience. It is 0 if the county was below the county level mean value of adult residents who had been enrolled in college at some point regardless of whether they graduated, and 1 otherwise.

The regression is weighted by pop_estimate16, the number of people in a county, to capture the influence of large counties.

Code and graphic

The code starts by specifying a model. We then create a plotly::plot_ly() object using the same variables. Next layer on the scattercloud, 3D surface, marginals, and labels. Note that regression notation uses $x_1$ and $x_2$ to represent the explanatory variables, and $y$ for the outcome. The plotly command denotes the explanatory variables as $x$ and $y$ , and the outcome variable is $z$ .

In this example, each call to add_marginals() specifies different constant values for each marginal effect using the arguments x1_constant_val and x2_constant_val. This highlights two key features of the regression surface.

The orange marginal effects of $x_1$ are linear for any given value of $x_2$ . The same is true for the red marginal effects of $x_2$ .
The size of the marginal effect of $x_1$ ( $x_2$ ) is different for the two values of $x_2$ ( $x_1$ ) plotted.

county_data$prcnt_unemployed_log <- log(county_data$prcnt_unemployed)

mymodel <- lm(prcnt_GOP16~ prcnt_unemployed_log * college_2cat_num, 
              data = county_data, weights = pop_estimate16)

plot_ly( data = county_data,
         x = ~prcnt_unemployed_log,
         y = ~college_2cat_num,
         z = ~prcnt_GOP16) %>%
  add_3d_surface(model = mymodel) %>%
  add_marginals(model = mymodel, 
                x1_constant_val = 2.75, x2_constant_val =0) %>%
  add_marginals(model = mymodel, 
                x1_constant_val = 1.25, x2_constant_val =1)

Click, hold, and drag the regression surface to spin it. See the article on linear models for more information on how to manipulate this graphic.

More on marginal effects

In a linear regression, the marginal effect of $x_1$ is constant for changing values of $x_2$ and vice versa. This is not true when an interaction term is added to the model. The next image highlights this fact by adjusting the image using the packages plotly, scales, and regress3d.

Using plotly functions and arguments, we add scatterpoints, a hovertemplate for each point, and better labels. We adjust the colors of the scatterpoints to emphasize the low (black) and high (blue) values of the education variable.
The package scales presents the numeric values and percentages in the hovertext more clearly.
regress3d lets us modify the marginal effects to highlight their changing values. In add_marginals(), we remove the marginal effect of $x_2$ (education) using omit_x2 and adjust the legend text by changing x1_direction_name. We also adjust the color of the lines using x1_color. As before, we adjust the location of the marginal effects using x2_constant_val.

library(scales)

county_data$prcnt_unemployed_log <- log(county_data$prcnt_unemployed)
mymodel <- lm(prcnt_GOP16~ prcnt_unemployed_log * college_2cat_num, 
              data = county_data, weights = pop_estimate16)

plot_ly(data = county_data, 
        x = ~prcnt_unemployed_log,
        y = ~college_2cat_num,
        z = ~prcnt_GOP16 ) %>%
  add_markers(size = ~pop_estimate16,
              hovertemplate = ~paste("<b>", county_state,"</b><br>",
                                     "County population: ",comma(pop_estimate16),
                                     "<br><br>", 
                                     "County unemployment (logged): ",
                                     round(prcnt_unemployed_log, 2),"<br>",
                                     "County unemployment: ",
                                     percent(prcnt_unemployed, accuracy = 0.1, scale = 1),"<br>",
                                     "County college experience: ",
                                     percent(any_college, accuracy = 0.1, scale = 1),"<br>",
                                     "Trump Vote %: ",
                                     percent(prcnt_GOP16, accuracy = 0.01, scale = 1),"<br>"),
              showlegend = F, name = "county",
              color = ~college_2cat,
              colors = c("deepskyblue", "dimgrey"))  %>%
  add_3d_surface(model = mymodel)%>%
  add_marginals(model = mymodel, ci = F,
                x2_constant_val =0,
                x1_color = "black",
                x1_direction_name = "Marginal effect of unemployment<br>for low education counties",
                omit_x2 = T)%>%
  add_marginals(model = mymodel, ci = F,
                x2_constant_val =1,
                x1_color = "blue",
                x1_direction_name = "Marginal effect of unemployment<br>for high education counties",
                omit_x2 = T) %>%
  layout( 
    title = "\nCounty Vote for Trump by\n unemployment & education",
    scene = list(xaxis = list(title = 'logged unemployment'),
                 yaxis = list(title = 'education'),
                 zaxis = list(title = '% vote for Trump 2016')),
    # width = 500, height = 500,
    legend = list(font = list(size = 8))
  )

Because $x_2$ (education) is binary, the numeric value of the marginal effect of $x_1$ (unemployment) when $x_2$ has the value of $0$ or $1$ is easily recoverable from the multiple regression with an interaction term. Either perform some algebra by substituting the values of 0 and 1 into the equation for predicted values, or estimate the regression on the subset of data where education has the value of 1, and again for 0. Note the following.

The estimated standard errors for unemployment in the simple regression estimated for college_2cat_num == 0 are different from those created in the multiple regression where college_2cat_num== 0. The same holds for college_2cat_num == 1.
The estimated value of the effect of unemployment in the simple regression estimated for college_2cat_num == 0 is the same as the marginal effect in the multiple regression for college_2cat_num == 0. The same holds for college_2cat_num == 1.

Numeric regression results

This graphic corresponds to the following numerical regression results. Three models are presented to allow the reader to examine three effects shown in the graphic.

The multiple regression equation with an interaction term for the regression surface in blue with grey confidence intervals.
A simple regression using only logged unemployment rate and counties below the national mean education level (low education) for the marginal effect shown with a black line. Note that while the black line reflects the effect size in the for both the simple regression and the multiple regression with an interaction term, the confidence intervals in the simple regression and multiple regression are different.
A simple regression using only logged unemployment rate and counties above the national mean education level (high education) for the marginal effect shown with an blue line. Note that while the blue line reflects the effect size in the for both the simple regression and the multiple regression with an interaction term, the confidence intervals in the simple regression and multiple regression are different.


	% of county that voted for Trump in 2016
	with interaction effect	only low education counties	only high education counties

% unemployed (logged)	-23.956^*	-23.956^*	-2.434
	(2.249)	(1.429)	(1.783)

County college experience (binary)	-57.255^*
	(4.618)

% unemployed (logged)*college experience	21.522^*
	(2.654)

Constant	103.939^*	103.939^*	46.684^*
	(4.081)	(2.593)	(2.735)


Observations	3,139	1,577	1,562
Adjusted R²	0.187	0.151	0.001

Note:	* p<0.05