I ran a Generalized Linear Mixed Model in R and included an interaction effect between two predictors. The interaction was not significant, but the main effects (the two predictors) both were. Now many textbook examples tell me that if there is a significant effect of the interaction, the main effects cannot be interpreted. But what if your interaction is not significant? Can I conclude that the two predictors have an effect on the response? Or is it better to run a new model where I leave out the interaction? I prefer not to do so, because I would then have to control for multiple testing.
$\begingroup$ If one of these answers works for you perhaps you might accept it or request a clarification. $\endgroup$
Commented Jul 23, 2012 at 14:31$\begingroup$ If the interaction is not significant, then you should drop it and run a regression without it. $\endgroup$
Commented Feb 6, 2015 at 15:56A little niggle
'Now many textbook examples tell me that if there is a significant effect of the interaction, the main effects cannot be interpreted'
I hope that's not true. They should say that if there is an interaction term, say between X and Z called XZ, then the interpretation of the individual coefficients for X and for Z cannot be interpreted in the same way as if XZ were not present. You can definitely interpret it.
Question 2
If the interaction makes theoretical sense then there is no reason not to leave it in, unless concerns for statistical efficiency for some reason override concerns about misspecification and allowing your theory and your model to diverge.
Given that you have left it in, then interpret your model using marginal effects in the same way as if the interaction were significant. For reference, I include a link to Brambor, Clark and Golder (2006) who explain how to interpret interaction models and how to avoid the common pitfalls.
Think of it this way: you often have control variables in a model that turn out not to be significant, but you don't (or shouldn't) go chopping them out at the first sign of missing stars.
Question 1
You ask whether you can 'conclude that the two predictors have an effect on the response?' Apparently you can, but you can also do better. For the model with the interaction term you can report what effect the two predictors actually have on the dependent variable (marginal effects) in a way that is indifferent to whether the interaction is significant, or even present in the model.
The Bottom Line
If you remove the interaction you are re-specifying the model. This may be a reasonable thing to do for many reasons, some theoretical and some statistical, but making it easier to interpret the coefficients is not one of them.
4,591 3 3 gold badges 33 33 silver badges 47 47 bronze badges answered Jun 24, 2012 at 19:29 conjugateprior conjugateprior 21.2k 2 2 gold badges 61 61 silver badges 89 89 bronze badges$\begingroup$ Sure. And if you're in R then you may find the package effects easier than working your way through the math, and also for generalisation to more complex models. $\endgroup$
Commented Jun 24, 2012 at 23:32 $\begingroup$ In your bottom line it depends on what you mean by 'easier'. $\endgroup$ Commented Jun 26, 2012 at 13:15$\begingroup$ Thank you so much for the Brambor, Clark and Golder (2006) reference! It's a very sane take at explaining interaction models. Very useful at understanding how to interpret (or NOT) the coefficients in such models. BTW, the paper comes with an internet appendix: Multiplicative Interaction Models, which comes in as a very handy overview of the discussion. $\endgroup$
Commented Feb 6, 2015 at 15:45$\begingroup$ Another interesting reference on this: tqmp.org/RegularArticles/vol16-1/p033/p033.pdf $\endgroup$
Commented Apr 24 at 14:52 $\begingroup$If you want the unconditional main effect then yes you do want to run a new model without the interaction term because that interaction term is not allowing you to see your unconditional main effects correctly. The main effects calculated with the interaction present are different from the main effects as one typically interprets them in something like ANOVA. For example, it's possible to have a trivial and non-signficant interaction the main effects won't be apparent when the interaction is in the model.
Let's say you have two predictors, A and B. When you include the interaction term then the magnitude of A is allowed to vary depending on B and vice versa. The reported beta coefficient in the regression output for A is then just one of many possible values. The default is to use the coefficient of A for the case when B is 0 and the interaction term is 0. But, when the regression is just additive A is not allowed to vary across B and you just get the main effect of A independent of B. These can be a very different values even if the interaction is trivial because they mean different things. The additive model is the only way to really assess the main effect by itself. On the other hand, when your interaction is meaningful (theoretically, not statistically) and you want to keep it in your model then the only way to assess A is looking at it across levels of B. That's actually the kind of thing you have to consider with respect to the interaction, not whether A is significant. You can only really see whether there's an unconditional effect of A in the additive model.
So, the models are looking at very different things and this is not an issue of multiple testing. You must look at it both ways. You don't decide based on significance. The best main effect to report is from the additive model. You make a decision on including or presenting the non significant interaction based on theoretical issues, or data presentation issues, etc.
(This is not to say that there are no potential multiple testing issues here. But what they mean depends a great deal on the theory driving the tests.)