Hello,
I am using Stata/MP 14.1 for Mac. I've been trying to analyze my data from repeated measure design experiment (n=6) with linear mixed models (Command mixed) under the setting like below.
- Dependent variable : volume_after
- Explanatory variable : approach (left, right), pressure (0, 5, 10, 15)
- Group variable : Index
And for better small sample analysis, I used "reml" and "dfmethod (kroger)" option described in Stata material.
(http://www.stata.com/meeting/columbus15/abstracts/materials/columbus15_yang.pdf)
I followed "top-down analysis" described in the textbook (Linear mixed models : A practical guide using statistical software, second edition).
Among three models described below, "lrtest" showed that Model 2. the better fitted model.
Model 1. mixed volume_after i.pressure approach_d i.pressure#c.approach_d || index: , covariance (identity) variance reml dfmethod(kroger)
Model 2. mixed volume_after i.pressure approach_d i.pressure#c.approach_d || index: pressure, covariance (unstruct) variance reml dfmethod(kroger)
Model 3. mixed volume_after i.pressure approach_d i.pressure#c.approach_d || index: pressure, covariance (unstruct) variance reml dfmethod(kroger) residuals(independent, by(approach_d))
* "approach_d" is a dummy variable for "approach"
The result of Model 2
Finally, I want to reduce non-significant fixed effect "approach_d" & "pressure#c.approach_d" with "testparm" command.
Here're questions...
1) With the "testparm" results above, can I just remove "i.pressure#c.approach_d" and get final model for estimates?
Final model(?)
mixed volume_after i.pressure i.pressure#approach_d ||index: pressure, covariance (identity) variance dfmethod(kroger) reml
2) If so, how can I understand my final model? I don't think I can say "approach is not relevant to volume_after".
3) I just tried Model 4. and compared with Final model (?) with AIC, BIC & "lrtest", which showed that final model is better than model 4.
** "lrtest" was performed without the option "reml" & "dfmethod (kroger)"
But, it seems that the result of Model 2. showed "approach_d" and "i.pressure#c.approach_d" is not signifincant"
How can I understand this result?
4)
I thought this problem is due to the small sample size,
I tried "Mann-Whitney U test" to remove "approach" factor before performing linear mixed model....then Model 4 will be my final model.
What are the limitations in this approach compared to the approach above? If I cannot increase my sample size, can I adopt this approach?
Model 4. mixed volume_after i.pressure || index: pressure, covariance (unstruct) variance reml dfmethod(kroger)
Thanks,
Jay
I am using Stata/MP 14.1 for Mac. I've been trying to analyze my data from repeated measure design experiment (n=6) with linear mixed models (Command mixed) under the setting like below.
- Dependent variable : volume_after
- Explanatory variable : approach (left, right), pressure (0, 5, 10, 15)
- Group variable : Index
And for better small sample analysis, I used "reml" and "dfmethod (kroger)" option described in Stata material.
(http://www.stata.com/meeting/columbus15/abstracts/materials/columbus15_yang.pdf)
I followed "top-down analysis" described in the textbook (Linear mixed models : A practical guide using statistical software, second edition).
Among three models described below, "lrtest" showed that Model 2. the better fitted model.
Model 1. mixed volume_after i.pressure approach_d i.pressure#c.approach_d || index: , covariance (identity) variance reml dfmethod(kroger)
Model 2. mixed volume_after i.pressure approach_d i.pressure#c.approach_d || index: pressure, covariance (unstruct) variance reml dfmethod(kroger)
Model 3. mixed volume_after i.pressure approach_d i.pressure#c.approach_d || index: pressure, covariance (unstruct) variance reml dfmethod(kroger) residuals(independent, by(approach_d))
* "approach_d" is a dummy variable for "approach"
The result of Model 2
Code:
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0: log restricted-likelihood = -69.838378
Iteration 1: log restricted-likelihood = -69.684868
Iteration 2: log restricted-likelihood = -69.680506
Iteration 3: log restricted-likelihood = -69.680503
Computing standard errors:
Computing degrees of freedom:
Mixed-effects REML regression Number of obs = 20
Group variable: index Number of groups = 6
Obs per group:
min = 3
avg = 3.3
max = 4
DF method: Kenward-Roger DF: min = 4.00
avg = 4.67
max = 5.28
F(7, 4.41) = 25.93
Log restricted-likelihood = -69.680503 Prob > F = 0.0023
---------------------------------------------------------------------------------------
volume_after | Coef. Std. Err. t P>|t| [95% Conf. Interval]
----------------------+----------------------------------------------------------------
pressure |
5 | 90.16037 65.08291 1.39 0.223 -75.89707 256.2178
10 | 368.853 121.7059 3.03 0.039 30.94312 706.7628
15 | 1015.671 187.8344 5.41 0.005 506.6321 1524.709
|
approach_d | -3.8654 28.93499 -0.13 0.899 -77.06403 69.33323
|
pressure#c.approach_d |
5 | -12.2128 92.04114 -0.13 0.899 -247.0535 222.6279
10 | 234.4865 172.1182 1.36 0.245 -243.3902 712.3632
15 | 31.96351 265.6379 0.12 0.910 -687.9255 751.8526
|
_cons | 17.02083 20.46012 0.83 0.441 -34.73841 68.78008
---------------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
index: Unstructured |
var(pressure) | 423.062 313.9038 98.81635 1811.253
var(_cons) | 190.4463 433.3242 2.20306 16463.38
cov(pressure,_cons) | -283.8496 388.1759 -1044.66 476.9611
-----------------------------+------------------------------------------------
var(Residual) | 1065.404 532.7942 399.7972 2839.153
------------------------------------------------------------------------------
LR test vs. linear model: chi2(3) = 17.49 Prob > chi2 = 0.0006
Note: LR test is conservative and provided only for reference.
. estat ic
Akaike's information criterion and Bayesian information criterion
-----------------------------------------------------------------------------
Model | Obs ll(null) ll(model) df AIC BIC
-------------+---------------------------------------------------------------
. | 20 . -69.6805 12 163.361 175.3098
-----------------------------------------------------------------------------
Note: N=Obs used in calculating BIC; see [R] BIC note.
Code:
. testparm i.pressure#c.approach_d
( 1) [volume_after]5.pressure#c.approach_d = 0
( 2) [volume_after]10.pressure#c.approach_d = 0
( 3) [volume_after]15.pressure#c.approach_d = 0
chi2( 3) = 28.59
Prob > chi2 = 0.0000
. testparm approach_d
( 1) [volume_after]approach_d = 0
chi2( 1) = 0.02
Prob > chi2 = 0.8937
. testparm i.pressure
( 1) [volume_after]5.pressure = 0
( 2) [volume_after]10.pressure = 0
( 3) [volume_after]15.pressure = 0
chi2( 3) = 107.60
Prob > chi2 = 0.0000
Here're questions...
1) With the "testparm" results above, can I just remove "i.pressure#c.approach_d" and get final model for estimates?
Final model(?)
mixed volume_after i.pressure i.pressure#approach_d ||index: pressure, covariance (identity) variance dfmethod(kroger) reml
2) If so, how can I understand my final model? I don't think I can say "approach is not relevant to volume_after".
Code:
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0: log restricted-likelihood = -70.546237
Iteration 1: log restricted-likelihood = -70.546229
Iteration 2: log restricted-likelihood = -70.546229
Computing standard errors:
Computing degrees of freedom:
Mixed-effects REML regression Number of obs = 20
Group variable: index Number of groups = 6
Obs per group:
min = 3
avg = 3.3
max = 4
DF method: Kenward-Roger DF: min = 4.21
avg = 6.30
max = 9.69
F(7, 7.13) = 29.66
Log restricted-likelihood = -70.546229 Prob > F = 0.0001
-------------------------------------------------------------------------------------
volume_after | Coef. Std. Err. t P>|t| [95% Conf. Interval]
--------------------+----------------------------------------------------------------
pressure |
5 | 90.16037 58.01232 1.55 0.165 -47.3103 227.631
10 | 368.853 104.6488 3.52 0.019 92.36201 645.3439
15 | 1007.622 161.5402 6.24 0.001 594.131 1421.112
|
pressure#approach_d |
0 1 | -3.8654 32.23463 -0.12 0.907 -76.00165 68.27085
5 1 | -16.0782 78.07926 -0.21 0.845 -213.4368 181.2804
10 1 | 230.6211 145.8365 1.58 0.185 -166.5587 627.8009
15 1 | 24.77871 227.0594 0.11 0.917 -562.6496 612.207
|
_cons | 17.02083 22.79332 0.75 0.473 -33.9872 68.02887
-------------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
index: Identity |
var(pressure _cons) | 303.438 195.4811 85.84362 1072.585
-----------------------------+------------------------------------------------
var(Residual) | 1255.169 657.3175 449.7154 3503.214
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 15.76 Prob >= chibar2 = 0.0000
3) I just tried Model 4. and compared with Final model (?) with AIC, BIC & "lrtest", which showed that final model is better than model 4.
** "lrtest" was performed without the option "reml" & "dfmethod (kroger)"
Code:
. lrtest model_final model4 Likelihood-ratio test LR chi2(8) = 22.32 (Assumption: model4 nested in model_final) Prob > chi2 = 0.0044
How can I understand this result?
4)
I thought this problem is due to the small sample size,
I tried "Mann-Whitney U test" to remove "approach" factor before performing linear mixed model....then Model 4 will be my final model.
What are the limitations in this approach compared to the approach above? If I cannot increase my sample size, can I adopt this approach?
Model 4. mixed volume_after i.pressure || index: pressure, covariance (unstruct) variance reml dfmethod(kroger)
Code:
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0: log restricted-likelihood = -98.097027
Iteration 1: log restricted-likelihood = -97.956087
Iteration 2: log restricted-likelihood = -97.946855
Iteration 3: log restricted-likelihood = -97.946798
Iteration 4: log restricted-likelihood = -97.946784
Iteration 5: log restricted-likelihood = -97.94678
Computing standard errors:
Computing degrees of freedom:
Mixed-effects REML regression Number of obs = 20
Group variable: index Number of groups = 6
Obs per group:
min = 3
avg = 3.3
max = 4
DF method: Kenward-Roger DF: min = 5.19
avg = 6.78
max = 8.83
F(3, 7.27) = 25.98
Log restricted-likelihood = -97.94678 Prob > F = 0.0003
------------------------------------------------------------------------------
volume_after | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
pressure |
5 | 84.05397 54.81305 1.53 0.160 -40.29687 208.4048
10 | 486.0962 92.6376 5.25 0.003 250.5715 721.6209
15 | 1023.594 148.682 6.88 0.000 659.7686 1387.42
|
_cons | 15.08813 25.43626 0.59 0.571 -44.90898 75.08525
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
index: Unstructured |
var(pressure) | 446.1804 324.3317 107.3411 1854.62
var(_cons) | 445.8629 1055.327 4.309887 46125.04
cov(pressure,_cons) | -446.0216 638.208 -1696.886 804.8431
-----------------------------+------------------------------------------------
var(Residual) | 3436.156 1472.698 1483.397 7959.545
------------------------------------------------------------------------------
LR test vs. linear model: chi2(3) = 11.67 Prob > chi2 = 0.0086
Note: LR test is conservative and provided only for reference.
. estat ic
Akaike's information criterion and Bayesian information criterion
-----------------------------------------------------------------------------
Model | Obs ll(null) ll(model) df AIC BIC
-------------+---------------------------------------------------------------
. | 20 . -97.94678 8 211.8936 219.8594
-----------------------------------------------------------------------------
Note: N=Obs used in calculating BIC; see [R] BIC note.
Jay