diff --git a/Comp/r-sas_ttest_1Sample.qmd b/Comp/r-sas_ttest_1Sample.qmd index caad67cd2..58986bb99 100644 --- a/Comp/r-sas_ttest_1Sample.qmd +++ b/Comp/r-sas_ttest_1Sample.qmd @@ -46,7 +46,7 @@ library(procs) # proc_ttest() ## Data -### Lognormal One Sample t-test +### Normal One Sample t-test and Lognormal One Sample t-test Sample Data We use a simulated sample data containing one numeric variable, `score`. The same data are used for the R and SAS examples. @@ -104,11 +104,14 @@ The following table shows the types of One Sample t-test analysis, the capabilit Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST`: -| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | -|--------------------|-----------|--------------|------------|-------|-------| -| Degrees of Freedom | 29 | 29 | 29 | Yes | | -| t value | 2.364306 | 2.364306 | 2.364306 | Yes | | -| p value | 0.0249741 | 0.0249741 | 0.0249741 | Yes | | +| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | +|--------------------|----------|--------------|------------|-------|-------| +| Degrees of Freedom | 43 | 43 | 43 | Yes | | +| t value | 2.8727 | 2.8727 | 2.87 | Yes | | +| p value | 0.006297 | 0.006297 | 0.0063 | Yes | | +| Mean | 34.8636 | 34.8636 | 34.8636 | Yes | | +| Lower 95% CL Mean | 31.44930 | 31.44930 | 31.44930 | Yes | | +| Upper 95% CL Mean | 38.27797 | 38.27797 | 38.2780 | Yes | | ### Lognormal Data {#lognormal} @@ -118,12 +121,12 @@ Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS | Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | |------------|------------|------------|------------|------------|------------| -| Degrees of Freedom | 29 | 29 | 29 | Yes | Calculated on log-transformed data | -| t value | 1.351675 | 1.351675 | 1.351675 | Yes | Calculated on log-transformed data | -| p value | 0.1869289 | 0.1869289 | 0.1869289 | Yes | Calculated on log-transformed data | -| Geometric Mean | 32.91447 | 32.91447 | 32.91447 | Yes | Back-transformed mean | -| Lower 95% CL Mean | 28.60623 | 28.60622 | 28.60623 | Yes | Back-transformed lower confidence limit | -| Upper 95% CL Mean | 37.87155 | 37.87155 | 37.87155 | Yes | Back-transformed upper confidence limit | +| Degrees of Freedom | 43 | 43 | 43 | Yes | Calculated on log-transformed data | +| t value | 1.6373 | 1.6373 | 1.64 | Yes | Calculated on log-transformed data | +| p value | 0.1089 | 0.1089 | 0.1089 | Yes | Calculated on log-transformed data | +| Geometric Mean | 32.84336 | 32.84336 | 32.8434 | Yes | Back-transformed mean | +| Lower 95% CL Mean | 29.3770 | 29.3770 | 29.3770 | Yes | Back-transformed lower confidence limit | +| Upper 95% CL Mean | 36.7187 | 36.7187 | 36.7187 | Yes | Back-transformed upper confidence limit | # Summary and Recommendation diff --git a/Comp/r-sas_ttest_2Sample.qmd b/Comp/r-sas_ttest_2Sample.qmd index f64d2c364..8a6fa7dbb 100644 --- a/Comp/r-sas_ttest_2Sample.qmd +++ b/Comp/r-sas_ttest_2Sample.qmd @@ -1,21 +1,105 @@ --- -title: "R vs SAS Two Sample T-Test" +title: "R vs SAS Two-Sample T-Test" --- -```{r} -#| label: setup +```{r setup} #| include: false knitr::opts_chunk$set(echo = TRUE) ``` +# Summary + +## Goal + +The goal of this comparison is to evaluate whether two-sample t-tests produce equivalent results in R and SAS for both normal and lognormal data. For normal data, results from `stats::t.test()`, `procs::proc_ttest()`, and SAS `PROC TTEST` are compared directly. For lognormal data, results from a log-transformation / back-transformation approach in R are compared with SAS `PROC TTEST` using the `DIST=LOGNORMAL` option. + +## Scope + +::::::: columns +:::: {.column width="45%"} +::: {.callout-note appearance="minimal" collapse="false"} +## Methodologies + +Two-sample t-test + +Lognormal two-sample t-test +::: +:::: + +:::: {.column width="55%"} +::: {.callout-note appearance="minimal" collapse="false"} +## Technical implementations + +SAS: `PROC TTEST` with and without the `DIST=LOGNORMAL` option + +R: `stats::t.test()` and log-transformation / back-transformation approach for lognormal data +::: +:::: +::::::: + +# Prerequisites + +## R packages + +```{r load-packages} +#| output: false +#| message: false +library(tibble) # for example data +library(stats) # t.test() +library(procs) # proc_ttest() +``` + +## Data + +### Normal two-sample t-test and Lognormal two-sample t-test Sample Data + +We use a simulated two-sample dataset containing weight gain measurements for two treatment groups. The same data are used in the R and SAS examples to compare the log-transformation / back-transformation approach in R with SAS `PROC TTEST` using the `DIST=LOGNORMAL` option. + +R: + +```{r} +d1 <- tibble::tribble( + ~trt_grp, ~WtGain, + "placebo", 94, "placebo", 12, "placebo", 26, "placebo", 89, + "placebo", 88, "placebo", 96, "placebo", 85, "placebo", 130, + "placebo", 75, "placebo", 54, "placebo", 112, "placebo", 69, + "placebo", 104, "placebo", 95, "placebo", 53, "placebo", 21, + "treatment", 45, "treatment", 62, "treatment", 96, "treatment", 128, + "treatment", 120, "treatment", 99, "treatment", 28, "treatment", 50, + "treatment", 109, "treatment", 115, "treatment", 39, "treatment", 96, + "treatment", 87, "treatment", 100, "treatment", 76, "treatment", 80 +) +``` + +SAS: + +``` sas +data d1; + length trt_grp $ 9; + input trt_grp $ WtGain @@; + datalines; +placebo 94 placebo 12 placebo 26 placebo 89 +placebo 88 placebo 96 placebo 85 placebo 130 +placebo 75 placebo 54 placebo 112 placebo 69 +placebo 104 placebo 95 placebo 53 placebo 21 +treatment 45 treatment 62 treatment 96 treatment 128 +treatment 120 treatment 99 treatment 28 treatment 50 +treatment 109 treatment 115 treatment 39 treatment 96 +treatment 87 treatment 100 treatment 76 treatment 80 +; + +run; +``` + # Two Sample t-test Comparison The following table shows the types of Two Sample t-test analysis, the capabilities of each language, and whether or not the results from each language match. -| Analysis | Supported in R | Supported in SAS | Results Match | Notes | +| Analysis | Supported in R | Supported in SAS | Results Match | Notes | |---------------|---------------|---------------|---------------|---------------| -| Two sample Student's t-test | [Yes](../R/ttest_2Sample.html#baseS) | [Yes](../SAS/ttest_2Sample.html#sas) | [Yes](#student) | In Base R, use `t.test()` function with `paired = FALSE` and `var.equal = TRUE`| -| Two sample Welch's t-test | [Yes](../R/ttest_2Sample.html#baseW) | [Yes](../SAS/ttest_2Sample.html#sas) | [Yes](#welch) | In Base R, use `t.test()` function with `paired = FALSE` and `var.equal = FALSE` | +| Two sample Student's t-test | [Yes](../R/ttest_2Sample.html#baseS) | [Yes](../SAS/ttest_2Sample.html#sas) | [Yes](#student) | In Base R, use `t.test()` function with `paired = FALSE` and `var.equal = TRUE` | +| Two sample Welch's t-test | [Yes](../R/ttest_2Sample.html#baseW) | [Yes](../SAS/ttest_2Sample_W.html#sas) | [Yes](#welch) | In Base R, use `t.test()` function with `paired = FALSE` and `var.equal = FALSE` | +| Two sample Lognormal t-test | [Yes](../R/ttest_2Sample.html#baseWlog) | [Yes](../SAS/ttest_2Sample_lognormal.html#sas) | [Yes](#lognormal) | | ## Comparison Results @@ -23,43 +107,54 @@ The following table shows the types of Two Sample t-test analysis, the capabilit Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST`: -| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | -|--------------------|-----------|--------------|------------|-------|-------| -| Degrees of Freedom | 30 | 30 | 30 | Yes | | -| t value | -0.6969002 | -0.6969002 | -0.6969002 | Yes | | -| p value | 0.4912306 | 0.4912306 | 0.4912306 | Yes | | - +| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | +|------------|------------|------------|------------|------------|------------| +| Degrees of Freedom | 30 | 30 | 30 | Yes | | +| t value | -0.6969 | -0.70 | -0.70 | Yes | | +| p value | 0.4912 | 0.4912 | 0.4912 | Yes | | +| Mean Difference | -7.9375 | -7.9375 | -7.9375 | Yes | | +| Lower 95% CL Mean Difference | -31.1984 | -31.1984 | -31.1984 | Yes | | +| Upper 95% CL Mean Difference | 15.3234 | 15.3234 | 15.3234 | Yes | | ### Welch's T-Test {#welch} In the Welch T-test the variance and effective degrees of freedom are calculated using Satterthwaite method. -Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST`for two example: +Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST` for this example. -Example with fairly equal variances: -| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | -|--------------------|-----------|--------------|------------|-------|-------| -| Degrees of Freedom | 29.69359 | 29.69359 | 29.69359 | Yes | | -| t value | -0.6969002 | -0.6969002 | -0.6969002 | Yes | | -| p value | 0.4912856 | 0.4912856 | 0.4912856 | Yes | | +Example with unequal variances: +| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | +|------------|------------|------------|------------|------------|------------| +| Degrees of Freedom | 29.694 | 29.694 | 29.694 | Yes | | +| t value | -0.6969 | -0.70 | -0.70 | Yes | | +| p value | 0.4913 | 0.4913 | 0.4913 | Yes | | +| Mean Difference | -7.9375 | -7.9375 | -7.9375 | Yes | | +| Lower 95% CL Mean Difference | -31.2085 | -31.2085 | -31.2085 | Yes | | +| Upper 95% CL Mean Difference | 15.3335 | 15.3335 | 15.3335 | Yes | | -Example with unequal variances: -| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | -|--------------------|-----------|--------------|------------|-------|-------| -| Degrees of Freedom | 18.137 | 18.137 | 18.137 | Yes | | -| t value | -1.54 | -1.54 | -1.54 | Yes | | -| p value | 0.1413 | 0.1413 | 0.1413 | Yes | | +### Lognormal Data {#lognormal} + +For lognormal data for two independent groups, a two-sample t-test can be performed in R by applying a natural log transformation to the measurements in each group and conducting the analysis on the log-transformed values. The resulting mean difference and confidence limits can then be exponentiated to obtain a geometric mean ratio and corresponding confidence limits. The table below compares results from `stats::t.test()`, `procs::proc_ttest()`, and SAS `PROC TTEST` with the `DIST=LOGNORMAL` option. + +Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST`: +| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | +|------------|------------|------------|------------|------------|------------| +| Degrees of Freedom | 30 | 30 | 30 | Yes | | +| t value | -0.8763 | -0.88 | -0.88 | Yes | | +| p value | 0.3878 | 0.3878 | 0.3878 | Yes | | +| Geometric Mean Ratio | 0.8381 | 0.8381 | 0.8381 | Yes | Back-transformed mean difference | +| Lower 95% CL Mean Ratio | 0.5553 | 0.5553 | 0.5553 | Yes | Back-transformed lower confidence limit | +| Upper 95% CL Mean Ratio | 1.265 | 1.2649 | 1.2649 | Yes | Back-transformed upper confidence limit | # Summary and Recommendation -For the Student's T-Test, the R two sample `t.test()` and **procs** `proc_ttest()` capabilities are comparable to SAS. -Comparison between SAS and R show identical results for the datasets tried. +For the Student's T-Test, the R two-sample `t.test()` and **procs** `proc_ttest()` capabilities are comparable to SAS. Comparison between SAS and R show identical results for the datasets tried. The **procs** package `proc_ttest()` function is very similar to SAS in the syntax and output produced. The `proc_ttest()` also supports by groups, where `t.test()` does not. -Likewise, for the Welch's T-Test, the R two sample `t.test()` and **procs** `proc_ttest()` capabilities are comparable to SAS. -Comparison between SAS and R show identical results for the datasets tried. +Likewise, for the Welch's T-Test, the R two-sample `t.test()` and **procs** `proc_ttest()` capabilities are comparable to SAS. Comparison between SAS and R show identical results for the datasets tried. +For the lognormal version of the two-sample t-test, equivalent results can be obtained in R by applying a natural log transformation to the measurements in each group, performing the two-sample t-test on the transformed data, and exponentiating the resulting estimates and confidence limits. Comparison with SAS `PROC TTEST` using the `DIST=LOGNORMAL` option showed that the results matched after back-transformation. Therefore, a separate package is not required for this analysis. # References @@ -67,4 +162,4 @@ R `t.test()` documentation: -SAS `PROC TTEST` Two Sample analysis documentation: +SAS `PROC TTEST` Two-Sample analysis documentation: diff --git a/Comp/r-sas_ttest_Paired.qmd b/Comp/r-sas_ttest_Paired.qmd index 9934277c0..4cd8ce9eb 100644 --- a/Comp/r-sas_ttest_Paired.qmd +++ b/Comp/r-sas_ttest_Paired.qmd @@ -46,7 +46,7 @@ library(procs) # proc_ttest() ## Data -### Lognormal One Sample t-test +### Normal Paired t-test and Lognormal Paired t-test Sample Data We use a simulated paired dataset containing systolic blood pressure measurements before and after treatment. The same data are used in the R and SAS examples to compare the log-transformation / back-transformation approach in R with SAS `PROC TTEST` using the `DIST=LOGNORMAL` option. @@ -97,9 +97,9 @@ library(procs) The following table shows the types of Paired t-test analysis, the capabilities of each language, and whether or not the results from each language match. | Analysis | Supported in R | Supported in SAS | Results Match | Notes | -|----|----|----|----|----| +|---------------|---------------|---------------|---------------|---------------| | Paired t-test, normal data | [Yes](../R/ttest_Paired.html#normal) | [Yes](../SAS/ttest_Paired.html#normal) | [Yes](#normal) | In Base R, use `paired = TRUE` on `t.test()` function | -| Paired t-test, lognormal data | [Yes](../R/ttest_Paired.html#lognormal) | [Yes](../SAS/ttest_Paired.html#lognormal) | [NA](#lognormal) | Apply a natural log transformation to both paired measurements, perform the paired t-test on the log-transformed data, and exponentiate the mean difference and the confidence limits. | +| Paired t-test, lognormal data | [Yes](../R/ttest_Paired.html#lognormal) | [Yes](../SAS/ttest_Paired.html#lognormal) | [Yes](#lognormal) | Apply a natural log transformation to both paired measurements, perform the paired t-test on the log-transformed data, and exponentiate the mean difference and the confidence limits. | ## Comparison Results @@ -107,11 +107,14 @@ The following table shows the types of Paired t-test analysis, the capabilities Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST`: -| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | -|--------------------|-----------|--------------|------------|-------|-------| -| Degrees of Freedom | 11 | 11 | 11 | Yes | | -| t value | -1.089648 | -1.089648 | -1.089648 | Yes | | -| p value | 0.2992 | 0.2992 | 0.2992 | Yes | | +| Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | +|-----------------|-----------|-----------|-----------|-----------|-----------| +| Degrees of Freedom | 11 | 11 | 11 | Yes | | +| t value | -1.089648 | -1.089648 | -1.09 | Yes | | +| p value | 0.2992 | 0.2992 | 0.2992 | Yes | | +| Mean Difference | -1.8333 | -1.8333 | -1.8333 | Yes | | +| Lower 95% CL Mean Difference | -5.5365 | -5.5365 | -5.5365 | Yes | | +| Upper 95% CL Mean Difference | 1.8698 | 1.8698 | 1.8698 | Yes | | ### Lognormal Data {#lognormal} @@ -120,13 +123,13 @@ For lognormal paired data, a paired t-test can be performed in R by applying a n Here is a table of comparison values between `t.test()`, `proc_ttest()`, and SAS `PROC TTEST`: | Statistic | t.test() | proc_ttest() | PROC TTEST | Match | Notes | -|----|----|----|----|----|----| +|------------|------------|------------|------------|------------|------------| | Degrees of Freedom | 11 | 11 | 11 | Yes | | -| t value | 1.094185 | 1.094185 | 1.094185 | Yes | | -| p value | 0.2972 | 0.2992 | 0.2972 | Yes | | -| Geometric Mean Ratio | 1.0146204 | 1.0146204 | 1.0146204 | Yes | Back-transformed mean difference | -| Lower 95% CL Mean Ratio | 0.98542 | 0.98542 | 0.98542 | Yes | Back-transformed lower confidence limit | -| Upper 95% CL Mean Ratio | 1.04468 | 1.04468 | 1.04468 | Yes | Back-transformed upper confidence limit | +| t value | -1.094185 | -1.094185 | -1.09 | Yes | | +| p value | 0.2973 | 0.2973 | 0.2973 | Yes | | +| Geometric Mean Ratio | 0.9856 | 0.9856 | 0.9856 | Yes | Back-transformed mean difference | +| Lower 95% CL Mean Ratio | 0.9572 | 0.9572 | 0.9572 | Yes | Back-transformed lower confidence limit | +| Upper 95% CL Mean Ratio | 1.0148 | 1.0148 | 1.0148 | Yes | Back-transformed upper confidence limit | # Summary and Recommendation diff --git a/R/ttest_2Sample.qmd b/R/ttest_2Sample.qmd index 9e0882caf..267382242 100644 --- a/R/ttest_2Sample.qmd +++ b/R/ttest_2Sample.qmd @@ -77,7 +77,6 @@ stats::t.test(d1p$WtGain, d1t$WtGain, var.equal = FALSE, paired = FALSE) ``` - ## Procs Package ### Student's T-Test and Welch's T-Test {#procs} @@ -105,7 +104,6 @@ Viewer Output: knitr::include_graphics("../images/ttest/twosample_rtest1.png") ``` - ## Example with unequal variances ```{r} @@ -140,8 +138,6 @@ Viewer Output: knitr::include_graphics("../images/ttest/twosample_rtest2.png") ``` - - ::: {.callout-note collapse="true" title="Session Info"} ```{r} #| echo: false @@ -154,4 +150,29 @@ si <- sessioninfo::session_info(c( )) si ``` -::: \ No newline at end of file +::: + +## Lognormal Data {#lognormal} + +The Base R `t.test()` function does not have an option for lognormal data. Likewise, the **procs** `proc_ttest()` function also does not have an option for lognormal data. + +Although Base R `t.test()` does not include a lognormal option, lognormal data can be analyzed by applying a natural log transformation to the data, performing a two-sample t-test on the log-transformed data, and exponentiating the group means and confidence limits. The back-transformed difference is interpreted as a ratio of geometric means. The standard deviation is not back-transformed. + +The following example applies a natural log transformation, performs the two-sample t-test on the log-transformed data, and exponentiates the mean and confidence limits. + +```{r} +# Apply natural log transformation to outcome variable +d1$log_WtGain <- log(d1$WtGain) + +# Perform two-sample t-test on log-transformed data +tt <- t.test(log_WtGain ~ trt_grp, data = d1) + +# Back-transform log means using exponential function +gm <- tapply(d1$WtGain, d1$trt_grp, function(x) exp(mean(log(x)))) + +# Back-transform confidence interval limits +ci <- exp(tt$conf.int) + +# Extract p-value from t-test result +p_value <- tt$p.value +``` diff --git a/SAS/ttest_2Sample.qmd b/SAS/ttest_2Sample.qmd index aed78b195..b0db1bda0 100644 --- a/SAS/ttest_2Sample.qmd +++ b/SAS/ttest_2Sample.qmd @@ -13,7 +13,7 @@ knitr::opts_chunk$set(echo = TRUE) The following data was used in this example. -```sas +``` sas data d1; length trt_grp $ 9; input trt_grp $ WtGain @@; @@ -34,12 +34,11 @@ run; The null hypothesis of the Independent Samples t-test is, the means for the two populations are equal. -In SAS the following code was used to test the mean comparison (mean of Weight Gain) of two independent treatment groups (Treatment and Placebo). -Both the Student's t-test and Welch's t-test (the Satterthwaite approximation is used to calculate the effective degrees of freedom and variance) in the same call. +In SAS the following code was used to test the mean comparison (mean of Weight Gain) of two independent treatment groups (Treatment and Placebo). Both the Student's t-test and Welch's t-test (the Satterthwaite approximation is used to calculate the effective degrees of freedom and variance) in the same call. For this example, we're testing the significant difference in mean of Weight gain (*WtGain*) between treatment and placebo (*trt_grp*) using PROC TTEST procedure in SAS. -```sas +``` sas proc ttest data=d1; class trt_grp; var WtGain; @@ -67,7 +66,7 @@ Note: Before entering straight into the t-test we need to check whether the assu 1. Normality: You can check for data to be normally distributed by plotting a histogram of the data by treatment. Alternatively, you can use the Shapiro-Wilk test or the Kolmogorov-Smirnov test. If the test is \<0.05 and your sample is quite small then this suggests you should not use the t-test. However, if your sample in each treatment group is large (say \>30 in each group), then you do not need to rely so heavily on the assumption that the data have an underlying normal distribution in order to apply the two-sample t-test. This is where plotting the data using histograms can help to support investigation into the normality assumption. We have checked the normality of the observations using the code below. Here for both the treatment groups we have P value greater than 0.05 (Shapiro-Wilk test is used), therefore the normality assumption is there for our data. -```sas +``` sas proc univariate data=d1 normal; qqplot WtGain; by trt_grp; @@ -113,12 +112,11 @@ Output: knitr::include_graphics("../images/ttest/variance_sas.png") ``` - ## Example with unequal variances For this example, it is important to use the Welch's t-test (the Satterthwaite approximation is used to calculate the effective degrees of freedom and variance) results. -```sas +``` sas data d2; length trt_grp $ 9; input trt_grp $ WtGain @@; @@ -141,7 +139,7 @@ run; Output: -```default +``` default Figure 1: Test results for independent t-test using PROC TTEST in SAS ``` @@ -152,3 +150,28 @@ Output: knitr::include_graphics("../images/ttest/test2.png") ``` +## Lognormal Data {#lognormal} + +The SAS two-sample t-test also supports lognormal analysis for a two-sample t-test. + +### Code + +Using the same d1 dataset as above, we will set the "DIST" option to "lognormal" and specify a CLASS variable to perform this analysis: + +``` sas +proc ttest data=d1 dist=lognormal; + class trt_grp; + var wtgain; +run; +``` + +Output: + +```{r} +#| echo: false +#| fig-align: center +#| out-width: 60% +knitr::include_graphics("../images/ttest/twosample_test2.png") +``` + +As can be seen in the figure above, the lognormal variation of the two-sample t-test provides results for geometric mean by group, geometric mean ratio, coefficient of variation, and 95% confidence limits for the ratio and coefficient of variation. diff --git a/images/ttest/twosample_test2.png b/images/ttest/twosample_test2.png new file mode 100644 index 000000000..7dec4dc29 Binary files /dev/null and b/images/ttest/twosample_test2.png differ