Pyrczak and Oh’s (2018) Making Sense of Statistics text, pages 132–133.
Chapter Objectives
The reader will be able to:
n Know how the results of
t tests are reported in various widely used
forms such as a table or statement using various phrases and
wordings
n Know the distinction between statistically significant and
practical significance
In Chapters 21–24 the use of the t test to test the difference between
two sample means for significance was considered. Obviously, the
values of the means should be reported before the results of the statistical
test performed on them are reported. In addition, the values of the
standard deviations and the number of cases in each group should be
reported first. This may be done within the context of a sentence or in
a table. Table 25.1 shows a typical table.
The samples for Groups A and B were drawn at random. The null
hypothesis states that the 3.50-point difference (6.00 – 2.50 = 3.50)
between the means of 2.50 and 6.00 is the result of sampling errors (i.e.,
errors resulting from random sampling) and that the true difference in
the population is zero.
Table 25.1 Means and Standard Deviations
m s N
Group A 2.50 1.87 6
Group B 6.00 1.89 6
Pyrczak, Fred, and Deborah M. Oh. Making Sense of Statistics : A Conceptual Overview, Taylor & Francis Group, 2018. ProQuest Ebook Central,
http://ebookcentral.proquest.com/lib/capella/detail.action?docID=5425416.
Created from capella on 2022-07-31 15:40:47.
Copyright © 2018. Taylor & Francis Group. All rights reserved.
Part F: Means Comparison
144
This is one way to
report the result of a
significant
t test.
The phrase
significant
at the .01 level indi-
cates that
p was equal
to or less than .01.Rejecting the null
hypothesis is the same
as
declaring statistical
significance.
In Example 2, the researcher has used a slightly different wording
to indicate that significance was obtained at the .01 level. The phrase
significant at the .01 level indicates that p was equal to or less than .01,
which is the probability that the null hypothesis is correct. Thus, the
null hypothesis was rejected.
Example 3 below provides the same information as Examples 1
and 2 but with a different wording. The sentence indicates that the
difference is statistically significant because rejecting the null hypothesis
is the same as declaring statistical significance.
Example 1
The difference between the means is statistically significant
(t = 3.22, df = 10, p < .01).
Example 2
The difference between the means is significant at the .01 level
(t = 3.22, df = 10).
Example 3
The null hypothesis was rejected at the .01 level
(t = 3.22, df = 10).
Any of the forms of expression illustrated in the previous three
examples is acceptable. However, authors of journal articles seldom
explicitly mention the null hypothesis. Instead, they tend to use the forms
of expression in Examples 1 and 2. In theses and dissertations, in
contrast, explicit references to the null hypothesis are more common.
The result of a significant t test may be described in several ways.
Below are some examples for the results in Table 25.1. The statement
in Example 1 below implies that the null hypothesis has been rejected
because the term statistically significant is synonymous with rejecting
the null hypothesis.
Statistically significant
is synonymous with
rejecting the null
hypothesis.
Pyrczak, Fred, and Deborah M. Oh. Making Sense of Statistics : A Conceptual Overview, Taylor & Francis Group, 2018. ProQuest Ebook Central,
http://ebookcentral.proquest.com/lib/capella/detail.action?docID=5425416.
Created from capella on 2022-07-31 15:40:47.
Copyright © 2018. Taylor & Francis Group. All rights reserved.
When researchers use the word significant in reporting the results
of significance tests, they should modify it with the adjective statistically.
This is because a result may be statistically significant but not of any
practical significance. For instance, suppose a researcher found a statis-
tically significant difference of 2 points in favor of a computer-assisted
approach over a traditional lecture/textbook approach. While it is
statistically significant, it may not be of practical significance if the
school district has to invest sizeable amounts of money to buy new
hardware and software. In other words, the cost of the difference may
be too great in light of the absolute size of the benefit.1
Now, consider how researchers report the results of a t test when
the difference between means is not statistically significant. Table 25.2
presents descriptive statistics. Examples 4 through 6 show some ways
to express the results of the insignificant t test for the data in the table.
The fact that p is greater than (>) .05 in Example 4 below indicates
that the null hypothesis was not rejected.
Chapter 25: Reports of the Results of
t Tests
145
The abbreviation
n.s.
means
not significant.
It is best to indicate
the specific probability
level at which the null
hypothesis was not
rejected.
Example 4
The difference between the means is not statistically significant
(t = 1.80, df = 12, p > .05).
Example 5
For the difference between the means, t = 1.80 (df = 12, n.s.).
A result may be
statistically significant
but not of any
practical significance.
Table 25.2 Means and Standard Deviations
m S N
Group A 8.14 2.19 7
Group B 5.71 2.81 7
The author of Example 5 has used the abbreviation n.s. to indicate
that the difference is not significant. Because the example does not
indicate a specific probability level, most readers will assume that it was
not significant at the .05 level—the most liberal of the widely used levels.2
Example 4 is preferable to Example 5 because the former indicates the
specific probability level that was used to test the null hypothesis.
Pyrczak, Fred, and Deborah M. Oh. Making Sense of Statistics : A Conceptual Overview, Taylor & Francis Group, 2018. ProQuest Ebook Central,
http://ebookcentral.proquest.com/lib/capella/detail.action?docID=5425416.
Created from capella on 2022-07-31 15:40:47.
Copyright © 2018. Taylor & Francis Group. All rights reserved.
Example 6 shows how the results of the test might be expressed
with explicit reference to the null hypothesis.
While reading journal articles, theses, and dissertations, you will
find variations in the exact words used to describe the results of t tests.
The examples in this chapter illustrate some of the most widely used
forms of expression.
Part F: Means Comparison
146
Example 6
The null hypothesis for the difference between the means was not
rejected at the .05 level (t = 1.80, df = 12).
Exercise for Chapter 25
Factual Questions
1. Which statistics should be reported before the results of a t test are reported?
2. Suppose you read this statement: “The difference between the means is statistically
significant at the .05 level (t = 2.333, df = 11).” Should you conclude that the null
hypothesis has been rejected?
3. Suppose you read this statement: “The null hypothesis was rejected (t = 2.810, df = 40,
p < .01).” Should you conclude that the difference is statistically significant?
4. Suppose you read this statement: “The null hypothesis was not rejected (t = –.926,
df = 24, p > .05).” Describe in words the meaning of the statistical term “p > .05.”
5. For the statement in Question 4, should you conclude that the difference is statistically
significant?
6. Suppose you read this statement: “For the difference between the means, t = 2.111 (df = 5,
n.s.).” Should you conclude that the null hypothesis has been rejected?
7. Which type of author seldom explicitly mentions the null hypothesis?
A. Authors of dissertations
B. Authors of journal articles
