D2.Psy.6.9-12: Collecting and Analyzing Data With Descriptive and Inferential Statistics

Overview

Lesson Procedure

60-70 minutes of class time across 5 phases.

1. Warm-Up5-10 minutes
2. Direct Instruction20 minutes
3. Guided Practice15 minutes
4. Independent Practice15 minutes
5. Closure5-10 minutes

1. Warm-Up5-10 minutes

   Warm-Up Prompt

   "A classmate says he can tell by feel which of two identical-looking coins is slightly heavier. In a test of 10 tries, he picks the heavier one 7 times. Are you convinced? What would you want to know before you believed him?"

   Collect answers. Students usually ask how the test was run (could he see a mark? did the tester know which coin was which?), and some ask "what would someone get by guessing?" Keep that last question on the board: it is the starting point of every inferential test in this lesson. Answer it at the end of Direct Instruction (Example 4).

2. Direct Instruction20 minutes

   A psychological question becomes answerable when it names what will be measured and what result would count as an answer. The data then get two kinds of treatment. Descriptive statistics summarize what happened in this sample. Inferential statistics ask whether the pattern is strong enough that chance alone is an unlikely explanation, so that it says something beyond these particular trials or people.

   The statistics toolkit for this lesson (examples written for this page)

   Tool	Kind	What it tells you
   Percentage (proportion)	Descriptive	The share of cases with some result, such as the percent of judgments that were right
   Mean	Descriptive	The sum of the values divided by how many there are; pulled by extreme values
   Median	Descriptive	The middle value once the data are in order; resistant to extreme values
   Range	Descriptive	Largest value minus smallest value: a quick measure of spread
   Standard deviation	Descriptive	The typical distance of values from the mean: a fuller measure of spread
   Null hypothesis	Inferential	The chance explanation: there is no real effect, and any pattern is luck
   Chance model	Inferential	What results look like if the null hypothesis is true, worked out from probability or by reshuffling
   p-value	Inferential	If the null hypothesis were true, the probability of a result at least as extreme as the one observed
   Statistical significance	Inferential	A small p-value (often below 0.05), so chance alone is an unlikely explanation; it does not mean the effect is large
   Effect size	Inferential	How big the difference is, for example in points or in standard deviations; needed alongside the p-value

   A chance model for yes/no judgments. If a person is only guessing between two answers, each judgment is right with probability 1/2. In n judgments, the number right by guessing averages n/2, with a standard deviation of √n / 2. A z-score measures how far a real count is from that average: z = (number right - n/2) / (√n / 2). A guesser lands 2 or more standard deviations above n/2 only about 2% of the time, so a z of 2 or more is good evidence that something other than guessing is at work. For small n, the exact chance probabilities can be listed instead, as in Diagram 1.

   The case. In the 1880s many psychologists held that there is a smallest difference between two sensations that a person can detect, a "difference threshold." Peirce and Jastrow designed an experiment to test that idea. Read Sources 1 and 2.

   1The physiological psychologists assume that two nerve excitations alike in quality will only produce distinguishable sensations provided they differ in intensity by an amount greater than a fixed ratio. The least perceptible difference of the excitations divided by half their sum is what they call the Unterschiedsschwelle. [...]

   [...]

   2We have experimented with the pressure sense, observing the proportion of errors among judgments as to which is the greater of two pressures, when it is known that the two are two stated pressures, and the question presented for the decision of the observer is, which is which? From the probability, thus ascertained, of committing an error of a given magnitude, the probable error of a judgment can be calculated according to the mathematical theory of errors. If, now, we find that when the ratio of the two pressures is smaller than a certain ratio, the erroneous judgments number one-half of the whole, while the mathematical theory requires them to be sensibly fewer, then this theory is plainly disproved, and the maximum ratio at which this phenomenon is observed the so-called Unterschiedsschwelle. If, on the other hand, the values obtained for the probable error are the same for errors varying from three times to one-fourth of the probable error (the smallest for which it is easy to collect sufficient observations), then the theory of the method of least squares is shown to hold good within those limits, the presumption will be that it extends still further, and it is possible that it holds for the smallest differences of excitation. But, further, if this law is shown to hold good for difference so slight that the observer is not conscious of being able to discriminate between the sensations at all, all reason for believing in an Unterschiedsschwelle is destroyed. [...]

   1The observations were conducted in the following manner: At each sitting three differential weights were employed. At first we always began and ended with the heaviest, but at a later period the plan was to begin on alternate days with the lightest and heaviest. When we began with the heaviest 25 observations were made with that; then 25 with the middle one, and then 25 with the lightest; this constituted one-half of the sitting. It was completed by three more sets of 25, the order of the weights being reversed. When we began with the lightest the heaviest was used for the third and fourth sets. In this way 150 experiments on each of us were taken at one sitting of two hours.

   2A pack of 25 cards were taken, 12 red and 13 black, or vice versa, so that in the 50 experiments made at one sitting with a given differential weight, 25 red and 25 black cards should be used. These cards were cut exactly square and their corners were distinguished by holes punched in them so as to indicate the scale of numbers (0, 1, 2, 3) used to designate the degree of confidence of the judgment. The backs of these cards were distinguished from their faces. They were, in fact, made of ordinary playing-cards. At the beginning of a set of 25, the pack was well shuffled, and, the operator and subject having taken their places, the operator was governed by the color of the successive cards in choosing whether he should first diminish the weight and then increase it, or vice versa. If the weight was to be first increased and then diminished the operator brought the pressure exerted by the kilogram alone upon the finger of the subject by means of the lever and cam mentioned above, and when the subject said "change" he gently lowered the differential weight, resting in the small pan, upon the pan of the balance. The subject, having appreciated the sensation, again said "change," whereupon the operator removed the differential weight. If, on the other hand, the color of the card directed the weight to be first diminished and then increased, the operator had the differential weight already on the pan of the balance before the pressure was brought to bear on the finger, and made the reverse changes at the command of the subject. The subject then stated his judgment and also his degree of confidence, whereupon the total pressure was at once removed by the cam, and the card that had been used to direct the change was placed face down or face up according as the answer was right or wrong, and with corner indicating the degree of confidence in a determinate position. By means of these trifling devices the important object of rapidity was secured, and any possible psychological guessing of what change the operator was likely to select was avoided. A slight disadvantage in this mode of proceeding arises from the long runs of one particular kind of change, which would occasionally be produced by chance and would tend to confuse the mind of the subject. But it seems clear that this disadvantage was less than that which would have been occasioned by his knowing that there would be no such long runs if any means had been taken to prevent them. At the end of each set the results were of course entered into a book.

   • From a question to hypotheses: Source 1, paragraph 2

     State the question Peirce and Jastrow test as a null hypothesis and an alternative.

     Result: Question: for very small differences between two pressures, are judgments of which is greater right more often than guessing allows? Null hypothesis: below some ratio, judgments are right only half the time (in Source 1, paragraph 2, errors would make up one-half of all judgments). Alternative: judgments stay right more than half the time, even for the smallest differences tested.

   • How the data were collected: Source 2, paragraph 2

     How did one pack of playing cards both set the order of the trials and record the results?

     Result: The color of each card set whether the weight was first increased or first decreased. A pack of 25 had 12 cards of one color and 13 of the other, so the 50 trials with one weight used 25 of each order. After each judgment the card was placed "face down or face up according as the answer was right or wrong," with a punched corner marking the confidence, so the record was made on the spot.

   • Descriptive statistics for a small data set

     Six invented scores, words recalled from a list of 15: 6, 8, 8, 9, 11, 12. Find the mean, median, range and standard deviation.

     Result: Mean = 54 / 6 = 9. Median = (8 + 9) / 2 = 8.5. Range = 12 - 6 = 6. Squared distances from the mean: 9, 1, 1, 0, 4, 9, sum 24; sample variance = 24 / 5 = 4.8, so the standard deviation is √4.8 ≈ 2.19 words.

   • The warm-up claim and a chance model

     If the classmate were only guessing, how likely is 7 or more right out of 10? What if he got 70 right out of 100?

     Result: Out of 1,024 equally likely patterns of 10 guesses, 120 + 45 + 10 + 1 = 176 have 7 or more right: 176 / 1,024 ≈ 17%. Guessing does this about 1 time in 6, so 7 of 10 is weak evidence. For 70 of 100: n/2 = 50, √100 / 2 = 5, z = (70 - 50) / 5 = 4. The same 70% is now strong evidence, because a guesser almost never lands 4 standard deviations high.

   • What "significant" means

     An invented study reports that a new study app raised quiz scores by 1 point out of 100, p = 0.01, in a sample of 5,000 students. Is the app important?

     Result: The small p-value says a 1-point difference would rarely arise by chance in a sample this large, so the difference is probably real. It says nothing about size: 1 point out of 100 is a tiny effect. Report both the p-value and the effect size, and judge importance from the size.

   Close the segment with Diagram 1, the chance model for 20 two-choice guesses that Activity 1 builds from real coin flips.

3. Guided Practice15 minutes

   Read Source 4 together. It is an invented study with invented data and a small sample: 20 volunteers from one school, so its results describe those students and the one article and music used.

   1Invented study summary, written for this page. A psychology class teacher wanted to know whether background music with lyrics changes how well students understand what they read. Twenty student volunteers took part, with their own and a parent's written permission, and anyone could stop at any time. A shuffled deck of 10 red and 10 black cards assigned each volunteer to a group: red read with a recording of songs with lyrics playing quietly, black read in silence. Everything else was the same: one 600-word article, 12 minutes of reading time, and the same 20-question comprehension quiz, scored by a teacher who did not know which group each quiz came from.

   2Quiz scores (out of 20), lyrics group: 11, 14, 9, 13, 12, 15, 10, 12, 8, 13. Quiz scores (out of 20), silent group: 12, 15, 11, 14, 13, 16, 10, 13, 12, 13.

   3To ask whether the difference could be due to chance alone, the teacher used a randomization test. If the music made no difference, each volunteer would have earned the same score in either group, and the split into groups would just be the luck of the shuffle. A computer listed every possible way to split the 20 scores into two groups of 10 and counted how often the silent group's mean beat the lyrics group's mean by at least as much as in the real data. That happened in 22,143 of the 184,756 possible splits, about 12%.

   4The teacher's summary: "In this small group of volunteers at one school, students who read in silence scored a little higher on average, but a difference this large would turn up fairly often by chance alone. These data cannot tell us whether the music matters. A larger study, with more students, other schools and other kinds of music, would be needed."

   1. Name the parts. Question: does music with lyrics change reading comprehension? Independent variable: lyrics or silence. Dependent variable: quiz score out of 20. Random assignment: the shuffled red and black cards. Blind scoring: the teacher did not know which group each quiz came from.
   2. Describe the lyrics group. In order: 8, 9, 10, 11, 12, 12, 13, 13, 14, 15. Mean = 117 / 10 = 11.7. Median = (12 + 12) / 2 = 12. Range = 15 - 8 = 7. Have pairs describe the silent group the same way; it returns in the quiz.
   3. Read the randomization test. The test asks: if the music did nothing, how often would a random split of these 20 scores give a gap at least as large as the real one? The answer is a count out of all possible splits, 22,143 of 184,756, which is the p-value written as a fraction. Ask pairs what a much smaller count, say 200 of 184,756, would have meant.
   4. Look at Diagram 2. Ask: do the two groups overlap? What does the overlap suggest about how much one student's score tells you about the group they were in?
4. Independent Practice15 minutes

   Students read Sources 3 and 5 on their own and use them, with Sources 1, 2 and 4, for the quiz and homework. Source 3 gives Peirce and Jastrow's own counts. Their study had only two subjects, the two authors, who took turns as subject and operator, so the results speak to many judgments by two trained adults, not to people in general. Source 5 is an invented post.

   1In 1,125 experiments (subject, Mr. Peirce) -- variations 15, 30, and 60 grams -- there occurred confidence of 3, 35 times (3 per cent.); of 2, 102 times (9 per cent.); of 1, 282 times (25 per cent.); of 0, 706 times (63 per cent.). In these experiments there were 332 (29 per cent.) errors committed, of which 1 (0.3 per cent.) was made in connection with a confidence 3; 10 (3 per cent.) with a confidence 2; 51 (15 per cent.) with a confidence 1; 270 (81 per cent.) with a confidence 0. From which we find that in connection with a confidence of 3 there occurred 1 error in 35 cases (3 per cent.); with a confidence of 2, 10 errors in 102 cases (10 per cent.); with a confidence of 1, 51 errors in 282 cases (18 per cent.); with a confidence of 0, 270 errors in 706 cases (38 per cent.).

   2In 1,975 experiments (subject, Mr. Jastrow) -- variations 15, 30, and 60 grams -- there occurred confidence of 3, 62 times (3 per cent.); of 2, 196 times (10 per cent.); of 1, 594 times (30 per cent.); of 0, 1,123 times (57 per cent.). In these experiments there were 451 (23 per cent.) errors committed, of which 2 (0.4 per cent.) were made in connection with a confidence of 3; 12 (3 per cent.) with a confidence of 2; 97 (22 per cent.) with a confidence of 1; 340 (75 per cent.) with a confidence of 0. Again, in connection with a confidence of 3, errors occurred twice in 62 cases (3 per cent.); with a confidence of 2, 12 times in 196 cases (6 per cent.); with a confidence of 1, 97 times in 504 cases (16 per cent.); with a confidence of 0, 340 times in 1,123 cases (30 per cent.).

   [...]

   3The quantity which we have called the degree of confidence was probably the secondary sensation of a difference between the primary sensations compared. The evidence of our experiments seems clearly to be that this sensation has no Schwelle, and vanishes only when the difference to which it refers vanishes. At the same time we found the subject often overlooked this element of his field of sensation, although his attention was directed with a certain strength toward it, so that he marked his confidence as zero. This happened in cases where the judgments were so much affected by the difference of pressures as to be correct three times out of five. [...]

   1Invented social media post, written for this page. "Huge news for anyone with a test tomorrow!!! A study found that students who took a memory test printed on green paper scored 'significantly' higher, p < 0.05. The green-paper group averaged 71% and the white-paper group averaged 69%. That's science, people. I'm printing all my flashcards on green paper."

   2Reply from a classmate: "Where's the study? How many students were in it, and how did they decide who got green paper? 'Significant' doesn't mean big. A 2-point difference might not matter even if it's real."

   Task: write three questions you would need answered before trusting the claim in Source 5, and say which statistic each answer would give you.

5. Closure5-10 minutes

   Exit ticket: "In one sentence each, say what a descriptive statistic told you today and what an inferential statistic told you."

   Teacher note on the source: Source 3 stops before the end of the paper's closing paragraph. There, Peirce and Jastrow speculate that faint sensations explain "the insight of females" and "telepathic" phenomena. These are 1885 speculations, not findings: they rest on a stereotype and on a claim modern research does not support, and the page leaves them out. If students read the full paper, discuss the passage as an example of a conclusion that goes beyond the data.

   Teacher note on data collection: the data students collect in this lesson come from coin flips and from a published paper, not from people. Any study of classmates or other people stays on paper unless your school's review process approves it.

Differentiation Strategies

Assessment Guidance

1. Question 1 of 20 · Multiple Choice

   Which of these is an inferential question rather than a descriptive one?

   AWhat was the median score in the silent group? BIs the gap between groups larger than chance? CHow many volunteers were in each of the groups? DWhat was the highest score in the lyrics group?

2. Question 2 of 20 · Multiple Choice

   In Source 1, paragraph 1, what do "the physiological psychologists" assume about two sensations of the same kind?

   AThey can be told apart only above a fixed ratio BThey can always be told apart if the observer pays close attention CThey are felt more strongly when the pressure is first increased DThey are judged more accurately by trained than by untrained people

3. Question 3 of 20 · Multiple Choice

   According to Source 1, paragraph 2, which result would leave no reason to believe in a difference threshold?

   AErrors equal to half of all judgments for every pair of pressures BAbove-chance judgments when no difference is felt CLarge differences in pressure judged correctly almost every time DTwo observers who disagree about which pressure was greater

4. Question 4 of 20 · Multiple Choice

   In Source 2, paragraph 2, why did the color of a shuffled card decide the order of each change?

   ASo the subject could not predict the order from a pattern BSo that heavier weights were always used before lighter ones CSo that the operator could choose the easiest order each time DSo that each sitting would last no longer than two hours

5. Question 5 of 20 · Multiple Choice

   Source 2, paragraph 2 admits that random order sometimes produced "long runs of one particular kind of change." Why did the authors accept this?

   ALong runs made each sitting shorter and less tiring BLong runs were too rare to happen in 50 trials CThe operator corrected long runs by hand as they occurred DPreventing runs would let the subject predict

6. Question 6 of 20 · Multiple Choice

   Source 2, paragraph 1 says the order of the three weights was reversed in the second half of each sitting. What does this design choice control for?

   ADifferences between the red and the black cards BThe subject's confidence in each judgment CPractice and fatigue within a sitting DWhether the weight was increased or decreased

7. Question 7 of 20 · Multiple Choice

   Use Source 3, paragraph 1. What percent of Mr. Peirce's 1,125 judgments were correct?

   A29.5% B63% C38% D70.5%

8. Question 8 of 20 · Multiple Choice

   Source 3, paragraph 2 gives Mr. Jastrow's confidence-1 count as "594 times" and later as "504 cases." Which is consistent with the rest of the paragraph?

   A504, because it appears in the later sentence B594, because 62 + 196 + 594 + 1,123 = 1,975 C504, because 451 errors must be subtracted DNeither, because the errors do not add up

9. Question 9 of 20 · Multiple Choice

   Source 3, paragraph 3 says that judgments made with confidence zero were still "correct three times out of five." What does this suggest?

   AThe subjects were guessing and got lucky on most judgments BThe confidence scale was reversed by mistake in those trials CJudgments used information the subjects did not notice DConfidence was the only thing the subjects really measured

10. Question 10 of 20 · Multiple Choice

    Use the invented data in Source 4, paragraph 2. What is the mean score of the silent group?

    A13 B12.9 C11.7 D129

11. Question 11 of 20 · Multiple Choice

    Source 4, paragraph 3 reports that 22,143 of 184,756 splits, about 12%, gave a gap at least as large as the real one. Which statement interprets this correctly?

    AThere is a 12% chance that music with lyrics affects reading BMusic with lyrics lowered the scores of about 12% of the readers CIf music did nothing, gaps this big would occur 12% of the time DThe study proves that music with lyrics has no effect on reading

12. Question 12 of 20 · Multiple Choice

    In Source 4, why did the shuffled cards and the teacher who "did not know which group each quiz came from" matter for the analysis?

    AThey made both groups score close to the same mean BThey made the data easier to put into a table CThey left only chance or the music to explain a gap DThey guaranteed that the result would be significant

13. Question 13 of 20 · Multiple Choice

    In Source 5, the post says the green-paper result was "significant" at p < 0.05. What does that tell you?

    AA 2-point gap would be unlikely by chance BGreen paper raises memory scores a lot CThe study used a large, random sample DGreen paper helps most students who use it

14. Question 14 of 20 · Multiple Choice

    Two invented studies test whether people can tell two tones apart. Both find 60% correct judgments, where guessing gives 50%. Study X used 25 judgments; Study Y used 900. Which statement is accurate?

    ABoth give equally strong evidence, since both found 60% BStudy X gives stronger evidence, because it is less work CNeither can be tested, because 60% is close to 50% DStudy Y gives much stronger evidence against chance

15. Question 15 of 20 · Short Answer

    Use Source 3, paragraph 1. Mr. Peirce made 706 judgments with confidence 0, and 270 of them were errors. Compute the percent correct. Then test it against chance: compute the expected number right for a guesser, the standard deviation √n / 2 and the z-score. What do you conclude?

    Show model answer

16. Question 16 of 20 · Short Answer

    Use the invented data in Source 4, paragraph 2 and your answer to question 10. Find the difference between the group means. Then count how many scores in each group are 13 or higher. Describe the two groups in two sentences, using these numbers and the overlap you can see in Diagram 2.

    Show model answer

17. Question 17 of 20 · Short Answer

    The teacher in Source 4, paragraph 4 says the data "cannot tell us whether the music matters." Explain why a result that is not statistically significant is not the same as evidence of no effect, and name two limits of this sample.

    Show model answer

18. Question 18 of 20 · Short Answer

    Source 5 reports that the green-paper group averaged 71% and the white-paper group 69%. Write two questions about how the data were collected that you would need answered before trusting the claim. For each, say why it matters.

    Show model answer

19. Question 19 of 20 · Short Answer

    Choose a psychological question that could be answered with two-choice judgments (for example, telling two shades of gray apart). On paper only, describe how you would collect the data and how you would analyze them with one descriptive and one inferential statistic.

    Show model answer

20. Question 20 of 20 · Short Answer

    Explain the difference between descriptive and inferential statistics, with one example of each from Source 3 or Source 4.

    Show model answer

These standards connect to D2.Psy.6.9-12: prerequisites to review first, parallel standards at the same level, and next steps that build on it.

Standard text: National Council for the Social Studies (NCSS), The College, Career, and Civic Life (C3) Framework for Social Studies State Standards: Guidance for Enhancing the Rigor of K-12 Civics, Economics, Geography, and History (Silver Spring, MD: NCSS, 2013). NCSS was not involved in the production of this lesson and does not endorse it.