Tess Dawber
Council for Aid to Education, United States
Olivia Cortellini
Council for Aid to Education, United States
Does Higher Education Teach Students to Think Critically?
8. Fields of study
Abstract
This chapter explores the relationships between student performance on the Collegiate Learning Assessment (CLA+) and fields of study at university or college. The fields of study are classified differently for the international and U.S. samples. Therefore, the results are presented separately. In addition, the international survey included questions about the format of instruction. These data are discussed first.
Instructional format
After completing the CLA+, the international students responded to a series of questions, including questions about the format of their instruction. Questions are not mutually exclusive; students were asked to select all that apply.
The table in the Annex shows the number and percentage of students that endorsed each of the instructional formats, shown in the column headings by field of study. The percentages are calculated based on the number of students for a given field of study. For example, reading across the row for liberal arts students, their instruction included lectures (54% of students), art studios (51%), and seminars (41%), to name a few formats. Overall, lectures, seminars and independent study were the most endorsed instruction formats across the fields of study.
Students’ performance on the CLA+ was examined for each instructional format. Table 8.1 shows the CLA+ mean scores by instructional format. It is important to note that there is overlap in the samples (e.g., students who participated in seminars may also have participated in independent studies). The mean total scores for students who participated in seminars, science laboratories or lectures were within the Proficient level of mastery. The mean total scores for the other instructional formats were within the Developing mastery level. Lecture and seminar formats are common across all fields of study, but the science laboratories were indicated most often by science students (see Annex). The results suggest that having students engage in seminars, science laboratories or lectures may facilitate the development of critical thinking skills. Figure 8.1 shows the average student CLA+ scores by instructional format.
Table 8.1. CLA+ scores by instructional format: International sample
Figure 8.1. Average CLA+ scores by instructional format for the international sample
Note: PT = Performance Task; SRQ = Selected-Response Questions
Fields of study for international students
The fields of study response options differed depending on whether students tested on the international platform or the domestic platform. The countries that tested on the international platform were Chile, Finland, Mexico and some United Kingdom students (22%). The domestic platform included the United States only. The average student score results are presented in tables as well as illustrated in figures.
Table 8.2 presents the CLA+ total score and section results by field of study for the international sample. The table shows the average scale score results for the entering and exiting students and the average difference between those scores. When mean difference results are positive, exiting students achieved higher scores on average than entering students; when mean difference results are negative, entering students achieved higher scores on average than exiting students. Generally, the exiting students achieved higher mean scores than entering students, with a few exceptions.
Fields of study with fewer than 100 students for either cohort were excluded to avoid reporting results that are not representative of the student groups. The three fields of study excluded were the general programme (liberal arts), law, and services (social services, nursing) programmes.
Looking at the CLA+ total score results, the average scores for science, humanities or arts, and social sciences were within the Proficient mastery level for entering students. All average scores for exiting students were classified as Proficient except for business and agriculture. The mean difference values show greater growth from entering to exiting status for the health or welfare and not-specified groups. Another observation is that the samples for entering students were much higher than those for the exiting students, roughly between 2 and 4 times higher.
To determine whether there were performance differences across the fields of study for the international students, analysis of variance (ANOVA) was performed. The results in Table 8.3 show that there were significant differences for both cohorts and for each CLA+ score, although the effect sizes were low.
Table 8.2. CLA+ scores by field of study for student cohorts: International sample
Note: “Mean difference” is calculated by subtracting the mean for “Entering” from the mean for “Exiting”.
To determine whether there were performance differences across the fields of study for the international students, ANOVA analyses were performed. The results in Table 8.3 show that there were significant differences for both cohorts and for each CLA+ score, although the effect sizes were low.
Table 8.3. One-way ANOVA for CLA+ scores across fields of study for international students
Bonferroni post hoc comparisons were conducted using a significance level of 0.05. For the CLA+ total score, the science and humanities students performed better than all or most of the other fields of study and all or most of the other fields of study performed better than business and agriculture for the entering cohort. For the exiting cohort, the humanities and science students outperformed the business and agriculture students, and all groups performed better than the agriculture group.
The CLA+ PT results show that humanities and arts students performed better than all other fields of study and all other fields of study performed better than agriculture for the entering cohort. For the exiting cohort, all fields of study outperformed the business and agriculture groups.
The CLA+ SRQ results show science students performed better than all other groups, and the business and agriculture students received lower scores on average than the other fields of study for the entering cohort. For the exiting cohort, engineering students performed better than health and welfare, business, and agriculture students; business performed better than agriculture but lower than most other groups, and agriculture students performed lower than all other groups.
These results suggest that there are consistent and meaningful differences across the fields of study. The business and agriculture students were found to have relatively low scores and the humanities, science and social science students were found to have relatively high scores compared to their peers in other fields of study. The implication of these findings is that students, particularly business and agriculture students, may benefit from some instructional workshops on critical thinking and written communication skills to increase their skills to the level of other fields of study.
Figure 8.2-Figure 8.4 show the international student results for the CLA+ total, PT, and SRQ scores, respectively.
Figure 8.2. International students’ CLA+ total scores by field of study
Figure 8.3. International students’ performance task scores by field of study
Figure 8.4. International students’ selected response scores by field of study
Fields of study for U.S. students
The fields of study for the United States consisted of five categories. Table 8.4 presents the results by cohort and field of study for each CLA+ score. The mean difference results are shown as well.
Looking at the CLA+ total score results for entering students, only the average score for sciences and engineering students was at the Proficient mastery level, and the average scores for the other fields of study were at the Developing mastery level. In contrast, the average scores for the exiting students were in the Proficient range for sciences and engineering, social sciences, and humanities and languages students, and in the Developing range for business and helping/services. It is interesting to note that the ranking of the fields of study remained the same for the entering and exiting students across all three CLA+ scores. Science and engineering students obtained the highest mean score, followed by social science, humanities and languages, business, and helping/services.
The mean difference results were positive for all CLA+ scores and fields of study, indicating that the exiting students consistently achieved higher scores on average than the entering students. The highest mean difference was observed for the helping/services field across all three CLA+ scores, which may suggest that studies in the helping/services area contribute to acquiring critical thinking and written communication skills.
Table 8.4. CLA+ scores by field of study for student cohorts: U.S. Sample
To determine whether there were significant performance differences across the fields of study for the U.S. students, ANOVA analyses were performed. The results in Table 8.5 show that there were significant differences for both cohorts for each CLA+ score although the effect sizes were relatively low. Figure 8.5, Figure 8.6 and Figure 8.7 show the mean scale score results for each CLA+ score by fields of study for the U.S. sample.
Table 8.5. One-way ANOVA for CLA+ scores across fields of study for U.S. students
Bonferroni post hoc tests were conducted at the 0.05 significance level. The same pattern of results emerged for both cohorts and for all CLA+ test scores. Specifically, sciences and engineering, and social sciences students outperformed all or most other fields of study; business and helping/services students performed lower than all or most other fields of study.
Figure 8.5. U.S. students’ CLA+ total scores by field of study
Figure 8.6. U.S. students’ performance task scores by field of study
Figure 8.7. U.S. students’ selected response scores by field of study
Conclusion
This chapter explored the relationships between student performance on the CLA+ and fields of study at university or college. In addition, students’ performance on the CLA+ was examined for each instructional format. On average, CLA+ scores for students who participated in seminars, science laboratories or lectures were within the Proficient level of mastery.
The fields of study for the international students were different from those for the U.S. students. For the international students, the science and humanities students tended to outperform their peers in other fields of study. For the U.S. sample, the sciences and engineering students, and the social science students outperformed their peers on average.
These results suggest that there are consistent and meaningful differences across the fields of study for the international and U.S. samples. The international sample is more diverse than the U.S. sample. As noted in Part III of this book, the chapters from Italy, Finland, the United Kingdom, Mexico and Latin American describe the introduction of the CLA+ within a specific context and for a specific purpose. Regardless of educational policies and accountability programmes, student groups may benefit from instruction to facilitate developing their critical thinking and written communication skills.
Annex 8.A. International sample: Instructional format by fields of study
Annex Table 8.A.1. Instructional format by fields of study for the international sample
Note: Percentages in the orange cells are between 25 and 50 percent; percentages in the grey cells are greater than 50 percent.