Anita Williams Woolley
Carnegie Mellon University
AI and the Future of Skills, Volume 1
Capabilities and Assessments
Educational Research and Innovation
6. Assessing collective intelligence in human groups
Abstract
This chapter summarises the research on collective intelligence (CI) in human groups over the last ten years. It describes key factors that lead to development of CI in human teams and discusses the approach taken to measuring it. It also examines different task taxonomies, including the McGrath Group Task Circumplex, and looks at the nature of interdependence in CI. Newer research on enhancing CI is explored, including possible roles for artificial intelligence (AI). The chapter discusses how these approaches could provide useful directions for shaping and evaluating AI, and particularly artificial social intelligence, for enabling teamwork in more complex settings.
Introduction
The ability to collaborate in teams is becoming increasingly important. Across every sector of society and the economy, there is growth in the delivery of emergency and medical services in teams (Hughes et al., 2016[1]), the management of business in teams (Pearce, 2004[2]) and even the development of new knowledge in science and technology in teams (Wuchty, Jones and Uzzi, 2007[3]).
At the same time, problems arise that are difficult for teams to address. These problems are due to complexity associated with geographic dispersion; language and cultural barriers; the large number of people required to co‑ordinate; and the broad diversity of expertise, beliefs and motivations needed to be aligned to make progress. The right knowledge and tools to manage those barriers might enable powerful engines of progress for society. However, a workforce would need tools for effective collaboration and support from tools and technology to help manage the associated complexity.
Enhancing collaboration to enable teams to address the toughest problems requires deep understanding of what helps teams perform well. Technology can likely play a role; however, knowing how to create technologies that will enable good teamwork consistently is also needed. This will require understanding both the individual characteristics and abilities that enable effective collaboration, and the group level features and behaviours that support good teamwork across a wide range of problems of varying complexity.
Research on collective intelligence (CI) has focused on conceptualising and measuring the capabilities underlying teamwork. Woolley et al. (2010[4]) used an analogy between the individual intelligence of a person and the CI of a group. Research literature commonly defines individual intelligence with a statistical factor (“g” for general intelligence) that captures and predicts how well a person will perform on a wide range of different tasks. Woolley and colleagues found that, similar to individual intelligence, a single factor predicted over 40% of the variance in performance when groups completed a range of different tasks. They called this factor “collective intelligence.”
Research on CI in human groups builds upon and extends traditional research on team performance. The latter is typically focused on elements that enable a team to perform well on a particular task. Conversely, research on CI examines what enables a group to perform tasks that vary in complexity over time. Accordingly, subsequent studies have tested and shown that measures of a group’s CI predicts future performance in a variety of settings (Woolley et al., 2010[4]; Engel et al., 2014[5]; Kim et al., 2017[6]; Aggarwal et al., 2019[7]; Glikson et al., 2019[8]).
What are the key factors that cultivate collective intelligence?
Several factors appear to be key drivers of CI in teams. These include individual characteristics, such as social perceptiveness, which enhance the quality of collaboration; group compositional features, such as various forms of diversity; and group process behaviours. These are discussed below.
Individual characteristics
Researchers have explored which individual characteristics can enable group CI. Most personality variables have relatively weak correlations with CI (Engel et al., 2014[5]; Woolley et al., 2010[4]). However, gender composition of the group emerged early as an important characteristic. Early studies showed a significant correlation between the proportion of women in the group and its CI (Woolley et al., 2010[4]; Engel et al., 2014[5]; Kim et al., 2017[6]).
The relationship between having more women in the group and CI is explained by another characteristic: social perceptiveness. Social perceptiveness is the ability to take another’s perspective and reason about their intentions, knowledge, beliefs and emotions (Premack and Woodruff, 1978[9]). In studies of CI, groups with higher average social perceptiveness are more collectively intelligent. Women, on average, tend to be more socially perceptive than men. Having more women in a group, then, raises CI because it raises the average level of social perceptiveness. Furthermore, groups with higher social perceptiveness tend to engage more in collaboration processes that enhance CI, which are discussed further below.
Group composition factors
In addition to the influence of specific individual characteristics, research on CI in groups finds several influential compositional factors. First, it appears that cognitive style diversity is important. Aggarwal et al. (2019[7]) evaluated individual member cognitive style using the Object-Spatial Imagery and Verbal Questionnaire (Blazenkova, Kozhevnikov and Motes, 2006[10]). They found that groups with moderate levels of cognitive style diversity reach the highest levels of CI. In related research, Aggarwal and Woolley (2018[11]) found that high levels of cognitive diversity were associated with group difficulties in settling on a shared task strategy. Task performers with different cognitive styles tended to approach the task in different ways. This finding has been illustrated in research in educational settings as well (Blazhenkova and Kozhevnikov, 2020[12]). Thus, to be collectively intelligent, a group needs a variety of perspectives but also a means of integrating them to produce better ideas or decisions and to act upon them.
Box 6.1. Cognitive styles and collective intelligence
Cognitive styles refer to consistencies in an individual’s manner of acquiring and processing information (Ausburn and Ausburn, 1978[13]). Research in the 1970s argued for a visual-verbal cognitive style dimension (Richardson, 1977[14]; Paivio, 1979[15]). However, neuroscience research demonstrates that the visual areas of the brain divide into two distinct pathways: the object (ventral) and the spatial (dorsal) pathways (Ungerleider and Mishkin, 1982[16]). Furthermore, researchers have documented a trade-off in the abilities associated with processing in these areas; individuals with above-average object visualisation abilities (such as artists) tend to have below-average spatial visualisation abilities. The inverse is true for those with above-average spatial visualisation abilities (scientists).
These observations led to the identification of two different cognitive styles associated with visualisation (i.e. object visualisation and spatial visualisation) in addition to the verbalisation cognitive style. In the past two decades, research has demonstrated these cognitive styles can be identified in children as young as eight years of age. They are also associated with related abilities and preferences for associated academic subjects (Blazhenkova, Becker and Kozhevnikov, 2011[17]).
A self-report survey instrument, the Object-Spatial Imagery and Verbal Questionnaire (Blazhenkova and Kozhevnikov, 2008[18]) has been developed and validated as a reliable measure of individual cognitive style in adults. A companion version has been developed for children (Blazhenkova, Becker and Kozhevnikov, 2011[17]). Items include statements such as, “I can easily imagine and mentally rotate three-dimensional geometric figures” as an indicator of spatial visualisation. Another statement is “When reading fiction, I usually form a clear and detailed mental picture of a scene or room that has been described” as an indicator of object visualisation.
In teams, diversity in cognitive styles has important implications for collaboration and collective intelligence. Teams with insufficient cognitive style diversity lack the abilities and perspectives to reach the highest levels of collective intelligence (Aggarwal et al., 2019[7]). Conversely, teams with high levels of cognitive style diversity can struggle to collaborate effectively (Woolley et al., 2008[19]; Aggarwal and Woolley, 2013[20]). This occurs because of different approaches taken by individuals with different cognitive styles to organise information and solve problems (Blazhenkova and Kozhevnikov, 2020[12]).
Highly cognitively diverse teams could be supported to collaborate effectively (Woolley et al., 2007[21]; Woolley et al., 2008[19]). However, this requires knowing the cognitive style strengths of all team members and helping them match members to the best task and roles. Artificially intelligent systems could be designed to sense the cognitive styles of individuals and represent information in a format best suited to their strengths. In addition, such systems could make cognitive style differences apparent to team members and aid in “translating” between them to facilitate co‑ordination. For example, a strong verbaliser might be trying to convey a project plan through long prose to a team member who is a strong spatial visualiser. Such a system could help the verbaliser translate the prose into a schematic or a diagram. Conversely, the same system could also sense when a team was too homogenous in cognitive style. It would thus prompt them to consider alternative approaches or seek other inputs to broaden their options. In these ways, artificial social intelligence could play an important role in both creating and facilitating diverse perspectives in teams.
There is evidence to suggest that racial diversity is beneficial for CI. Chikersal et al. (2017[22]) found that CI was higher when partners were of different ethnic backgrounds. In that study, the researchers examined facial expression synchrony, or the degree to which partners were mirroring the positivity or negativity of each other’s facial expressions during their interaction. They found CI was higher in pairs with greater synchrony in facial expressions. The level of racial diversity of each pair enhanced synchrony in facial expressions. This is likely because people exhibit increased attention to interaction partners who are of a different race. They are more likely to attend to information they supply (Phillips and Loyd, 2006[23]) and process information more carefully in their presence in general (Sommers, 2006[24]). This heightened attention can translate into higher CI levels.
Group process
In addition to the characteristics of the people in the group, interaction processes in which group members engage advance CI. In the last few decades, there has been growing recognition that team cognition plays an essential role in enabling effective teamwork (DeChurch and Mesmer-Magnus, 2010[25]). Similarly, Gupta and Woolley (2021[26]) recently articulated a Transactive Systems Theory of Collective Intelligence. It describes the interconnected memory, attention and reasoning systems that operate at the individual and collective levels to enable the emergence of CI.
These transactive cognitive systems give rise to three observable group processes: the group’s level of effort, their task strategy and their use of member knowledge and skill (Hackman, 1987[27]). Effort relates to the total amount of work that members put towards their collective task. Task strategy encompasses a group’s decisions regarding aspects of its work. This includes what gets completed first and what tasks to divide among members versus what tasks to do all together, etc. Use of member knowledge and skill captures a group’s proficiency at achieving agreement between relative member skills and their contributions to work on a task. Groups that make better use of member talents achieve higher levels of CI. Groups that engage in higher levels of effort, who engage in better task strategies, and who use the specific knowledge and skills of members more effectively develop higher levels of CI. Each of these pathways to CI suggest specific ways in which AI might be developed to enhance CI, as will be discussed further below.
Measuring collective intelligence
The primary approach to measuring CI in teams focuses on capturing group performance on a variety of group tasks that require different modes of interaction for completion. A group’s performance scores across a diverse battery of tasks then leads to an inference about the group’s CI. This approach is modelled after the traditional psychometric approach to measuring intelligence in individuals.
A measurement battery should incorporate tasks that are sufficiently diverse to draw an inference of CI. Diversity is more important for the collaboration processes needed for completion than for content.
Identifying task types
Figure 6.1. McGrath’s Group Task Circumplex
Source: Adapted from McGrath (1984[28]).
Several different task taxonomies have been proposed to capture and describe the essential differences in the types of tasks groups are asked to perform. McGrath’s Group Task Circumplex (see Figure 6.1); (McGrath, 1984[28]), for example, articulates four major types of tasks: generate, choose, negotiate and execute. These types vary in terms of the degree to which they require collaboration versus resolution of conflicting preferences. “Generate” tasks require teams to think as broadly and divergently as possible to develop a wide range of ideas. “Choose” tasks, by contrast, require groups to pool information or perspectives to converge on the single best or correct answer. “Negotiate” tasks require the resolution of conflicting objectives for group members to arrive at a solution that works best for the group. “Execute” tasks require the careful co‑ordination of physical inputs to accomplish a specified product as quickly and accurately as possible.
In addition to sampling task types, the level of task interdependence needed for the team to accomplish the task successfully should vary in a diverse battery. In the context of teamwork, one major driver of task complexity is the level and number of interdependencies a task involves (Thompson, 1967[29]; Steiner, 1972[30]; Wageman, 1995[31]). The inference of CI in a group is most strongly supported by evidence that the group can handle tasks of varying complexity.
Thompson (1967[29]) conceptualises three levels of interdependence (see Figure 6.2). This framework specifies the ways in which task contributors must combine their efforts to accomplish a task.
“Reciprocal” interdependence is the highest level. Here, the final product cannot be traced back to the inputs of any one member. The solution to a complex problem often results from reciprocal interdependence. Different contributors provide different perspectives or pieces of information that get combined synergistically.
“Sequential” interdependence represents an intermediate level. Here, each member works on and hands off their work to another member, who passes it down the line (e.g. a typical factory assembly line).
Figure 6.2. Interdependence of group members’ intelligence
“Pooled” is the lowest level. Here, each member works relatively independently and then contributes their part to a shared product (e.g. independently written book chapters in an edited volume, which are typically assembled with little to no interaction among authors).
The task types identified in the McGrath Group Task Circumplex can vary in their level of interdependence. For instance, “generate” tasks are generally best completed primarily using pooled interdependence; such tasks benefit from a broader range of divergent perspectives; however, there can be problems for which members need to integrate knowledge to generate solutions, which would involve higher levels of interdependence. “Choose” and “execute” tasks can also vary in the interdependence employed. A choose task completed with pooled interdependence could be an election or a crowd prediction of a future event such as a stock price. A choose task completed with sequential interdependence would benefit from multiple reviewers checking and correcting, such as a complex math problem. A choose task completed with reciprocal interdependence might be a hiring decision, where many different individuals need to give input and integrate information.
Box 6.2. Task types, interdependence and the workplace
Most jobs in organisational settings involve a variety of task types and interdependence levels. The number of different task types or interdependence levels vary by job type. For instance, a custodian’s job might consist mostly of execute tasks. It could involve a small amount of decision making (i.e. choose) and maybe some occasional planning or creative problem solving (i.e. generate), conducted mostly with pooled or sequential interdependence with co-workers. A management position might incorporate more task types and more variation in interdependence. In these roles, individuals frequently shift across task types (e.g. developing options, making a choice, negotiating with others). They also shift between independent work and highly collaborative, reciprocally interdependent work. Performance in complex jobs, particularly those requiring ongoing learning of new functions, is shown to be the most strongly correlated with measures of individual intelligence and cognitive ability (Schmidt and Hunter, 1998[32]; Murphy, 1989[33]). Complex jobs involve a high level of social skill to facilitate high interdependence. Wages for these kinds of jobs have grown significantly faster over the last four decades than for jobs that require technical skills alone (Deming, 2017[34]).
Developing test items
The Test of Collective Intelligence (TCI) (Riedl et al., 2021[35]; Kim et al., 2017[6]) aims to sample different task types that vary in levels of interdependence. It also includes items that tap into both verbal and non‑verbal content. The latter has become a larger priority over time. The inclusion of more non-native English speakers in studies suggested the CI of a group might be underestimated if all test items depended too much on facility with English.
The Platform for Online Group Studies (POGS), a web-based platform, facilitates the administration of the TCI. It enables collaborators to work together from anywhere in the world, seeing the work of their fellow group members in real time. POGS regulates how much time groups work on each task and provides a chat feature to facilitate co‑ordination. Administering the tasks on the platform enables the capture of detailed information about each member’s activities. This allows for a more accurate calculation of process measures in evaluating how different collaboration behaviours contribute to CI (Riedl et al., 2021[35]).
“Generate” items
Many standard examples of items from the “generate” quadrant of the McGrath Group Task Circumplex resemble standard brainstorming tasks. This is true as well for items on the TCI that tap into this quadrant. Figure 6.3 provides a verbal example; others in the TCI battery are more mathematical (i.e. brainstorm equations that satisfy specified constraints). Groups do best on tasks of this type when members work relatively independently to come up with as broad a range of ideas as possible. This finding is consistent with extant research on group brainstorming [e.g. Paulus and Yang (2000[36])]. Groups are scored based on both the number of unique ideas generated and the creativity of their ideas. Creativity is measured based on how often the same idea appears within the corpus of ideas submitted by groups across all samples.
Figure 6.3. Brainstorming task for the “generate” quadrant of the McGrath Group Task Circumplex
Note: A screenshot from the Platform for Online Group Studies (POGS).
Some “generate” tasks in the TCI vary in the level of interdependence required. For instance, some problems require that ideas later in the list incorporate information from the prior item in the list (e.g. use one of the same numbers in the equation). This then requires groups to work with at least a sequential, if not reciprocal, level of interdependence to generate a large list of ideas effectively.
“Choose” items
Common examples of tasks in the TCI related to the “choose” category resemble problem-solving and decision-making tasks used in a variety of studies of both individual and team performance. Some of these involve problems with demonstrably correct answers, such as unscrambling a word or solving a puzzle. Others involve matters of judgement, such as rating the quality of poems or photographs.
Many “choose” tasks can be completed by groups using pooled or sequential interdependence. For example, members may independently suggest an answer and the group goes with the one favoured by most (pooled). In another scenario, a member attempts to solve the problem and others review it afterward to check their work (sequential). Some “choose” tasks require reciprocal interdependence, such as a Sudoku puzzle designed to permit more than one solution; any one member’s choice of number has implications for the numbers other members select in the grid.
Figure 6.4. Suduko task for the “choose” quadrant of the McGrath Group Task Circumplex
Note: A screenshot from the Platform for Online Group Studies (POGS).
“Negotiate” items
As described above, “negotiate” tasks involve the resolution of conflicting motives. In these cases, an action that would provide better rewards for one member might compromise the outcomes of other members of the group or the group as a whole, and vice versa. A number of behavioural economics games facilitate the operationalisation of such situations, such as the minimum effort game (Anderson, Goeree and Holt, 2001[37]). In this game, each member chooses from a set of options without communicating with other members. Their payoff is determined based on the combination of their choice and the lowest choice of anyone else in the group. If they can trust other members to choose the highest number (i.e. 5), then they can win the maximum points. However, if they choose 5 and anyone chooses 1, they lose points (see Figure 6.5). This type of game creates incentives for co‑operation; in some other games, such as “the prisoner’s dilemma”, individuals are rewarded for defecting.
Figure 6.5. Minimum effort game for the “negotiate” quadrant of the McGrath Group Task Circumplex
“Execute” items
Tasks that fall into the ”execute” category typically involve carefully co‑ordinated psychomotor movements so the team can quickly and accurately complete a specified task. In the TCI, for example, team members need to retype a given text as quickly as possible into a shared online document. Members need to be careful not to type over each other or to skip over large blocks of text; missing text results in loss of points. In a variation meant to increase interdependence, different members can only type specified words, which forces team members to work together more closely in reproducing the text.
Box 6.3. The psychometric structure of collective intelligence
In their initial studies, Woolley et al. (2010[4]) found strong evidence of a single general factor in the performance scores of teams performing tasks sampled from existing team task taxonomies, such as the McGrath Group Task Circumplex. That factor was replicated in subsequently published studies (Engel et al., 2014[5]; Meslec, Aggarwal and Curşeu, 2016[38]; Kim et al., 2017[6]; Aggarwal et al., 2019[7]). Some researchers have raised questions about the factor structure of collective intelligence (CI), questioning if there is truly a single factor or rather multiple factors (Barlow and Dennis, 2016[39]; Credé and Howardson, 2017[40]). Others have also raised questions about the size of the role of individual intelligence (Bates and Gupta, 2017[41]).
A more recent analysis illustrated the group process was more than twice as important for predicting CI as individual skills. Riedl and colleagues (2021[35]) meta-analytically examined data from 22 different samples, including more than 1 300 teams that worked on a range of tasks such as those above. Their dataset included detailed process information describing how team members worked together on the tasks, as well as what level of skill each member possessed for working on the tasks independently. While they showed group process to be over twice as important as individual skills for predicting CI, they also identified variance in the degree to which team process predicted performance on different individual tasks of different types compared to individual skills. This suggested that studies showing a larger role for individual ability tended to oversample types of tasks that depend more upon individual skill. Furthermore, Riedl et al. (2021[35]) found that individual characteristics such as social perceptiveness were influential because they led teams to engage in the processes most strongly predictive of CI. In addition, it appears that whether a single factor or multiple factors of CI emerge depends on some methodological choices. These include whether one team member or multiple team members can record answers to problems simultaneously, and whether all task types are represented in the battery used to measure CI.
Using artificial intelligence to enhance collective intelligence
Can technology be used to enhance human group CI? Given the important role of group process and individual skill in creating CI, the answer is almost certainly “Yes”.
Technological tools could enhance both individual skill and group process in a variety of ways. Gupta and colleagues (2019[42]) have experimented with “digital nudges”. Like other types of nudges (Thaler and Sunstein, 2009[43]), digital nudges are subtle changes in a digital environment that make it more likely that people will make certain choices over others – in this case, choices about how to do that results in group processes resembling those known to enhance CI.
In their studies, the researchers deployed “bots” and other tools to prompt teams to consider critical aspects of group process. For example, a bot facilitator would ask the team questions that would lead them to consider whether they were using information about which member was good at what in deciding how to divide a task, or whether they had co‑ordinated their task strategy to cover all parts of the task. The researchers found that technological nudges helped encourage some group processes, particularly those related to getting groups to use information about individuals’ skills and abilities in structuring work.
In a similar study, Glikson et al. (2019[8]) found that use of a digital nudge that provided feedback about the relative effort of members enhanced group CI. This was especially true in teams with members who were low in conscientious. The nudge effectively served as a “conscience” to help lesser-contributing members realise they needed to increase their effort.
Given the consistent role of characteristics related to social intelligence in advancing group CI, another avenue for enhancing CI might be to help improve the social and communications skills of collaborators. How much these abilities can be enhanced in individuals is an ongoing debate. Some researchers have shown efficacy of interventions in improving individual skills in these areas through training (Kidd and Castano, 2013[44]; Hodzic et al., 2018[45]; Kotsou et al., 2019[46]). Others question that evidence (Panero et al., 2016[47]; Panero et al., 2017[48]; Kidd and Castano, 2017[49]).
In still other areas, researchers are experimenting with technological tools to augment individuals’ natural ability to pick up on social cues. This helps individuals interpret otherwise ambiguous situations [e.g. Voss et al. (2016[50])]. Therefore, as technology develops, CI may be enhanced in the context of human‑computer systems. Specifically, AI might enable technology to augment the individual skills and group collaboration processes that enable CI to emerge.
Using collective intelligence to evaluate or enhance artificial intelligence
Can the same approach to evaluating CI also be used to assess AI capabilities? Arguably, such an approach could complement perspectives on evaluating the utility of AI. This section explores the role of production and co‑ordination technologies, and artificial social intelligence (ASI) in diagnosis and assessment.
Production and co‑ordination technologies
Much of the work to date on AI has focused on development of production technologies. These are technologies designed to produce more accurate or higher-quality output more effectively or efficiently than traditional approaches. Decision aids that improve medical diagnoses or autonomous vehicles that can drive more safely than humans are examples of production technologies.
Some argue that developing AI as a co‑ordination technology has even more potential for enhancing CI. Such technology operates by co‑ordinating and combining the inputs of other contributors. Examples that exist today include shared ride systems such as Uber or Lyft, or social network platforms that connect people with similar interests or complementary needs. As we think about evaluating and increasing the level of intelligence in technology, we might consider the types of tasks and levels of interdependence the technology can facilitate as an important indicator. High levels of intelligence would be evident in technology that could sense that different group members had conflicting goals or preferences and help resolve them or facilitate problem solutions requiring high levels of knowledge integration. Such a capability will only become possible as the field of AI makes progress in developing artificial social intelligence.
Artificial social intelligence
The next frontier in AI is artificial social intelligence (ASI) – the capacity of technology to pick up on the knowledge, beliefs, goals and emotions of users to anticipate their needs or potential response to events. To facilitate high CI, an ASI system would need to perform several kinds of functions that humans with high levels of social and emotional intelligence perform naturally. A high-functioning ASI technology would accomplish this by facilitating collective memory, attention and reasoning processes (Gupta and Woolley, 2021[26]).
ASI enhances collective memory by:
sensing the skills and abilities of the human participants
suggesting roles based on ability
conducting or facilitating information transfer so that all participants know who is doing what and have the relevant information for their task.
ASI enhances collective attention by:
reallocating work if some members are overloaded while others are underutilised
helping manage the work cue based on group priorities
drawing the group together to discuss tasks or problems as needed.
ASI enhances collective reasoning by:
sensing when individual teammates are feeling frustrated, unmotivated or distracted, and intervening or prodding the team to address it
sensing when participants are annoyed with or ignoring the ASI system itself and altering its approach accordingly.
Groups in which collective memory, attention and reasoning are functioning at a high level are characterised by several factors. They demonstrate a strong ability to match member skills with tasks and roles; they co‑ordinate task strategy so their work gets accomplished efficiently; and their members exhibit uniformly high and consistent levels of effort and commitment (Riedl et al., 2021[35]; Gupta et al., 2019[42]). The intelligence level of an ASI agent or system could be tested by its ability to enhance CI in human groups working across a range of different tasks and levels of interdependence. Specifically, it would be tested on its ability to enhance collective memory, attention and reasoning processes, and how well it led the group to exhibit optimal skill use, task strategy and high collective effort.
Agents with ASI could play a valuable role to facilitate CI in more challenging systems. Teamwork can become more complicated with more members or dispersion of members across different locations. In these contexts, agents with ASI could help facilitate CI in ways that humans alone could not manage. ASI abilities would need to be tested in highly complex environments, where teams are adapting to different types of tasks and levels of interdependence, and the development of collective memory, attention and reasoning is a significant challenge.
References
[11] Aggarwal, I. and A. Woolley (2018), “Team creativity, cognition and cognitive style diversity”, Management Science, Vol. 65/4, http://dx.doi.org/10.1287/mnsc.2017.3001.
[20] Aggarwal, I. and A. Woolley (2013), “Do you see what I see? The effect of members’ cognitive styles on team processes and performance”, Organizational Behavior and Human Decision Processes, Vol. 122, pp. 92-99, https://doi.org/10.1016/j.obhdp.2013.04.003.
[7] Aggarwal, I. et al. (2019), “The impact of cognitive style diversity on implicit learning in teams”, Frontiers in Psychology, Vol. 10/112, http://dx.doi.org/10.3389/fpsyg.2019.00112.
[37] Anderson, S., J. Goeree and C. Holt (2001), “Minimum-effort coordination games: Stochastic potential and logit equilibrium”, Games and Economic Behavior, Vol. 34/2, pp. 177-199, http://people.virginia.edu/~cah2k/mineff.pdf.
[13] Ausburn, L. and F. Ausburn (1978), “Cognitive styles: Some information and implications for instructional design”, Educational Communications & Technology Journal, Vol. 26, pp. 337-354.
[39] Barlow, J. and A. Dennis (2016), “Not as smart as we think: A study of collective intelligence in virtual groups”, Journal of Management Information Systems, Vol. 33/3, pp. 684-712, http://dx.doi.org/10.1080/07421222.2016.1243944.
[41] Bates, T. and S. Gupta (2017), “Smart groups of smart people: Evidence for IQ as the origin of collective intelligence in the performance of human groups”, Intelligence, Vol. 60, pp. 46-56, https://doi.org/10.1016/j.intell.2016.11.004.
[10] Blazenkova, O., M. Kozhevnikov and M. Motes (2006), “Object-spatial imagery: A new self-report imagery questionnaire”, Applied Cognitive Psychology, Vol. 20/2, pp. 239-263, http://dx.doi.org/10.1002/acp.1182.
[17] Blazhenkova, O., M. Becker and M. Kozhevnikov (2011), “Object–spatial imagery and verbal cognitive styles in children and adolescents: Developmental trajectories in relation to ability”, Learning and Individual Differences, Vol. 21/2, pp. 281-287, https://doi.org/10.1016/j.lindif.2010.11.012.
[12] Blazhenkova, O. and M. Kozhevnikov (2020), “Creative processes during a collaborative drawing task in teams of different specializations”, Creative Education, Vol. 11/9, p. 1751, http://dx.doi.org/10.4236/ce.2020.119128.
[18] Blazhenkova, O. and M. Kozhevnikov (2008), “The new object-spatial-verbal cognitive style model: Theory and measurement”, Applied Cognitive Psychology, Vol. 23/5, pp. 638-6563, http://dx.doi.org/10.1002/acp.1473.
[22] Chikersal, P. et al. (2017), “Deep structures of collaboration: Physiological correlates of collective intelligence and group satisfaction”, Proceedings of the 20th ACM Conference on Computer-Supported Cooperative Work and Social Computing, pp. 873-888, https://doi.org/10.1145/2998181.2998250.
[40] Credé, M. and G. Howardson (2017), “The structure of group task performance – a second look at ’collective intelligence’: Comment on Woolley et al. (2010)”, Journal of Applied Psychology, Vol. 102/10, pp. 1483-1492, https://doi.org/10.1037/apl0000176.
[25] DeChurch, L. and J. Mesmer-Magnus (2010), “The cognitive underpinnings of effective teamwork: A meta-analysis”, Journal of Applied Psychology, Vol. 95/1, pp. 32-53, http://dx.doi.org/10.1037/a0017328.
[34] Deming, D. (2017), “The growing importance of social skills in the labor market”, The Quarterly Journal of Economics, Vol. 132/4, pp. 1593-1640, http://dx.doi.org/10.1093/qje/qjx022.
[5] Engel, D. et al. (2014), “Reading the mind in the eyes or reading between the lines? Theory of mind predicts collective intelligence equally well online and face-to-face”, PLoS ONE, Vol. 9/12, p. e115212, http://dx.doi.org/10.1371/journal.pone.0115212.
[8] Glikson, E. et al. (2019), “Visualized automatic feedback in virtual teams”, Frontiers in Psychology, Vol. 16 April, https://doi.org/10.3389/fpsyg.2019.00814.
[42] Gupta, P. et al. (2019), “Digital nudging team processes to enhance collective intelligence”, Proceedings of Collective Intelligence 2019.
[26] Gupta, P. and A. Woolley (2021), “Articulating the role of artificial intelligence in collective intelligence: A transactive systems framework.”, No. 65, Proceedings of the Human Factors and Ergonomics Society.
[27] Hackman, J. (1987), “The design of work teams”, in Lorsch, J. (ed.), Handbook of Organizational Behavior, Prentice Hall, Englewood Cliffs, NJ.
[45] Hodzic, S. et al. (2018), “How efficient are emotional intelligence trainings: A meta-analysis”, Emotion Review, Vol. 10/2, pp. 138-148, http://dx.doi.org/10.1177/1754073917708613.
[1] Hughes, A. et al. (2016), “Saving lives: A meta-analysis of team training in healthcare”, Journal of Applied Psychology, Vol. 10/9, pp. 1266-1304, http://dx.doi.org/10.1037/apl0000120.
[49] Kidd, D. and E. Castano (2017), “Panero et al. (2016): Failure to replicate methods caused the failure to replicate results”, Journal of Personality and Social Psychology, Vol. 11/3, pp. e1–e4, http://dx.doi.org/10.1037/pspa0000072.
[44] Kidd, D. and E. Castano (2013), “Reading literary fiction improves theory of mind”, Science, Vol. 342/6156, pp. 377-380, http://dx.doi.org/10.1037/apl0000120.
[6] Kim, Y. et al. (2017), “What makes a strong team? Using collective intelligence to predict performance of teams in League of Legends”, Proceedings of the 20th ACM Conference on Computer-Supported Cooperative Work and Social Computing, Vol. 30 November, http://dx.doi.org/doi.org/10.5465/ambpp.2016.13564abstract.
[46] Kotsou, I. et al. (2019), “Improving emotional intelligence: A systematic review of existing work and future challenges”, Emotion Review, Vol. 11/2, pp. 151-165, http://dx.doi.org/10.1177/1754073917735902.
[28] McGrath, J. (1984), Groups: Interaction and Performance, Prentice-Hall, Englewood Cliffs, NJ.
[38] Meslec, N., I. Aggarwal and P. Curşeu (2016), “The insensitive ruins it all: Compositional and compilational influences of social sensitivity on collective intelligence in groups”, Frontiers in Psychology, Vol. 9 May, http://dx.doi.org/doi.org/10.3389/fpsyg.2016.00676.
[33] Murphy, K. (1989), “Is the relationship between cognitive ability and job performance stable over time?”, Human Performance, Vol. 2/3, pp. 183-200, http://dx.doi.org/10.1207/s15327043hup0203_3.
[15] Paivio, A. (1979), Imagery and Verbal Processes, Lawrence Erlbaum Assoc Inc., Hillsdale, NJ.
[48] Panero, M. et al. (2017), “No support for the claim that literary fiction uniquely and immediately improves theory of mind: A reply to Kidd and Castano’s commentary on Panero et al. (2016)”, Journal of Personality and Social Psychology, Vol. 112/3, http://dx.doi.org/10.1037/pspa0000079.
[47] Panero, M. et al. (2016), “Does reading a single passage of literary fiction really improve theory of mind? An attempt at replication”, Journal of Personality and Social Psychology, Vol. 111/5, pp. e46–e54, http://dx.doi.org/10.1037/pspa0000064.
[36] Paulus, P. and H. Yang (2000), “Idea generation in groups: A basis for creativity in organizations”, Organizational Behavior and Human Decision Processes, Vol. 82/1, pp. 76-87, http://dx.doi.org/doi.org/10.1006/obhd.2000.2888.
[2] Pearce, C. (2004), “The future of leadership: Combining vertical and shared leadership to transform knowledge work”, Academy of Management Executive, Vol. 18/1, pp. 47-57, http://dx.doi.org/10.5465/ame.2004.12690298.
[23] Phillips, K. and D. Loyd (2006), “When surface and deep-level diversity collide: The effects on dissenting group members”, Organizational Behavior and Human Decision Processes, Vol. 99/2, pp. 143-160, http://dx.doi.org/10.1016/j.obhdp.2005.12.001.
[9] Premack, D. and G. Woodruff (1978), “Does the chimpanzee have a theory of mind?”, Behavioral and Brain Sciences, Vol. 1/4, p. 515, http://dx.doi.org/10.1017/S0140525X00076512.
[14] Richardson, A. (1977), “Verbalizer-visualizer: A cognitive style dimension”, Journal of Mental Imagery, Vol. 1/1, pp. 109-126.
[35] Riedl, C. et al. (2021), “Quantifying collective intelligence in human groups.”, Proceedings of the National Academy of Sciences of the United States of America, Vol. 118/21, http://dx.doi.org/10.1073/pnas.2005737118.
[32] Schmidt, F. and J. Hunter (1998), “The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings”, Psychological Bulletin, Vol. 124/2, p. 262, http://dx.doi.org/10.1037/0033-2909.124.2.262.
[24] Sommers, S. (2006), “On racial diversity and group decision making: Identifying multiple effects of racial composition on jury deliberations”, Journal of Personality and Social Psychology, Vol. 90/4, pp. 597-612, http://dx.doi.org/doi.org/10.1037/0022-3514.90.4.597.
[30] Steiner, I. (1972), Group Process and Productivity, Academic Press, New York.
[43] Thaler, R. and C. Sunstein (2009), Nudge: Improving decisions about health, wealth, and happiness, HeinOnline.
[29] Thompson, J. (1967), “The structure of complex organizations”, in Organization Theory, Penguin Books, Harmondsworth.
[16] Ungerleider, L. and M. Mishkin (1982), “Two cortical visual systems”, in Ingle, D., M. Goodale and R. Mansfield (eds.), Analysis of Visual Behavior, MIT Press, Cambridge, MA.
[50] Voss, C. et al. (2016), “Superpower glass: Delivering unobtrusive real-time social cues in wearable systems”, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. New York, pp. 1218-1226, https://arxiv.org/abs/2002.06581v1.
[31] Wageman, R. (1995), “Interdependence and group effectiveness”, Administrative Science Quarterly, Vol. 40, pp. 145-180, http://dx.doi.org/doi.org/10.2307/2393703.
[4] Woolley, A. et al. (2010), “Evidence for a collective intelligence factor in the performance of human groups”, Science, Vol. 330/6004, pp. 686-688, http://dx.doi.org/10.1126/science.1193147.
[19] Woolley, A. et al. (2008), “Bringing in the experts: How team composition and work strategy jointly shape analytic effectiveness”, Small Group Research, Vol. 39/3, pp. 352-371, http://dx.doi.org/10.1177/1046496408317792.
[21] Woolley, A. et al. (2007), “Using brain-based measures to compose teams: How individual capabilities and team collaboration strategies jointly shape performance”, Social Neuroscience, Vol. 2, pp. 96-105, http://dx.doi.org/10.1080/17470910701363041.
[3] Wuchty, S., B. Jones and B. Uzzi (2007), “The increasing dominance of teams in production of knowledge”, Science, Vol. 316/5827, pp. 1036-1039, http://dx.doi.org/10.1126/science.1136099.