The Transdisciplinary Orientation Scale: Factor Structure and Relation to the Integrative Quality and Scope of Scientific Publications
- 1. Urban Affairs and Planning, School of Public and International Affairs, Virginia Tech, USA
- 2. Planning, Policy, and Design & Psychology and Social Behavior, School of Social Ecology, University of California, Irvine, USA
- 3. Statistical Technology Center, Monsanto, USA
Abstract
The collaborative success of cross-disciplinary scientific teams depends in part on the intrapersonal, interpersonal, and intellectual orientations individual team members bring to the group. This paper extends earlier conceptualizations of Transdisciplinary Orientation (TDO) --- defined as the values, attitudes, beliefs, conceptual skills and knowledge, and behavioral repertoires that predispose an individual to collaborating effectively in cross-disciplinary scientific teams---by developing a new metric to assess individual team members’ TDO. In study 1, we tested the internal consistency and factor structure of the TDO scale. Strongest support was found for a two-factor correlated model of transdisciplinary orientation with two dimensions—Values, Attitudes, and Beliefs (VAB) and Conceptual Skills and Behaviors (CSB). In study 2, we examined the relationships between individuals’ self-reported TDO and the intellectual qualities of their scientific outputs and their past experiences in cross-disciplinary teams. Individuals reporting higher TDO had published significantly more interdisciplinary research articles with higher potential societal impact as judged by independent raters. Past experience in interdisciplinary teams was found to be significantly and positively related to TDO. The TDO scale provides a useful metric for evaluating changes in individuals’ TDO and a team’s collective TDO that may result from their educational and collaborative research experiences.
Keywords
• Transdisciplinary orientation scale
• Scientific outputs
• Science teams
• Scientific collaboration
Citation
Misra S, Stokols D, Cheng L (2015) The Transdisciplinary Orientation Scale: Factor Structure and Relation to the Integrative Quality and Scope of Scientific Publications. J Transl Med Epidemiol 3(2): 1042.
ABBREVIATIONS
TDO: Transdisciplinary Orientation; SciTS: Science of Team Science; ID: Interdisciplinary; TD: Transdisciplinary; MD: Multidisciplinary
INTRODUCTION
Team-based scientific initiatives have dominated individualbased research across the natural sciences, engineering, social sciences, and humanities over the past five decades [1]. University scientists must increasingly work across disciplinary, institutional, sectoral, and geographic boundaries on complex social, environmental, and health problems with the goal of generating novel conceptual frameworks or translational outcomes that integrate and transcend the substantive concerns of any one discipline. Given the diverse perspectives, worldviews, and philosophical and methodological orientations represented in these science teams, and the complex research challenges being addressed, these collaborative initiatives are highly labor-intensive and conflict prone [2,3]. Moreover, the effectiveness of cross-disciplinary science teams is highly variable and depends on a web of intrapersonal, interpersonal, organizational, environmental, technological, and socio-political factors that facilitate or hinder team performance and collaboration [4].
Among the array of antecedent conditions known to facilitate or hinder collaborative processes and outcomes of science teams, members’ intrapersonal orientations toward cross-disciplinary research have been emphasized in recent studies [5-7]. Transdisciplinary orientation (TDO) is conceptualized as an intrapersonal disposition that emerges over the course of one’s scholarly career and predisposes an individual to engage in cross-disciplinary team-based or independent research [8]. A clear understanding of the personal qualities that constitute an individual’s TDO and how TDO can be calibrated is crucial for guiding educational and training efforts designed to promote the next generation of students’ and scholars’ engagement in cross-disciplinary collaborative research. Further, large-scale team science ventures could benefit from optimizing the design and management of the team to maximize their potential for success.
This study extends earlier conceptualizations of TDO by (1) creating a measurement tool to assess scholars’ TDO, (2) testing its reliability and factor structure, and (3) developing and testing hypotheses concerning the relationship of TDO to researchers’ past experiences in cross-disciplinary team science initiatives and the intellectual quality and scientific and societal impact of their research products as rated by independent evaluators. We first provide an overview of the TDO construct and its core dimensions. We then develop specific hypotheses linking scholars’ TDO to the outcomes of their scientific work and past experiences in science teams. The remainder of the paper presents the findings from two separate studies designed to test the reliability and factor structure of the newly developed TDO scale; and examines empirical links between an individuals’ TDO, prior team-based experiences, and the intellectual and integrative quality of their scientific work.
Core components of TD orientation
Following Stokols’ [8] conceptualization, we define transdisciplinary orientation as an intrapersonal disposition that emerges over the course of one’s scholarly career and predisposes an individual to engage in cross-disciplinary team-based or independent research. An individual’s intellectual orientation is shaped through her/his exposure to educational and research settings such as college, graduate school, postdoctoral departmental and university settings, cross-cultural field work, and experiences in community-based organizations or research initiatives and other professional settings. Graduate and postgraduate research and training experiences are known have a particularly strong impact on the development of one’s intellectual orientation [8, 9,10-18].
Rosenfield’s [19] and Kessel et al.’s [20] definition of transdisciplinary research focuses on the collaborative aspects of cross-disciplinary research -- an overarching term used to describe a continuum of research forms ranging from unidisciplinary to transdisciplinary research. These research forms vary according to the degree of communication and coordination among participants of the research team and the integrative quality of their research products. According to this conceptualization, transdisciplinary research involves close coordination and communication between team members leading to the development of shared conceptual frameworks or new methodologies that integrate and extend two or more disparate disciplines. While multidisciplinary and interdisciplinary research also bring together researchers from different disciplines collaborating on a common problem, they entail lower levels of interaction and coordination and do not synthesize disciplines to the extent that TD research does, or transcend the boundaries of any one discipline. Whereas this distinction is often applied to team-based research, it is important to note that multidisciplinary, interdisciplinary, and transdisciplinary research can also be pursued by individual researchers [2,21]. Thus, in the present conceptualization, TD orientation can be expressed both through individual as well as team-based scholarship.
We acknowledge that the characteristics of multidisciplinary, interdisciplinary, and transdisciplinary orientations partly overlap and scholars and researchers often shift their orientations depending on the demands of their research projects. Certainly, creative and significant scientific contributions can be achieved through unidisciplinary (UD), multidisciplinary (MD), and interdisciplinary (ID) intellectual orientations [22]. However, our conceptualization focuses on the qualities of TD orientation based on a robust body of research that points to the distinctive advantages of possessing broad-based knowledge from diverse disciplines, a variety interests and experiences that are integrated into scientific work, and the capacity for integrative and pluralistic forms of thinking [23-25]. Therefore, we posit that individuals with TD orientation are more likely to generate creative scientific outputs that transcend the contours of any one discipline, as compared to those with an ID, MD, or UD intellectual orientation. Moreover, a TD orientation may better enable the understanding of complex social and environmental problems that necessitate new syntheses of multiple knowledge bases and methodologies and more effective engagement in teams, as compared to UD, MD, or ID orientations [26-29].
Based on Stokols’ [8] conceptualization, TD Orientation is hypothesized to be composed of five personal attributes: TD values, TD beliefs, TD attitudes, TD conceptual skills, and TD behaviors.
(1) TD Values are the core guiding principles that incline an individual to participate and work effectively in cross-disciplinary teams and incorporate theories and methodologies from other fields into one’s own research. Individuals exhibiting TD values appreciate the importance of collaborative research in addressing social and environmental issues; are inclusive of worldviews and paradigms divergent from their own; are open to learning about theories and methods with which they are unfamiliar; and are respectful and tolerant toward other points of view.
(2) TD Attitudes: TD values support a variety of attitudes conducive to cross-disciplinary scholarship such as the willingness to invest time in learning about fields other than one’s own and to tackle complex problems even though doing so requires expending additional time and effort.
(3) TD Beliefs: The NCI Research Orientation Scale measures the degree to which individuals believe that the benefits of team science outweigh its costs and that they are more (or less) productive when they work in teams versus alone [30]. Further, team members’ beliefs about the theoretical and empirical discoveries and valuable translational outcomes resulting from cross-disciplinary research have been linked to improved collaborative outcomes among members of geographically dispersed teams in prior research [31]. These exemplify the kinds of beliefs associated with a TD Orientation.
(4) TD Conceptual Skills and Knowledge: Certain analytic skills and knowledge particularly exemplify a TD Orientation, such as the ability to view research problems and questions holistically from distinct vantage points and traverse multiple levels of analysis [32,33]. The capacity to create conceptual frameworks that account for the manifold causes and consequences of research problems, build on multiple fields, and synthesize diverse philosophical paradigms is yet another skill that is advantageous to cross-disciplinary work [10,13,34,35]. Other knowledge categories that are conducive to a TD orientation are systems thinking skills, knowledge management strategies [26,36], as well as stakeholder analysis and anticipatory governance skills for engaging in community-based participatory research [32,37].
(5) TD Behaviors: Behavioral routines and habits reflecting a TD Orientation include reading journals and books from different fields, attending conferences and meetings outside one’s own field, and engaging with colleagues from other disciplines to share and integrate ideas with the intent of collaborating on research. Further, using research methods from other fields, communicating respectfully online and offline with colleagues, and gaining experience in working in cross-disciplinary teams are behaviors conducive to cross-disciplinary team work [13,30,38- 40].
We propose that the synergistic combination of one’s TD values, attitudes, beliefs, conceptual skills and knowledge, and behavioral repertoires reinforce her/his capacity to excel in collaborative research projects, conduct independent broad-based integrative scholarship, and generate highly creative scientific products and societal innovations. Prior research has provided evidence for some of the benefits associated with separate facets of the TD Orientation. Having identified the core dimensions of TDO, we next provide an overview of the research findings linking various features of TD Orientation to collaborative processes and scientific outcomes.
TD orientation and its relationship to collaborative processes and outcomes
The findings from earlier studies reveal the influence of personal attributes on group processes and outcomes in science teams. In particular, four intrapersonal dimensions (psychological traits) have been found to impact team science processes and outcomes in prior research.
Value dimensions: Inclusionary and pluralistic values enable individuals to manage certain challenges that arise in cross-disciplinary teams such as resisting in-group and outgroup biases and the usual tendency to associate primarily with team members whose perspectives are highly similar to one’s own [41,42,43].
Attitudinal dimensions such as preferences for conducting traditional research situated within a single discipline, unwillingness to adjust one’s own disciplinary conceptual schemas to fit the demands of team work, resistance to understanding and accepting divergent paradigms, values, worldviews, and methodological approaches may undermine collaboration in cross-disciplinary teams [44]. Lack of dedication and commitment necessary for cross-disciplinary team work 3,5,34,38,45-50], and low levels of trust concerning the competence and reliability of team members [51,52] have been found to weaken interpersonal trust among team members and lead to conflict.
Conceptual dimensions like the inability to approach problems and questions from a holistic perspective and to creatively integrate concepts or methods from different disciplines can undermine team members’ capacity to generate scientific and societal innovations [46,53-56].
Behavioral dimensions such as failure to communicate openly with team members about ideas, linguistic barriers to communication, poor conflict resolution skills, lack of patience with labor- and time-intensive collaborative ventures, and lack of experience in working in cross-disciplinary scientific teams pose significant barriers to collaborative research [38,45,46,49,50].
Additionally, the NCI Research Orientation Scale [30] measures individuals’ inclination to participate in cross-disciplinary research teams, ranging from a unidisciplinary research orientation to a transdisciplinary research orientation. A transdisciplinary research orientation is characterized by attitudes and beliefs that predispose individuals toward engaging in collaborative research. In NCI’s evaluation study of the Transdisciplinary Research on Energetics and Cancer (TREC) initiative, investigators with a transdisciplinary research orientation engaged in more cross-disciplinary collaborative activities such as reading journals and attending conferences outside of one’s field and establishing connections with researchers from other disciplines that lead to collaborative work.
The TDO scale developed here builds on prior conceptualizations of the values, attitudes, beliefs, conceptual skills, and behaviors that predispose individuals to cross-disciplinary research, and on earlier findings linking certain intrapersonal attributes to the processes and outcomes of cross-disciplinary research. Although there is evidence of intrapersonal and behavioral dimensions influencing collaborative processes in cross-disciplinary teams, there is no research on how these values, attitudes, beliefs, conceptual skills and behaviors jointly influence group processes and outcomes. Moreover, little is known about the influence of a composite TD orientation (encompassing multiple value, attitude, belief, conceptual, and behavioral dimensions) on the qualities of scholarly products generated by individual researchers and teams. The next section summarizes progress to date on the evaluation of cross-disciplinary research products.
Evaluating the scientific products of collaborative research
Some studies have employed bibliometric indicators of productivity and scientific impact such as publication rates, journal impact factors, and number of co-authors to assess the effectiveness of transdisciplinary research teams and training programs [57, 58]. However, there is very little research on the qualitative attributes of research products emanating from science teams and individual scholars. Mitrany and Stokols [59] gauged the transdisciplinary qualities of doctoral dissertations by having independent raters assess the contextual scope of the research topic addressed in the written dissertation, the variety of research methods incorporated, the number of levels of analysis bridged, and the potential for translating the research into practical problem solving solutions. Later, Hall, Stokols, et al. [30] developed a Written Products Protocol (WPP) to evaluate the intellectual scope and integration level of developmental project proposals written by members of the National Cancer Institute’s (NCI) Transdisciplinary Research on Energetics and Cancer (TREC) centers. The NCI WPP assesses criteria such as the number of disciplines represented in a research proposal, the levels of analysis bridged, and type of cross-disciplinary integration achieved ranging from unidisciplinary to transdisciplinary.
Misra, Harvey, et al. [34] adapted the NCI-TREC Written Products Protocol and Mitrany and Stokols’ [59] evaluation criteria to assess the intellectual and integrative quality and creativity of students’ projects and papers. They compared the integrative and intellectual qualities of research projects and papers of undergraduate students who had undergone a training program in interdisciplinary theories and methods with another set of undergraduate students who had not received interdisciplinary training. Independent raters assessed the degree of integration of students’ papers and projects on a 10-point scale. Highly integrative papers bridged ideas from multiple disciplines, developed conceptual frameworks and theories that synthesized more than one discipline, and/or combined multiple methodological approaches. More recently, the National Academies Keck Futures Initiative (NAKFI) evaluated the scientific and societal outcomes of the NAKFI interdisciplinary seed grant program. A new rating instrument was developed based on the NCI-TREC Written Products Protocol and used by expert reviewers from relevant fields to estimate the scholarly qualities of seed grant proposals (NCI, 2012). In addition to the measurement criteria used in the NCI-TREC research [30], the NAKFI study incorporated assessments of the creativity, and potential scientific and societal impact of the research described in the grant proposal.
The present research draws on and connects these lines of research regarding: (1) intrapersonal dimensions that influence collaborative processes and outcomes of team science; and (2) the evaluation of the transdisciplinary qualities of scholars’ and research teams’ scientific products such as grant proposals, student papers and projects. We develop a measurement tool --- the Transdisciplinary Orientation (TDO) Scale --- to assess the combined impact of TD values, attitudes, beliefs, conceptual skills and behaviors, and investigate the empirical links between TDO and the integrative and scholarly qualities of researchers’ journal articles. Three sets of hypotheses were tested:
Hypothesis 1: Academics’ self-reported TDO is positively related to the extent of cross-disciplinary integration reflected in scientists’ research articles. That is, academics who report a high level of TDO also produce more interdisciplinary research products as rated by independent raters.
Hypothesis 2: Transdisciplinary orientation is positively related to researchers’ past experience in team science initiatives. That is, academics who report higher TDO scores are more likely to have participated in cross-disciplinary team based initiatives than those with lower TDO scores.
Hypothesis 3: Similarly, researchers’ who have led cross-disciplinary research teams are more likely to report higher levels of TDO.
MATERIALS AND METHODS
The following sections describe the participants, procedures, and measures for the two studies conducted to test the aforementioned hypotheses.
Study 1: Testing the reliability and factor structure of the TDO scale
Participants and Procedures: We invited academics and researchers to participate in a study of factors that influence the success of interdisciplinary teams by posting an online version of the TDO Scale on a variety of academic mailing lists representing a broad array of disciplines. The survey took about 15 minutes to complete. Our online sample included academics from the liberal arts, social sciences, natural sciences, and engineering. A total of 150 researchers and academics participated in this study. The mean age of the sample was 48.8 years (SD= 13.4). Sixty-four percent of the sample was male. Twenty-four percent of the sample was at the Associate Professor level, 20 percent at the Professor level, 19 percent was at the Assistant Professor level, and the remaining 37 percent was at the Lecturer or Research Scientist level, or in another research or academic position.
Measure
Transdisciplinary Orientation: The 12-item TDO scale (α=0.93; M=4.09; SD=.74) was administered to participants. Items in this scale pertained to scholars’ thoughts, expectations, and behaviors about their research to date. On a five-point Likert scale, participants were asked to indicate how strongly they agree with statements such as: “My research to date reflects my openness to diverse disciplinary perspectives when analyzing particular problems” and “I generally approach scientific problems from a multi-level perspective that encompasses both micro- and macro- level factors.” This 12-item scale assessed TD values, attitudes, and beliefs (6-items, α= 0.87) and TD conceptual skills and behaviors (6 items, α=0.88). A copy of the TDO Scale is provided in Appendix A.
Data Analytic Framework
After creating the summated scale for transdisciplinary orientation encompassing values, attitudes, beliefs, conceptual skills, and behaviors, we conducted an item analysis to assess item-to-total and inter-item correlations among the TDO scale items. These tests also were conducted for the values, attitudes, and beliefs (VAB) and conceptual skills and behaviors (CSB) sub-scale items. None of the item-to-total correlations were less than .5 and none of the inter-item correlations were less than .3, the thresholds for deletion [60]. The hypothesized factor structure of the TDO construct was tested through confirmatory factor analyses. Finally, a series of multiple regression analyses were conducted to test the hypothesized relationships among independent and dependent measures.
RESULTS
Confirmatory Factor Analyses
Confirmatory factor analysis was conducted using the Maximum Likelihood method with JMP 10.0.0. Strongest support was found for the two-factor correlated model of transdisciplinary orientation with two dimensions—Values, Attitudes, and Beliefs (VAB) and Conceptual Skills and Behaviors (CSB). Several goodness-of-fit statistics were applied to this two-factor correlated model (Figure 1).
Figure 1 Two-Factor Correlated Model of Transdisciplinary Orientation with Values, Attitudes and Beliefs and Behaviors and Conceptual Skills as the Two Correlated Factors. The goodness of fit parameters are as follows:Ratio of the minimum discrepancy and degrees of freedom= 2.833; adjusted goodness of fit index(AGFI)=0.76; Incremental Fit Index =0.90; Normed Fit Index ( Bentler-Bonett NFI; Bentler&Bonett, 1980), also known as Tucker-Lewis Index=0.85; Non-normed Fit Index (Bentler-Bonett Non-normed Index)=0.89; Comparative Fit Index (Bentler Comparative Fit Index)=0.90; Parsimonious NFI=0.66; RMSEA=0.12. The covariance and correlation between the two factors are 0.592 and 0.872.
Figure 1 also shows the unstandardized factor loadings for each variable and covariance between the two factors in the model. The standardized factor loadings for the two-factor model were found to be large, and the critical ratio of the regression weights was statistically significant at p<.001 for all items, suggesting convergent validity. The covariance between VAB and CSB was 0.59 and the correlation was 0.87, which is high but still suggests adequate discriminant validity.
Study 2: Relationship of TDO to the integrative quality and scope of scientists’ publications and past team-based experiences
Participants and Procedures: We compiled a list of faculty members in the social, biological, agricultural, and engineering science departments from a large public university in Virginia, USA. Disciplines and/or multidisciplinary fields represented in this list included computer science, geosciences, geography, fish and wildlife conservation, political science, urban planning, economics, forest resources and conservation, biomedical engineering, civil engineering, education, science, technology, and society, psychology, agricultural economics, business management, human nutrition, food and exercise, public administration, medicine, government and international affairs, and sociology. From this list, a total of 223 faculty members were randomly selected representing diverse academic fields or disciplines. These faculty members were invited to respond to an online survey. The survey took 15 – 20 minutes to complete. Seventy-six individuals (mean age=48.41 years; SD=11.90; 64.47 percent Male) responded to the survey, yielding a response-rate of 34.08 percent.¹
Selection, Preparation, and Rating of Executive Summaries: For each of the 76 individuals who responded to the survey from the university-based sample, we compiled a list of their publications within the last five years (2009 to 2013) in which they were the lead or sole author. Then we randomly selected one research article from this list of publications. The first author of the present paper and two graduate students reviewed each article thoroughly and distilled it into a onepage executive summary describing the background, research questions or hypotheses, method, results, and conclusions. Reviewers were blind to scholars’ scores on the TDO Scale. This was done to facilitate raters’ evaluation of the integrative and intellectual qualities of the research.
Three trained graduate student reviewers independently rated the integrative qualities and scientific contributions of the research summaries. These raters were not subject experts, but they were trained in evaluating the cross-disciplinary qualities of research products. The training involved a workshop on the science of team science (SciTS), in which participants were assigned readings of foundational papers and chapters in the SciTS field. These publications were discussed during the workshop. Additionally, participants learned about different metrics and measures used to evaluate research outcomes of team-based scientific enterprises, completed exercises that involved rating research papers, and developed a common understanding of the different types and levels of cross-disciplinary research. After the training, each rater reviewed all the executive summaries and rated them on a number of dimensions including extent of integration, disciplines represented, levels of analyses mentioned, and potential scientific and societal contributions of the research.
Measures
Independent Variable: Transdisciplinary orientation: The 12-item TDO scale as described in Study 1 was used for Study 2 (α=0.92; M=3.94; SD=.77).
Dependent Variables
Extent of cross-disciplinary integration: Independent raters assessed each executive summary on nominal scale where 1= unidisciplinary research; 2= multi-disciplinary research; 3=inter-disciplinary research; and 4= trans-disciplinary research, adapted from the NCI-TREC and NAKFI studies [30, 61]. Rosenfield’s [19] definitions of different types of cross-disciplinary integration were used in this item. Ratings were averaged across the three raters (M=2.05; SD=.81).
Number of levels of analysis mentioned or implied in the article: Independent raters were asked to indicate all levels of analysis mentioned or implied as part of the project background, purpose, methods, analysis, and results in the executive summary. These levels of analysis included the nano, cellular & molecular, individual, group & interpersonal, organizational and institutional, community and regional, societal and national and global levels. This item was adapted from the NCI-TREC and NAKFI studies [30, 61]. The number of levels of analysis were added to calculate the total number of levels of analysis mentioned or implied in the summary and averaged across the three raters (M=1.71; SD=.58).
Contributions of the research to theory, methodology, and practice: Raters were asked to indicate whether the research described in the executive summary had the potential to make an important contribution to the development of a new theory, extension of an existing theory, development of a new methodology, and / or the development of a new practical application. This item was adapted from the NAKFI study on the evaluation of seed grant proposals [61]. The total number of scientific contributions of the research was calculated by adding the individual contributions to theory, methodology, and/or practice and averaged across the three raters (M=1.11; SD=.35).
Number of disciplines represented: The number of disciplines represented in the research described in the summary were listed by each independent rater and then summed to yield the total number of disciplines and averaged across the three raters (M=2.25; SD=.74). This item was adapted from the NCITREC and NAKFI studies [30, 61].
ID experience: Participants were asked to report how many interdisciplinary teams they have been a member of in the past (M=5.54; SD=5.13). Participants also were asked to report the number of interdisciplinary teams they had led (M=3.25; SD=4.04).
Covariates: Age (M=48.41 years; SD=11.90), gender (35.5 percent female; females were coded as 1 and males as -1 in regression analysis), and number of years in academia (M=13.8 years; SD=10.5) were entered as covariates in these analyses.
RESULTS
Correlations between Key Study Variables
Correlations were estimated using Restricted Maximum Likelihood (REML) method in JMP (Table 1).
Table 1: Correlations between Key Study Variables (n=76). |
||||||||||
|
TDO Score |
Total Number of Disciplines |
Total Number of Levels of Analysis |
Total Number of Scientific Contributions |
ID Experience |
Age |
Degree of Cross Disciplinary Integration |
Potential Scientific Impact
|
Potential Societal Impact
|
Number of Years in Academia |
TDO Score
|
1.00 |
-0.08 |
0.12 |
0.15 |
0.24 |
0.18 |
0.13 |
0.01 |
0.12 |
0.28 |
Total Number of Disciplines
|
-0.08 |
1.00 |
0.34* |
-0.01 |
0.18 |
-0.00 |
0.54* |
-0.01 |
0.09 |
-0.24 |
Total Number of Levels of Analysis
|
0.12 |
0.34* |
1.00 |
-0.10 |
0.01 |
0.08 |
0.49* |
0.03 |
0.11 |
0.11 |
Total Number of Scientific Contributions
|
0.15 |
-0.01 |
-0.10 |
1.00 |
-0.21 |
-0.16 |
0.35* |
0.32* |
0.16 |
-0.23 |
ID experience
|
0.24 |
0.18 |
0.01 |
-0.21 |
1.00 |
0.31 |
0.15 |
-0.20 |
0.06 |
0.14 |
Age |
0.18 |
-0.00 |
0.08 |
-0.16 |
0.31 |
1.00 |
-0.11 |
-0.28 |
-0.04 |
0.84* |
Degree of Cross Disciplinary Integration |
0.13 |
0.54* |
0.49* |
0.35* |
0.15 |
-0.11 |
1.00 |
0.17 |
0.36* |
-0.23 |
Potential Scientific Impact
|
0.01 |
-0.01 |
0.03 |
0.32* |
-0.20 |
-0.28 |
0.17 |
1.00 |
0.00 |
-0.14 |
Potential Societal Impact
|
0.12 |
0.09 |
0.11 |
0.16 |
0.06 |
-0.04 |
0.36* |
0.00 |
1.00 |
-0.12 |
Number of Years in Academia |
0.28 |
-0.24 |
0.11 |
-0.23 |
0.14 |
0.84* |
-0.23 |
-0.14 |
-0.12 |
1.00 |
*p<.05; There are 31 missing values. The correlations are estimated by REML method. Abbreviations: TDO: Transdisciplinary Orientation; ID: Interdisciplinary |
Using REML is a default method to estimate the covariance or correlation structure when there are missing data and the number of observations and the number of variables are not very large. This method uses all the available data and is less biased than pairwise correlations when there are missing data.
Multiple Regression Analysis
Multiple regression analyses, in which age, gender and number of years in academia were entered as covariates, were used to test the hypotheses in this study. Tables 2, 3, and 4
Table 2: Multiple Regression Analysis between TDO Score and Extent of Cross-Disciplinary Integration of the Scientific Publication (n=76). |
||||
|
Extent of Cross Disciplinary Integration |
|||
Variable |
b |
SE(b) |
t ratio |
df |
Intercept |
-0.33 |
0.99 |
-0.33 |
34 |
Age |
0.05 |
0.02 |
2.34** |
34 |
Gender[Female] |
0.20 |
0.13 |
1.51 |
34 |
Time in academia in years |
-0.07 |
0.02 |
-3.15** |
34 |
TDO score |
0.34 |
0.16 |
2.09** |
34 |
** p<.01 (one-sided), R2=0.27. Abbreviations: TDO: Transdisciplinary Orientation |
Table 3: Multiple Regression Analysis between TDO Score and Potential Societal Impact of the Research Reported in the Article (n=76). |
||||
|
Potential societal impact |
|||
Variable |
b |
SE(b) |
t ratio |
df |
Intercept |
1.15 |
1.06 |
1.09 |
34 |
Age |
0.01 |
0.02 |
0.67 |
34 |
Gender[Female] |
-0.12 |
0.14 |
-0.87 |
34 |
Time in academia in years |
-0.03 |
0.02 |
-1.43 |
34 |
TDO score |
0.32 |
0.17 |
1.87* |
34 |
*p<.05 (one-sided), R2=0.16. |
Table 4: Statistical Regression Analysis between TDO Score and ID Experience. |
||||
|
ID Experience |
|
||
Variable |
b |
SE(b) |
t ratio |
df |
Intercept |
-17.03 |
6.90 |
-2.47** |
28 |
Age |
0.43 |
0.14 |
3.10** |
28 |
Gender[Female] |
-0.03 |
0.92 |
-0.03 |
28 |
Time in academia in years |
-0.47 |
0.15 |
-3.17** |
28 |
TDO score |
2.21 |
1.10 |
2.01* |
28 |
** p<.01 (one-sided); p<.05 (one-sided); R2=0.32. |
present the results of these analyses. We found that researchers reporting higher levels of TDO produced scientific papers that were rated to be significantly more interdisciplinary by independent raters (b=.34; p<.01). Participants who reported more experience in participating in cross-disciplinary team science ventures also reported significantly higher level of TDO (b=2.21; p<.01). Participants’ prior experience in leading interdisciplinary teams was not found to have a significant relationship to their TDO scores. Further, higher self-reported level of TDO was significantly and positively correlated with independent ratings of the potential societal impact of the research reported in the scholar’s article (b=.32; p<.05).
Our other hypotheses were partially supported by the data. TDO score was positively and marginally significantly related to raters’ evaluations of the creativity (b=.28; p<.07) and intellectual quality of scientific publications (b=.18; p<.08). We did not find any significant relationships between TDO scores and the number of disciplines mentioned or implied in the research, the number of levels of analysis, the scientific impact, or the total number of scientific contributions of the research.
CONCLUSION
This study contributes to the science of team science (SciTS) literature in three respects. First, it provides a reliable and preliminarily validated tool to measure scientists’ and scholars’ personal disposition toward transdisciplinary research. We found support for the hypothesized two-factor structure of the transdisciplinary orientation construct, composed of two correlated sub-constructs Values, Attitudes, and Beliefs and Conceptual Skills and Behaviors. Second, this study evaluated the empirical links between a scholar’s TDO and the integrative and intellectual quality of his/her scholarly outputs. As predicted, scholars reporting higher levels of TDO produced scientific outputs that were judged to be more interdisciplinary and transdisciplinary in nature as rated by independent evaluators- --that is, they were more successful in synthesizing concepts, ideas, or methods from multiple disciplines and extending the boundaries of any single discipline, compared to scholars who did not possess values, attitudes, conceptual skills, and behavioral routines exemplifying a TDO. TD oriented researchers also produced scientific outputs that were judged to have greater translational, policy, and practical relevance. Third, the study found that prior experience in participating in cross-disciplinary team science is positively linked to researchers’ TDO.
We established only partial support for our other hypotheses, which should be reexamined in the future with larger samples and different research designs. The scholarly outputs of TD-oriented scholars were judged to be marginally more creative and of higher intellectual quality compared to less TD-oriented scholars. Other indicators of transdisciplinarity such as the number of disciplines mentioned in a published article, the number of levels of analysis (nano through societal), and the number and type of scientific contributions (e.g., advances in theory, methodology, or practice) were not found to be significantly linked to researchers’ TDO.
The reported findings must be considered in light of certain limitations of this research. First, it is possible that the findings would be more robust if domain experts (scholars with expertise in the topic areas addressed in each publication) rated the papers rather than non-expert raters with training in interdisciplinary team science. Presumably, expert researchers would be more capable of assessing the scientific contributions and the future scientific impact of an article as compared to non-expert evaluators. One explanation for the lack of significant support for our hypothesis concerning the relationship of TDO to the potential scientific impact of research articles is that the independent raters reported difficulty in assessing the scientific impact of articles because of their lack of knowledge about the state of the art in the relevant field/s. This issue should be explicitly addressed in future research.
Second, our findings are based on rating only one randomly selected publication of each scholar within the past five years. Whereas it is reasonable to assume that a scholar’s intellectual orientation, unidisciplinary or transdisciplinary, will be reflected in all his/her publications in which s/he is the lead or primary author, it is possible that at least some researchers engage in different types of scholarship throughout their careers. Some projects may be more unidisciplinary than others; some may be collaborative and others more independent. Further, junior scholars may focus on more unidisciplinary work in the beginning of their scholarly careers to satisfy tenure and promotion criteria as compared to later in their careers when they have more liberty to pursue other interests and undertake projects that do not fit squarely into any single discipline or domain. Thus, the findings would have been strengthened had we rated two or three research articles for each respondent. Also, it would further benefit the conclusions drawn here if we had evaluated other kinds of products, such as grant proposals and books, in addition to research articles. Often, grant proposals present very timely and cutting edge research ideas. Similarly, books afford a more comprehensive and in-depth exploration of research topics and are more amenable to the development of theory than research articles, which are more circumscribed and must comply with manuscript length constraints imposed by journal editors.
Third, although these findings offer initial evidence for the concurrent and content validity of the TDO construct, we did not test its predictive validity or track its change over time with exposure to different educational and professional settings. TD orientation is conceptualized as a personal disposition that develops over the course of an individual’s scholarly career. Future studies should track changes in scholars’ TDO (at both individual and team levels) over time to gauge the efficacy of interdisciplinary training programs through panel studies. Moreover, longitudinal research designs should examine the predictive power of individual team members’ TDO on the effectiveness of cross-disciplinary teams.
Limitations notwithstanding, this research makes a number of theoretical and practical contributions. It extends prior measurements of interdisciplinary or transdisciplinary intellectual values, beliefs, and attitudes, such as those included in NCI Research Orientation Scale [30], and the Collaborative Activities Scale (CAS) and Interdisciplinary Attitudes (IA) scale [34] by developing a comprehensive instrument based on Stokols’ [8] conceptualization of TDO, which is composed more broadly of TD values, attitudes, beliefs, conceptual skills, and behaviors. This is the first paper to develop a holistic conceptualization of this personal disposition and provide a composite tool for calibrating it. Further, it builds on earlier research [24] suggesting the benefits of encouraging broad gauged, integrative, and pluralistic forms of thinking and conceptual, interpersonal, and management skills among budding scholars and researchers.
This research has several practical implications. For instance, to the extent that these findings are replicated among different samples, new curricula as well as short- and longer-term training strategies should be designed to cultivate TDO among undergraduate, graduate, and postdoctoral students and scholars [22,62]. Secondly, information about potential team members’ and research partners’ TDO can be useful in strategically designing cross-disciplinary science teams to maximize their probability for success [4,46]. As noted earlier in the paper, intrapersonal traits such as lower levels of openness to other disciplinary concepts and methods, being less amenable to learning about different disciplines, and interpersonal tensions have been found to be detrimental to collaborative processes and outcomes team science [45,51, 56].
Promising directions for future research include: (1) investigation of the influence of individual members’ TDO on team processes and outcomes using both quantitative and qualitative methods; (2) investigation of the collective TD orientation of teams (for example, is TD orientation additive across individual team members?); and (3) development of a tool for measuring TDO among undergraduate and graduate students. As an initial step toward those goals, the present research contributes a reliable and partially validated tool for evaluating changes in individual and collective levels of TDO over the course of scientific projects; and an empirical foundation for future efforts to fine-tune the composition of cross-disciplinary science teams, as well as their collaborative processes and outcomes.
Appendix A: Transdisciplinary Orientation Scale
The following items pertain to your thoughts, expectations, and behaviors about your research to date. Please indicate how strongly you agree with each of the following statements.
1. My research to date reflects my openness to diverse disciplinary perspectives when analyzing particular problems. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
2. My research to date reflects my interest in learning about disciplinary concepts and theories in addition to the ones I am most familiar with. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5(strongly agree) |
0 | 0 | 0 | 0 | 0 |
3. My research to date reflects my interest in learning about new research methods that are different from the ones I am most familiar with. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
4. I would describe myself as someone who values interdisciplinary collaboration. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
5. I am willing to invest the time required for learning about fields that are different from my own. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
6. I enjoy tackling the challenges posed by working on complex problems, even if doing so requires me to expend extra time and effort. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
7. I generally approach scientific problems from a multi-level perspective that encompasses both micro- and macro- level factors. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
8. My research projects to date reflect my ability to conceptualize complex problems by identifying various situation-specific factors that account for those problems. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
9. My research to date reflects my ability to create conceptual frameworks that bridge multiple fields. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
10. My research projects to date reflect my ability to think broadly about complex problems. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
11. In my own work, I incorporate perspectives from fields that are different from my own. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
12. In my own research, I use research methods drawn from more than one discipline rather than relying exclusively on a single disciplinary approach. | ||||
1 (strongly disagree) | 2 | 3 | 4 | 5 (strongly agree) |
0 | 0 | 0 | 0 | 0 |
ACKNOWLEDGEMENTS
This research was supported by a grant to Dr. Shalini Misra from the Institute for Society, Culture, and Environment (Virginia Tech). We would like to thank Jamie Genevie and Meliha Aljabar for their help in data collection for this study.
REFERENCES
15. National Academy of Sciences. Facilitating interdisciplinary research. Washington, DC. 2005.
31. Olson GM, Zimmerman A, Bos N. Scientific collaboration on the Internet. The MIT Press. 2008.
41. Byrne D. The attraction paradigm. New York: Academic Press. 1971.
42. Tajfel H. Social identity and intergroup behavior. Cambrige, UK: Cambridge University Press. 1982.
48. Giri AK. The calling of a creative transdisciplinarity. Futures. 2002; 34: 103-115.
61. National Cancer Institute. NCI Team Science Toolkit. 2012.