Scope

All campus-level faculty scholarly activity systems employed on the Indiana University Bloomington campus for the evaluation of activities of a faculty member, department or unit.

Back to top

Policy Statement

1. Definitions and principles
   The evaluation of creative and scholarly activities is best performed by peer review with the input of experts in the field of the individual or unit being evaluated. The Bloomington Faculty Council recognizes that the output of systems which collect and allow the comparison of information on scholarly and creative activities can be useful adjuncts to peer review. These outputs are best used as just one of several inputs to the peer review process. Specifically, it is recognized that the output of these systems is only as good as the input data, the algorithms used to compute the output data, and the methods utilized to display/present the data. This policy establishes requirements surrounding the selection and use of technology systems used to evaluate individual faculty, departments, schools, or colleges at Indiana University, Bloomington.
   
   
   1. Definitions
      Faculty scholarly activity systems are any systems that monitor and aggregate information on activities related to performance in research, teaching, service, and creative activity and used for peer or administrative evaluation. These include faculty profile systems (such as those used for faculty annual review), research information systems (e.g., euroCRIS), and other research output systems in which individual faculty control the input of data, as well as external systems which provide individual or aggregated external data. These systems may provide certain summary indicators of quality or performance. Included within the scope of this policy are all systems that provide non-anonymized data at the individual, department, school, or college level, regardless of whether they are intended for individual evaluation or the evaluation of programs and departments.
   2. Principles
      The principles listed below are informed by the Leiden Manifesto, an international generated statement of principles guiding research evaluation¹.
      1. Disciplinary difference. Scholarship is increasingly heterogeneous. Faculty activity systems that generate performance or quality indicators must attempt to take disciplinary differences into account in both the inputs and outputs of the system, using field normalized indicators where possible and providing explicit acknowledgement of disciplinary biases in the system. Information on these biases, when known, must be provided so specific fields can judge what particular disciplinary biases were included, and apply these accordingly. If the systems generate comparisons to other faculty or units, either within the campus, University, or elsewhere, the contextual/background, size, nature and composition of the comparison groups need to be clearly identified, and should be comparable in terms of the discipline and ideally the sub-discipline.
      2. Complementary methods. Indicators are inherently reductionist and should only be used to supplement peer review provided by individual, departmental or School reviews, rather than replace other forms of review (such as peer review) that more fully contextualize the varied nature of academic performance.
      3. Transparency. The source of data and output indicators for faculty activity systems should be made transparent. The scope and use of the system must be clearly described prior to its use. This description of use must specifically include how the input data are transformed to the output aggregates and how any output indicators are generated from the input data.
      4. Accuracy. Faculty activity systems must contain accurate data. In order to ensure the data provided are accurate, data from the system should be accessible or accessible upon request to faculty and administrators. If any system data are deemed to be inaccurate by the individual or group to which it corresponds, opportunity to redress any incorrect data must be provided.
2. Implementation
   
   
   1. Selection of New Systems. In order to be considered to have valid data for the evaluation of faculty creative, research, service, and teaching activities, faculty activity systems selected for purchase or use at Indiana University Bloomington must meet the principles outlined in Section 1b. To safeguard these principles, any new faculty evaluation technology system must have its scope and use approved, prior to acquisition, by either a Bloomington Faculty Council Committee charged with oversight of that review process or a campus committee consisting of at least one representative of the Research Affairs Committee of the Bloomington Faculty Council, one member of the Faculty Affairs Committee, one member from the Technology Policy Committee, and one member from or appointed by the Vice Provost for Faculty and Academic Affairs office. The user of the system may provide supplementary information to the review committee to evaluate during the review process if there are extenuating circumstances.
   2. Review of Systems in Use. If a faculty member or administrator believes that a system currently employed on campus does not meet these principles, and the matter cannot be resolved through consultation with the Office of the Vice Provost for Faculty and Academic Affairs, they should bring their concerns to the attention of the Executive Committee of the BFC. The Executive Committee will decide whether to initiate a review of the system, either by referring this to an existing Bloomington Faculty Council Committee or convene an ad hoc subcommittee comprising at least one member each from the Research Affairs Committee, the Faculty Affairs Committee, and the Technology Policy Committee, and an appointee of the Vice Provost for Faculty and Academic Affairs office, as well as other external members as deemed necessary to offer guidance. The subcommittee will examine the system and determine the degree to which the system adheres to the principles of this policy. If failures are identified, this group will negotiate remediation with the executive sponsor of the system or their technical designate.
   3. Recommendation. Any acquisition or use of IUB campus-level faculty scholarly activity systems should comply with the principles of this policy or have remediation plans that will align with these principles. If remediation cannot be made to bring the system into alignment the BFC Executive Committee will evaluate the issue and bring it to full Council for a vote on continued use of the system. This recommendation will then be forwarded to the executive sponsor of the system for compliance. This policy will be reviewed at least every 8 years by the Research Affairs, Technology Policy, and Faculty Affairs Committees.
   4. Related Policies. This policy is related and complementary to the University Policy (DM-01) Management of Institutional Data, which focuses specifically on the use and misuse of data.

^1. http://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351 

Back to top

History

Approved: BFC 04/26/2016, 02/06/2018

Previous  versions of the policy: 04/25/2016 - 02/05/2018

Back to top