Home | Sitemap | Contact | NRF Home

Monitoring & Evaluation

 

Enhancing the quality of research in South Africa
Evaluation and rating
 
Administrative and decision-making structures
Specialist committees
NRF Assessors
Assessment Panels
Executive Evaluation Committee (EEC)
Appeals Committee
NRF Monitoring & Evaluation
 

Specialist Committees

Composition
Three to six respected members of the South African research community in each of the following fields of research:

List of NRF specialist committees 2009
 Animal and Veterinary Sciences

 Information Technology

 Anthropology, Development Studies,  Geography, Sociology and Social Work

 Law
 Biochemistry, Molecular and Cell Biology  Literary Studies, Languages and Linguistics
 Chemistry  Mathematical Sciences
 Communication, Media Studies, Library and Information Sciences  Microbiology and Plant Pathology
 Earth Sciences  Performing and Creative Arts, and   Design
 Economics, Management, Administration and Accounting  Physics
 Education  Plant Sciences
 Engineering  Political Sciences, Policy Studies and Philosophy
 Health Sciences  Psychology
 Historical Studies  Religious Studies

Appointment
New members are appointed by the responsible NRF Executive member, after one or more of the following steps:
 A call for nominations.
 Consultation with the appropriate, already constituted, Specialist Committee.
 Consultation with the relevant NRF Executive Directors.
 Consultation with any suitable person in the South African research community.

Period of appointment
Normally for three consecutive years, with members retiring annually by rotation.

Convener
Appointed by the NRF Executive member with delegated authority by the NRF President.

Terms of reference
 To receive documentation from the NRF in respect of the evaluation and rating of individual researchers falling within the appropriate field of research. On the basis of this documentation, to select the most appropriate reviewers.
 To carefully scrutinise and assess the contents of applications of individual researchers.
 To assess the reports by reviewers.
 In consultation with colleagues on the Specialist Committee, to decide on an appropriate rating for applicants based on the reports of reviewers, for discussion at the Assessment Panel meeting.
 To identify constructive feedback from the reports by reviewers that could be conveyed to applicants.
 To rate the reports by reviewers.
 To consider the evidence presented upon appeals and to make recommendations on the appeal to the Executive Evaluation Committee (EEC).
 To act as advisors in matters concerning the evaluation and rating of individual researchers, particularly the selection of peers as reviewers.
 To recommend improvements to the evaluation and rating process.
 
To alert the NRF Executive to needs and priorities within their area of expertise.Top of page

NRF Assessors

Appointment
A limited number of Assessors are appointed annually, normally for a period of three consecutive years. Assessors are selected from the ranks of highly respected researchers who have served on Specialist Committees in the past, who are therefore familiar with the process and who have earned a reputation during their tenure on these Specialist Committees for their wisdom and objective judgements. Assessors must ensure that the assessment process is fair and independent and that the same criteria are applied consistently by all the Specialist Committees. Assessors are appointed by the NRF Executive member, who has been delegated this task by the NRF President.

Terms of reference
 To receive documentation from the NRF in respect of the evaluation and rating of individual researchers within any of the fields of research.
 To carefully scrutinise and assess the contents of the evaluation statements by the individual applicants.
 To carefully scrutinise and assess the reports by reviewers.
 To independently propose an appropriate rating, based on the reports of reviewers, for discussion at the Assessment Panel meetings.
 To ensure that the same standards in respect of ratings are maintained across the different fields of research.
 To consider the evidence presented upon appeal and to make recommendations to the Executive Evaluation Committee (EEC).
 To recommend improvements to the evaluation and rating process.Top of page

Assessment Panels

Composition
 An NRF Executive member or a researcher of international repute.
 An independent
Assessor.
 Three to six members of a particular
Specialist Committee
 In attendance:
– Manager: Evaluation Centre
– Secretariat: Evaluation Centre
– Appropriate NRF Programme Managers by invitation

Chairperson
The NRF Executive member or a researcher of international repute.

Appointment
Chairpersons are appointed annually by the NRF President, normally for a period of three consecutive years.

Terms of reference
 To consider the proposed ratings of the Assessor and the Specialist Committee and to try to reach agreement on each rating.
 To debate issues if there is no consensus on a particular rating and to try to reach consensus.
 To refer cases where consensus cannot be reached to the Executive Evaluation Committee (EEC).
 To refer cases to the EEC where a principle decision is required.
 To forward nominations for the A and P categories the EEC together with the Assessment Panel’s recommendations.
 To refer all applications of researchers who previously fell in the A category, who are deemed to no longer fall into this category, to the EEC.
  To act as advisors to the NRF Executive in matters concerning the evaluation and rating of individual researchers, particularly the selection of peers as reviewers, and to recommend improvements to the evaluation and rating process.
 To alert the NRF Executive to the needs and priorities of particular disciplines.

Top of page

Executive Evaluation Committee (EEC)

Composition
 NRF President
 NRF Vice-President
 NRF Executive Director with responsibility for evaluation and rating
 Chairpersons of the Assessment Panels
 Relevant NRF Executive Directors on a rotational basis
 One of the conveners of the Assessment Panels on a rotational basis annually
 In attendance:
  – Manager: Monitoring & Evaluation
  – Secretariat: Monitoring & Evaluation

Chairperson
NRF President

Quorum
 Three external members (i.e. not NRF staff)
 Three NRF staff members including either the NRF President or the Vice-President

Appointment
 The Chairpersons of the Assessment Panels and the external members are appointed annually by the NRF President, normally for a period of three consecutive years.
 The NRF Executive members are appointed by the NRF President.

Terms of reference
 To consider appeals against decisions of Assessment Panels.
 To consider cases where consensus could not be reached at the meetings of the Assessment Panels.
 To take decisions about applicants nominated for the A and P categories.
 To take decisions on all applicants who were previously in the A category and who are nominated for placement in another category.
 To take decisions on all applications referred by the Assessment Panels where a principle decision is required.
 To make recommendations on any aspect of the evaluation and rating process to the full NRF Executive.
 To commission investigations about the evaluation and rating process where necessary.
 To refer amendments to policy issues to the NRF Executive for decisions.

Top of page

The Appeals Committee

Composition
 A senior researcher of repute (either present or past) who is or has been thoroughly familiar with the NRF evaluation and rating system.
 Chairperson of the Executive Evaluation Committee (EEC) (ex officio).
 One member who has considerable experience with the NRF evaluation and rating process (e.g. NRF Assessor/past EEC member who has not been involved in the case being appealed).
 One or more additional members who are or who have been researchers/ scholars of stature.
 In attendance:
  – Manager: Monitoring & Evaluation
  – Secretariat: Monitoring & Evaluation

Chairperson
The Chairperson is a senior researcher of repute (either present or past) who is or has been thoroughly familiar with the NRF rating system. He/she need not be a member of the NRF Board.

Appointment
 The Chairperson will be appointed by the NRF Board for a period of three years with the possibility of reappointment for a further maximum period of three consecutive years.
 Other members will be appointed by the NRF President in consultation with the Chairperson of the Appeals Committee.

Terms of reference
 To consider appeals lodged against decisions on evaluation and rating taken by the EEC.
 To ensure that the correct and complete process has been followed in the specific evaluation and rating against which an appeal has been lodged.
 To ensure that fairness and objectivity prevails in the interpretation of reviewers’ reports.
 To ensure that NRF ratings are interpreted correctly.
 To make a pronouncement on the fairness and correctness of the rating decision that is being appealed.
 To refer the case back to the EEC and to point out specific shortcomings in the process that warrant review in cases where the Appeals Committee is at variance with decisions of the EEC.

Appeals process
Written appeals against ratings allocated to applicants must be lodged by the appropriate authorities at employing institutions on behalf of applicants within three months of the date on which the NRF informed the institution and the applicant about the rating. Appeals are screened by Monitoring & Evaluation and returned to the employing institution for modification if necessary. Monitoring & Evaluation divides the appeals into two groups, those that are lodged against decisions taken by the Assessment Panels  and those that are lodged against decisions taken by the EEC.

Appeals against decisions taken by the Assessment Panels are heard by the EEC. Documents relevant to the appeal are first referred to the Specialist Committee and Assessor who were involved in the rating decision of the applicant lodging the appeal. Their comments together with the appeal documents (which consist of: the applicant’s original submission, the reports by reviewers, a record of the meeting of the Assessment Panel and the letter of appeal) are then forwarded to the EEC for a final decision. The decision of the EEC is communicated to the employing institution lodging the appeal.

Should there be an appeal against a decision of the EEC, it is heard by the Appeals Committee. Before appeals are considered by the Appeals Committee, Monitoring & Evaluation will consult with the NRF President or Vice-President about the grounds for the appeal. The NRF President or Vice-President will make a pronouncement on whether a particular appeal will be heard or not. In cases where an appeal is deemed invalid, the President or Vice-President will communicate with the appropriate authority at the institution providing reasons why the appeal as submitted is not regarded as being valid.

Valid appeals are then considered by the Appeals Committee, which takes the following into account:
 meeting records as well as the decisions taken by the relevant Assessment Panel and the EEC
 the application of the researcher
 reports by reviewers and
 the letter of appeal by the employing institution as well as
 any further documentation for clarification of issues.

The Appeals Committee judges the fairness and correctness of the evaluation and rating procedure. It does not make a pronouncement on a rating. If the Appeals Committee is at variance with decisions of the EEC it refers these cases back to the EEC pointing out specific shortcomings in the process that warrant review. After reviewing the process and reaching a decision, the EEC provides feedback to the Chairperson of the Appeals Committee. The decision of the EEC is communicated to the employing institution lodging the appeal.
Top of page

Monitoring & Evaluation

Composition of staff
Currently, Monitoring & Evaluation has 15 staff positions of which twelve devote some or all of their time to the evaluation and rating process.

Terms of reference
 To interact with individual researchers and/or employing institutions and/or NRF staff about the strategies, logistics and closing dates for submission of applications for evaluation and rating.
 To screen applications for evaluation and rating for completeness and to acknowledge receipt thereof.
 To approach the reviewers selected by the Specialist Committees for each applicant, to receive and screen the reports and to acknowledge receipt of reports.
 To ensure that a sufficient number of reviewers’ reports have been solicited and received for each applicant.
 To make all logistic arrangements for meetings of committees/workshops (dates, venues, transport etc).
 To act as secretariat for Specialist Committees, Assessors, Chairpersons, Assessment Panels, the Executive Evaluation Committee (EEC) and the Appeals Committee.
 To inform employing institutions and applicants of rating outcomes.
 To capture and maintain all relevant data on the NRF’s databases.
 To provide annual statistics.
 To inform the NRF management about matters pertaining to the evaluation and rating of researchers.
 To develop new procedures and processes and to refine existing ones.
 To advise the NRF Executive on all matters pertaining to the evaluation and rating process and to make recommendations on issues that need attention.

 

This page was last updated on 16 October 2009