Home | Sitemap | Contact | NRF Home 

 

Monitoring & Evaluation

Enhancing the quality of research in South Africa
Evaluation and rating
Panel Information
Guidelines to specialist committees
Selection of reviewers
Assessment of reviewers' reports
Committee members for 2009 and meeting dates
 

Guidelines to specialist committees
The selection of appropriate reviewers constitutes the very essence of the peer-review system that supports the evaluation and rating of individuals. The Specialist Committees and the applicants for evaluation and rating are thus expected to show great circumspection in nominating reviewers.

Applicants are requested to supply names of at least six but not more than ten research active reviewers who are best able to assess the scope and impact of their recent research and other scholastic outputs, activities and contributions. They are also requested to indicate their relationship with the reviewer and to give reasons for each nomination to provide the Specialist Committee with additional information for the selection of reviewers. Applicants are also given the opportunity to indicate which reviewers should not be approached by the NRF.

The Specialist Committees are requested to nominate six reviewers of whom, generally speaking, three should be selected from those listed by the applicant and another three provided, independently, by the Specialist Committee. (NB: This is a general guideline and does not preclude scenarios where for example, all six reviewers could be chosen from the applicant’s list.)

Persons who serve on the Specialist Committees should be “wise” in the broader context of their fields with extensive national and international networks to assist them in the identification of suitable reviewers. These persons should make research contributions in their own right and be capable of fair evaluation. There is no substitute for the wisdom of members of the Specialist Committees who are ultimately responsible for the selection of reviewers and whose task it is to select a mix of reviewers from whose reports the impact of applicants’ research in their fields and in broader fields can be gleaned.Top of page

Selection of reviewers

General guidelines
 Specialist Committee members should consult closely with one another, especially with the Convener of the Specialist Committee, regarding the selection of reviewers.
 Where Specialist Committee members have difficulties or uncertainties regarding reviewers for particular applicants or fields, they should consult colleagues (locally or abroad) who would be able to make suggestions about suitable reviewers.

Specific guidelines
 The Specialist Committee should affirm that reviewers nominated by applicants are genuine peers and that they are experts in the field of the applicant (either by reputation, citation, publications, members of editorial boards of journals in which applicants publish the majority of their work, etc). It should be noted that some applicants do, in fact, nominate inappropriate reviewers. Once it has been established that the persons listed are real peers, three reviewers from the applicant’s list should be selected, finding a balance between those who have been collaborators and those who are independent of the applicant.
 Three independent reviewers should be chosen who are not on the applicant’s list, again ensuring that they are true peers and active in the field of research of the applicant. Care should be exercised when approaching reviewers from the same department or institution as the applicant.
 The standing of reviewers should be taken into account: to take two obvious examples,
– Nobel Prize winners should not be approached for an applicant most likely to be placed in the Y category.
– International leaders should be approached for applicants currently in the A category (or for applicants where panel members feel there is a strong possibility that they may be placed in the A category).
 In some cases an applicant’s work may cover several fields. Reviewers should therefore be chosen to ensure that the scope and impact of the work is adequately covered. It may be necessary to consult with other Specialist Committees or to approach more than six reviewers in such cases especially if the fields are very divergent.
 Although applicants are requested to include South African reviewers it is not essential to nominate such reviewers. The main criteria of proper acquaintance and expertise in the field of the applicant and being a true peer must take precedence.
 In some cases where the applicant works in a specialised field and a large number of names of reviewers are supplied by him/her, it can be difficult to find appropriate additional reviewers. In such instances, and provided the given reviewers are sufficiently independent of the applicant, the use of more than three of the reviewers as supplied by the applicant may be justified (and the so-called ‘independent’ reviewers decreased accordingly).
 Reviewers who are generalists and those who are aware of the ‘broader picture’ are essential in the evaluation of researchers who do descriptive research because they can place the research into a wider context.
 Care must be taken not to approach the same reviewer too often. Where a particular person is suitable for several applicants he/she could be approached for some of them but could also be asked to suggest names of suitable reviewers for the other applicants.
 Generally speaking the same reviewer should not be approached more than twice consecutively to review a particular applicant.
 When approaching reviewers in industry it is important that the chosen reviewers are persons who are active in research.
 Assessors should not be approached as reviewers in those cases where they act as Assessor for a particular applicant.
 Members of Specialist Committees should not be asked to act as reviewers of applicants.
 In cases where an applicant’s research straddles more than one Specialist Committee, the evaluation should be handled by the most applicable Specialist Committee as identified by the applicant, the NRF or the members of Specialist Committees. Members of this Specialist Committee will consult with other relevant Specialist Committees if and when the need arises (in particular during the process of identification of reviewers).Top of page

Assessment of reviewers' reports

Reports by reviewers are assessed by Specialist Committees as follows:

Specialist Committee members must play an interpretative role when they assess reviewers’ reports. For example, if a reviewer states that the applicant is publishing in top journals, yet the journals are in the bottom tier of journals in the field, then the validity of the reviewer’s report must be called into question. Similarly Specialist Committee members should recognise the weakness of reviewers’ reports which overly praise the importance of the applicant’s work and where the work concerned is clearly not of the calibre suggested in the report.

Reports by reviewers are assessed by Specialist Committees and rated as follows:

1Good to excellent report. Reviewer gives a critical analysis of the recent research outputs and comments critically on the quality of the research outputs of the last seven years as well as the international/national standing of the applicant. Reviewer has read the most important outputs of the last seven years.

(Note: A ‘good to excellent’ rating should only be awarded to reviewers who make it clear that they have actually read or have a sound knowledge of the applicant’s research outputs. Hence care should be taken not to overrate reports which state, for example,
• It appears that the applicant’s work is…
• I believe…
• I have heard from one of my colleagues…
• I am told…
• It seems that his/her recent research outputs are…
• I have not read any of the publications…
• My research is not in the field of Prof X but I will nevertheless try to address your questions…)
2 Satisfactory report. Reviewer gives an analysis of recent research outputs. Reviewer is familiar or has familiarised him/herself with some of the outputs of the last seven years (e.g. by having acquainted him/herself with at least two or three recent research outputs).
3 Unsatisfactory report. Report by reviewer is superficial, or contains sweeping and/or uncorroborated statements, or over-generalises, or provides irrelevant information, or fails to focus on last seven years.

(Note: Unsatisfactory reports should not be taken into account when deciding on a rating.)
4 Inconclusive report. Report which makes it difficult to decide whether the reviewer should be approached again in future. The reviewer has replied but cannot oblige with the present request for various reasons (ill, going abroad, going on sabbatical for a lengthy period, time constraints, etc.)
5 Inappropriate report. Report is inappropriate for a particular applicant, e.g. the reviewer is not active in the applicant’s field of research. Such a reviewer should not be approached again for the applicant in question.
6 Inappropriate reviewer. Reviewer should not be approached again for any applicant. Reviewers in this group include:
• a reviewer whose report is not credible or is biased (whether positively or negatively);
• a reviewer who is no longer active in the research field, e.g. as a result of retirement

Top of page Annual Statistics 2005
as gathered at 17 March 2006

Committees for the 2009 round of evaluation and rating
* names of conveners of Specialist Committees are underlined

Animal and Veterinary Sciences
14 and 15 October 2009

Dr G Bronner
Prof ML Hamer
Prof AN Hodgson
Dr IV Nsahlai
Prof BL Penzhorn

Anthropology, Development Studies, Geography, Sociology and Social Work
9 and 10 November 2009

Prof P Alexander
Prof P Bond
Prof P le Roux
Prof B Maharaj
Dr I Pikirayi

Dr E van der Spuy
Biochemistry, Molecular and Cell Biology
26 and 27 October 2009
Prof NN Illing
Prof A Kramvis

Prof D Litthauer
Chemistry
2 and 3 November 2009
Prof SA Bourne
Prof CB de Koning
Prof KR Koch
Prof S Lotz
Communication, Media Studies, Library and Information Sciences
16 September 2009
Prof DN Ocholla
Prof L Strelitz
Earth Sciences
21 and 22 October 2009
Prof PG Eriksson
Prof JC Hughes
Dr KA Kuman

Prof G Stevens

Economics, Management, Administration and Accounting
17 and 18 November 2009

Prof JF Kirsten
Prof JM Luiz

Prof NS Terblanché
Prof MJD Ward
Education
13 and 14 October 2009
Dr RA Deacon
Prof RC Laugksch
Prof RM Setati
Engineering
26 and 27 November 2009
Dr ES Boje
Prof DJ de Beer

Prof HC Ferreira
Prof WW Focke
Prof A Zingoni
Health Sciences
Health Sciences 1: 29 and 30 October 2009
Health Sciences 2: 23 and 24 November 2009
Prof VA Corfield
Prof B Kramer
Prof MI Lambert
Prof BV Mendelow
Prof I Truter
Prof
 HH Vorster
Prof C Williamson
Historical Studies
11 September 2009
Dr TM Dedering
Prof N Erlank
Prof CC Saunders
Information Technology
16 October 2009
Prof ITJ Brown
Prof AB Engelbrecht
Prof P Kotzé
Law
5 and 6 November 2009
Prof T Boezaart
Prof H Mostert
Prof NC Steytler
Prof HA Strydom
Literary Studies, Languages and Linguistics
19 and 20 November 2009
Prof AJ Coetzee
Prof BKJPeterson
Prof GB van Huyssteen
Prof AJ van Rooy
Mathematical Sciences
14 and 15 September 2009
Prof J Banasiak
Prof MS Finkelstein
Prof JM-S Lubuma
Microbiology and Plant Pathology
5 and 6 October 2009
Prof A Botha
Prof TA Coutinho
Prof EP Rybicki
Performing and Creative Arts, and Design
8 September 2009
Dr SJvZ Muller
Prof W Nebe
Prof B Schmahmann
Physics
17 and 18 September 2009
Prof AG Every
Prof RC Kraan-Korteweg
Prof FPetruccione
Prof HC Swart
Plant Sciences
12 and 13 November 2009
Dr E Barros
Prof DK Berger
Prof MT Hoffman
Prof T McLellan
Political  Sciences and Philosophy
23 October 2009
Prof AB du Toit
Prof H Hudson
Prof RB Mattes
Prof AA Van Niekerk
Psychology
2 October 2009
Prof SA Kagee
Prof CI Macleod
Prof CG Tredoux
Religious Studies and Theology
21 and 22 September 2009
Prof CD Roux
Prof AI Tayob
Prof DF Tolmie
Special Assessment Panel for the L rating
10 December 2009

Dr SD Selematsela (Chairperson)
Dr R Maharaj
Prof H Ngambi
Prof D Pillay

Assessors Prof HW Dirr
Prof JR Midgley
Prof RG Morrell
Prof JP Muller
Prof TJ Robinson
Chairpersons of Assessment Panels Prof CM de la Rey
Prof RJ Haines
Prof H Huismans
Prof NW Pammenter
Dr G von Gruenewaldt
Executive Evaluation Committee (EEC)
EEC meeting dates:
23 September 2009
28 October 2009
25 November 2009
9 December 2009
Dr AS van Jaarsveld (Chairperson)
Chairpersons of Assessment Panels (see above)
Prof VA Corfield
Dr AM Kaniki
Dr R Maharaj
Dr SD Selematsela
Prof AJ van Rooy
Appeals Committee

Prof D Mitchell (Chairperson)
Prof W du Plessis
Prof RG Morrell
Dr AS van Jaarsveld (ex officio)

 

Page last updated: 16 October 2009