Home | Sitemap | Contact | NRF Home

Monitoring & Evaluation

 

Enhancing the quality of research in South Africa
Evaluation and rating
Information for panel members
Guidelines to specialist committees
Selection of reviewers
Assessment of reviewers' reports
Committee members for 2008 and meeting dates
Committee members for 2007 and meeting dates
Committee members for 2006 and meeting dates
Committee members for 2005 and meeting dates
Committee members for 2004 and meeting dates
Committee members for 2003 and meeting dates
 

Guidelines to specialist committees
The selection of appropriate reviewers constitutes the very essence of the peer-review system that supports the evaluation and rating of individuals. The Specialist Committees and the applicants for evaluation and rating are thus expected to show great circumspection in nominating reviewers.

Applicants are requested to supply names of at least six but not more than ten research active reviewers who are best able to assess the scope and impact of their recent research and other scholastic outputs, activities and contributions. They are also requested to indicate their relationship with the reviewer and to give reasons for each nomination to provide the Specialist Committee with additional information for the selection of reviewers. Applicants are also given the opportunity to indicate which reviewers should not be approached by the NRF.

The Specialist Committees are requested to nominate six reviewers of whom, generally speaking, three should be selected from those listed by the applicant and another three provided, independently, by the Specialist Committee. (NB: This is a general guideline and does not preclude scenarios where for example, all six reviewers could be chosen from the applicant’s list.)

Persons who serve on the Specialist Committees should be “wise” in the broader context of their fields with extensive national and international networks to assist them in the identification of suitable reviewers. These persons should make research contributions in their own right and be capable of fair evaluation. There is no substitute for the wisdom of members of the Specialist Committees who are ultimately responsible for the selection of reviewers and whose task it is to select a mix of reviewers from whose reports the impact of applicants’ research in their fields and in broader fields can be gleaned.Top of page

Selection of reviewers

General guidelines
 Specialist Committee members should consult closely with one another, especially with the Convenor of the Specialist Committee, regarding the selection of reviewers.
 Where Specialist Committee members have difficulties or uncertainties regarding reviewers for particular applicants or fields, they should consult colleagues (locally or abroad) who would be able to make suggestions about suitable reviewers.

Specific guidelines
 The Specialist Committee should affirm that reviewers nominated by applicants are genuine peers and that they are experts in the field of the applicant (either by reputation, citation, publications, members of editorial boards of journals in which applicants publish the majority of their work, etc). It should be noted that some applicants do, in fact, nominate inappropriate reviewers. Once it has been established that the persons listed are real peers, three reviewers from the applicant’s list should be selected, finding a balance between those who have been collaborators and those who are independent of the applicant.
 Three independent reviewers should be chosen who are not on the applicant’s list, again ensuring that they are true peers and active in the field of research of the applicant. Care should be exercised when approaching reviewers from the same department or institution as the applicant.
 The standing of reviewers should be taken into account: to take two obvious examples,
– Nobel Prize winners should not be approached for an applicant most likely to be placed in the Y category.
– International leaders should be approached for applicants currently in the A category (or for applicants where panel members feel there is a strong possibility that they may be placed in the A category).
 In some cases an applicant’s work may cover several fields. Reviewers should therefore be chosen to ensure that the scope and impact of the work is adequately covered. It may be necessary to consult with other Specialist Committees or to approach more than six reviewers in such cases especially if the fields are very divergent.
 Although applicants are requested to include South African reviewers it is not essential to nominate such reviewers. The main criteria of proper acquaintance and expertise in the field of the applicant and being a true peer must take precedence.
 In some cases where the applicant works in a specialised field and a large number of names of reviewers are supplied by him/her, it can be difficult to find appropriate additional reviewers. In such instances, and provided the given reviewers are sufficiently independent of the applicant, the use of more than three of the reviewers as supplied by the applicant may be justified (and the so-called ‘independent’ reviewers decreased accordingly).
 Reviewers who are generalists and those who are aware of the ‘broader picture’ are essential in the evaluation of researchers who do descriptive research because they can place the research into a wider context.
 Care must be taken not to approach the same reviewer too often. Where a particular person is suitable for several applicants he/she could be approached for some of them but could also be asked to suggest names of suitable reviewers for the other applicants.
 Generally speaking the same reviewer should not be approached more than twice consecutively to review a particular applicant.
 When approaching reviewers in industry it is important that the chosen reviewers are persons who are active in research.
 Assessors should not be approached as reviewers in those cases where they act as Assessor for a particular applicant.
 Members of Specialist Committees should not be asked to act as reviewers of applicants.
 In cases where an applicant’s research straddles more than one Specialist Committee, the evaluation should be handled by the most applicable Specialist Committee as identified by the applicant, the NRF or the members of Specialist Committees. Members of this Specialist Committee will consult with other relevant Specialist Committees if and when the need arises (in particular during the process of identification of reviewers).Top of page

Assessment of reviewers' reports

Reports by reviewers are assessed by Specialist Committees as follows:

Specialist Committee members must play an interpretative role when they assess reviewers’ reports. For example, if a reviewer states that the applicant is publishing in top journals, yet the journals are in the bottom tier of journals in the field, then the validity of the reviewer’s report must be called into question. Similarly Specialist Committee members should recognise the weakness of reviewers’ reports which overly praise the importance of the applicant’s work and where the work concerned is clearly not of the calibre suggested in the report.

Reports by reviewers are assessed by Specialist Committees and rated as follows:

1Good to excellent report. Reviewer gives a critical analysis of the recent research outputs and comments critically on the quality of the research outputs of the last seven years as well as the international/national standing of the applicant. Reviewer has read the most important outputs of the last seven years.

(Note: A ‘good to excellent’ rating should only be awarded to reviewers who make it clear that they have actually read or have a sound knowledge of the applicant’s research outputs. Hence care should be taken not to overrate reports which state, for example,
• It appears that the applicant’s work is…
• I believe…
• I have heard from one of my colleagues…
• I am told…
• It seems that his/her recent research outputs are…
• I have not read any of the publications…
• My research is not in the field of Prof X but I will nevertheless try to address your questions…)
2 Satisfactory report. Reviewer gives an analysis of recent research outputs. Reviewer is familiar or has familiarised him/herself with some of the outputs of the last seven years (e.g. by having acquainted him/herself with at least two or three recent research outputs).
3 Unsatisfactory report. Report by reviewer is superficial, or contains sweeping and/or uncorroborated statements, or over-generalises, or provides irrelevant information, or fails to focus on last seven years.

(Note: Unsatisfactory reports should not be taken into account when deciding on a rating.)
4 Inconclusive report. Report which makes it difficult to decide whether the reviewer should be approached again in future. The reviewer has replied but cannot oblige with the present request for various reasons (ill, going abroad, going on sabbatical for a lengthy period, time constraints, etc.)
5 Inappropriate report. Report is inappropriate for a particular applicant, e.g. the reviewer is not active in the applicant’s field of research. Such a reviewer should not be approached again for the applicant in question.
6 Inappropriate reviewer. Reviewer should not be approached again for any applicant. Reviewers in this group include:
• a reviewer whose report is not credible or is biased (whether positively or negatively);
• a reviewer who is no longer active in the research field, e.g. as a result of retirement

Top of page Annual Statistics 2005
as gathered at 17 March 2006

Committees for the 2008 round of evaluation and rating
* names of conveners of Specialist Committees are underlined

Animal and Veterinary Sciences
Panel meeting date: 22 October 2008

Prof NC Bennett
Prof AN Hodgson
Dr IV Nsahlai
Prof BL Penzhorn

Dr MJ Smale

Anthropology, Development Studies, Geography, Sociology and Social Work
Panel meeting date: 21 November 2008

Prof P Alexander
Prof
P le Roux
Prof B Maharaj
Dr I Pikirayi
Prof WF van Delft
Dr E van der Spuy
Biochemistry, Molecular and Cell Biology
Panel meeting date: 16 October 2008
Dr C Frost
Prof D Litthauer
Prof DB McIntosh
Chemistry
Panel meeting date: 9 October 2008
Prof CB de Koning
Prof
D Jaganyi
Prof KR Koch
Prof JR Moss
Communication, Media Studies, Library and Information Sciences
Panel meeting date: 5 September 2008
Dr MC Nassimbeni
Prof L Strelitz
Prof RE Teer-Tomaselli
Earth Sciences
Panel meeting date: 22  September 2008
Prof PG Eriksson
Prof KM Rowntree
Prof G Stevens
Prof NJ van der Merwe
 

Economics, Management, Administration and Accounting
Panel meeting date: 18 November 2008

Prof JF Kirsten
Prof JM Luiz

Prof NS Terblanché
Prof MJD Ward
Education
Panel meeting date: 10 October 2008
Prof RA Deacon
Prof H Janks
Prof RC Laugksch
Prof RM Setati
Engineering
Panel meeting date: 30 October 2008
Dr ES Boje
Prof JP du Plessis
Prof HC Ferreira
Prof WW Focke
Prof A Zingoni
Health Sciences
Panel meeting date: 5 November 2008
Prof VA Corfield
Prof B Kramer
Prof BV Mendelow
Prof
TD Noakes
Prof M Poggenpoel
Prof I Truter
Prof HH Vorster
Prof C Williamson
Historical Studies
Panel meeting date: 1 September 2008
Dr TM Dedering
Prof F Pretorius
Prof CC Saunders
Information Technology
Panel meeting date: 29 August 2008
Prof ITJ Brown
Prof P Kotzé
Prof MS Olivier
Law
Panel meeting date: 9 January 2009
Prof J Neethling
Prof AL Stander
Prof NC Steytler
Prof HA Strydom
Literary Studies, Languages and Linguistics
Panel meeting date: 25 September 2008
Prof AJ Coetzee
Prof R Mesthrie

Prof BKJ Peterson
Prof AJ van Rooy
Mathematical Sciences
Panel meeting date: 2 October 2008
Pro fJ Banasiak
Prof MS Finkelstein
Prof JM-S Lubuma
Microbiology and Plant Pathology
Panel meeting date: 29 October 2008
Prof TJ Britz
Prof TA Coutinho
Prof MEC Rey
Performing and Creative Arts, and Design
Panel meeting date: 8 October 2008
Prof M Fleishman
Dr SJvZ Muller
Prof B Schmahmann
Physics
Panel meeting date: 18 September 2008
Prof OC de Jager
Prof AG Every
Prof FG Scholtz
Prof HC Swart
Plant Sciences
Panel meeting date: 11 November 2008
Prof TAJ Hedderson
Dr SJE Midgley
Prof DJ Mycock
Prof CM Shackleton
Political  Sciences and Philosophy
Panel meeting date: 13 November 2008
Prof AB du Toit
Prof H Hudson
Prof RB Mattes
Prof AA van Niekerk
Psychology
Panel meeting date: 16 September 2008
Prof SA Kagee
Prof CI Macleod
Prof CG Tredoux
Religious Studies and Theology
Panel meeting date: 2 September 2008
Prof PK Penumala
Prof CD Roux
prof DF Tolmie
Special Assessment Panel for the L rating
Panel meeting date: 3 December 2008

Dr AM Kaniki (Chairperson)
Dr R Maharaj
Prof H Huismans
Prof D Pillay
Prof HH Vorster

Assessors Prof HW Dirr
Prof JR Midgley
Prof RG Morrell
Prof JP Muller
Prof N Pammenter
Prof TJ Robinson
Chairpersons of Assessment Panels Prof CM de la Rey
Prof RJ Haines
Dr AM Kaniki
Dr AS van Jaarsveld
Dr G von Gruenewaldt
Executive Evaluation Committee (EEC)
Panel meeting date: The EEC meets several times throughout the year
Dr MP Mangaliso (Chairperson)
Prof CM de la Rey
Prof RJ Haines
Dr AM Kaniki
Dr R Maharaj
Dr AS van Jaarsveld
Dr G von Gruenewaldt
Appeals Committee

To follow

 

Page last updated: 9 June 2008