• May 1, 2009 Ruth Lilly Auditorium, Riley Hospital

Purpose: This study utilized a systematic review to determine how scholarly productivity of faculty in health-related professions is measured.

Method: A review of the literature was conducted using multiple databases to search pertinent keywords. Hand searches of article references were also used to ensure that as many relevant articles as possible were identified. The database searches and hand searches yielded approximately 3000 articles. This was narrowed down to 344 relevant abstracts (67.37% of these were found through hand searches). No randomized controlled trials were found within the 344 relevant articles. Only articles that discussed academic productivity as opposed to clinical productivity were included, resulting in a total of 334 articles. Articles were narrowed to those that included inferential statistical comparison of at least two groups and p-values revealing a total of 54 articles. 3 articles were eliminated because they did not meet essential criteria resulting in a final count of 51 articles. The final 51 articles were scored by 6 researchers using the MacDermid's Guidelines after they were blinded by an unbiased research assistant. The MacDermid's Guidelines had no previous reliability statistics published. Researchers therefore completed test-retest analysis of the MacDermid's Guidelines by rescoring 5 blinded articles. In addition, journal impact scores and citation index data were also tracked for all articles. The journal impact and citation index look at the quality of the article and the journal in which it was originally published.

Results: Within the 51 articles, 54 different faculty productivity outcome variables were identified. When the inclusion criteria and time formats for these variables were integrated the number of outcome variables expanded to 389. 78.4% of the articles did not report reliability of test instruments. 68.6% did not report any validity of the instruments and 19.6% reported content validity based on expert opinion. The MacDermid's Guidelines test-retest reliability had a Cronbach's alpha reliability for internal consistency ranging from .831 to .982. ANOVA found only one article pair had a significant difference between the test-retest sessions; there was no statistical difference between the scores for the other four article pairs. For the 51 articles assessed the highest possible score using MacDermid's Guidelines was 48. The scores ranged from 33.5 to 13.0 with the top 2 ranking articles measuring productivity by counting publications. The article with the highest journal impact score (52.59) also had the highest number of citations in the citation index (106). Journals achieve high impact scores as a direct result of specific articles that have high citation counts.

Conclusion: Throughout this systematic review, 90 years of information was reviewed. Only 1 article of the 51 assessed addressed the validity of productivity measurement. It has become apparent that studies are needed that specifically address the actual measurement tools. For nearly a century, there has been a lack of evidence regarding the measurement of faculty productivity. A uniform way of measuring productivity is desperately needed due to its implications on promotion and tenure of faculty members, school performance, and for evidence based practice within health professions.