HOW DOES UCF RANK?

A COMPARISON OF UCF AND SUS
INSTITUTIONS IN PUBLISHED RANKINGS

Catherine Baltunis

OEAS-TR-02-001

February 2002

Operational Excellence and Assessment Support
University of Central Florida
12424 Research Parkway, Suite 225
Orlando, FL 32826-3207
(407) 882-0275
FAX: (407) 882-0288

ABSTRACT

The purpose of this Technical Report is to provide an overview of how UCF compares to other SUS institutions in a variety of published rankings. Published rankings can play an important role in the decision to attend a university.  Included in this report are descriptions of each of the surveys and methodologies used and a series of tables that indicate UCF’s position in each of the rankings.  Only rankings that include UCF are addressed in this document.

Dr. Julia Pet-Armacost, Interim Director

1.0  INTRODUCTION AND SCOPE

The purpose of this Technical Report is to provide an overview of how the University of Central Florida (UCF) compares to other SUS institutions in a variety of published rankings.  This report is partly the result of UCF appearing in the fourth tier of the “Best Colleges” guide produced by US News and World Report.  The US News and World Report Rankings is a popular source of information for prospective students and their families and may be a critical factor in the decision to attend or not to attend a university.  Although US News is one of the most popular ranking resources, perspective students also use a number of other published rankings when deciding on which college to attend.  Additionally, many rankings are used for public relations purposes, promotional materials, governing boards, and higher education groups.  The rankings often provide information on quality, accountability, and performance indicators for higher education institutions.

A primary focus of this study is the question of how UCF compares with SUS institutions. Information for this comparison was obtained directly from the published rankings, either through websites or paper copies.  Only rankings that include UCF have been included in this document. This report contains a description of each of the published rankings and methodologies used, as well as a series of tables that indicate UCF’s position compared to other SUS institutions. 

2.0  A COMPARISON OF UCF AND SUS INSTITUTIONS IN PUBLISHED RANKINGS

2.1  THE CENTER –TOP AMERICAN RESEARCH UNIVERSITIES (UF)

This ranking identifies the top public and private research universities in the United States based on nine quality measures.  For this analysis, The Center only includes research institutions with at least $20 million in federal research expenditures per year

Institutions that have federal research expenditures as reported to NSF of at least $20 million and that fall within the top 25 on at least one of the nine measures are classified as top research universities.

2.1.1   Process Used to Develop Ranking

The raw data used in this ranking were obtained from federal agencies and national organizations. 

The Center believes that top universities have strength in research, private support, faculty, graduate, and post-doctorate programs, and high quality undergraduates.

For this analysis, institutions in the top group rank among the top 25 on all nine measures of quality.  These institutions have high scores on all the criteria included in the ranking. The second group includes universities that rank in the top 25 on eight measures, and so on for the rest of the groups. 

2.1.2   Nine Quality Measures

In order to determine the Top American Research Universities, institutions are clustered and ranked on nine different quality measures.  An additional measure of quality, National Merit Scholars, is provided in the results, but not used in the actual ranking.  The Center finds the average SAT score to be a broader based and more reliable measure of overall undergraduate quality.

• Total Research
  • Includes all expenditures on research reported to NSF
• Federal Research Expenditures
  • Non-science and engineering research in fields such as law, education, humanities, business, journalism, and fine arts have been excluded.
• Endowment Assets
  • Endowment reflects generations of gifts to the institution and the investment growth of those gifts
• Annual Giving
  • Includes gifts received during the institution's fiscal year.
• National Academy Members
  • Faculty quality is one of the primary sources of an institution’s strength as a competitive enterprise.
• Faculty Awards
• Doctorates Granted
  • The number of Doctor of Ed., Doctor of Juridical Science, Doctor of Public Health, or Doctor of philosophy awarded in a one year period.
• Postdoctoral Appointees
• Median SAT Scores

2.1.3   Ranking Results

Control Rank - Rank among all public institutions
National Rank - Rank among all public and private institutions

2.2  THE GOURMAN REPORT – A RATING OF UNDERGRADUATE PROGRAMS IN AMERICAN AND INTERNATIONAL UNIVERSITIES

The Gourman Report ranking takes a qualitative approach to determine what constitutes academic excellence or quality in American institutions. 

2.2.1  Process Used to Develop Ranking

• Information used for the rankings is drawn from both internal, as well as, external sources
  • Internal sources: Educators; administrators; and persons chosen for their academic qualifications, their published works, and their interest in improving the quality of higher education evaluate their programs only.
  • External sources: Includes funding for public universities as authorized by legislative bodies; required filings by schools to meet standards of nondiscrimination; and material provided by the institutions about faculty makeup and experience, fields of study offered, and physical plant. Additionally, individuals, associations, and agencies, whose business it is to make projections of the success of graduates will enjoy, are an important part of the research.
• Rating Categories –Assigns a numerical score to each school and undergraduate program

2.2.2  Dimensions of Measurement

• Auspices, control and organization of an institution
• Total educational programs and degrees awarded
• Age of the institution, individual program, discipline, or division
• Faculty qualifications, experience, intellectual interests, attainments, and professional productivity
• Students’ quality of scholastic work and records of graduates
• Requirements for admission of students
• Student enrollment
• Curriculum of the program, discipline, or division
• Standards and quality of education (including teaching load)
• Quality of Administration’s attitudes and policy toward teaching, research and scholarly production, and the amount of administrative research done
• Quality and availability of non-departmental areas such as counseling and career placement
• Quality of physical plant devoted to undergraduate, graduate, and professional levels
• Institutional finances, including budgets, investments, expenditures and sources of income
• Library facilities, including number of volumes, appropriateness of materials to individual disciplines, and accessibility of materials
• Computer facilities, includes the ability to support current research activities for both faculty and students
• Funding for research equipment and infrastructure
• Number of teaching and research assistantships
• Academic-athletic balance

2.2.3   Ranking Results

Business Administration (Ranked 1-45) All 730 degree granting programs evaluated

Engineering (Ranked 1-10) 22 degree granting programs were evaluated

2.3    THE TEMPLETON GUIDE: COLLEGES THAT ENCOURAGE CHARACTER DEVELOPMENT

This guide profiles 405 exemplary college programs in ten categories that inspire students to lead ethical and civic-minded lives.

2.3.1   Process Used to Develop guide

Colleges That Encourage Character Development is divided into three major sections: 405 exemplary programs listed in 10 categories, 50 presidents, and 100 colleges and universities that inspire students to lead ethical and civic-minded lives

The John Templeton Foundation sent a nomination packet to the president, public information officer, and vice president for academic affairs and vice president for student life at all four-year accredited colleges and universities in the United States. Participants were invited to nominate character-development programs that deserved special recognition.

Each program nominated was mailed an application form that required the contact person to respond in writing to a list of questions that addressed specific selection criteria. Over 2,500 programs and 1,000 institutions were included in the research process.

2.3.2   Selection Process

In collaboration with members of the project's Advisory Board, the Institute on College Student Values developed a set of rigorous selection criteria for each of the 10 Exemplary Program categories, the Presidential Leadership recognition, and the Templeton Honor Roll designation. The Institute's research team read, reviewed, and rated each application to determine to final results.

2.3.3  Categories for Exemplary Programs

• First-year programs
  • 60 college programs that offer students the tools to develop a moral compass to navigate between increased personal freedom and new responsibilities
• Academic Honesty Programs
  • 35 college programs that effectively communicate the values of honesty, trust, respect, responsibility, integrity, and fairness in the classroom
• Faculty and Curriculum Programs
  • 45 college programs that offer opportunities in the classroom for students to examine, reflect on, and articulate a set of moral ideals and commitments
• Volunteer Service Programs
  • 60 college programs that provide opportunities for students to learn through serving others in their communities and in the world
• Substance-Abuse Prevention Program
  • 35 college programs that place character development at the heart of their alcohol- and drug-abuse prevention efforts
• Student Leadership Programs
  • 40 college programs that help students develop the competencies, conscience, and compassion required of leaders in a civil society
• Spiritual Growth Programs
  • 40 college programs, not all church-related, that provide opportunities for students to develop a coherent vision of moral integrity that connects belief to behavior
• Civic Education Programs
  • 40 college programs that encourage students to develop the skills and habits of mind to become active, well-informed, responsible citizens in a democratic society
• Character and Sexuality Programs
  • 20 College programs that help students to learn, appreciate, and apply the core virtues of self-control, respect, responsibility and integrity in their relationships
• Senior-year Programs
  • 30 college programs that help seniors reflect on, connect, and attach meaning to their undergraduate experience.

2.3.4   Ranking Results

UCF was rated among the top 40 institutions for “Student Leaderships Programs." No SUS institutions were included in the ten exemplary programs evaluated.

2.4  U. S. NEWS AND WORLD REPORT 2002 COLLEGE RANKINGS

The US News and World Report Rankings is a popular source of information for prospective students and their families and may be a critical factor in the decision to attend or not to attend a university. Additionally, college and university administrators, governing boards, and higher education groups have used USNWR and similar rankings as measures of quality, accountability, and performance indicators for higher education institutions.

Information for this comparison was obtained directly from the US News and World Report rankings. 

2.4.1   Process Used to Develop Undergraduate Rankings

• The bulk of the data is collected from the universities through a survey distributed by US News and World Report. Incomplete information from the survey was supplemented from other sources or the previous years scores.
• Colleges in each category are ranked against their peers, based on their composite weighted score.
• The top 50 schools are ranked in descending order, while the remainders are grouped in tiers. Each tier is organized alphabetically.
• The majority of the data is collected from the universities through a survey distributed by US News and World Report. Incomplete information from the survey was supplemented from other sources or the previous years scores.
• Colleges in each category are ranked against their peers, based on their composite weighted score.
• The top 50 schools are ranked in descending order, while the remainders are grouped in tiers. Each tier is organized alphabetically.

2.4.2   Seven Dimensions of Measurement

• Academic reputation: (25% weight) A reputation survey is administered to presidents, provosts, and deans of admission at institutions in the same category. 
• Graduation & Retention: (20% weight) Consists of two components of measurement, which include a six-year graduation rate (80% of retention score) and its freshman retention rate (20%)
•  Faculty resources:(20% weight) Five factors are used to assess a school’s commitment to superb instruction
  • Class size (40%) –represents the proportion of classes with fewer than 20 students and of classes with more than 50.
  • Faculty salary (35%)
  • Proportion of professors with the highest degree in their field (15%)
  • Proportion of full-time faculty (5%)
  • Student/Faculty ratio (5%)
• Student selectivity: (15% weight)
  • SAT or ACT test scores of enrollees (40%)
  • Proportion of enrolled freshmen who graduated high school in the top 10 percent for the national institutions (35%)
  • Acceptance rate (15%) or the ratio of students admitted to applicants
  • Yield (10%) or the ratio of students who enroll to those admitted
• Financial resources: (10% weight) Average spending per student on instruction, research and education-related services annually.
• Graduation rate performance: (5% weight) Measure between a school’s six-year graduation rate and the predicted rate for the class.
• Alumni giving rate (5% weight) The percentage of alumni who give to their school is an indicator of satisfaction.

2.4.3   Ranking Results

(*)  Indicates schools with information found in the Master's Universities ranking.  All others are from the National Universities.

Italics indicates information was obtained from sources other than US News

(NA) Information is not available

2.5   YAHOO—WIRED COLLEGES 2001

This ranking attempts to determine which schools best utilize the vast array of available networked technology.  To keep up with the rapid changes in technology, this years ranking uses a number of new factors in reaching a final score for each institution.

2.5.1   Process Used to Develop Rankings

An on-line 52-part survey attempts to find out which schools best utilize the vast array of available networked technology.  

Each schools information-technology department received password access to the site.  All 2&4 year institutions (3631), including community and vocational colleges were evaluated.  Ranking is organized into universities/research schools, colleges/baccalaureate schools, two-year institutions, and less selective baccalaureate colleges

2.5.2 Dimensions of Measurement

• Infrastructure (28% of score)
  • Percentage of new public computers purchased in each of the last five years Percentage of public computers available 24/7
  • Percentage of dorm rooms with high-speed networks
  • Percentage of classrooms with high-speed networks
  • Percentage of classrooms with active Ethernet ports per desk
  • Percentage of network ports in campus facilities for mobile computing
  • Gigabit networking
  • Frequency of network outages
• Student Resources(22% of score)
  • Online registration and drop/add
  • Online grade report
  • Online career-counseling and advising
  • Online bookstore
  • Online Course catalog and evaluations
  • Online campus news
  • Web-based tuition payments
  • Campus supported web space and shared network file space
  • Computer purchasing plans
  • Equipment available for temporary use by students
• Web Portal (22% of score)
  • Design and navigability
  • Usefulness of admissions website
  • Availability of online application
  • Availability of information for current/prospective students and parents
  • Online alumni resources
• E-learning (12% of score)
  • Available technologies for instruction
  • Net-related courses
  • Library services available to faculty
  • Library digital archiving projects
• Tech support (11% of score)
  • Hours of free tech support
  • Web-based tech support
  • On-site tech support
  • Human-taught computer and Internet orientation for students/faculty
  • Availability of trained student employees or volunteers in dormitory housing
• Wireless Access (5% of score)
  • Availability of wireless data networking to institution buildings/grounds including on-campus housing and academic buildings

2.5.3  Ranking Results

Not all SUS institutions were included in the ranking.

2.6   THE 2000 NATIONAL DOCTORAL PROGRAM SURVEY

The National Doctoral Program Survey is an assessment of implementation of educational best practices in graduate education, as advocated by the Association of American Universities (1998), the Woodrow Wilson National Fellowship Foundation (1998), the Modern Language Association (1997), and the National Research Council's Committee on Science, Engineering and Public Policy (1995) among others.

2.6.1   Process Used to Develop Rankings  

This survey is based on the “Graduate School Survey” on the web and reflects the opinions of self-selected participants. Over 32,600 current and recent doctoral students participated. Each section contained a number of questions to be answered on a four-point Likert scale, as well as a free response box to allow respondents to expand upon their response. Scoring is based on a point system and assigned a letter grade. Only programs with 10 responses or more are included in the ranking. This is an observational study, not a controlled experiment. The respondents do not constitute a random sample of all doctoral students.

2.6.2   Dimensions of Measurement

• Information for prospective students
• Curricular breadth and flexibility
• Teaching
• Professional Development
• Career guidance and placement services
• Time to degree
• Faculty Mentoring
• Program Climate
• Overall Satisfaction

2.6.3   Ranking Results

Two of UCF’s programs were included in the ranking.

Computer Science School

No other SUS institutions were included in this category

Optics School

No other SUS institutions were included in this category

Institution	Professional Development	Guidance and placement	Controlling time to degree	Faculty mentoring	Program climate
UCF	52 C	61 B-	58 C+	66 B	71 B
Average US Engineering General/other	52 C	56 C+	54 C	68 B	61 B-

3.0 SUMMARY AND CONCLUSIONS

The various rankings presented in Section 2 represent the published rankings in which UCF has been included.  These rankings provide insight into UCF’s position related to SUS institutions and other institutions included in the analysis.

Because published rankings are often a popular source of information for prospective students and their families, as well as a source of information on quality and accountability, it is important that we take into consideration how UCF is reflected in these rankings. 

Progress and quality performance is an ongoing quest of our institution.  Therefore, a follow up study needs to be conducted that provides an in-depth examination of what changes our institution can make in order to improve our standing in various ranking publications.