3 Deriving Scales from Theory

Topics: Using theoretical arguments to derive measures from indicator data. Axiomatic Analysis. Dimensional Analysis

Required reading:

3.1 Seminar

For this assignment, we are going to think about measures that relate the number of students to the number of teachers in the context of a university.

In the UK, these are commonly reported as “student-staff ratios” for the institution overall. In the US, statistics on this are variously reported, but sometimes focus on the average or other features of the distribution of course sizes. As of August 2019, Harvard’s FAQ on undergraduate education presented the following statement: “Some introductory courses as well as several other popular courses have large enrollments. Yet, the median class size at Harvard is 12. Of the nearly 1,300 courses offered last fall, for example, more than 1,000 of them enrolled 20 or fewer students.” Princeton advertises a “5:1 student to faculty ratio”. “The University of California - Berkeley advertises that 71% of undergraduate classes have fewer than 30 students.

You might reasonably suspect that there is a lot of selective choice of statistics to find ones that sound good. Clearly there are a range of measures that we could use. This assignment is going to lead you through thinking about which ones make the most sense from the perspective of a student trying to form expectations about how large their modules/courses will be. You have been provided with a dataset that includes the number of students enrolled in all of the POLS00XX undergraduate modules/courses taught by the UCL political science department in 2018-19. All full unit modules appear twice, once for each term, so all modules listed are half-unit.

  1. The student-staff ratio statistic is conventionally calculated as the ratio of the total number of students to the total number of staff. For the UCL political science department, there were 161 undergraduate students who were in the only political science department degree programme, the Philosophy, Politics and Economics BSc, in 2018-19 (in years 1, 2 and 3 combined), and there were 30 staff who taught on the courses in the data file. The student-staff ratio is therefore 161 students / 30 staff = 5.36. Why is this measure not very informative? Identify at least one potential problem with both the numerator and the denominator of the ratio. If you did this calculation at the level of a university, instead of a department, to what extent would that help?
  1. Use the data file to compare UCL political science courses to the statistics advertised above. What is the median class size? What proportion of classes enrolled 20 or fewer students? What proportion of classes enrolled 30 or fewer students?
  1. Compare the mean and median class sizes for UCL political science courses. Why might universities tend to prefer to report the median class size to the mean class size? Is one of these obviously more relevant from the student perspective?
  1. We are now going to think about what statistic is most relevant at capturing the student experience. To constrain the problem a bit, let’s think in terms of some kind of average/mean. Where \(m\) is the number of classes taught, and \(\textrm{s}_j\) is the size / number of students of class \(j\), the mean class size statistic is: \[ A_1 = \frac{\sum_{j=1}^m \textrm{s}_j}{m} \] Come up with a “toy example” that illustrates why this could be a very bad representation of students’ experience of class size. Make your example as simple as possible to illustrate the problem and explain what the problem is.
  1. I propose that the following is a better measure of the concept that we are attempting to measure, the average class size from the perspective of a student: \[ A_2 = \frac{\sum_{j=1}^m \textrm{s}_j^2}{\sum_{j=1}^m \textrm{s}_j} \] What is the value of this measure for your toy example? Does this seem like a better representation of the average experience of students in that example?
  1. State some axioms that you would want a measure to satisfy. Does the measure that I have proposed satisfy those axioms?
  1. What are the units of the measure that I have proposed?
  1. Why does this measure work better? Hint: try thinking about a simple example like one class with two students and one class with six students.
  1. What is the value of this measure for the UCL political science department data set? How does it compare to the median and mean number of students per class?
  1. What is the value of this measure for Harvard? Is this value surprising given the advertised class sizes quoted above? How can these things all be true?