Current situation >> Methodology - research questions
This page describes the operationalization of the research questions and the methods used to gather and analyse data on the bachelor programs. It first describes in detail how we measured the extent to which the current bachelor programs meet the four dimensions of a suitable curriculum. Second, it explains the choice of curricula included in this study and the general comparative approach.
If you are looking for information on the data collection process, variables and data treatment, click here. If you are considering doing such research in your own country/university, we have documented our method, which you are more than welcome to use.
Research (sub-)questions
As set out in the theoretical framework, this research builds on four identified requirements for a proper economics education. This section operationalizes and expands on the four sub-questions, and will methodologically explain how each of the four sub-questions will be answered (click the overview figure to expand it).
As this image shows, the four sub-questions have been operationalized into several smaller sections, in order to analyse the programs course by course, categorizing each course on a number of variables related to the 4 dimensions of interest. For the full questionnaire used, click here.
Sub-question 1: what research skills do we as students learn?
This question is separated into four broad categories; quantitative methods, qualitative methods and mathematical techniques.
In each of these categories, preliminary research on a sample of course descriptions served to identify the most common subcategories. For example, under quantitative research, this resulted in the subcategories regression analysis, factor analysis, descriptive statistics, survey and questionnaire design, data selection and evaluation, experimental economics, and applied econometrics. The categories quantitative methods, qualitative methods and mathematical techniques are measured on a 4-point Likert scale.
Sub-question 2: what economic theoretical approaches and other social sciences are taught, and in what proportions?
A second aspect of pluralism is the diversity in theoretical approaches that is taught within a single program. We measured this by scanning the course descriptions for keywords, concepts and names signaling what theoretical approaches are taught in the course. The second main aspect of this sub-question concerns how much other social sciences were treated, both in separate courses and in combination. This provides insight in how multi- and interdisciplinary the programs are.
The basic categorization here consists of the categories history of economic thought, ten theoretical economic approaches and several forms of inter- and multidisciplinarity with other social sciences. If a certain approach was treated in a course, the extent to which it was treated was noted on a 4-point Likert scale. This enabled us to capture both the diversity and mix of theory, and the theoretical centre of gravity. In other words, this method provided a complete picture of the approaches treated in a certain course or program, as it creates a percentage-wise breakdown of the time spent on each approach.
This report distinguishes the following economic approaches: Austrian school, Behavioral economics, Classical political economy, Complexity economics, Ecological economics, Feminist economics / Social economics, Neoclassical economics, Original institutional economics, Post-Keynesian economics, Radical economics. This categorization of economic approaches is further explained in this appendix file.
[1]: /definitions-and-explanations-of-important-concepts To get a more precise overview of what economic ideas are taught in economics curricula, the different sub-branches of neoclassical economics taught are also measured. Table 5 provides an overview of the categorization of the sub-branches of neoclassical economics used in this research. To indicate how those sub-branches are categorized, in appendix 2 section Q2.2 two theoretical models and three important economists of each sub-branch of neoclassical economics are listed.
Sub-question 3: how much attention is spent on leaving the ivory tower, exploring the real world economy?
The third sub-question deals with the extent to which we as students gain familiarity with real-world economic processes. This sub-question refers to the opposite of what Ronald Coase (2012, p. 19) has called “blackboard economics”, which are mainly thought experiments to support a theoretical argument. This report explores the question from three different angles: real world economic sectors, real world economic problems, real world economic history.
First, it assesses to what extent courses start from the actual economy, casting the theory in a supportive role, focusing on a specific sector or phenomenon rather than a theoretical theme. Examples are the economic consequences of climate change, rising wealth inequality, extreme hunger, a lack of education and development of human capital, gender inequality within economic relationships, or diseases and health problems.
Second, it looks at the amount of courses that are built around one specific economic sector or field. This often concerns the structure of companies within specific sectors, the labour market, housing market, financial sector, energy economics, the informal economy, etc.
Third and finally, it identifies how many courses devote substantial attention to economic history. Examples include historically differently organized economies, past processes of industrialisation and globalisation, or past financial crises.
It is important to note here that the course descriptions may not correspond exactly to the actual contents of the courses. However, we believe that this method guarantees a relatively conservative measurement. After all, any potential bias in terms of course descriptions’ under-reporting of ‘real world economics’ may be quite balanced out by the inverse phenomenon. In the experience of the authors, course descriptions more frequently over-report than under-report the extent of ‘real world economics’ in the course, promising more than is delivered. Indeed, the same phenomenon frequently occurs in the marketing descriptions of entire bachelor programs.
On the whole, we believe this report presents a fairly accurate picture of the degree to which courses deal with real-world economics.
Sub-question 4: do curricula stimulate students to develop and maintain a critical attitude?
The final question is whether the education is an academic one, developing us into critical and creative thinkers, able to look at problems with a fresh and open view, receptive to various types of information and various points of view. This one is the hardest to answer through the selected method of reviewing course outlines, because it depends most strongly on the attitude of the lecturer, and on other unobservable things like class size and in what way material is being discussed.
Fortunately, not everything in this regard is unobservable. We collected information on the following topics: philosophy of science, ethics and economic methodology.
Second, attention to ethics helps us in developing a critical mind-set as ethics enables us as students to reflect on dimensions of morality and justice that are endogenous to certain economic processes and outcomes.
Third, the degree of attention to economic methodology in courses is captured. Here, economic methodology does not refer to applied quantitative or qualitative methodological data gathering skills; rather than a means to gather data, ‘economic methodology’ reflects on methodological issues and decisions. Such an approach increases a student’s awareness of the nature and consequences of methodological choices it makes, and thereby enhances critical thinking.
In order to see whether critical thinking is stimulated, we also compared the didactic methods used in economics bachelor programs.
Such proxy data on the didactic methods used allow a more detailed picture of the degree to which a course causes us as students to think critically. Unfortunately, these are quite crude proxies, but they are the best that were available from the data. Any suggestions on better measurement techniques for this topic will be highly appreciated.
See more
Findings: Theoretical Approaches
Findings: Critical Thinking
What is economics?