New insights on skills from the application of economic complexity to UK job postings data

In an ever changing world of work, having timely and accurate information about the latest trends in the labour market is essential. Policymakers and education providers can use this information to address skills gaps, improve matching between job seekers and vacancies and better understand returns on skills investments.

Thanks to advances in technology, it is now possible to apply big data techniques to online job adverts to answer some of these questions. Online job adverts offer insights on the skills, knowledge and behaviours most in demand among employers as well as contextual information on a number of different dimensions such as occupation, salary and location.

The purpose of this paper is to bring the analysis of online job postings data one step further to answer questions about the substance of the skills required in the adverts and not just their frequency. The paper proposes to do so by applying for the first time the concept of economic complexity to online job postings data. Introducing the concept of skills complexity, this paper looks at a new way of extracting information about skills from online job postings, approaching skills from four different perspectives: specialisation, diversity, ubiquity and proximity.

Each of these dimensions offers a different angle on the labour market, allowing us to answer questions such as: what level of Excel skills are required in a particular occupation? What are the most intensive skills required for an Automation Engineer role? Is demand for an occupation becoming more complex and specific over time? Is there proximity between the industrial or occupation make-up of different communities?

The metrics introduced in this paper have been built using proprietary data on online job postings from Lightcast (formerly Emsi Burning Glass) and the methodology of economic complexity first developed in 2009 by Hidalgo and Hausmann.

Lightcast job postings library contains over 77 million job postings for the UK, collected since 2012. It is built by scraping on a daily basis over 1,000 online job boards, newspapers and employer sites. The job ads are then deduplicated to ensure only one posting is counted for each opening -regardless of how many places it is advertised in. The job postings are then classified by location, employer, occupation, skills required and any other type of relevant information that can be extracted from the ad. Compared to traditional sources of data, job postings data allows for a detailed, real-time look at the labour market and what employers need. However, the data is dependent on employers advertising their openings online, which is more popular in higher-skilled professional occupations than in lower-skilled ones.

This research then applied to this data the methodology introduced by Hidalgo and Hausmann in 2009 to identify countries’ competitive advantage by analysing their exports. The authors initially used this concept to capture the process of knowledge creation and diffusion in an economy. They measured economies against two metrics: (1) their diversity -i.e. how many different sectors they specialise in and (2) their ubiquity -i.e. how many places specialise in that particular sector. From the combination of these two metrics, they created the economic complexity index, a relative measure of the complexity of each economy. Over the years, this index was then reviewed and applied to different dimensions -such as measuring the complexity of local economies -and Hidalgo produced a comprehensive review of all the applications of this concept in 2021.

In the context of the labour market, economic complexity is a measure of the knowledge in an occupation as expressed by the skills it requires. The complexity of an occupation is calculated based on the diversity of skills that occupation requires and their ubiquity -the frequency with which they appear in different occupations (and their complexity). Occupations that require a diverse range of skill know-how, including sophisticated, unique know-how, are more complex than others.

The intuition behind the computation of economic complexity in this context is the application of learning vectors. Learning vectors explain the structure of a specialisation matrix and the usage of matrices of networks to create representations of a complex system. For the purpose of this research, the following metrics were created:

1. Two specialisation matrices: one using online job postings and skills required and the second using online job postings and the occupations required in a specific location;
2. Binary specialisation matrices
3. Diversity and ubiquity metrics

From the individual metrics, it is then possible to calculate the Economic Complexity Index (ECI) and unpack it into its four different components:

• Specialisation: a measure that captures the significant presence or absence of a skill in a specific occupation.
• Diversity: a measure of how many different types of skills an occupation requires. A skill requires a specific set of know-how, therefore, an occupation’s total diversity is another way of expressing the amount of collective know-how held within that occupation.
• Ubiquity: a measure of the number of occupations that require a specific skill. More complex knowledge diffuses with more difficulty.
• Proximity: a measure of the probability that an occupation requires a skill ‘A’ given that it requires skills ‘B’, or vice versa. Given that an occupation requires one skill, proximity captures the ease of obtaining the know-how needed to move into another occupation.

This measure formalises the intuitive idea that the intensity/ability required of a skill can be revealed by looking at other skills required in a job posting.

The innovative way to look at skills presented in this paper has a number of important practical implications. Firstly, measuring the level of complexity required of a skill in a specific occupation and how this has evolved over time can help education providers better plan their courses, going beyond the concept of skills frequency. Secondly, by looking at the relatedness between different occupations and areas, this measure allows us to identify labour flows and help workers understand which occupations are most similar to their current ones. Thirdly, by understanding the complexity of different occupations, local and regional economic development organisations can shape their policy responses to promote economic growth in their area.

The rest of the paper is structured as follows: Section 1 will provide an overview of the literature on the topics of economic complexity, online job postings and big data and skills. Section 2 will outline the data used in the report. Section 3 will present the methodology and Section 4 will present the findings of the research, before setting out concluding remarks in Section 5.

References

Hidalgo C.A. (2021) “Economic complexity theory and applications”, Nature Review Physics 3, 92-113

Hidalgo C.A. and Hausmann R. (2009)“The Building Blocks of Economic Complexity’, Centre for International Development at Harvard University, working paper no. 186

Mealy, P., Farmer, J. D., & Teytelboym, A. (2019). “Interpreting economic complexity. Science advances”, 5(1), eaau1705.

Giabelli, A., Malandri, L., Mercorio, F., & Mezzanzanica, M. (2020). “GraphLMI: A data driven system for exploring labor market information through graph databases”. Multimedia Tools and Applications, 1-30.

Colombo, E., Mercorio, F., & Mezzanzanica, M. (2018). “Applying machine learning tools on web vacancies for labour market and skill analysis.” Terminator or the Jetsons? The Economics and Policy Implications of Artificial Intelligence.

World Economic Forum Boston Consulting Group (BCG). (2018). “Towards a reskilling revolution: A future of jobs for all”. In World Economic Forum.