The Programme for International Adult Assessment of Competencies (PIAAC) – the Organisation for Economic Cooperation and Development (OECD ) also calls it the Survey of Adult Skills – is an international effort to assess human skills and further develop our understanding of human capital, started in 2012 in 22 countries (listed below) and coordinated by the OECD. The list of participating countries will be extended to new participants during this decennial as new rounds of the PIAAC assessment are being implemented (participants of the second round of PIAAC are also listed).
Intro video from the OECD | http://www.oecd.org/site/piaac/
Following in the footsteps of the International Adult Literacy Survey (IALS – 1994) and the Adult Literacy and Lifeskills Survey (ALL – 2003), PIAAC (2012) proposes to measure skill levels in a technology-rich environment (shortened to TRE) and to compare them across many OECD countries. With a wealth of common socio-economic indicators, PIAAC opens the door for extensive comparative research on education and its social outcomes (also read Education and Health: building indicators in international comparison, RISS – IJHS, 4(1)), which is extremely useful for further research on the determinants of health.
Surveys like PIAAC , which include psychometric elements, are complex to develop. To appropriately measure one’s skills, these tests must minimize any variable that would affect the performance of the interviewees. Tests, taken in person in the home environment, require a continued effort for an extended period of time, hence have to account for tiredness and boredom – variables we wish to exclude from the skills assessment as they may alter an individual’s performance. Other indicators are in place at the time of the assessment to keep record of the conditions of the test (length of time per item, overall duration of the test, presence of relatives or children).
Two elements compose the PIAAC survey:
- A background questionnaire with a common core for all countries (to standardize socio-economic characteristics of individuals) and a limited set of questions established by each country.
- Three psychometric tests with sets of “items” that will serve to establish a score for (1) literacy skills, (2) numeracy skills and (3) problem-solving skills in TRE.
The whole survey and its various components were developed under the leadership of Education Testing Services (ETS), an American-based company that built its experience with the previous national and international surveys (IALS & ALL), and the Graduate Record Examination (GRE) and the Test of English as a Foreign Language (TOEFL). These psychometric items remain the intellectual property of ETS and may only be consulted or used with their permission. ETS always welcomes researchers to submit their project. Funding may also be available to expand the use and reach of the programme’s database.
In most countries, national agencies and ministries were responsible for the implementation of PIAAC: from sampling to collection and presentation of data. Some countries resorted to private firms (ex: Singapore). Each country was also able to add their own (limited) set of questions to the background questionnaire, in the perspective of further national work on adult skills. This extra set of data may be available for research use through each national agency or ministry.
To limit the duration of the test, each individual would only be tested on two of the three type of skills: one would not be tested on her/his literacy, numeracy and problem-solving skills (as shown in Figure 1). This scheme allows interviewers to alleviate some of the mental stress put upon the interviewees and, presumably, test more accurately individual skills.
Figure 1: Generally, seven possibilities to do the PIAAC assessment (computer-based). The psychometric tests are colored in yellow. Individuals who declare very little or no computer experience in the Background Questionnaire are tested on their literacy and numeracy skills but not on their problem-solving skills in TRE.
Yet, to preserve the comprehensiveness of such skills assessment and allow for comparison, a set of 10 plausible values would act for the missing empirical data in further analysis. The plausible values for one type of skills are chosen randomly within a range determined by (1) the score obtained in the two other tested skills for each individual and by (2) the scores of other individuals obtaining similar scores in the common area of testing (literacy, numeracy or problem-solving). The Australian Bureau of Statistics offers more information on the making and use of these plausible values.
It is also important to note that, when comparing between countries, the sample in each national dataset is further weighted to their respective population. Researchers often use Canada as a point of reference for accurate data, in consideration of the fact that it has the largest national sample – it was built to provide analytically significant estimate at the provincial/territorial level and for specific sub-populations. This allows for precise global estimates of literacy, numeracy and problem-solving levels and for better comparison.
Accessing and analyzing PIAAC data
PIAAC data are publicly available either through the OECD online software (the International Data Explorer) or by downloading Public User Data Files (PUF). PUF are available for download for every participating country (except Australia*) and can be analyzed with SPSS, SAS, Excel or STATA. Data compendia and instructions are also presented.
*Access to Australian PUF may be requested through the Australian Bureau of Statistics. Australian data are available in the OECD online software.
Software developed by stakeholders of PIAAC such as the OECD and the International Association for the Evaluation of Educational Achievement (IEA) were made to guide analysts in the computation of PIAAC results. They process the plausible values and sample weights automatically, which makes them very useful for quicker access to compiled PIAAC data.
For data visualization and analysis:
- The OECD developed the International Data Explorer (PIAAC IDE), an online software. Cyprus data are not available through the PIAAC IDE.
- The IEA developed the International Database (IDB) Analyzer, available for download. Because the IDB Analyzer uses PUF, Australian data are not be available for this software.
Both software are publicly available.
Quick useful resources
The OECD website also lists a handful of resources and links for interested data analysts.
The Canadian report on PIAAC: Skills in Canada: First results from the Programme for the International Assessment of Adult Competencies, 2012 (final)
“Skills assessment, PIAAC and the adult literacy agenda” was published in the peer-reviewed International Journal of Lifelong Education (Volume 33, Issue 2) in 2014.
Feel free to leave a comment below if you want to recommend resources about PIAAC available at your organization.
National agency or ministry
Please note that the agencies’ website are not necessarily available in English. As much as possible, the English name of the institution is listed here whenever provided by the institution itself. Italicized countries participate to the second round of PIAAC assessment.
- Australia: Australian Bureau of Statistics
- Austria: Statistik Austria
- Belgium (Flanders): Universiteit Gent
- Canada: Statistics Canada, Council of Ministers of Eduction, Canada (CMEC)
- Chile (assessment taking place in June-November 2014): Centro de Medición de la Universidad Católica de Chile (MIDE UC)
- Cyprus: Centre of Educational Research and Evaluation
- Czech Republic: Mezinárodní výzkum dospělých
- Denmark: SFI – Det Nationale Forskningscenter for Velfærd
- Estonia: Haridus- ja Teadusministeerium
- Finland: Statistics Finland, University of Jyväskylä and the Finnish Institute for Educational Research
- France: Institut national de la statistique et des études économiques
- Germany: Deutsche Institut für Erwachsenenbildung
- Greece (assessment taking place from April 2013 to May 2016): National Organisation for the Certification of Qualifications & Vocational Guidance (EOPPEP) (Greece’s PIAAC website: http://www.piaac.gr )
- Indonesia (assessment expected during the second round or PIAAC, between 2012 and 2016): not found, unfortunately.
- Ireland: Central Statistics Office
- Israel (assessment expected during the second round or PIAAC, between 2012 and 2016): Central Bureau of Statistics
- Italy: Istituto per lo sviluppo della formazione professionale dei lavoratori
- Japan: National Institute for Educational Policy Research (Ministry of Education, Culture, Sports, Science and Technology)
- Lithuania (assessment expected during the second round or PIAAC, between 2012 and 2016): not found, unfortunately.
- South Korea: Korea Research Institute for Vocational Education and Training (KRIVET)
- Netherland: Expertisecentrum Beroepsonderwijs (ecbo)
- New-Zealand (assessment taking place in April-December 2014): National Research Bureau
- Norway: Statistics Norway
- Poland: Educational Research Institute
- Russian Federation: Institute for Educational Studies
- Singapore (assessment taking place in April-December 2014): Institute for Adult Learning
- Slovak Republic: Národný ústav celoživotného vzdelávania
- Slovenia (assessment taking place between January 2013 and June 2015): Slovenian Institute for Adult Education; their dedicated website for PIAAC is http://piaac.acs.si
- Spain: National Statistics Institute
- Sweden: Statistics Sweden
- Turkey (assessment expected during the second round or PIAAC, between 2012 and 2016): Ministry of Labour and Social Security
- United Kingdom (England and Northern Ireland): Department for Business, Innovation & Skills
- United States: National Center for Education Statistics
Should you see any mistake in this list, please do not hesitate to leave a comment below: I will be notified automatically.
Latest posts by Gatien de Broucker (see all)
- eHealth governance: Google, dot-health and privatizing the Internet - 26 / 02 / 2015
- Introduction to PIAAC – measuring adult literacy and skills - 27 / 08 / 2014
- Foreword – Volume 4, Issue 1 - 26 / 08 / 2014
- Statistics and health 2014 [special issue] - 25 / 06 / 2014
- Networking event on March 31 | uOttawa - 01 / 04 / 2014