Programme for the International Assessment of Adult Competencies

The Programme for the International Assessment of Adult Competencies (PIAAC) is a worldwide study by the Organisation for Economic Co-operation and Development (OECD) in 24 countries of cognitive and workplace skills. The main aim is to be able to assess the skills of literacy, numeracy and problem solving in technology-rich environments, and use the collected information to help countries develop ways to further improve these skills. The focus is on the working-age population (between the ages of 16 and 65). The first data was released on October 8, 2013.^[1][2] A new PIAAC survey is expected to be published in 2021/2022.^[3]

Since the early 1990s the need for assessing literary skills in developed countries has been addressed by two large international surveys. The first was the International Adult Literacy Survey (IALS) which was implemented in 1994, 1996, and 1998. The second was the International Adult Literacy and Life Skills Survey carried out in 2003, and between 2006 and 2008.^[4]

Three central basic skills are assessed in PIAAC: literacy, numeracy and problem solving in technology-rich environments.^[5]

Literacy encompasses the ability to understand, use and interpret written texts. Literacy is a prerequisite for developing one's knowledge and potential and participating in society. The literacy domain in PIAAC includes tasks such as reading and understanding a drug label or a brief newspaper article. In addition, there are tasks that involve digital media, such as reading an online job posting.

Below Level 1 – 0 – 175 The tasks at this level require the respondent to read brief texts on familiar topics to locate a single piece of specific information. There is seldom any competing information in the text and the requested information is identical in form to information in the question or directive. The respondent may be required to locate information in short continuous texts. However, in this case, the information can be located as if the text were non-continuous in format. Only basic vocabulary knowledge is required, and the reader is not required to understand the structure of sentences or paragraphs or make use of other text features. Tasks below Level 1 do not make use of any features specific to digital texts.^[6]

Level 1 – 176 – 225 Most of the tasks at this level require the respondent to read relatively short digital or print continuous, non-continuous, or mixed texts to locate a single piece of information that is identical to or synonymous with the information given in the question or directive. Some tasks, such as those involving non-continuous texts, may require the respondent to enter personal information onto a document. Little, if any, competing information is present. Some tasks may require simple cycling through more than one piece of information. Knowledge and skill in recognizing basic vocabulary determining the meaning of sentences, and reading paragraphs of text is expected.^[6]

Level 2 – 226 – 275 At this level, the medium of texts may be digital or printed, and texts may comprise continuous, non-continuous, or mixed types. Tasks at this level require respondents to make matches between the text and information and may require paraphrasing or low-level inferences. Some competing pieces of information may be present. Some tasks require the respondent to

• cycle through or integrate two or more pieces of information based on criteria;
• compare and contrast or reason about information requested in the question; or
• navigate within digital texts to access and identify information from various parts of a document.^[6]

Level 3 – 276 – 325 Texts at this level are often dense or lengthy, and include continuous, non-continuous, mixed, or multiple pages of text. Understanding text and rhetorical structures become more central to successfully completing tasks, especially navigating complex digital texts. Tasks require the respondent to identify, interpret, or evaluate one or more pieces of information, and often require varying levels of inference. Many tasks require the respondent to construct meaning across larger chunks of text or perform multi-step operations in order to identify and formulate responses. Often tasks also demand that the respondent disregard irrelevant or inappropriate content to answer accurately. Competing information is often present, but it is not more prominent than the correct information.^[6]

Level 4 – 326 – 375 Tasks at this level often require respondents to perform multiple-step operations to integrate, interpret, or synthesize information from complex or lengthy continuous, non-continuous, mixed, or multiple type texts. Complex inferences and application of background knowledge may be needed to perform the task successfully. Many tasks require identifying and understanding one or more specific, non-central idea(s) in the text in order to interpret or evaluate subtle evidence-claim or persuasive discourse relationships. Conditional information is frequently present in tasks at this level and must be taken into consideration by the respondent. Competing information is present and sometimes seemingly as prominent as correct information.^[6]

Level 5 – 376 – 500 At this level, tasks may require the respondent to search for and integrate information across multiple, dense texts; construct syntheses of similar and contrasting ideas or points of view; or evaluate evidence based arguments. Application and evaluation of logical and conceptual models of ideas may be required to accomplish tasks. Evaluating reliability of evidentiary sources and selecting key information is frequently a requirement. Tasks often require respondents to be aware of subtle, rhetorical cues and to make high-level inferences or use specialized background knowledge.^[6]

Numeracy refers to the ability to access, use and interpret everyday mathematical information in order to manage mathematical demands in daily life. This is measured, for example, with items involving the evaluation of a special offer or the interpretation of numerical information in figures and tables.

Below Level 1 0 – 175 Tasks at this level require the respondents to carry out simple processes such as counting, sorting, performing basic arithmetic operations with whole numbers or money, or recognizing common spatial representations in concrete, familiar contexts where the mathematical content is explicit with little or no text or distractors.^[6]

Level 1 176 – 225 Tasks at this level require the respondent to carry out basic mathematical processes in common, concrete contexts where the mathematical content is explicit with little text and minimal distractors. Tasks usually require one-step or simple processes involving counting, sorting, performing basic arithmetic operations, understanding simple percent's such as 50%, and locating and identifying elements of simple or common graphical or spatial representations.^[6]

Level 2 226 – 275 Tasks at this level require the respondent to identify and act on mathematical information and ideas embedded in a range of common contexts where the mathematical content is fairly explicit or visual with relatively few distractors. Tasks tend to require the application of two or more steps or processes involving calculation with whole numbers and common decimals, percents and fractions; simple measurement and spatial representation; estimation; and interpretation of relatively simple data and statistics in texts, tables and graphs.^[6]

Level 3 – 276 – 325 Tasks at this level require the respondent to understand mathematical information that may be less explicit, embedded in contexts that are not always familiar and represented in more complex ways. Tasks require several steps and may involve the choice of problem-solving strategies and relevant processes. Tasks tend to require the application of number sense and spatial sense; recognizing and working with mathematical relationships, patterns, and proportions expressed in verbal or numerical form; and interpretation and basic analysis of data and statistics in texts, tables and graphs.^[6]

Level 4 326 – 375 Tasks at this level require the respondent to understand a broad range of mathematical information that may be complex, abstract or embedded in unfamiliar contexts. These tasks involve undertaking multiple steps and choosing relevant problem-solving strategies and processes. Tasks tend to require analysis and more complex reasoning about quantities and data; statistics and chance; spatial relationships; and change, proportions and formulas. Tasks at this level may also require understanding arguments or communicating well-reasoned explanations for answers or choices.^[6]

Level 5 – 376 – 500 Tasks at this level require the respondent to understand complex representations and abstract and formal mathematical and statistical ideas, possibly embedded in complex texts. Respondents may have to integrate multiple types of mathematical information where considerable translation or interpretation is required; draw inferences; develop or work with mathematical arguments or models; and justify, evaluate and critically reflect upon solutions or choices.^[6]

PIAAC is the first international survey to implement problem solving in technology-rich environments (ICT). This key skill is defined as the ability to successfully use digital technologies, communication tools and networks to search for, communicate and interpret information. The first wave of PIAAC focuses on how persons access and make use of information in a computer-based environment. Items include sorting and sending e-mails, filling out digital forms, and evaluating the informational content and credibility of different websites.

Level 1 – 241 – 290 At this level, tasks typically require the use of widely available and familiar technology applications, such as e-mail software or a web browser. There is little or no navigation required to access the information or commands required to solve the problem. The problem may be solved regardless of the respondent's awareness and use of specific tools and functions (e.g. a sort function). The tasks involve few steps and a minimal number of operators. At the cognitive level, the respondent can readily infer the goal from the task statement; problem resolution requires the respondent to apply explicit criteria; and there are few monitoring demands (e.g. the respondent does not have to check whether he or she has used the appropriate procedure or made progress towards the solution). Identifying content and operators can be done through simple match. Only simple forms of reasoning, such as assigning items to categories, are required; there is no need to contrast or integrate information.^[6]

Level 2 – 291 – 340 At this level, tasks typically require the use of both generic and more specific technology applications. For instance, the respondent may have to make use of a novel online form. Some navigation across pages and applications is required to solve the problem. The use of tools (e.g. a sort function) can facilitate the resolution of the problem. The task may involve multiple steps and operators. The goal of the problem may have to be defined by the respondent, though the criteria to be met are explicit. There are higher monitoring demands. Some unexpected outcomes or impasses may appear. The task may require evaluating the relevance of a set of items to discard distractors. Some integration and inferential reasoning may be needed.^[6]

Level 3 – 341 – 500 At this level, tasks typically require the use of both generic and more specific technology applications. Some navigation across pages and applications is required to solve the problem. The use of tools (e.g. a sort function) is required to make progress towards the solution. The task may involve multiple steps and operators. The goal of the problem may have to be defined by the respondent, and the criteria to be met may or may not be explicit. There are typically high monitoring demands. Unexpected outcomes and impasses are likely to occur. The task may require evaluating the relevance and reliability of information in order to discard distractors. Integration and inferential reasoning may be needed to a large extent.^[6]

PIAAC was initiated by the OECD member states in 2008 and, like PISA, it is designed as a multi-cycle programme. Round 1 took place in 2008–13 (main study in 2011^[4]), supplementary Round 2 in 2012–16, and Round 3 in 2014–18 (main study in 2016–17).^[7] Subsequent cycles will allow future changes in adult skills to be monitored and analysed and will provide first indications of where improvements have been achieved and deficits persist. Twenty-four countries participated in PIAAC Round I. Nine additional countries also expressed interest in participating in PIAAC. At least 5 000 randomly selected respondents between the ages of 16 and 65 were interviewed and assessed in each participating country. The survey was carried out as a personal interview comprising a questionnaire followed by a skills assessment, a computer- or paper-based version of which was independently completed by the respondent in the presence of the interviewer; the entire interview (including the assessment) took between 1 1/2 and 2 hours to administer.^[5]

The results were published in 2013, together with summaries in 25 languages.^[8]

Notes:

1. Sub-national entities (Belgium and UK) are placed at the end followed by two partner countries (Cyprus and Russia) who took part. The figures for Russia are preliminary and do not include Moscow or any items in the 'Missing' category.
2. Participants were ranked at 5 levels (3 in problem solving). Level 1 corresponds to 176 points. Those who scored less than level 1 are listed above as well as those marked as 'Non-starters' above or 'Missing' in the report, referring to 'literacy-related non-responses' due to mental or learning disabilities or language difficulties. These were marked at 85 (out of 500) in calculating the mean figures.
3. The last three columns show the extra points scored on average by the first group compared with the second. The last compares native-born native language speakers with foreign-born foreign language speakers.
4. France, Italy, Spain and Cyprus did not participate in the Problem Solving test, which is officially described as 'Problem Solving in technology-rich environments'. The 'No ICT' column includes those who had no computer experience, opted out or failed a basic IT competence test.
5. Cyprus only refers to that part under control of the Republic of Cyprus.

• Missing: Individuals at this level were unable to complete the background questionnaire.
• Below Level 1: can read brief texts on familiar topics and locate a single piece of specific information identical in form to information in the question or directive.
• Level 1: (176 points) can complete simple forms, understand basic vocabulary, determine the meaning of sentences, and read continuous texts with a degree of fluency.
• Level 2: (226 points) can integrate two or more pieces of information based on criteria, compare and contrast or reason about information and make low-level inferences
• Level 3: (276 points) can understand and respond appropriately to dense or lengthy texts, including continuous, non-continuous, mixed, or multiple pages.
• Level 4: (326 points) can perform multiple-step operations to integrate, interpret, or synthesise information from complex or lengthy continuous, non-continuous, mixed, or multiple-type texts that involve conditional and/or competing information.
• Level 5: (376 points) can perform tasks that involve searching for and integrating information across multiple, dense texts; constructing syntheses of similar and contrasting ideas or points of view, or evaluating evidence and arguments.^[6]

For details of the numeracy and ICT tests see OECD 2013.

Gallup principal economist Jonathan Rothwell assessed results collected by PIAAC during 2012 - 2017 for a 2020 economic impact analysis^[9] commissioned by the Barbara Bush Foundation for Family Literacy,^[10] surmising that, of the 33 OECD nations surveyed, the U.S. had placed sixteenth for literacy, with about half of Americans surveyed, aged 16 to 74, demonstrating a below sixth-grade reading level.^[9]

Programme for International Student Assessment

• Official PIAAC web site
• Download PIAAC result and summary at GESIS – Leibniz Institute for the Social Sciences
• Literacy Lost: Canada's Basic Skills Shortfall, Canada West Foundation, December 2018
• 2018 International Yearbook of Adult Education, Trends and Issues in Canadian Adult Education Research, wbv.de