Educational assessment practices and emerging technology trends

Assessment is a powerful tool in education which can influence the way in which students respond to teaching, and their overall learning experience.  Innovative technologies have also paved the way for alternative assessment formats, including video-based evaluations and interactive tools like VR.

A research team from Singapore Management University conducted a meta-analysis of the existing research on this topic, revealing four growing groups of technology trends relating to learning environments and assessment.

Read their original research:  https://ink.library.smu.edu.sg/sis_research/7595

Transcript:

Hello and welcome to ResearchPod. Thank you for listening and joining us today.

In this episode, we will delve into the research of Tristan Lim, a Doctor of Engineering student at the Singapore Management University, in collaboration with his research advisors Professor Swapna Gottipati and Professor Michelle Cheong. The research investigates how emerging technologies are shaping assessment practices in higher education.

Assessment is a powerful tool in education which can influence the way in which students respond to teaching, and their overall learning experience. Nowadays, there is increasing pressure on higher education institutions to advance their methods of assessment based on current technologies.

The research team evaluate how emerging technologies are shaping assessment practices within higher education, focusing on research published between 2011 and 2021.

Advancements in technology have significantly impacted various facets of education. For instance, mobile devices like laptops, tablets, and smartphones have facilitated the administration of assessments beyond traditional exam halls and into diverse environments. Students can now take exams from the comfort of their homes, which is not only convenient but also more accessible for those living at a distance from their educational institutions. Conducting assessments at home also eliminates challenges related to in-person submission, presentation, and storage. In addition, mobile technologies allow for examinations in more practical settings, such as medical professionals completing tests within healthcare environments.

Innovative technologies have also paved the way for alternative assessment formats, including video-based evaluations, digital presentations, and electronic portfolios. Furthermore, contemporary assessments can incorporate various interactive tools, like virtual reality and gamification, which provide a more engaging and immersive experience for learners.

To better understand the emergent technologies that are influencing educational assessment, the research team have conducted what is called a ‘meta-analysis’ of the existing research on this topic. A meta-analysis gathers together all findings on a research question for a given period and puts them together into one overarching analysis. It offers a powerful approach for synthesizing existing research to provide more reliable and generalizable insights, inform decision-making, and guide future research directions.

To conduct their meta-analysis, the research team chose to use publications from the Horizon Reports – an annual prediction of key global technology developments in education, in relation to assessment practices – between the years 2011 and 2021.

They followed several steps to conduct this important meta-analysis. First, they made a visual representation of the trends in recent technologies used in higher education institutions using the Horizon Reports predictions. To do this they grouped technologies according to their similarities, and then created a detailed view of each group.

Second, they performed a bibliometric analysis to measure the impact of predicted technologies in assessment practices. This analysis was performed by searching and analysing the number of relevant papers published between 2011 and 2021 of each technology group in bibliographic databases.

The results of this method revealed that there were four groups of technology trends based on the Horizon Reports. These were technology trends related to ubiquitous, adaptive, and immersive learning technologies, and technology trends related to learning objects.

Of these four groups, immersive learning technologies generated the highest publication interest in 2021. Immersive learning technologies are technologies that create an immersive environment for the purpose of learning and assessment. Examples of these include a digital avatar learning experience, or a game-based virtual world of augmented reality or mixed reality, such as Lord of the Rings Online. These technologies are becoming popular amongst students, as they create a fun and engaging learning environment.

The publication numbers of immersive learning technologies were consistently around 800 to just under 950 per year between 2011 and 2021, and these technologies were mentioned in nine out of the 11 years under study in the Horizon Reports. This demonstrates consistent research in immersive learning technologies over the past decade and beyond, including game-based environments and immersive environment-related education and assessments.

One issue with immersive learning technologies is that the cost of creating and setting up these assessment tools can be high, relative to other technologies or assessment methods. These high costs are arguably offset by the cheap adaptation of the content for different scenarios and assessments, the lower maintenance costs, the ease of transporting the technology around, and the smaller space requirements, compared to other assessment methods.

Immersive learning technologies have what is called ‘high ecological validity’, which means they can be a good reflection of real-life situations, potentially making them more accurate assessment tools. Despite this, immersive learning technologies still suffer from several issues, including immaturity of technology, uncertain skill transfer, and lack of specialisation and testing of systems.

The results of the research team’s meta-analysis revealed that Adaptive Learning Technologies had the second highest publication interest in 2021. Adaptive learning technologies use artificial intelligence as a method to advance assessment practices.

There are lots of ethical debates surrounding the use of artificial intelligence in education and assessment, particularly around the topic of grading. However, there are several great advantages to using artificial intelligence in education and assessment, such as improved analysis of performance and feedback on assessments.

The researchers note that while the amount of interest garnered in adaptive learning technologies was less than 50% of the research interest in immersive learning technologies in 2011, by 2021 there was only a 10% difference between adaptive learning technologies and immersive learning technologies. In fact, the interest in adaptive learning technologies almost tripled between 2011 and 2021. Based on the existing trends published in the Horizon reports, it is expected that adaptive learning technologies will take on a larger role within education and assessment in the years to come.

The results of the research team’s meta-analysis also showed that ubiquitous learning technologies generated the third highest publication interest in the Horizon Report between 2011 and 2021. The number of ubiquitous learning technology publications started at 107 in 2011 and ended at 323 in 2021, tripling over a ten-year period.

Ubiquitous learning technologies apply certain technology methods to the real world to integrate people with their environment. Ubiquitous learning assessments are important because they have the potential to provide authentic situational assessment experiences.

Ubiquitous learning technologies are faced with some scepticism because of the concern that they may make cheating easier through their poor monitoring systems. However, most threats of cheating are manageable and ubiquitous learning technologies offer a great advantage to other technologies by enhancing the authenticity of assessments in using real-world settings. Authentic assessments refer to evaluation methods that require students to apply their knowledge and skills in real-world or practical contexts. The fact that the number of publications in ubiquitous learning technologies has tripled over the past decade suggests that these technologies are an area with great potential for education researchers to explore in the future.

Finally, the results of the research team’s meta-analysis showed that publications relating to learning objects had the lowest numbers in 2021. Learning objects can be any entity, digital or non-digital, which can be used during learning. For example, learning objects can include computer-based training systems, interactive learning environments, distance learning systems, and collaborative learning environments.

There has been growing research on learning objects between 2011 and 2021, with a surge in interest since 2019. Although, it is important to note that learning objects still had the least amount of publication interest among the four groups of technology trends. This increased interest potentially reflects the COVID-19 pandemic, from which point there was greater interest in using online learning platforms, and other learning objects, to improve online assessment.

The research team recommend that future research focuses on providing more detailed guidance on how each area of technology can influence educational assessment practices. It would also be useful to study research relating to the technology cycles for new emergent technologies, to inform and prepare educators and researchers about potential technological changes in the upcoming years.

The research team have shown that Horizon Reports predictions can be used to study trends in technologies that are shaping educational assessment, at least within the higher education sector. As might be expected, trends in educational assessment technologies have been affected by the COVID-19 pandemic in recent years – with a surge in publication interest relating to educational assessment technologies generated after 2019.

The bibliometric statistics suggest that adaptive learning technologies, whilst currently the second most highly published area, will likely surpass immersive learning technologies in the future. Similarly, having tripled in publication numbers over the past ten years, ubiquitous learning technologies may also be a future focus for educational assessment researchers.

The meta-analysis reported by the researchers has shown that technology within educational assessment is growing at a rapid pace. The research has highlighted that there are several different areas of educational technological advancements, including immersive, adaptive, and ubiquitous learning technologies, and learning objects. These areas are all growing at different rates and will continue to do so in the future.

That’s all for this episode – thanks for listening. Don’t forget to subscribe to ResearchPod for more detailed breakdowns of the latest academic research.

See you next time!

Also published on: https://researchpod.org/education-training/educational-assessment-practices-emerging-technology-trends

Podcast is also available on Spotify, Apple iTunes, Google Podcasts, and many more (please use search term “ResearchPod”).