Large Language Models in Education:
Vision and Opportunities

Abstract

With the rapid development of artificial intelligence technology, large language models (LLMs) have become a hot research topic. Education plays an important role in human social development and progress. Traditional education faces challenges such as individual student differences, insufficient allocation of teaching resources, and assessment of teaching effectiveness. Therefore, the applications of LLMs in the field of digital/smart education have broad prospects. The research on educational large models (EduLLMs) is constantly evolving, providing new methods and approaches to achieve personalized learning, intelligent tutoring, and educational assessment goals, thereby improving the quality of education and the learning experience. This article aims to investigate and summarize the application of LLMs in smart education. It first introduces the research background and motivation of LLMs and explains the essence of LLMs. It then discusses the relationship between digital education and EduLLMs and summarizes the current research status of educational large models. The main contributions are the systematic summary and vision of the research background, motivation, and application of large models for education (LLM4Edu). By reviewing existing research, this article provides guidance and insights for educators, researchers, and policy-makers to gain a deep understanding of the potential and challenges of LLM4Edu. It further provides guidance for further advancing the development and application of LLM4Edu, while still facing technical, ethical, and practical challenges requiring further research and exploration.

I Introduction

With the rapid development of big data [1, 2], artificial intelligence, and Web 3.0 [3, 4], large language models (LLMs) [5, 6, 7, 8] have become a research hotspot. LLMs are deep learning models that learn the underlying patterns and rules of language by training on large-scale corpora. They possess powerful capabilities in generating and understanding natural language and have been widely applied in natural language processing (NLP) [9], machine translation [10], dialogue systems [11], AI-generated content (AIGC) [9], social cognitive computing, among other fields. Education is a significant domain that plays a crucial role in the development and progress of human society. Traditional educational models face challenges such as individual differences among students, insufficient allocation of teaching resources, and the assessment of teaching effectiveness [12]. Therefore, incorporating LLMs into the field of education holds the potential to provide support for personalized learning [13], intelligent tutoring, adaptive assessment [14], and other aspects, thereby improving the quality of education and the learning experience.

In the digital era, the education field currently faces various challenges [13], including low student engagement [15] and unequal distribution of teaching resources [16]. Traditional classroom teaching struggles to meet the personalized needs of different students. LLMs, as powerful natural language processing tools, have the potential to revolutionize traditional teaching models by enabling personalized learning and intelligent tutoring. Furthermore, with the advent of the big data era, the education field has accumulated a vast amount of learning data [17]. Utilizing this data for in-depth analysis and mining can reveal learners’ patterns [18], evaluate learning outcomes [19], and provide personalized recommendations [20, 21]. LLMs have advantages in processing and analyzing large-scale data, making their application in the education field capable of providing deeper learning support and personalized education.

Large models refer to models with a massive number of parameters and computational capabilities [22]. LLMs are one type of large models, often involving billions of parameters. The essence of large models lies in their ability to handle complex tasks and large-scale data, enabling them to learn richer language patterns and knowledge representations [21]. This makes large models highly applicable in the field of education. Smart/intelligent education refers to the provision of personalized, adaptive, and intelligent educational services through the utilization of technologies such as artificial intelligence and big data. For smart education, educational large models (EduLLMs) refer to educational application models based on LLMs. By learning from extensive educational data and corpora, EduLLMs can provide personalized learning support [23], intelligent tutoring [24], and educational assessment capabilities to students [25]. The research status of EduLLMs demonstrates significant potential and opportunities. Firstly, EduLLMs can identify students’ learning patterns and characteristics by learning from massive educational data, enabling the provision of personalized learning support and recommendations for educational resources. Secondly, EduLLMs can be applied to intelligent tutoring, providing real-time problem-solving, learning advice, and academic guidance through dialogue and interaction with students. Moreover, EduLLMs have the potential for educational assessment, automatically evaluating students’ knowledge mastery, learning outcomes, and expressive abilities, thereby providing more comprehensive student evaluation and teaching feedback to educators.

However, the research on LLM4Edu still faces challenges and issues. Firstly, social cognitive learning is challenging in LLM4Edu. Data privacy and security are crucial considerations to ensure the protection of students’ personal information [26]. The interpretability and fairness of LLM4Edu are also focal points [27], requiring the large models’ decision-making processes to be interpretable and avoiding unfair biases caused by data. Moreover, the development and deployment of educational large models need to fully consider educational practices and teachers’ professional knowledge to ensure the models are closely integrated with actual teaching [28].

This paper is a systematic summary and analysis of the research background, motivation, and applications of educational large models. By reviewing existing research, we provide an in-depth understanding of the potential and challenges of educational large models for education practitioners, researchers, and policymakers, offering guidance and insights for further advancing the development and application of EduLLMs. The main contributions of this article are as follows:

• This paper first reviews the background of education, LLMs, and smart education, respectively. It then introduces the connection between LLMs and education and also discusses mart education (Section II).

• This paper provides an in-depth understanding of the key technologies of EduLLMs, including natural language processing (NLP), machine learning, data mining, computer vision, etc. (Section III).

• We also discuss how LLMs empower education from the perspective of various applications of education under LLMs (Section IV-A). It further exhibits several distinct characteristics of education under LLMs (Section IV-B).

• We also summarize the key points in EduLLMs, including training data and preprocessing, the training process, and integration with various technologies (Section V).

• Finally, we highlight some key challenges existing in LLM4Edu (Section VI-A), and discuss potential future directions for LLM4Edu in more detail (Section VI-B).

II Education and LLMs

II-A Background of Education

Education is a conscious process of facilitating and guiding individual development [29]. It involves imparting knowledge, fostering skills, and shaping attitudes and values, with the aim of promoting holistic growth and self-realization in learners. The goal of education is to cultivate intellectual, emotional, moral, creative, and social adaptability in individuals, enabling them to make positive contributions to society.

Education takes various forms, including but not limited to:

• School education: Traditional school education is the most common and widely accepted form, where students receive organized instruction from teachers and acquire knowledge and skills.

• Online education: With the advancement of digital technologies, the internet and online platforms provide new forms of education [30]. Students can engage in learning through online courses, distance education, and other digital avenues.

• Community education: It refers to educational activities conducted within a community, providing specific training and learning opportunities to meet the educational needs of community members.

• Self-directed learning: Learning is the key to education. Self-directed learning emphasizes the ability of students to explore and learn autonomously, acquiring knowledge and skills through self-motivation and self-management.

In general, education involves various roles, including but not limited to:

• Teachers: Teachers have a core role in education. They are responsible for organizing, imparting knowledge, and guiding student learning and development.

• Students: Students are the recipients of education. They acquire knowledge and skills through learning and practice, aiming for personal development and growth.

• Parents: As an important supportive and guardianship role in education, they are concerned with their children’s learning and development, providing necessary resources and environments.

• Educational institutions: Schools, universities, training organizations, and other educational institutions provide educational resources and environments, organizing and managing educational activities.

• Government and society: They play roles in education policy-making, resource allocation, and social support, providing necessary support and safeguards for education.

II-B Background of LLMs

What is a large language model (LLM) [5, 22]? What are its characteristics? What is the relationship between large models and AI, data science, and other interdisciplinary fields? What are the key technologies employed in large models? A LLM possesses powerful language generation and understanding capabilities. Its objective is to train on massive amounts of language data to learn the statistical patterns and semantic relationships within the language, to generate coherent and accurate text, and to understand and respond to human queries [31]. Here are several characteristics of LLMs:

1. Natural language generation: LLMs can generate high-quality, coherent natural language text. They can understand the context and generate appropriate responses, articles, stories, and more based on input prompts or questions [32].

2. Semantic understanding: LLMs can comprehend the semantic relationships within human language, including vocabulary, syntax, and context [33]. They can parse and understand complex sentence structures, extract key information, and generate relevant responses.

3. Context awareness: LLMs can perform language understanding and generation based on context [34]. They can understand the history of a conversation and generate responses that are coherent and related to the context.

4. Wide range of applications: LLMs have extensive applications in natural language processing, virtual assistants [35], intelligent customer service [36], and intelligent writing [37], among others. They can provide language generation and understanding support for various tasks and scenarios.

5. Continuous learning: LLMs can continuously learn and update themselves by training on new data [38]. They can accumulate new language knowledge and patterns by learning from fresh data, improving their performance and capabilities.

Large models employ several key technologies. Here, we describe five of them in detail:

1. Transformer model: It serves as the foundational architecture for large models [39]. It utilizes self-attention mechanisms to handle the dependency relationships within input sequences [40]. It effectively captures long-range dependencies, enabling the model to better understand and generate text.

2. Pre-training and fine-tuning: Large models typically employ a two-stage approach of pre-training [41] and fine-tuning [42]. In the pre-training stage, the model undergoes self-supervised learning using a large-scale unlabeled corpus to learn the statistical patterns and semantic relationships of language. In the fine-tuning stage, the model is further trained and adjusted using labeled task-specific data to adapt to specific task requirements.

3. Large-scale datasets: Large models require massive language datasets for training [43]. These datasets often include text data from the internet, books, news articles, and more. The use of large-scale data provides abundant language inputs and enhances the model’s generalization ability.

4. High computational resources [44]: Large models necessitate significant computational resources for training and inference. High-performance graphics processing units (GPUs) or specialized deep learning accelerators, such as TPUs, are commonly used to accelerate computations and achieve efficient model training and inference.

5. Iterative optimization algorithms: Large models are typically trained using iterative optimization algorithms such as stochastic gradient descent (SGD) [45] and adaptive optimization algorithms like ADMA [46]. These algorithms update the model’s parameters through backpropagation, minimizing the loss function and optimizing the model’s performance.

In addition to the aforementioned key technologies, research on large models also involves aspects such as scaling up model size [47], data handling and selection [48], model compression and acceleration [49], and more. With advancing technology, the application of large models in natural language processing, intelligent dialogues, text generation, and other fields will become more extensive and mature.

II-C Smart Education

Smart education refers to an educational model that utilizes advanced information technology and the theories and methods of educational science to provide personalized, efficient, and innovative learning and teaching experiences. Its core idea is to leverage the advantages of information technology to offer intelligent and personalized learning environments and resources, thereby promoting students’ comprehensive development and enhancing learning outcomes.

Smart education is closely related to artificial intelligence (AI) and LLMs [50]. AI is the scientific and engineering field that aims to simulate and mimic human intelligence, while LLMs are a type of deep learning model with the capability to handle large-scale data and complex tasks. Through the applications of AI and LLMs, smart education can achieve more accurate learning analysis and assessment, personalized learning support and guidance, automated learning resource recommendations, and innovative teaching methods. However, smart education currently faces several issues and challenges:

• Shift in roles for teachers and students: Smart education involves transforming the roles of teachers and students from traditional transmitters and receivers of knowledge to collaborators and explorers [51]. This requires teachers to possess new teaching philosophies and skills to adapt to and guide students in the learning approaches and needs within a smart education environment.

• Data privacy and security [52, 53]: Smart education involves the collection and analysis of large amounts of student data to provide personalized learning support and assessment. However, this raises concerns about student privacy and data security [54]. It is crucial to establish robust data management and protection mechanisms to ensure the safety and lawful use of student data.

• Technological infrastructure and resources: Implementing smart education requires adequate technological infrastructure and resource support, including network connectivity, computing devices, educational software, etc. However, some regions and schools may face challenges regarding technological conditions and resource scarcity, limiting the widespread adoption and application of smart education.

• Ethical and moral issues: The application of smart education raises ethical and moral questions, such as data privacy, algorithm bias, and fairness in artificial intelligence. It is necessary to establish guidelines and regulations to ensure that the application of smart education not only yields educational benefits but also adheres to ethical principles and social fairness.

• Balancing personalization and social equity: Smart education aims for personalized learning support, but excessive reliance on personalization may widen the gaps between learners. It is essential to strike a balance between personalization and social equity, ensuring that the application of smart education does not exacerbate educational inequalities but instead provides equal learning opportunities for all learners.

In conclusion, smart education refers to an educational model that utilizes advanced information technology and the theories and methods of educational science to provide personalized, efficient, and innovative learning and teaching experiences. It is closely related to AI and large models. However, unlike mere technological applications, smart education also involves a range of issues and challenges, including the transformation of teacher roles, data privacy and security, technological infrastructure and resources, ethical and moral concerns, balancing personalization and social equity, and innovation in educational content and assessment systems. Addressing these issues and promoting the sustainable development of education requires collaborative efforts from the education sector, technology industry, and society as a whole.

II-D LLMs for Education

Large models have close relationships with artificial intelligence, data science, and other interdisciplinary fields. Large models are an important research direction within the field of artificial intelligence. They use deep learning and large-scale data training methods to simulate human language capabilities and achieve natural language processing tasks. In the field of data science, large models can be applied to tasks such as text mining, sentiment analysis, machine translation, and extracting valuable information from text data. Furthermore, large models involve computer science, machine learning, cognitive science, and other interdisciplinary fields. Through the study of language and intelligence, they drive the cross-fertilization and development between these disciplines.

In recent years, the emergence of LLMs, such as GPT-3, has sparked widespread attention and discussion. LLMs are AI technologies based on deep learning that possess powerful language generation and understanding capabilities. At the same time, the field of education faces many challenges and opportunities, such as personalized learning, educational resource inequality, and instructional effectiveness assessment. As a result, the education sector has begun to explore how to integrate LLMs with education to enhance teaching quality and effectiveness. Here are the significance and several ongoing practical areas, which can be depicted in Fig. 1:

Figure 1: Architecture of LLMs for education (LLM4Edu).

1. Personalized learning: Large models can provide personalized learning content and recommendations based on students’ learning needs and interests. By analyzing students’ learning data and behavioral patterns [55, 56], large models can design unique learning paths and resources for each student [57], helping them learn and grow more efficiently.

2. Instructional support tools: LLMs can serve as assistants to teachers, providing intelligent instructional support tools and platforms [58]. Teachers can utilize the generated content and recommendations from LLMs to design teaching activities, monitor students’ learning progress, and provide personalized teaching support.

3. Educational assessment and feedback: LLMs can analyze students’ assignments, exams, and other learning data to provide assessment and feedback on their learning progress. By automatically generating comments and suggestions, LLMs can help teachers gain a more accurate understanding of students’ learning achievements and challenges, and provide corresponding guidance and support.

4. Educational resource and content creation: LLMs can be used for the creation and generation of educational resources and content. They can generate teaching materials, exercises, case studies, and more based on instructional goals and needs, providing teachers with a rich array of resources and inspiration.

III Key Technologies for EduLLMs

Educational LLMs involve several key technologies. Here are 10 key technologies related to educational large language models (EduLLMs), along with detailed descriptions for each:

1. Natural language processing (NLP): NLP is one of the core technologies behind EduLLMs. It encompasses techniques such as text analysis, semantic understanding, and sentiment analysis, enabling the models to comprehend and process human language [59]. NLP enables EduLLMs to understand student queries, generate language responses, and extract important information from text.

2. Deep learning (DL): DL is a branch of machine learning [60] that involves constructing and training deep neural network models for learning and inference [61]. EduLLMs often rely on deep learning architectures such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to process and analyze educational data and generate meaningful outputs. Many DL techniques have been developed.

3. Reinforcement learning (RL) [62]: RL trains an agent to make decisions through trial and error and reward mechanisms. In EduLLMs, reinforcement learning can be employed to optimize model responses and recommendations, allowing the models to adjust based on student feedback and outcomes to provide more accurate and effective learning support [63].

4. Data mining (DM) [64, 65]: DM is the process of extracting useful information and patterns from large datasets. EduLLMs can utilize data mining techniques to discover student learning patterns, behavior trends, and knowledge gaps, providing the foundation for personalized learning and offering insights for educational research.

5. Computer vision (CV): The powerful CV technologies enable computers to understand and interpret images and videos. In education, EduLLMs can employ computer vision techniques to analyze students’ facial expressions, postures, and behaviors, providing more accurate emotion analysis and learning feedback [66].

6. Speech recognition and synthesis: Speech recognition technology converts speech into text, while speech synthesis technology converts text into speech. EduLLMs can utilize these technologies to engage in speech interactions with students, offering support for oral practice, speech assessment, and pronunciation correction [67].

7. Multimodal learning [68]: It involves the fusion of various sensors and data sources, such as text, images, audio, and video. EduLLMs can process and analyze multimodal data to gain a more comprehensive understanding of students’ learning situations and needs [69].

8. Personalized recommendation systems: They utilize ML and DM techniques to provide students with personalized learning resources and suggestions based on their interests, learning history, and learning styles [70]. EduLLMs can play a significant role in personalized recommendation systems, leveraging student data and behavior patterns to recommend suitable learning materials, courses, and activities.

Therefore, the combination of these key technologies enables EduLLMs to offer personalized, adaptive, and targeted educational support. The applications foster innovation in education, improving learning outcomes and teaching quality. However, these applications of EduLLMs also face challenges, such as privacy protection, data bias, and algorithm transparency. These need to be appropriately addressed in technological development and practical implementation.

IV LLM-empowered Education

IV-A Applications of Education under LLMs

Possible applications of LLMs for education can be found in various educational scenarios, providing personalized learning, teaching assistance, and educational research support. Here are 12 potential application scenarios of LLM4Edu, along with specific descriptions and examples, as shown in Table I:

TABLE I: Several applications of LLM4Edu

Function	Description
Learning assistance tools	Provide support in problem-solving, generating study materials, and organizing knowledge.
Personalized learning experience	Recommend related learning materials.
Content creation and generation	Generate teaching outlines, practice questions, and lesson plans.
Language learning and teaching	Provide grammar and vocabulary exercises and enhance their language communication abilities.
Cross-language communication and translation	Provide real-time translation services.
Educational research and data analysis	Offer employment prospects, career development paths, and advice on relevant skill development.
Virtual experiments and simulations	Provide virtual experiment and simulation environments.
Career planning and guidance	Offer employment prospects, career development paths, and advice.
Exam preparation and test-taking support	Offer practice questions, explanations, and strategies.
Academic writing assistance	Provide guidance on structuring essays, citing sources, refining arguments, and enhancing overall clarity and coherence.
Interactive learning experiences	Create interactive and immersive learning experiences.
Lifelong learning and continuing education	Enable them to acquire new skills, explore new fields, and pursue personal development.

1. Learning assistance tools: EduLLMs can serve as learning assistance tools, providing support to students in problem-solving, generating study materials, and organizing knowledge. For example, students can ask the model for solution methods to mathematical problems, and the model can generate detailed explanations and step-by-step processes to help students understand and master the concepts.

2. Personalized learning experience: EduLLMs can offer personalized learning content and suggestions based on students’ learning needs and interests. For instance, the model can recommend related reading materials, practice questions, and learning resources based on students’ learning histories and interests, catering to their individualized requirements.

3. Content creation and generation: EduLLMs can assist educators and content creators in generating educational materials and resources. For example, the models can automatically generate teaching outlines, practice questions, and lesson plans, providing educators with diverse and enriched teaching resources.

4. Language learning and teaching: LLM-empowered education has potential applications in language learning and teaching. For instance, the models can provide grammar and vocabulary exercises to help students improve their language skills. The models can also generate dialogue scenarios for students to practice real-life conversations, enhancing their language communication abilities.

5. Cross-language communication and translation: LLMs can assist in cross-language communication and translation in smart education. For instance, the large models can provide real-time translation services, helping students and educators overcome language barriers and facilitating cross-cultural communication and collaboration.

6. Educational research and data analysis: EduLLMs can analyze extensive educational data (aka educational data mining) [71] and provide deep insights and research support. For example, the models can assist researchers in analyzing student’s learning behaviors and performances, discovering effective teaching methods and strategies, and providing evidence for educational policy-making.

7. Virtual experiments and simulations: EduLLMs can provide virtual experiment and simulation environments, allowing students to engage in practical experiences. For example, the models can offer virtual chemistry laboratories, enabling students to conduct chemical experiments in safe and controlled environments, honing their practical skills and scientific thinking.

8. Career planning and guidance: EduLLMs provide career planning and guidance to students. For instance, the models can offer employment prospects, career development paths, and advice on relevant skill development based on student’s interests, skills, and market demands, assisting students in making informed career planning decisions.

9. Exam preparation and test-taking support: EduLLMs can assist students in preparing for exams and improve their test-taking skills. They can offer practice questions, explanations, and strategies for different types of exams, helping students familiarize themselves with the format, content, and techniques required for successful performance.

10. Academic writing assistance: LLMs can aid students in improving their academic writing skills. They can provide guidance on structuring essays, citing sources, refining arguments, and enhancing overall clarity and coherence. These models can also assist students in developing critical thinking and analytical skills necessary for academic success.

11. Interactive learning experiences: EduLLMs will create interactive and immersive learning experiences. For example, they can simulate historical events, scientific experiments, or virtual field trips, allowing students to engage actively and learn through realistic scenarios. These interactive experiences can enhance student engagement and deepen their understanding of complex concepts.

12. Lifelong learning and continuing education: Educational LLMs can support lifelong learning [72] and continuing education initiatives. They can provide resources, courses, and learning opportunities for individuals outside traditional educational settings, enabling them to acquire new skills, explore new fields, and pursue personal or professional development at any stage of life.

The versatility of educational LLMs allows for their application across a wide range of educational contexts, from K-12 classrooms to higher education institutions, vocational training, and beyond. By leveraging the capabilities of these models, educational stakeholders can enhance the quality, accessibility, and effectiveness of teaching and learning experiences. In summary, the applications of EduLLMs encompass learning assistance tools, personalized learning experiences, content creation and generation, language learning and teaching, student assignment evaluation, cross-language communication and translation, educational research and data analysis, virtual experiments and simulations, learning content recommendations, and career planning and guidance. These scenarios demonstrate the potential of EduLLMs to provide personalized, efficient, and innovative educational services. However, it is crucial to balance technological advancements with ethical considerations in the application of EduLLMs, ensuring that their usage aligns with educational goals and values while prioritizing individual privacy and data security.

IV-B Characteristics of Education under LLMs

Education under large language models (LLMs) exhibits several distinct characteristics, as shown in Fig. 2:

Figure 2: The characteristics of education under LLMs.

1. Personalized learning: LLMs have the ability to process and analyze vast amounts of data, allowing for personalized learning experiences. They can adapt instructional content, pacing, and assessments to match the unique needs and preferences of individual learners. This personalization enhances the effectiveness and engagement of the learning process.

2. Adaptive feedback: LLMs can provide immediate and adaptive feedback to learners. They can identify areas of weakness or misconceptions and offer tailored explanations and guidance. This real-time feedback helps learners to understand concepts more effectively and make progress at their own pace.

3. Access to diverse resources: For smart education, LLMs have access to a vast amount of information and knowledge. They can provide learners with a wide range of resources, including texts, images, videos, and interactive materials. This access to diverse resources enhances the depth and breadth of learning, enabling learners to explore various perspectives and engage with rich content.

4. Natural language interaction: LLMs are proficient in understanding and generating human language. Learners can engage in natural language conversations with LLMs, asking questions, seeking clarifications, and discussing ideas. This natural language interaction promotes a more conversational and interactive learning experience.

5. Continuous learning support: LLMs can provide continuous learning support beyond traditional classroom hours. Learners can access educational materials, review lessons, and seek assistance from LLMs at any time. Note that this flexibility in learning support accommodates different schedules and learning preferences.

6. Content generation and creation: LLMs can assist in generating educational content. They can automate the creation of quizzes, exercises, and learning materials based on specific learning objectives. This content generation capability reduces the burden on educators and allows for the creation of diverse and customized learning resources.

7. Multilingual capabilities: LLMs are capable of processing and generating content in multiple languages [73]. This enables learners from different linguistic backgrounds to access educational materials in their native languages, promoting inclusivity and accessibility.

8. Analyzing learning data: Educational LLMs can analyze learning data and provide insights into learners’ progress, strengths, and areas for improvement. Educators can utilize these analytics to gain a deeper understanding of learners’ learning patterns, adjust instructional strategies, and provide targeted interventions.

9. Ethical considerations: Education under LLMs raises ethical considerations. It is essential to ensure transparency, accountability, and privacy in the use of learner data. Clear guidelines and safeguards should be in place to protect learners’ privacy and prevent potential biases or misuse of data.

10. Collaboration between humans and LLMs: LLMs are tools that can enhance and augment human teaching and learning [74]. They are not meant to replace human educators but rather to collaborate with them. Educators can leverage LLMs to provide personalized support, curate content, and facilitate meaningful learning experiences.

V Key Points in LLMsEdu

V-A Training Data and Preprocessing

Preprocessing steps applied to the data before training may include tokenization, normalization, and other data cleaning techniques. Tokenization involves breaking the text into smaller units, such as words or subwords, to facilitate processing. Normalization may include converting text to lowercase to ensure uniformity and remove case-specific variations. Other cleaning techniques may involve removing irrelevant HTML tags, special characters, or noisy data to enhance the quality of the training data. For educational purposes, when training models to understand and generate text in an educational context, it is crucial to curate datasets that include diverse educational content. This can range from textbooks and scholarly articles to educational websites and forums. These preprocessing steps should be tailored to preserve the educational context, ensuring that the model learns to generate coherent and contextually relevant educational content.

V-B Training Process

Pre-training and fine-tuning play a key role in the construction of educational LLMs. First, in the pre-training stage, the model is initialized through large general text data to achieve the learning of general language features such as syntax, semantics, and logical relationships. This provides the model with broad language understanding capabilities, allowing it to understand and process a variety of language tasks. Next, in the fine-tuning phase, fine-tuning is performed by collecting domain-specific data according to specific task requirements in the education field. This ensures that the model can better adapt to the tasks and show superior performance in the education field. During the fine-tuning process, pre-trained model weights are used for initialization, which provides a strong foundation for the model to learn specific tasks. Adjust model parameters through supervised learning to adapt them to the specific requirements of the task, and ensure that the model reaches a satisfactory level on educational tasks through performance evaluation. Hyperparameter tuning further optimizes model performance, such as by adjusting the learning rate and batch size. Ultimately, by saving the fine-tuned model, it becomes a powerful tool that can be deployed and applied to specific educational tasks. Therefore, the entire training process enables the model to achieve excellent results in a wide range of language understanding and specific educational tasks, providing a powerful language processing tool for smart education.

V-C Integration with Educational Technologies

Finally, they can be seamlessly integrated into various practical applications within educational technology to enhance the overall learning experience. LLMs can power chatbots, providing personalized support by addressing queries related to course content, assignments, or general information, with the added advantage of 24/7 availability. LLMs can be incorporated into intelligent tutoring systems, delivering personalized learning experiences by offering customized guidance and recommendations to students. They can also automate the generation of educational content, including quizzes, tests, and study materials, thereby saving educators valuable time. Moreover, LLMs have applications on language learning platforms, facilitating conversational practice through realistic dialogue simulations and offering real-time feedback on grammar usage. These technologies can extend to virtual labs and simulations, enhancing students’ practical learning experiences through natural language interactions. Overall, the application of LLMs in educational technology necessitates considerations for ethical issues, data privacy, and potential biases in the models. Continuous user feedback and improvement are crucial for optimizing learning outcomes.

VI Challenges and Future Directions

VI-A Challenges and Issues

The application of LLMs for education brings forth numerous potential challenges and issues. Here are 10 possible challenges related to LLM4Edu, along with detailed descriptions:

1. Privacy protection [53, 75]: In general, EduLLMs deal with a vast amount of student data, including personal information, learning records, and behavioral data. This raises concerns regarding privacy protection. Ensuring the security and privacy of student data becomes a significant challenge, necessitating rigorous data security measures and privacy policies to safeguard student rights.

2. Data bias: The data used during the training process of EduLLMs may contain biases, which can result in biased outputs from the models [76]. For instance, if there are biases in the training data concerning gender or race, the models may reflect these biases and have unfair effects on students. Eliminating data bias is an important challenge to ensure the fairness and reliability of the models.

3. Algorithm transparency: EduLLMs often consist of complex neural network models, and their decision-making processes can be difficult to interpret and understand. Algorithm transparency refers to the extent to which the model’s decision-making process can be explained and understood [77]. In education, students and teachers need to understand how the models make recommendations and evaluations to trust and utilize them.

4. Technical feasibility: Educational LLMs typically require substantial computational resources and storage space for training and inference. In certain educational environments, especially in resource-constrained schools or regions, these requirements may not be met. Hence, ensuring the technical feasibility of EduLLMs to operate reliably in various educational settings is a critical challenge.

5. Human interaction and emotion: Education involves rich human interactions and emotional experiences. EduLLMs still face challenges in simulating human teacher-student interactions. For example, in terms of emotion analysis, models may struggle to accurately understand students’ emotional states and provide appropriate support [78]. Addressing these challenges, especially in the Metaverse [79, 80], requires further research and technological innovation.

6. Accessibility: The application of EduLLMs should have broad accessibility to meet the needs of diverse learners. This includes support for students with disabilities, such as assistive features for visually and hearing-impaired students. Ensuring that accessibility needs are considered in the design and implementation of EduLLMs is a significant challenge.

7. Credibility and quality assessment: Ensuring the credibility and quality assessment of EduLLMs is crucial. Students and teachers need to have confidence that the recommendations and feedback provided by the models are accurate and reliable [81]. Therefore, conducting credibility and quality assessments of EduLLMs is an important challenge. This involves establishing evaluation criteria and metrics to validate the model’s performance and effectiveness while ensuring its reliability in educational practice.

8. Teacher roles and professional development: The use of EduLLMs may impact teacher roles and professional development. Firstly, EduLLMs can provide instructional assistance and personalized learning support, alleviating teachers’ workload. Secondly, teachers need to adapt to and master the technologies and tools related to EduLLMs to collaborate and work effectively with them. This presents new requirements and challenges for teacher professional development.

VI-B Future Directions

Here are some possible research directions for EduLLMs in the future, along with detailed descriptions:

1. Model interpretability: Educational LLMs often consist of complex neural network structures, and their decision-making processes can be difficult to interpret and understand. To establish the credibility and acceptability of EduLLMs, further research is challenging on how to explain the model’s decision-making process, enabling teachers, students, and other stakeholders to comprehend and trust the model’s recommendations and evaluations.

2. Personalized learning support: One major application of EduLLMs is to provide personalized learning support. Future research can explore how to better utilize models to understand students’ learning needs, interests, and learning styles, in order to offer more accurate and personalized learning suggestions and resources.

3. Emotional intelligence: Education involves emotional factors such as students’ emotional states and experiences. Future research can focus on integrating emotional intelligence into EduLLMs, enabling the models to accurately recognize and understand students’ emotional states and provide appropriate emotional support and guidance when needed.

4. Evaluation and assessment: Evaluating the effectiveness and impact of EduLLMs is important. Future research can focus on establishing effective evaluation methods and metrics to assess the influence of EduLLMs on students’ learning outcomes, learning processes, and learning experiences.

5. Social equity: The application of EduLLMs in providing personalized learning may raise issues of social equity. Future research can explore how to address these issues through the design and implementation of models, ensuring that their applications do not exacerbate educational inequalities but instead promote a fair and inclusive learning environment.

6. Educational ethics: The application of EduLLMs raises ethical issues such as privacy protection, data usage, and the model’s moral responsibility. Future research can focus on establishing appropriate ethical guidelines and frameworks to guide the development, use, and evaluation of EduLLMs.

7. Cross-cultural adaptability: The research and application of EduLLMs need to consider the needs and differences of learners from different cultures and backgrounds. Future research can focus on making EduLLMs cross-culturally adaptable to better meet the needs of learners worldwide.

8. Long-term learning and development: Research on EduLLMs should not only focus on short-term effects during the learning process but also consider students’ long-term learning and development. Future research can explore how EduLLMs can support students’ long-term learning goals, facilitate continuous growth, and promote lifelong learning.

VII Conclusion

The application of LLMs in the field of education has broad prospects. This review provides a systematic summary and analysis of the research background, motivation, and application of educational large models. It first introduces the research background and motivation of LLMs and explains the essence of large models. It then discusses the relationship between intelligent education and educational LLMs, and summarizes the current research status of educational LLMs. Finally, by reviewing existing research, this article provides guidance and insights for educators, researchers, and policy-makers to gain a deep understanding of the potential opportunities and challenges of educational LLMs, and provides guidance for further advancing the development and application of educational LLMs. However, the development and applications of educational LLMs still face technical, ethical, and practical challenges, requiring further research and exploration.

With the advancement of technology and the evolution of educational needs, educational large models will play an increasingly important role in providing more efficient and personalized support and services for education. We believe that AI-driven education is one of the most innovative and forward-looking directions in the field of education today. It can be foreseen that in the future, with the continuous development and improvement of artificial intelligence, the future of smart education will be more digitalized and humanized, as well as more diverse and personalized.