The impact of training on digital citizenship skills in developing students’ attitudes towards sustainable development at the university level

Main Article Content

Safar Bakheet Almudara
Mohamed Mahmoud El-Gammal
Maiada Hasan Ali
Mohamed Sayed Abdellatif
Adel Ibrahim Abdullah Elshazly
Sherif Ahmed Ibrahim
Anwar Hammad Al-Rashidi

Abstract

The research aimed to investigate the impact of training on digital citizenship skills in developing attitudes toward sustainable development among university students. The study involved 25 first-year students from Prince Sattam bin Abdulaziz University, including 13 Medical Sciences College students in the experimental group and 12 Engineering College students in the control group. The researcher administered the following tools: the training program and the Sustainable Development Attitude Scale. A quasi-experimental design was employed, and data were analyzed statistically using the Mann-Whitney and Wilcoxon tests. The research results revealed statistically significant differences between the mean ranks of the scores of individuals in the experimental and control groups on the Sustainable Development Attitude Scale after the implementation of the program in favor of the experimental group. Additionally, statistically significant differences were found between the mean ranks of the scores of individuals in the experimental group on the Sustainable Development Attitude Scale in the pre-test and post-test measures in favor of the post-test. However, no statistically significant differences were found between the mean ranks of the scores of the experimental group on the Sustainable Development Attitude Scale in the post-test and follow-up measures (after one month). The researcher provided a comprehensive discussion of the study variables: attitudes towards sustainable development and its components, as well as the training program based on digital citizenship skills in the current study. Furthermore, the researcher presented some recommendations and proposed further research.

Downloads

Download data is not yet available.

Article Details

How to Cite
Almudara, S. B. ., El-Gammal, M. M. ., Ali, M. H. ., Abdellatif, M. S., Elshazly, A. I. A. ., Ibrahim, S. A. ., & Al-Rashidi, A. H. . (2024). The impact of training on digital citizenship skills in developing students’ attitudes towards sustainable development at the university level. Research Journal in Advanced Humanities, 5(3). https://doi.org/10.58256/8jrvr939
Section
Articles
Author Biographies

Safar Bakheet Almudara, Prince Sattam Bin Abdulaziz University

Department of Educational Sciences, College of Education 

Mohamed Mahmoud El-Gammal , Hamad Bin Khalifa University

Department of Islamic Studies, College of Islamic Studies

Maiada Hasan Ali, Al-Iraqia University

Department of English, College of Arts

Mohamed Sayed Abdellatif, Prince Sattam Bin Abdulaziz University,

Department of Psychology, College of Education

Adel Ibrahim Abdullah Elshazly, King Faisal University in Al-Ahsa

Department of Curriculum and Teaching Methods, College of Education

Sherif Ahmed Ibrahim, University of Bisha

Department of Educational Technology, College of Education

Anwar Hammad Al-Rashidi, Prince Sattam Bin Abdulaziz University

Department of Psychology, College of Education

How to Cite

Almudara, S. B. ., El-Gammal, M. M. ., Ali, M. H. ., Abdellatif, M. S., Elshazly, A. I. A. ., Ibrahim, S. A. ., & Al-Rashidi, A. H. . (2024). The impact of training on digital citizenship skills in developing students’ attitudes towards sustainable development at the university level. Research Journal in Advanced Humanities, 5(3). https://doi.org/10.58256/8jrvr939

References

Abbasi, K. R., Hussain, K., Haddad, A. M., Salman, A., & Ozturk, I. (2022). The role of financial development and technological innovation towards sustainable development in Pakistan: fresh insights from consumption and territory-based emissions. Technological Forecasting and Social Change, 176, 121444.‏ DOI: 10.1016/j.techfore.2021.121444

Abu Almajd, M. A., & Al-Youssef, I. Y. (2018). Social media networks and their employment methods in enhancing digital citizenship dimensions among students of the College of Education at King Faisal University. Al-Majallah Al-Tarbiyah, 56, 691-722. https://doi.org/10.21608/edusohag.2018.23934

Al-Abdullatif, A., & Gameil, A. (2020). Exploring students’ knowledge and practice of digital citizenship in higher education. International Journal of Emerging Technologies in Learning (iJET), 15(19), 122-142. https://doi.org/10.3991/ijet.v15i19.15611

Alamri, H., & A Alqahtani, A. (2022). Digital Citizenship among Saudi Community Colleges’ Undergraduates: Reality and Requirements for Enablement. European Online Journal of Natural and Social Sciences, 11(4), 1144-1165. www.european-science.com

Al-Dhuhli, R., Alkindi, L, & Al-Taani, W. (2022). Digital Technology and its Role in Achieving Sustainable Development Requirements in Basic Education Schools in the Sultanate of Oman. Jordan Journal of Applied Science Humanities Science Series (JJOAS), 31(1), 1-15. https://digitalcommons.aaru.edu.jo/jjoas-h/vol31/iss1/4

Al-Hadif, N. B. M. (2021). The level of availability of digital citizenship skills among postgraduate students at the College of Education, Qassim University: Their perspectives and its relationship to some variables. Umm Al-Qura University Journal of Educational and Psychological Sciences, 13(4), 189-219.

Al-Naqbi, A. K., & Alshanna, Q. (2018). The status of education for sustainable development and sustainability knowledge, attitudes, and behaviors of UAE University students. International Journal of Sustainability in Higher Education, 19(3), 566-588. https://doi.org/10.1108/IJSHE-06-2017-0091

Al-Qarni, Z. (2021). The role of Saudi universities in promoting digital citizenship values: An analytical study of the websites of Saudi universities. Journal of King Abdulaziz University, Arts and Humanities, 29(2), 247–290. http://dx.doi.org/10.4197/Art.29-2.10

Al-Samadi, H. S. I. (2017). Students' perceptions of digital citizenship: A field study on a sample of students at Qassim University. Journal of Psychological and Educational Studies, 1(18), 175-184. DOI: 10.35156/1192-000-018-013

Al-Waly, M. F. R. (2023). The Role of Technology in Achieving Sustainable Development in the Light of Egypt's Vision 2030. International Journal of Contemporary Educational and Human Sciences, 2(1), 1–21.

AlZebidi, A. A., & Alsuhaymi, D. S. (2021). Investigating Students' Digital Citizenship Practices for Undergraduate Students at Al-Qunfudhah University College. Ilkogretim Online, 20(5), 2089-2104. doi: 10.17051/ilkonline.2021.05.230

Ambusaidi, A., & Al Washahi, M. (2016). Prospective teachers’ perceptions about the concept of sustainable development and related issues in Oman. Journal of Education for Sustainable Development, 10(1), 3-19. https://doi.org/10.1177/0973408215625528

Andersson, K. (2017). Starting the pluralistic tradition of teaching? Effects of education for sustainable development (ESD) on pre-service teachers’ views on teaching about sustainable development. Environmental Education Research, 23(3), 436-449. https://doi.org/10.1080/13504622.2016.1174982

Aseeri, M. A. I. (2023). The role of social media networks in developing digital citizenship skills among postgraduate students at Jazan University. Master's thesis. Jazan University, Jazan.

Bal, E., & Akcil, U. (2024). The Implementation of a Sustainable Online Course for the Development of Digital Citizenship Skills in Higher Education. Sustainability, 16(1), 445. https://doi.org/10.3390/su16010445

Biasutti, M., & Frate, S. (2017). A validity and reliability study of the attitudes toward sustainable development scale. Environmental Education Research, 23(2), 214-230. https://doi.org/10.1080/13504622.2016.1146660

Bocar, A., & Ancheta, R. (2023). Exploring students’ digital citizenship: Its importance, benefits, and drawbacks. Journal of Business, Communication & Technology, 2(2),28-33. http://doi.org/10.56632/bct.2023.2203

Boonlab, S., & Pasitpakakul, P. (2023). Developing a Teaching and Learning Model to Foster Digital Citizenship in General Education Undergraduate Courses. Journal of Social Studies Education Research, 14(3), 287-304.

Brynjolfsson, E., & McAfee, A. (2014). The second machine age: Work, progress, and prosperity in a time of brilliant technologies. WW Norton & Company.

Choi, M. (2016). A concept analysis of digital citizenship for democratic citizenship education in the internet age. Theory & research in social education, 44(4), 565-607.

Clayton, T., & Radcliffe, N. (2018). Sustainability: a systems approach. Routledge.

Couros, A, & Hildebrandt, K. (2015). Digital Citizenship Education in Saskatchewan Schools: A policy planning Guide for School Divisions and schools to implement Digital Citizenship Education from kindergarten to Grade 12. Saskatchewan Ministry of Education.

Del Río Castro, G., González Fernández, M. C., & Uruburu Colsa, Á. (2021). Unleashing the convergence amid digitalization and sustainability towards pursuing the Sustainable Development Goals (SDGs): A holistic review. Journal of Cleaner Production, 280(1), 1-15. https://doi.org/10.1016/j.jclepro.2020.122204

ESCAP, U. (2022). Frontier ICTs for sustainable development for digital leaders: submodule C: Internet of Things. United Nations ESCAP.

Ferguson, A. M., McLean, D., & Risko, E. F. (2015). Answers at your fingertips: Access to the Internet influences willingness to answer questions. Consciousness and Cognition: An International Journal, 37, 91–102. https://doi.org/10.1016/j.concog.2015.08.008

Gatti, L., Ulrich, M., & Seele, P. (2019). Education for sustainable development through business simulation games: An exploratory study of sustainability gamification and its effects on students' learning outcomes. Journal of Cleaner Production, 207, 667-678. https://doi.org/10.1016/j.jclepro.2018.09.130

Ghosn-Chelala, M. (2019). Exploring sustainable learning and practice of digital citizenship: Education and place-based challenges. Education, Citizenship and Social Justice, 14(1), 40-56. https://doi.org/10.1177/1746197918759155

Giddour, H. B. (2016). The impact of the digital revolution and the extensive use of social networks in drawing the new image of the concept of citizenship: from the ordinary citizen to the digital citizen. The Books of Law, 15(5), 720-735.

Goralski, M. A., & Tan, T. K. (2020). Artificial intelligence and sustainable development. The International Journal of Management Education, 18(1), 100330.

Greesh, D. S. (2023). The effectiveness of a training program based on video modeling for developing some sustainable development skills among gifted children with autism spectrum disorder "Savant Syndrome. Journal of the College of Education, 39(2), 66-110. https://dx.doi.org/10.21608/mfes.2023.294800

Gu, M. M., Huang, C. F., & Lee, C. K. J. (2023). Investigating university students’ digital citizenship development through the lens of digital literacy practice: A Translingual and transemiotizing perspective. Linguistics and Education, 77, 101226. https://doi.org/10.1016/j.linged.2023.101226

Guandalini, I. (2022). Sustainability through digital transformation: A systematic literature review for research guidance. Journal of Business Research, 148, 456-471. http://dx.doi.org/10.1016/j.jbusres.2022.05.003

Hollandsworth, R., Dowdy, L., & Donovan, J. (2011). Digital citizenship in K-12: It takes a village. TechTrends: Linking Research and Practice to Improve Learning, 55 (4), 37-47. https://eric.ed.gov/?id=EJ926721

Hollandsworth, R., Dowdy, L., & Donovan, J. (2011). Digital Citizenship in K-12: It Takes a Village. Techtrends. 55. 37-47. http://dx.doi.org/10.1007/s11528-011-0510-z

Hui, B. (2013, May). Teaching digital citizenship by fostering digital citizens. In Proceedings of the 18th Western Canadian Conference on Computing Education (WCCCE), Vancouver, BC, Canada (pp. 3-4).

Hussainy, S. S., & Jamalullah, S. R. (2021). A study on factors affecting digital citizenship among college faculties in India. International Journal of Teaching, Education and Learning, 4(3), 49-61. http://dx.doi.org/10.20319/pijtel.2021.43.4961

Illahaqi, A. A. D., Nurcahyo, H., & Panjaitan, M. H. (2021, March). Advancing students’ environmental sustainability awareness through science mobile learning: A literature review. In 6th International Seminar on Science Education (ISSE 2020) (pp. 795-800). Atlantis Press.

Jones, L. M., & Mitchell, K. J. (2016). Defining and measuring youth digital citizenship. New media & society, 18(9), 2063-2079. https://doi.org/10.1177/1461444815577797

Jovanović, M., Dlačić, J., & Okanović, M. (2018). Digitalization and society’s sustainable development–Measures and implications. Zbornik radova Ekonomskog fakulteta u Rijeci: časopis za ekonomsku teoriju i praksu, 36(2), 905-928. http://dx.doi.org/10.18045/zbefri.2018.2.905

Joyce, A. (2018). How can Information and Communication Technologies support education for sustainable Development? A Critique. In EDULEARN18 Proceedings (pp. 9245-9254). IATED. http://dx.doi.org/10.21125/edulearn.2018.2170

Judge, J. P., Morgan, K. N., & Clifton, A. (2022). Video Modeling to Support Physical Activity in Children with Autism Spectrum Disorder: A Systematic Review. Journal of the American Academy of Special Education Professionals, 34, 48. https://eric.ed.gov/?id=EJ1353982

Kalsoom, Q., & Khanam, A. (2017). Inquiry into sustainability issues by preservice teachers: A pedagogy to enhance sustainability consciousness. Journal of cleaner production, 164, 1301-1311. https://doi.org/10.1016/j.jclepro.2017.07.047

Kara, N. (2018). Understanding university students’ thoughts and practices about digital citizenship: A mixed methods study. Journal of Educational Technology & Society, 21(1), 172-185.

Karaduman, H., & Ozturk, C. (2014). The effects of activities for digital citizenship on students’ attitudes toward digital citizenship and their reflections on students’ understanding about digital citizenship. Journal of Social Studies Education Research, 5(1), 38-78.

Langley, D. J. (2022). Digital product-service systems: The role of data in the transition to servitization business models. Sustainability, 14(3), 1303. https://doi.org/10.3390/su14031303

Leca, A. (2022). What is digital citizenship and why is it important? https://usidhr.org/what-is-digital-citizenship-and-why-is-it-important/#:~:text=Digital%20Citizenship%20Education%20 encourages%20young,more%20aware%20of%20internet%20safety

Literacy with ICT. (2009): Digital Citizenship. Retrieved January .10, 2009, from: http://lwictdigitalcitizenship.wikispaces.com

Logan, A. G. (2016). Digital citizenship in 21st century Education. Master Thesis, Dominican University of California.

Lozano-Díaz, A., & Fernández-Prados, J. S. (2020). Educating digital citizens: An opportunity to critical and activist perspective of sustainable development goals. Sustainability, 12(18), 7260. https://doi.org/10.3390/su12187260

Lu, H., Fu, K., Liu, X., & Hu, W. (2023). Digital citizen participation of college students: Reality and optimization path. Problems of Education in the 21st Century, 81(2), 244-257. https://doi.org/10.33225/pec/23.81.244

Lyons, R. (2012). Investigating student gender and grade level differences in digital citizenship behavior (Doctoral dissertation, Walden University).

Mahadir, N. B., Baharudin, N. H., & Ibrahim, N. N. (2021). Digital Citizenship Skills among Undergraduate Students in Malaysia: A Preliminary Study. International Journal of Evaluation and Research in Education, 10(3), 835-844. http://dx.doi.org/10.11591/ijere.v10i3.21277

Martin, F., Gezer, T., & Wang, C. (2019). Educators’ perceptions of student digital citizenship practices. Computers in the Schools, 36(4), 238-254. https://doi.org/10.1080/07380569.2019.1674621

Michalos, A. C., Creech, H., McDonald, C., & Kahlke, P. M. H. (2011). Knowledge, attitudes and behaviours. Concerning education for sustainable development: Two exploratory studies. Social indicators research, 100, 391-413. https://doi.org/10.1007/s11205-010-9620-9

Mills, K. L. (2016). Possible effects of Internet use on cognitive development in adolescence. Media and Communication, 4(3), 4-12. https://doi.org/10.17645/mac.v4i3.516

Mohammed, M. A. H. (2022). A proposed program in light of the dimensions of sustainable development and green economy and its impact on developing sustainable thinking, cognitive balance, and sustainable attitudes among science stream students at the College of Education. Journal of the College of Education, 38(3), 106–170. https://doi.org/10.21608/mfes.2022.228315

Mora, H., Pujol-López, F. A., Mendoza-Tello, J. C., & Morales-Morales, M. R. (2020). An education-based approach for enabling the sustainable development gear. Computers in Human Behavior, 107, 105775, 1-7. https://doi.org/10.1016/j.chb.2018.11.004

Nambisan, S., Wright, M., & Feldman, M. (2019). The digital transformation of innovation and entrepreneurship: Progress, challenges and key themes. Research policy, 48(8), 103773. https://doi.org/10.1016/j.respol.2019.03.018

Nousheen, A., Zai, S. A. Y., Waseem, M., & Khan, S. A. (2020). Education for sustainable development (ESD): Effects of sustainability education on pre-service teachers’ attitude towards sustainable development (SD). Journal of Cleaner Production, 250, 119537. https://doi.org/10.1016/j.jclepro.2019.119537

Ohler, J. B. (2010). Digital community, digital citizen. Corwin Press.

Öztürk, G. (2021). Digital citizenship and its teaching: A literature review. Journal of Educational Technology and Online Learning, 4(1), 31-45. http://dx.doi.org/10.31681/jetol.857904

Paiola, M., Schiavone, F., Grandinetti, R., & Chen, J. (2021). Digital servitization and sustainability through networking: Some evidences from IoT-based business models. Journal of Business Research, 132, 507-516. https://doi.org/10.1016/j.jbusres.2021.04.047

Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using IBM SPSS. Routledge. ‏

Park, Y. (2016). An Innovative Digital Citizenship Initiative in Singapore and Korea. In SITE-Society for Information Technology & Teacher Education International Conference. Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/173241/

Pigola, A., da Costa, P. R., Carvalho, L. C., Silva, L. F. D., Kniess, C. T., & Maccari, E. A. (2021). Artificial intelligence-driven digital technologies to the implementation of the sustainable development goals: A perspective from Brazil and Portugal. Sustainability, 13(24), 13669. https://doi.org/10.3390/su132413669

Prasetiyo, W. H., Sumardjoko, B., Muhibbin, A., Naidu, N. B. M., & Achmad, M. İ. (2023). Promoting digital citizenship among student-teachers: The role of project-based learning in improving appropriate online behaviors. Participatory Educational Research, 10(1), 389-407. https://doi.org/10.17275/per.23.21.10.1

Probst, L., Bardach, L., Kamusingize, D., Templer, N., Ogwali, H., Owamani, A., ... & Adugna, B. T. (2019). A transformative university learning experience contributes to sustainability attitudes, skills and agency. Journal of Cleaner Production, 232, 648-656. https://doi.org/10.1016/j.jclepro.2019.05.395

Pua’at, A. A. (2023). A study on awareness, exposure and attitude towards digital citizenship among university students in Malaysia. Information Management and Business Review, 15(1 (I) SI), 190-203. https://doi.org/10.22610/imbr.v15i1(I)SI.3397

Rahman, H. (Ed.). (2021). Human-computer Interaction and Technology Integration in Modern Society. IGI Global.

Ribble, M, (2011). Digital Citizenship in Schools. ISTE (International Society for Technology in Education).

Ribble, M. (2009). Passport to Digital Citizenship: Journey toward Appropriate Technology Use at School and home. Learning & Leading with Technology, 36(4), 14-17. https://eric.ed.gov/?id=EJ904288

Ribble, M. (2012). Digital citizenship for educational change. Kappa Delta Pi Record, 48(4), 148-151. https://doi.org/10.1080/00228958.2012.734015

Ribble, M. (2014). The Importance of Digital Citizenship: Why schools should help, young people navigate the digital landscape. District Administration.

Ribble, M. (2015). Digital citizenship in schools: Nine elements all students should know. International Society for technology in Education.

Ribble, M., & Bailey, G. (2007). Digital Citizenship in Schools. Washington DC: ISTE.

Ribble, M., & Miller, T. N. (2013). Educational leadership in an online world: Connecting students to technology responsibly, safely, and ethically. Journal of asynchronous learning networks, 17(1), 137-145.

Ribble, M., & Park, M. (2022). The digital citizenship handbook for school leaders: Fostering positive interactions online. International Society for Technology in Education.

Sahin, E., Ertepinar, H., & Teksoz, G. (2012). University Students' Behaviors Pertaining to Sustainability: A Structural Equation Model with Sustainability-Related Attributes. International Journal of Environmental and Science Education, 7(3), 459-478.

Salam, A. (2020). Internet of things for sustainable community development: introduction and overview. In Internet of Things for sustainable community development (pp. 1-31). Springer, Cham.

Sandoval, Z. V. (2019). DIGITAL CITIZENSHIP IN HIGHER EDUCATION STUDENTS. Issues in Information Systems, 20(4).

Saputra, M., & Al Siddiq, I. H. (2020). Social media and digital citizenship: The urgency of digital literacy in the middle of a disrupted society Era. International Journal of Emerging Technologies in Learning (Online), 15(7), 156.

Sari, D. P., Sapriya, S., & Fitriasari, S. (2022, January). Analysis of students’ digital citizenship practices on distance learning activities during pandemic Covid-19. In Annual Civic Education Conference (ACEC 2021) (pp. 507-513). Atlantis Press. https://doi.org/10.2991/assehr.k.220108.091

Shenishen, L. M. R. (2023). Exploring the level of digital citizenship awareness among female students in the information science department at Umm Al-Qura University. Information Development, 39(3), 402-424. https://doi.org/10.1177/02666669231184578

Sherman, L. E., Michikyan, M., & Greenfield, P. M. (2013). The effects of text, audio, video, and in-person communication on bonding between friends. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 7(2), Article 3. https://doi.org/10.5817/CP2013-2-3

Simsek, E., & Simsek, A. (2013). New literacies for digital citizenship. Contemporary educational technology, 4(2), 126-137. https://doi.org/10.30935/cedtech/6097

Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: Cognitive consequences of having information at our fingertips. science, 333(6043), 776-778.

Suleiman, F. R. A. (2020). A proposed unit in light of sustainable development and the green economy to impart to students of the scientific departments at the College of Education some concepts related to green economy and sustainable trends. Educational and Psychological Studies, 10(108), 85-149. DOI: 10.21608/sec.2020.102858

Sung, A., Leong, K., Cunningham, S. (2020). Emerging Technologies in Education for Sustainable Development. In: Leal Filho, W., Azul, A., Brandli, L., Özuyar, P., Wall, T. (eds) Partnerships for the Goals. Encyclopedia of the UN Sustainable Development Goals. Springer, Cham. https://doi.org/10.1007/978-3-319-71067-9_61-1

Tan, T. (2011). Educating digital citizens. Leadership, 41(1), 30-32. https://eric.ed.gov/?id=EJ965963

Teppers, E., Luyckx, K., Klimstra, T. A., & Goossens, L. (2014). Loneliness and Facebook motives in adolescence: A longitudinal inquiry into directionality of effect. Journal of Adolescence, 37(5), 691-699. https://doi.org/10.1016/j.adolescence.2013.11.003

Tomas, L., Girgenti, S., & Jackson, C. (2017). Pre-service teachers’ attitudes toward education for sustainability and its relevance to their learning: Implications for pedagogical practice. Environmental Education Research, 23(3), 324-347. http://dx.doi.org/10.1080/13504622.2015.1109065

Tsai, C. Y. (2018). The effect of online argumentation of socio-scientific issues on students' scientific competencies and sustainability attitudes. Computers & Education, 116, 14-27. http://dx.doi.org/10.1016/j.compedu.2017.08.009

Ull, M. A., Martínez-Agut, M. P., Piñero, A., & Aznar-Minguet, P. (2014). Perceptions and attitudes of students of teacher-training towards environment and sustainability. Procedia-Social and Behavioral Sciences, 131, 453-457. https://doi.org/10.1016/j.sbspro.2014.04.147

UNCTAD. (2015). Science, Technology and Innovation for Sustainable Urbanization, UNCTAD Current Studies on Science, Technology, and Innovation no. 10. United Nations publication.

UNCTAD. (2018). Technology and innovation report 2018: Harnessing frontier technologies for sustainable development. United Nations publication.

Vasiliu-Feltes, I., & Thomason, J. (Eds.). (2021). Applied Ethics in a Digital World. IGI Global.

Young, D. (2014). A 21st-century model for teaching digital citizenship. Educational Horizons, 92(3), 9-12. http://dx.doi.org/10.1177/0013175X1409200304

Zuwein, S. H. M. (2017). The effectiveness of using electronic blogs in teaching geography on developing digital citizenship skills for student teachers at the College of Education. College of Education Journal, 33(9), 461-531.