SEEALL: INTERACTIVE SYSTEM TO SUPPORT COMPETENCE ASSESSMENT

Valkiria Venancio
venancio@lsi.usp.br

Roseli de Deus Lopes

roseli@lsi.usp.br Universidade de So Paulo Av. Prof. Luciano Gualberto, travessa 3 N 158 Engenharia Eltrica

ABSTRACT Today, we live in a Learning Society with the active participation of ICT in knowledge construction, enabling new ways of thinking; it is a digital learning ecosystem where the school is inserted. Assessment in this complex environment still lies on the student; it is necessary to follow each individual among many students, and this is not easy. Modern curriculum programs are focusing on learners competences, skills and attitudes, in other words, on their global education. SeeAll is an interactive system to manage competences assessment and support learning planning and monitoring. KEYWORDS Learning Management. Competence assessment. Digital Learning Ecosystem.

1. INTRODUCTION
The rapid development of Information and Communication transformed relationships and behaviors, enabling interactions among humans and information, previously impossible. Nowadays, Information Communication Technologies (ICT) participate actively in information acquisition for knowledge construction. The Knowledge Society is the Learning Society, learning is no longer limited to schools [Delors, 1999] [Assman, 2000]. New ways of thinking and dealing with information arises with the evolution of digital media. The school became part of a digital learning ecosystem [Ficheman,2008] [Gtl,2009]. Assessment in this ecosystem remains focused on students and on their effort to learn. The classroom assessment dynamic is complex, it is necessary to follow each individual pathways in a collective space [De Sordi, 2009] [Behar, 2009]. In the current educational system curriculum, organized in cycles, teachers have the same goals, continuing or resuming the work of his/her colleague [Perrenoud, 2007]. And the formative evaluation focused on skills and competences, guides teachers choices activities and they can to pay more attention to students diversity [Hadji, 2009]. Seeking an effective school, there are external assessments such as the International Student Assessment Program (PISA) and national Evaluation System Basic Education (SAEB), concomitantly, there are electronic management, content delivery and reporting system for on-line or classroom courses; both to help teachers in their work. This paper presents the SeeAll, an interactive system to manage competences assessment and support learning planning and monitoring. It is designed to complement to the school digital learning ecosystem. Conceptual details are provided in section 2, its implementation in section 3, a preliminary evaluation in section 4 and final considerations in section 5.

2. SEEALL
A digital learning ecosystem is a complex flow of information and relationships among species in an educational environment supported by the media [Ficheman, 2008]. According to Ficheman (2008), to conduct the requirement analysis of a digital learning ecosystem, it is necessary to identify, to specify, to define and to describe its components, connections, interactions and influence, the functions of which would target systems authoring, collaboration, immersion and mobility characteristics. SeeAll was designed considering the digital learning ecosystem approach and the existing educational electronic systems. And aims to be a digital learning sub-ecosystem to support competence assessment with its species, populations and communities interrelating, interacting, recording and and suffering internal and external influences the system.

2.1 Requirement Analysis

To understand teachers content an initial survey was conducted with seventeen questions about how teachers use ICT, teach and plan, and assess their students. Its content analysis was divided into: digital fluency, teaching and evaluation. Some survey results are presented. In digital fluency topic, teachers inform that they used the computer to perform other habitual tasks, but not to register or monitor the studants performance. In the teaching topic, teachers inform they use curriculum matrices to plan. In the evaluation topic, they inform that they have some difficulties in individually monitor students for appropriate intervention. SeeAll should be easy to use; it must be based on national curriculum guidelines and, finally, help teachers with monitoring individuals. Also it tries to overcome the shortcomings of other educational electronics systems. Examples of systems: Learning Content Management System (LCMS) for content delivery, Virtual Learning Environment (VLE) for management of learning and results performances, Learning Management System (LMS) that adds communication tools and Managed Learning Environment (MLE) for management in schools [Weller, 2007] [Page, 2009] [Merino, 2006]. However, there is a gap. We did not find any electronic systems that offers tools that follows the learning process based on students competences and skills or that provides a student overview in all areas concomitantly Thus, SeeAll has the following characteristics functional and nonfunctional: users and technologies mobility; interactivity; allows authoring; allows contribution between users; flexibility; it is an Open Source software; has multi-platform and interoperability. SeeAll presents a conceptual assessment not only lessons results, but it allows we to record and monitor the students skills and progress in carrying out learning experiences.

2.2 Functional Description

The main features of the prototype are: organizes users and groups; lists attitudes, competences, skills and formal and informal activities, expressed by the learners, we call learning experiences; allows the authorship of these lists; combines learning experiences, skills, competences and attitudes; stores the information obtained; presents individual data visualization in different ways; electronically organizes the learners record of use while the system is used; sends messages to groups and individuals; communicates with other systems such as LMS or VLE through Sharable Content Object Reference Model (SCORM) [ADL, 2004].

SeeAll manages the assessment process via competences and skills, the functionalities of which are: register the system organizes users through their profiles. description and learners areas expected list - the system organizes competences, skills, attitudes and learning experiences created by teachers or managers in different areas (Figure 1);

Figure 1. Skill descriptions

storage and retrieval - the system stores information and allows their retrieval; information visualization - the system allows viewing students achievement through the Map Use (Figure 2). It uses colored figures to identify students or their class resulting skills, competences and attitudes ; so far in paper prototype (Figure 3) [Snyder, 2003];

Attitudes

Skills

Competences

Figure 3. Map Use (paper prototype)

Figure 2. Student achievement visualization (Map Use)

Each skill has an Opportunity Scale. It shows by color, the results of the skills developed through learning experiences in different areas of knowledge (Figure 4).

Figure 4. Opportunity Scale

The colors are the ones teachers usually use in schools to register learners grade [Carvalho; Marcos, 2010]: white - did not work; yellow - did not achieve the objectives and worked a few times; red - did not achieve the objectives and worked a lot of times; orange - achieved the objectives and worked a few times; blue - achieved the objectives and worked a lot of times. records of use the system organizes learners records of use with his/ her information visualization: map use, opportunity scale and learning experience all along the time they use the system;

communication the system allows users to send and receive messages; interoperability allows communications between electronic systems (LMS, VLE or MLE). In SeeAll, the teacher registers learning experiences and the students achievement. These learning experiences are linked with skills that are linked with competences. Only skills and competences appear in map use, not activities or learning experiences.

3. PRELIMINARY EVALUATION
The SeeAll evaluation had three steps: functional tests, tests with potential users and analyses.

3.1 Tests
The functional tests performed by the author intended to verify that all prototype features were operating as expected. After this, the necessary changes were implemented, such as records, language adaptation and improvements in the interface. The potential users tests were developed with five people, a sufficient number for evaluation to find problems and measure satisfaction [Nielsen, 1993]: two teachers, two managers and one expert on technology in education. The objectives of the potential users tests are: their understanding of the conceptual principle system; easy information visualization in the paper prototyping [Snyder, 2003]; SeeAll prototype ease of use; satisfaction and acceptability level. For this, the following script was used: brief system presentation in netbook; brief manipulation with tasks set; semi-structured interview. The test was performed individually in a specific room, recorded and filmed for analysis.

3.2 Analysis
After observations content analysis, and interviews, two large categories were established: Pedagogical Characteristics and System Acceptability. The interpretative reading Pedagogical Characteristics category highlights the evaluation, organization, individuality and information visualization. Evaluation according to educators, SeeAll can contribute to a deeper understanding of concepts competences and skills or contents and objectives. And the relation between learning experiences, competences and skills with feedback facilitates planning and re-planning reflection, reviving and revising teaching actions; Organization and Individuality SeeAll contributes towards the register organization making it easier to search for individuals information. This record format approaches the planning assessment and the action towards its goals. Information Visualization the Map Use and Opportunity Scale with color subtitles allows both teachers and parents to better understand the full learners use and allows his/ her monitoring, and teachers safe diagnosis of their performance. However, tests showed that everything depends on the amount of information on screen. The interpretative reading System Acceptability category highlights the educators comments: Acceptability - It is easy, applicable, natural, of easy manipulation. Doubts email or name, register or record, how to save.

4. DISCUSSION
This work allowed the SeeAll design as a support tool for teaching and learning, planning and monitoring within a digital learning ecosystem.

It provides another kind of results, that display with the concepts competence, skill and attitude in several areas of knowledge simultaneously, for individuals or a class. It also can be used for long of time by the learner, not only in a year scholar. Its implementation with user's interaction and tests showed us a positive assessment of its potential as a tool that contribute with the teacher. Foe example, what the direction the teacher can to take after a learning disability or its worsens is detected. Some characteristics were quoted by the teachers how important: ease of use, organization and recovery of data, the teachers authorship, results optimization and analysis. After tests some changes must be made, and we implement resources besides and to add others; for example, the Self-evaluation for mature learners. For future work would be interesting a detailed study of quantity and quality of information offered to acquire the skills and abilities that can to be incorporated into the system.

REFERENCES
ADL Advanced Distributed Learning initiative.2004 Sharable Content Object Reference Model (SCORM). Available <http://www.adlnet.gov>. Accessed: jul, 2010. Assman, H. 2000. A Metamorfose do Aprender na Sociedade da Informao. Cincia da Informao, IBICT, Braslia, v. 29, n. 2, p. 7-15. Available: <http://www.scielo.br/pdf/ci/v29n2/a02v29n2.pdf>. Accessed: may.2010. Behar, P.A.; Bassani, P.S. 2009. Os Ambientes Virtuais e a nova era da Avaliao. Ptio Revista Pedaggica, Porto Alegre: Artmed Editora, ano XIII, n. 50, p.16 19. Carvalho, E.S.; Marcos, A.F. 2009. Visualizao da Informao. Guimares: Centro de computao gafica. Available: <http://repositorium.sdum.uminho.pt/handel/1822/8863>. Accessed: dec, 2010. Delors, J. (chairman), 1999. Learning: The Treasure within. Report to UNESCO of the International Commission on Education for the Twenty-first Century. UNESCO publishing. Available: <http://www.unesco.org/delors/delors_e.pdf>. Accessed: jan, 2011. De Sordi, M.R.; Ldke, M. 2009. A Avaliao nos Novos Contextos e Paradigmas Educacionais. Ptio Revista Pedaggica, Porto Alegre: Artmed Editora, ano XIII, n. 50, p. 12-15. Ficheman, I.K.; Lopes, R. de D. 2008. Digital Learning Ecosystems: Authoring, Collaboration, Immersion and Mobility. 8. IEEE International Conference on Advanced Learning Technologies. Espanha. ____. 2008. Mobility in Digital Learning Ecosystem. IADIS International Conference of Mobile Learning. Portugal. Glt, C.; Chang, V. 2009. The use of Web 2.0 tecnologies and services to support e-learning ecosystem to develop more effective learning environments. Available: <http://espace.library.curtin.edu.au/R/?func=dbin-jumpfull&object_id=128535&local_base=GEN01-ERA02>.Accessed: mar, 2010. Hadji, C. 2009. Muito alm dos Indicadores. Ptio Revista Pedaggica, Porto Alegre: Artmed Editora, ano XIII, n. 50, p.20-22. Merino, P.J.M; Kloos, C.D.R.S.; Garca, R.M.C. 2006. Rating the Importance of Different LMS Functionalities. 36th ASEE/IEEE Frontiers in Education Conference. San Diego, CA. USA, p.28 31. Nielsen, J. 1993. Usability Engineering. New Jersey: Academic Press, USA. Page, T. 2009. An Evaluation in the Use of Internet-Based Instructional Resources and MLE in Support of theTeaching and Learning of CAD/CAM. The Journal of Design and Technology Education, Volume 9, N.2. Available: <http://ojs.lboro.ac.uk/ojs/index.php/JDTE/article/viewFile/685/657 >. Accessed: oct, 2010. Perrenoud, P.2007. Diez Nuevas Competencias para Ensear. Barcelona: Gra Biblioteca de Aula, Spain. Snyder, C. 2003. Paper Prototyping: The fast and easy way to design and refine user interfaces. Boston. MorganKaufman. Weller, M. 2007. Virtual Learning Environment: using, choice and developing your VLE. Routledge, NY, USA