Analyzing self-reflection by Computer Science students to identify bad study habits

Self-reflection performed by students of programming courses on the study habits and acquired skills through the B-Learning supported by an automatic judge

Filomena Sustelo
University of Algarve Faro, Portugal fsustelo@gmail.com

Pedro Guerreiro
University of Algarve Faro, Portugal pguerr@gmail.com

Abstract We present some preliminary results and the main conclusions of a study that we conducted at the University of Algarve, for a third programming subject of the first year of Computer Science Programming Course. We analyzed the selfreflections made by programming students about their study habits and skills acquired in it using a methodology of blendedlearning supported by an automatic judge, during the period of study for the subject examination, on a sample of students who performed the examination. The research data were obtained through questionnaires. We took into account data from other instruments about the previous study carried out by students, as an important factor to acquire programming skills and balance, to overcome any limitations in the study during the period of preparation for the exam, as well as their performance. We intended to ascertain to what extent the planning, motivation, previous study or knowledge about the type of examination influenced final results. The results suggest measures to be performed in future editions of the subject, for the benefit of students to come. Keywords Blended-Learning, automatic judge, self-reflections, study habits and acquired skills of programming students, learning constrains.

I.

INTRODUCTION

from a cognitive point of view [2], which aim the gradual and progressive application of andragogy 1 principles (adult education) and not only of pedagogical principles and to identify factors that can condition the development of the necessary competences for programming. So, in the academic part of the subject Algorithm and Data Structures (ADS) of this course, was adopted a Blended-Learning method based on the availability of the supports needed to learn and evaluate, supported by an automatic evaluation tool of the programs written by students [1] and so releasing teachers to more gratifying pedagogic tasks. Being ADS a subject of the first cycle, first year, but the second of CSP programming, the students were supposed to master previous competences, and be able to apply them and learn new ones, such as: skills on C programming with the deep demanded competence of the use of data structures libraries, and programming of main data and algorithm structures, as well as the resolution of the typical problems of programming contests. However, Computer Science teachers recognize the students difficulties on acquiring these competences, and try to help them identifying their fragilities, roots and ways to surpass them. In a way to have a perception of these difficulties, we aim this study to a vision of the necessary competences for a full success (that is, to obtain results above minimum grade). A. Cognitive competences of programming Studies based on characterizing cognitive competences of university students refer Bloom [3] taxonomy, formed by six levels, and indicate that students, in general, initiate their learning by a superficial approach, which correspond to the first three levels, then evolving in a gradual and progressive way to the deep approach, to which correspond the last three levels. A more recent version of this taxonomy [4] differs from the original one on the terms used to name competences, and this way it reinforces mind activity at each level trough a verb (know, understand, apply, analyze, evaluate and create) instead of trough a substantive (knowledge, comprehension, application, analyzes, synthesis and evaluation). The most
1 According to Waal and Telles there are five andragogy principles: autonomy, experience, learning interest, learning use and motivation to learning.

In general, the Portuguese or foreigner student population enrolling in the Computer Science Programme (CSP) of the University of Algarve (UALg) is mostly composed by young adults (say, 18-19 year olds) and some adults. These students bring with them, from the beginning of secondary education and also from other experiences, knowledge, skills, attitudes, study habits, motivations, interests and expectations, and also the weaknesses referred in [1], that we must not ignore. Still, these motivations, interests, and expectations do not always have the desired effect in achieving academic success within the prescribed time. Furthermore, it has been observed that accumulated failures as a result of difficulties in developing the necessary skills for writing programs that require more than superficial knowledge lead students to stay at the university as adults (say, with more than 24 years of age). We believe that it is the teachers responsibility to encourage students in building a deep, responsible and autonomous knowledge for the adoption of rich, diverse, motivating and demanding strategies

visible alteration is verified on the change of the two last levels. As showed on the following picture the synthesis level stop to exist on the new version, being replaced by create a new competence. Original version
Evaluation Synthesis Analysis Application Comprehension Knowledge

Recent version
Creating Evaluating Analysing Applying Understanding Remembering

material created, that students improve the critic and decision competences. It seems natural that this is the way human beings are directed in learning and problem solving, whether of everyday or programming ones. But if this is the natural course of learning, what are the reasons for so many difficulties in acquire highest level competences in programming [6, 7]? It was examined problems and development possibilities of these competences in students for a better understanding of these difficulties. 1) Impeditive factors to acquire high level competences In this point, impeditive factors for the development of the necessary critical and creative competences to reach full success, on ADS subject were pondered. This way the characteristics of a programming student with weak progression possibilities on these competences are: 1. Lack of effort and necessary persistence to apply the knowledge acquired on different contexts, originated by the lack of taste for math or the self-motivation necessary to fulfil these tasks, whether by the perception of difficulties or incapacity, or even caused by other interests (hedonism,); 2. Satisfaction on the fulfilment of the minimum requirement for the subject approval, for the reasons stated on the previous point; 3. Adoption of a preferential style that worked in the past, but that does not respect programming basic principles and good practices; 4. Limits at the level of abstraction and logical thinking, of mathematical knowledge or other underlying to the presented programming problems; 5. Weaknesses on planning and/or execution competences of the tasks on the school time associated to Bologna model; 6. Contrarieties caused by the quantity of necessary effort to develop all the necessary competences. 2) Possibilities to foment the development of high level competences In an attempt to find out solutions to some of these problems, we headed for a first analysis of a classic article [8], often quoted at educational level, where Dijkstra argues that learning is a slow and gradual transformation of "new to usual, reinforcing the idea that learning programming is what he calls the "great news". Jenkins in [6], supports the argument of Dijkstra and adds that the programming is a matter difficult to acquire, but that it should not be impossible to overcome this obstacle. Other studies were observed to promote deep learning [9, 10], where we highlight [9], which sought to demonstrate that the strategy "peer review" promotes deep learning in programming engineering courses. The author argues from experience that when students evaluate the work of their colleagues they think deeply, see as others solve the problems, learn to criticize constructively improving the capabilities of critical thinking. In this article, the author refers other strategies to force deep learning, stressing the importance of the implementation of reflective learning in higher education based on the application of knowledge to new situations or in different contexts. The present work is inserted on a research line which intended to evaluate the efforts results on the application of B-

Figure 1. Changes on Blooms taxonomy

Despite the innovation, we believe that programming is still viewed as a creative activity for analysis and synthesis [5], and a critical assessment and synthesis activity, or self-assessment or when generating some discussion and reflection on the resolution problems through programs, or on decision-making supported by deep reflection to justify the selection of the more efficient programming solution for the original problem. So, for the present work, we relied on Bloom's original taxonomy, which seemed more convenient to describe the natural course of an interested, motivated and hardworking college student, dedicated to learning the principles and good programming practices. The student, when confronted with a hierarchy of skills and consequent assessment, will certainly first familiarize him/herself with lower level competences, which means to know and understand syntactic and semantic rules of language and how to apply them to solve at least the more trivial programming problems presented. Usually the exercises presented here should be helpful to: Diagnose weaknesses knowledge; in students at background

Evaluate the level of student difficulties in more trivial programming tasks; Elucidate and train students, preferably through effective teaching strategies, that provide them sequential action guides (say, "storylines" with "step by step" tasks) with the necessary information and clues for gradual and progressive development of high-level programming skills.

For the application of highest level competences, the student should be lead to analyze defying problems, which will consist on how to interpret the question problem with focus and attention, bearing in mind the principle of "Divide to Conquer". It is about to decompose the problem into parts, as independent as possible of each other, and analyze their relationships. The junction of the parts, well related, with the aim of reaching the solution of the original problem is the synthesis competence. Creativity in programming will reside in these two skills. Usually it is on self-assessment or value judgment of the

Learning teaching and learning strategies, supported by an automatic judge [1] that assesses the programs written by students, and thus freeing teachers for other more rewarding tasks. In this article we report the case of the ADS subject, analyzing self-reflection exercise made by students in the postexamination period of the academic year 2008-2009, their study and learning habits through B-Learning supported by an automatic evaluation tool, during the period of preparation for the exam, which roughly corresponded to a week between the end of classes and the day of the exam. We proceeded to the analysis in an attempt to identify core competencies in planning the study, effort and action in accordance with the planning and execution of the exam, as factors to develop or maintain in future editions of the subject. The research data were obtained through questionnaires collected from a sample of students who attended the examination. We took into account data from another sample of students on the previous study conducted in the teaching of the subject, as important factor in the acquisition of key programming skills and of balance, to overcome any limitations in the study during the period of preparation for the exam, so as their performance. To this effect were made available in Moodle two surveys, both designed in Google Docs [11] [12], the first at the end of the subject course work, on the preliminary study; and the second after the examination, on self-reflection after examination. These data were treated in terms of characterizing the profile of the group of students from ADS, both in terms of individual profile for the placement of each student at the findings of the group and desired skills for ADS for the key competencies identified. The goal of this research is to instil in coming students an awareness of the most common weaknesses, or critical points, to the development of high level competences. This way the teacher will have a concrete mean to advise or help students in acquiring key competences, which will allow them to jump for learning progresses and, in this case, on the acquisition of competences on the third programming subject of the course. Following we present our case study II. STUDY CASE

there, some already adults 2. The justifications more invoked for the previous fails were: a) lack of basis on mathematic, other subjects or others, by 12 (52%) respondents; b) short school period associated to the academic regime of five weeks classes, adopted by UALg, by 11 (48%); c) lack of motivation, little effort and persistence on learning, by 10 (43%); d) high levels of uncertainty specially around exams, by 7 (30%). So we have grouped some respondents aware of their past weaknesses, within reach of high level programming competences, but that have shown signs of: a) encouragement on study persistence, on the opinion of 13 (54%); b) frustration and anxiety on the opinion of 11 (46%); and c) trends to drag the subject and the consequent postponement of success, on the opinion of 9 (38%); besides other less voted reasons. In this case, the reactions did not point out towards successive fails constituting an immediate motive for academic abandonment, with the exception of 5 (21%) that have shown some signs. It was also verified that 16 (70%) respondents reveal taste by programming, but 13 (57%) did not liked mathematic and, curiously, only 7 (30%) were motivated by both. It was also detected 3 (13%) more critical cases of lack of taste whether for programming or for mathematic. These were concerning results, as in [1]. Then it was analyzed the satisfaction degree of students about subject activities, whether trough Moodle platform, whether trough an automatic evaluation tool, Mooshak 3 , besides the competences acquired, personal interests, programming skills, ADS expectations and their relation with the results obtained in the mid-exam (an average of 12.4 values), before the final exam. 1) Functioning of the subject on B-Learning modality, supported by an automatic evaluation tool B-Learning, supported by Mooshak, was the learning modality adopted for ADS, to hold, for the one hand, presential learning and, for the other hand, to foment autonomous work, which resulted in merging the existing human resources with the technology available. Based on planning the C contents and evaluation materials, were made available, from the beginning, information on learning goals, way of work, as well as evaluation methods. Theoretic presential classes were filmed to be available on the platform in a way to reinforce learning and stimulate autonomous work at the rhythm of every student. Continuous evaluation was made by: 4 "storylines" in total, with practical sequential tasks, information and clues necessary to the gradual and progressive of high level competences; 3 problems of usual style on programming competitions to apply those competences; and self-evaluation quizzes, after each theory class, to consolidate the matter learned. In this case, 3 (13%) respondents tried to make it all, 8 (35%) considered them useful, 1 (4%) fun and 7 (30%) have agreed that they helped remembering the matters, while 3 (13%) considered some questions a bit lingering. All tasks had an automatic evaluation return, either through Mooshak 4, and either by the platform. The exception was the storylines reports submitted
2 Therefore one of the competences to improve, on future editions of the subject, will be to avoid success postponement. 3 Open-source automatic judge applicable to programs with text entrances and exits, developed by Jos Paulo Leal, from Porto University. 4 After the submission of each task, this automatic judge returned one of the feedbacks: Accepted, Presentation Error, Runtime Error, Compile Time Error, Time Limit Exceeded, Memory Limit Exceeded, Wrong Answer or Invalid Function.

A. Previous study made by students of the ADS subject On the analysis to the previous study made by the students, were considerate 23 questionnaires, gather at the end of the five school weeks of ADS and before the period of preparation for the examination. Despite being predicted more students for exam from a total of 136 enrolled and 56 that actually attended classes. We highlight 17 (74%) male students, 15 (65%) fulltime students, and the rest with some kind of part-time occupation (but less than 8 hours/day), 14 (60%) with ages between 18 and 24 and the rest with ages superior to 24 years. It was only registered 4 (17%) respondents with a single subject enrolment, 7 (30%) with two and 12 (53%) with four or more. Apparently part of the ones who quit on the first week has shown signs of lack of preparation and courage to attend the third programming subject of the course. So it is visible the trend to withdrawal from this subject by students that start young and after some failure years or dropouts still continue

in the platform in video, that were commented by the teacher trough the e-mail of every student. It was verified 14 (61%) agreements and 7 (30%) hesitations about the rules and way of evaluation. On the interaction established between teachers and students were more agreements, 16 (69%), with the effective of communication trough the platform than those, 12 (52%), which agreed with the teacher presential feedback to works, a situation similar to the one of [1]. However, there were 8 (35%) contrarieties and 7 (30%) respondents revealed easiness on time management. Likewise, 16 (69%) agreed that learning activities have taken more time, than previously expected. Some respondents said to have acquired, trough Mooshak, platform and other technology, essential study competences, such as: a) more careful on planning activities and meeting deadlines, with 15 (65%) agreements and 5 (21%) hesitations; b) more rigor on the activities fulfilled and submitted, expressed by 15 (65%) agreements and 6 (26%) hesitations; c) more motivated to fulfil the activities, with 16 (70%) agreements and 3 (13%) hesitations; d) 14 (61%) have felt some learning progresses and 6 (26%) hesitated; e) 16 (70%) developed critical spirit, creativity and problem resolution capacity and 6 (26%) hesitated, f) 12 (52%) have recognized the justice of an evaluation system that have grant them the wanted transparency from the beginning and 6 (26%) hesitated; g) 10 (43%) agreed that the several evaluation exercises have measured their knowledge and competences, with 9 (39%) hesitations; and h) 11 (48%) said to have improved their academic performance, despite the 8 (35%) hesitations. Based on the calculus of the average ranking (AR), trough the pondered average (PA), of the four items of the Likert 5 Scale, presented in the previous table, it was showed quite favourable behaviours (with values above 3) on the respondents attitudes in relation to the study:
TABLE I. AVERAGE RANKING OBTAINED IN THE PREDISPOSITION IN
RELATION TO THE STUDY

of hedonism, except for 1 (4%), 3 (13%) of popularity and indifference towards the study, 1 (4%) with desire to exert authority, influence and enhance public image. Yet, despite the visibly favourable results, study commitment beyond favourable predisposition requires task planning, effort and actions in accordance with the plan, still that in programming action is, most of the times, fruit of students abilities. So, based on the next table, were verified the abilities acquired in programming:
TABLE II. AVERAGE RANKING MDIO OBTAINED ON PROGRAMMING
ABILITIES

Abilities in programming 1. Im able to solve the most trivial problems of the subject. 2. I know and understand the rules of the C language and I apply them in the most trivial exercise. 3. Id acquired skills for problem solving typical programming contests. 4. Im able to analyze challenging problems, its decomposition, relationships, joint parties with the aim of reaching the solution. 5. My logical or mathematical reasoning has been effective in most evaluation work. 6. Id acquired skills to decide on the best programming solution to a unique problem. 7. Rarely lapses in accuracy in programming.

AR 3,6

3,5

3,4

3,3

3,5

3,3

2,1

Comportment Favourable (with 26% of uncertain) Favourable (with 17% of uncertain) Favourable (with30% of uncertain) Favourable (with 39% of uncertain) Favourable (with 30% of uncertain) Favourable (with 35% of uncertain) Unfavourable (with 30% of uncertain)

Aptitudes in relation to previous study 1. Actually I face the study with seriousness and as an obligation to satisfy my interests 2. Usually I fulfil the terms of my study plans, even if arising calls for fun. 3. When I cant fulfil the obligations (delivery of works...) I'm frustrated. 4. Not pass on subjects is something that bothers me a lot.

AR 4,1

Comportment Favourable (with 17% of uncertain) Favourable (with 17% of uncertain) Favourable (with 17% of uncertain) Favourable (with 13% of uncertain)

4,0

4,1

4,3

In this case, 19 (83%) have said, some (10) with more conviction than others (9), that have faced study with seriousness and as an obligation to satisfy their interests, 4 (17%) have revealed hesitations and 1 (4%) depreciated the study. The respondents have presented interests of personal fulfilment, by 18 (78%), and of autonomy and independence from the family, by 19 (82%). In this case there were no signs
5 The scale was scored in a way that 0 was dont know how to answer, 1 to strongly disagree and successively until 5 for strongly agree. A value lower than 3 for each item corresponds to a favorable comportment of the students; a value below 3 to an unfavorable comportment caused by some contrariety; and value 3 corresponds to an uncertainty comportment, that can be a sign of fragility or indifference.

The table presents an analysis of data provided by respondents in relation to their ability to programming. It can be inferred about the existence of positive behaviours, although these are not as massive as those found in table I, as shown by AR variables and the high rates of response with hesitation over the remarks. It deserves some highlight the fact that the uncertainty has increased and the AR decreased with the increase in demand and consequent complexity of the exercises. The results point to greater difficulties in acquiring high-level skills, visible in sections 3, 4, 5 and 6 of Table II, fundamental to the full success in the subject. The last variable implies unfavourable behaviours, despite the uncertainty of 30%, where it appears that most respondents admitted committing precision faults on programming. Following the analysis it became relevant to know the expectations of these students in relation to the final results in the subject, after the examination. In this respect it was found that 13 (57%) were not very ambitious to be satisfied in achieving the minimum for approval, 5 (22%) undecided and the remaining 5 (22%) expressed dissatisfaction in getting minimum levels. In this case it was seen a distinct modesty of respondents in the satisfaction by obtaining the minimum requirements for approval 6, possibly caused by too many past failures and by the awareness of weaknesses in programming skills.

6 To avoid satisfaction in obtaining minimum grade on the subject is other competence to be developed in future editions.

B. Self-reflection of students after the ADS examination After the ADS exam were collected data from 10 questionnaires, from a set of 28 possible respondents who attended the exam. This subject allowed 37 students to take the exam, with relatively good performances (average of 12.4 values) compared with the final results after the examination. The average obtained in the exam was 8 values, with 12 successful students, 6 of them with grades above or equal to 12 values (12, 13, 13, 15, 15, 18 respectively) and the remaining 6 with a grade between 10 and 11 values (10, 11, 11, 11, 11, 11, respectively), 16 failed and 9 who were absent. These results are no less disturbing especially for their persistence over the years, where it is difficult to find another explanation than poor study habits of students in the third programming subject of the degree. This is the cause of the emphasis in the exercise of selfreflection of students to their study habits in the postexamination in relation to the planning done in the prepreparatory study, the effort and action conducted in preparation for examination, the taking of the exam and postsurvey weighting on key issues to maintain or improve in the way they prepared themselves, between the end of lessons and the day of the exam, and if the effort was worth it. The data indicated that on the exam, 9 respondents were male, 6 young adults and 4 adults over 24 years. 1) Planning carried out during the pre-preparatory study The planning of the study is a key skill for any student consisting of a reflection of preparation, visualization and design of the task. At this point it was noted the following questions: Qpl_1) hot trends, of 80% of the respondents, that were likely to be at the exam; Qpl_2) incidence of 50% of interests in the questions frequently asked by the teacher; Qpl_3) more profitable time of day; Qpl_4) the favourite place to study, Qpl_6) ways to avoid distracting factors, Qpl_7) definition of learning goals, materials or technology necessary were also the second key aspect most valued, with 50% interest; Qpl_5) profitable study hours were aspects seen by 40%; Qpl_8) finally, the components distribution of study and rest times and what strategies to use to avoid anxiety states obtained 40% of records. In the respondents reflections after having seen the exam was verified that only 20% of them, corresponding to R1 and R2 of the following table, defended the way they prepared themselves, since considering to have obtained a good result, despite R2 admitting a final result below his/her expectations. The key issues envisaged by all respondents, and advised to be improved by some or maintained by others, are in the following table:
TABLE III.
Qpl_1) R1 R2 R3 Respondents R5 R6 R8 R9 R10 Qpl_2) R1 R5 R6 R8 R9 -

2) Action and effort in preparing for the exam The better the planning and the more consolidated are the matters seized in the course work certainly the greater the effectiveness of the result of the study will be. Usually the action and effort taken during this period are used to review matters, overcome difficulties, solve tests that have been done, think of new exercises, etc. At the end of the study, and before the test, it is essential to make a self-assessment of learning, so students are self-accountable for their study habits and results obtained. In this case all respondents used the preparation time to study, as shown in the following table, and the study was always or almost always conducted with the support of materials and technology available on the platform:
TABLE IV.
R1
Days / Hours of study in a week

TIME DEDICATED TO STUDY WHILE PREPARING FOR THE

EXAM

R2
4/80

R3
2/14

R4
5/15

R5
5/45

R6
3/12

R7
3/12

R8
3/42

R9
3/15

R10
2/10

3/40

neither agree nor disagree

neither agree nor disagree

neither agree nor disagree

fully agree

agree

agree

agree

agree

agree

QUESTIONS PONDERED ON STUDY PLANNING

Qpl_3) R1 R2 R3 R4 R6 Qpl_4) R2 R3 R4 R6 R8 Qpl_5) R2 R6 R8 R9 Qpl_6) R1 R2 R3 R6 R8 R10 Qpl_7) R1 R3 R5 R7 Qpl_8) R4 R5 R7 R4 -

The data confirmed that the answerer R1 was the one who was surer that the hours dedicated to study were profitable. On the other hand, it was verified that in relation to the questions presented here: Qac_1) all respondents reviewed the theoretical materials; Qac_2) 70% analyzed exams made in the subject; Qac_3 50% resolved the evaluation exercises of the course work; Qac_4) only 10% thought of new exercises from the ones made available on the platform, the remaining 50% despite agreeing to the effectiveness of their study strategy did not show it clearly and there were 40% of hesitation; Qac_5) no respondent claimed to have solved the proposed exercises that showed no solution; Qac_6) 50% revealed, but without much conviction, to have made self-assessment of their learning, 10% hesitated and the remaining 40% denied it; Qac_7) 30% of respondents said that throughout the study were able to formulate their own questions and that therefore there was no reason for jitters before and during the exam, 30% hesitate and 40% disagreed; Qac_8) interestingly, 80% of respondents studied alone while the others in group; Qac_9) for doubts that arose, 40% relied on colleagues (in person or in forums), 30% referred to other means and no one appealed to teachers during this period. Notice the importance that 40% of respondents indicated that the effort in the preparation paid off for having gotten a better note than the one they would have got if they had not prepared themselves. The effort and action conducted by these respondents is showed in the next table:
TABLE V.
Qac_1 R1 R3 R6 R7 Qac_2) R1 R3 R6 R7 R3 R7

ACTION AND EFFORT WHILE STUDYING FOR THE EXAM

Qac_3) Qac_4) R7 Qac_5) Qac_6) R1 R6 R7 Qac_7) R1 R7 Qac_8) R1 R3 R6 R7 Qac_9) R6 -

By analyzing the Table V, the respondent R7 distinguished him/herself from the other for, besides all the effort and action in the preparation for the exam, having been the only one to

agree

Hours devoted to the study were profitable

reaffirm that he/she equated new exercises from the ones available at the platform and for presenting him/herself as the more convinced that his/her study strategy was effective to relax and avoid anxiety. It was found that, like R7, R1 also maintained a positive spirit during study and examination. In these two students are found skills of self-motivation to formulate their own questions where normally those who enunciate them, must have a thorough knowledge on the matter to give the appropriate answer. To be added that at the end of the study, respondents R1, R6 and R7 performed the selfassessment of their learning taking self-responsibility for their actions in the study. However 40% of the respondents felt that the effort of preparation was not worth it because they obtained a mark below their expectations. The effort and action conducted by these respondents are given below:
TABLE VI.
Qac_1 R2 R5 R6 R8 Qac_2) R2 R5 R6 R5 R8

ACTION AND EFFORT WHILE STUDYING FOR THE EXAM

Qac_3) Qac_4) Qac_5) Qac_6) R5 R6 Qac_7) Qac_8) R5 R6 Qac_9) R5 R6 R8

the respondents was time, being this one of the key competences to be improved on future editions. Like on other areas, for programming exams it is always necessary a deep study of the matter so that right answers appear faster with the necessary self-confidence and self-control. These are fundamental aspects that help to dissolve fragilities of time management, nerves, etc. The exam was structured into five groups, as following: 1st with questions like the ones of selfevaluation questionnaires referred on the previous study; 2nd with a trivial question over the output generated by a certain test function; 3rd and 4th with questions to apply the knowledge acquired in the context of new functions; and in the last group a problem with a complete program, like the tasks submitted to Mooshak, typical of programming contests. This way, we analyzed the key competences to maintain or improve on the resolution of the 1st, 2nd, 3rd and 5th groups of the exam, respectively:
TABLE VII. STRONG AND CRITICAL POINTS IN THE RESOLUTION OF THE
THREE FIRST GROUPS OF THE EXAM

In this case, no respondent stood up with enough selfmotivation to formulate their own questions, although Table IV shows that these were the ones who have dedicated more time to study during preparation time. It is verified that respondent R5 also took personal responsibility for his/her actions and it is worth noting that R6 respondent considered that this effort allowed him/her to get a better grade than he/she would have had if he/she did not make some effort, being however unsatisfied for the grade below his/her expectations. 3) Making the ADS exam During the exam it was noticed that the study strategies used specially by R1 and R7 worked as a key competence to grant success at the subject. Next, we present a graphic vision over the main difficulties felt by this group of respondents during the exam:
Other difficulties No difficulties I was nervous I had no time I had no detailed knowledge I didnt go to that class or didnt have time I made a misinterpretation of what was asked Dont manage time effectively There were things that I didnt know Didnt know how to solve some exercises Not always I understand what was asked 0 R7 1 R5 1 R2 R4 R5 R7 R10 5 R6 R10 2 R6 R9 R10 3 R7 R8 2 R3 R4 R9 R10 4 R6 R10 2 R3 R6 R8 R10 4 R5 R8 2 R10 1 R1 1 0 1 2 3 4 5 6

Key competencies to maintain (bold) or to enhance (normal) Many difficulties Withdrawal at first attempt First time I do these exercises I kept calm and was careful

1st Group R6 R1 R1, R3, R4, R6, R8, R9, R10 R6, R10 R6, R10 R1, R3, R4, R5, R6, R7, R8 R2, R6

2nd Group R1, R4, R6, R7, R8, R10 R10 R2, R3, R5, R6, R7, R9 R10

3rd Group R8, R10 R4, R6, R10 R4, R8, R10 R1, R2, R3, R6, R7, R9 R5

For not to make mistakes I preferred not to respond Not studied in detail I memorized this matter I recognized the matter and was able to apply it in context of these exercises I had no detailed knowledge that would allow me to respond well

On the execution of the 1st group, concerning the questionnaire, we highlight respondents R3, R4 and R8 with more key competences to maintain. In these exam groups, were valorised the item over matter recognition in the context of the questions presented, following on the next level R1, R5 and R7. On the other hand, R6 was the one who presented more weaknesses on the questionnaire, followed by R2 and R10. On the two following groups, to implement functions, the highlight goes to R6 and R7, with key competences to maintain, followed by R2, R3, R9 and R1. On these exercises, R10 was the one with more difficulties, followed by R8, R4 and R5.
TABLE VIII. STRONG AND CRITICAL POINTS, ON A RESOLUTION OF A PROBLEM TO APPLY HIGH LEVEL COMPETENCES

1st time that I solved some of exercises 1st time that I solved an exam of this type

Key competencies to maintain (bold) or to enhance (normal) Many difficulties First time I do these exercises Withdrawal at first attempt I kept calm and was careful For not to make mistakes I preferred not to respond Not studied in detail I recognized the matter and was able to apply it in context of this

5th Group
R9, R10 R10 R10 R1 R10 R7, R10 R1, R2, R7

Figure 2. Dificuldades no exame de AED

R7 was the only to affirm not to have felt any difficulties, while R10 was the one who felt them the most; R5 assumed some nerves and the most limitative factor for the generality of

problem I had no detailed knowledge that would allow me to respond well I should have studied the matter thoroughly to respond quickly I correctly interpreted the statement of the problem I identified the data to solve the problem

I was able to build a mental algorithm to solve the problem I decomposed the problem into parts independent of each other I analyzed the relationships of the parties, which could resolve I joined the parties well linked, with the aim of reaching the solution of the problem Im aware that I presented the programming solution more efficiently I had difficulty in applying the knowledge in the context of the problem I had difficulties in my reasoning I couldnt reach an initial settlement proposal I wasnt able to understand what was meant There were steps which I couldnt perform

R3, R6, R8, R10 R3, R10 R1, R4, R6 R1, R2, R5, R6, R7, R8, R10 R1, R6, R8 R1, R2, R3, R6, R7 R1 R1, R7 R5 R3 R3, R4, R5, R10 R10 R6

things they did not know. However, both R4 and R10, which presented more weaknesses at the level of acquired key competences, considered to change their preparation way by making more exercises; 60%, concerning R1, R2, R3, R4, R7 and R10, considered rewarding the effort made, since they liked programming, with the exception of R2. On the reflection on the aspects they would change on the way they prepared themselves, R3 referred he/she would study more deeply certain parts of the matter, and R7 that he/she would have made more exercises; 30%, referring to R1, R3 and R7, defended that their effort paid off, since they got a better grade than the one they would have gotten if they were not prepared, despite the fact that R3 did not presented signs of a good grade; 40%, concerning R2, R5, R6 and R8, considered the effort and action in vain, since they had a grade lower than they expected. On the reflections of R5 and R8 on what they would change in the way they prepare themselves was also valorised the effort trough making more exercises, while R6 would have paid more attention to the questionnaires in the academic part of the subject; At last, also R9 did not felt any advantages on the effort, considering the lack of time, however referred that he/she would have made more exercises.

In the last group of the examination, on a typical problem of programming contests, who stood up by positive was R1, for the evidence of deeper understanding of the matter and its application in the context of the problem presented, despite not having clearly demonstrated suitability for the selection of the more efficiently programming solution for the original problem. This is a core competence to be developed by all students of this case study, as well as in the previous study was found that students often tend to commit precision faults on programming. R7 stood in a significant distance from R1, followed by R2 and R6. In this group, as in others, the respondent who had more difficulties was R10, followed by R3, R4 and R5 who also showed reduced ability to apply knowledge to new situations. So, there is no wonder that the more superficial approaches to the study lead students, whenever asked differently, to difficulties in understanding the meaning of the problems. In the exam, in general, the one who stood up with the largest number of core competencies, to maintain, and with self-motivation to develop the matter, was clearly R1, followed by R7 and then R2 and R6, although the latter two have presented some weaknesses in the 1st group. The respondent with more difficulties in terms of the various groups was clearly R10, followed by R3, R4 and R5, while the latter three, with surface trends, present skills to keep on the questionnaire and R3 also in the 2nd and 3rd groups. 4) Post-exam reflection over key aspects to maintain or improve, or if the effort was worth it After the reflections over how study habits have influenced exam performance, the respondents reflected over key aspects to maintain or improve in the way they prepared themselves. It was verified the following: 20%, referring to R1 and R2, defend the way they prepare themselves, since they consider to have obtained a good result, despite R2 admitted that the effort did not paid for, since he/she obtained a grade below his/her expectations. On the opposite, R1, R4 and R10 considered their effort rewarding, since they learned

In this case we can conclude that a significant percentage of the respondents assumed the responsibility for bad study habits, caused not by the lack of study hours, but for the lack of a deep study on matters that require more than superficial knowledge. In this case they all agreed with an exam within reach for most the respondents, and that covered the matters of the theory and practical classes of the subject. III. CONCLUSION

The article presents a set of results that may not necessarily ensure the representation of the reality of what is happening in Computer Science Degrees in Portugal, but that are nonetheless significant. The study reveals that, despite concerns expressed by respondents for personal achievement, through the attainment of academic and professional success, and autonomy and independence of families, besides dispositions favourable for the study, only one part, 21% of students of the third subject of programming in CSP degree of University of Algarve has achieved competencies and skills necessary for writing programs with the required in-depth understanding, which requires more than a superficial knowledge, for full success in this subject. The exercise of self-reflection done by these students on their study and learning habits through BLearning, supported by an automatic judge, is also reflected in skills of self-knowledge that are part of the experiments and experiences in college, being therefore key skills for the current knowledge society. It is important that the student is aware of both the critical points for the development of high-level skills or strengths to decide how to manage the academic path and thus ensure a better preparation to face challenges and goals. Interesting to note that, if a student considers lack of effort as one of his/hers critical points, this reading will favour him/her,

since he/she will work harder. Nevertheless, if he/she considers lack of appetence for programming, which difficultly would be improved, will feel that he/she will not be able to control it and will have less chances to succeed. Now, we present the competences of the more successful respondents of this case study: a) self-motivated to carry out subject activities, to formulate their own questions and to work independently, b) lack of self-motivated by obtaining minimum score, only to approval and rescheduling of activities, b) self-disciplined and rigorous in planning and implementing activities, as well as in meeting the deadlines, c) with habits and effective study strategies, d) with ability to program, e) with handling capabilities of the technology available on the platform, f) with a desire to learn and participate in the forums or in the classroom, g) with the ability to easily communicate through the technology available on the platform or in person. REFERENCES
[1] F. Sustelo, P. Guerreiro, Estratgias de combate das fragilidades e contrariedades comuns nos estudantes que ingressam num curso de informtica do Ensino Susperior 2009. Actas da 4 Conferncia Ibrica de Sistemas e Tecnologias de Informao , 17-20 Junho 2009, Pvoa de Varzim, Portugal. (Em Livro, pp. 601-608. ISBN: 978-989-96247-0-2). P. Ramsden, Learning to teach in Higher Education, 1992, London, Routledge Press. http://ultibase.rmit.edu.au/Articles/june97/learn1.htm Bloom, Benjamin S. & David R. Krathwohl. (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. Handbook 1: Cognitive domain. New York , Longmans. Pohl, Michael. (2000). Learning to Think, Thinking to Learn: Models and Strategies to Develop a Classroom Culture of Thinking. Cheltenham, Vic.: Hawker Brownlow ctp.di.fct.unl.pt/~pg/docs/telegramas_s.pdf T. Jenkins, On the difficulty of learning to program, in Proc. of the 3rd Annu. LTSN_ICS Conf., Loughborough University, United Kingdom, August 2002, pp. 53-58.
Gomes, A. and Mendes, A. , "Learning to program difficulties and solutions", International Conference on Engineering Education, September 2007

[2] [3]

[4]

[5] [6]

[7] [8]

Edsger W. Dijkstra. On the Cruelty of Really Teaching Computing Science. Comm. ACM, Vol.32, pp 1398-1404, 1989. http://www.cs.utexas.edu/users/EWD/ewd10xx/EWD1036.PDF [9] Jirarat Sitthiworachart and Mike Joy, "Using Web-based Peer Assessment in Fostering Deep Learning in Computer Programming", Proceedings of the International Conference on Education and Information Systems, Technologies and Applications (EISTA 2004), Orlando, USA, pp. 231-236, 2004. http://www2.warwick.ac.uk/fac/sci/dcs/research/edtech/publications/sitt hiworachart04b.pdf. [10] Campbell, Teaching Strategies to Foster Deep Versus Surface Learning, http:// www.uottawa.ca/academic/cut/options/Nov_98/TeachingStrategies_en.h tm, 2004 [11] http://spreadsheets.google.com/viewform?formkey=cjg5dEtlek04SDJ6 WEVMYzd2R281THc6MA.. [12] http://spreadsheets.google.com/viewform?formkey=dHRUUEJVYVI0U 1FDM0x5UVlwUE1qQUE6MA..