I. INTRODUCTION
Asking questions is considered to be an essential element of human communicative behavior that is propelled by curiosity and the need to acquire more knowledge about a given situation; feedback, then, can be an important aspect of the questioning process in a given communicative situation as it-feedback- may dictate modification and adjustment to the structure and content of the posed questions. In the teaching and learning processes, questions and questioning are inevitable because, generally, educators are supposed to seek feedback, from the part of their students and leaners about the has been learning situation, over a span of time; feedback, in this educational sense, enables those educators to make judgments in regard to students and learners' achievement as well as about the teaching practices that have been followed in a given educational situation. As a result, questions, of a given exam or a test, constitute valuable tools to cast feedback that is highly needed to get a formidable view about how far the teaching and the learning processes have achieved what they are supposed to achieve of educational objectives and goals; moreover, and in accordance with previous related literature, it can be stated that students, learners, and exam-takers' comprehension capacity correlates with the specifications of the questions of a given exam, and, as a result, then, their level of achievement in that given exam.
Accordingly, and educationally speaking, it can be proposed that the structure, the language, and, more importantly, the specifications of an exam question correlate with the type, the amount of, and the quality of the feedback that is acquired, in reverse, through that given exam, and also the comprehension capacity and the level of achievement; in other words, furthermore, how much effort is exerted towards producing exam questions that are well-anchored to a given taxonomy of specifications, how fruitful the feedback, comprehension capacity, and the level of achievement would be, and how reliable and valid the exam, as a whole, would also be.
Depending on these premises, this study investigates the status and specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University- Kadoorie (PTUK)\Tulkarm campus; this study considers questions of an achievement exam as a mere educational communicative act in-between both teachers and students or learners; in this regard, this study also proposes that exam questions, when prepared and written, should be referenced to a well-established taxonomy, i.e. Blooms' cognitive taxonomy, so as to yield desired results that can offer guidance towards developing and elevating the processes of teaching and learning.
Although this study bears the limitation of restricting research to the teaching staff-members of Palestine Technical University – Kadoorie (PTUK), its investigative magnitude transcends its boundaries in the sense that, as far as the researchers have known, this is the first study to investigate such a topic, and that the educational nature of the topic can be generalized to other educational situations and environments. To achieve the objective of this study, the researchers followed a quantitative approach in order to investigate and reveal the teaching staff-members' points of view through deciding those highly considered maxims and specifications they- the teaching staff-members- abide by when writing or constructing achievement exam questions, and, then, as a result, drawing a line of understanding in regard to the achievement exams status and the specifications of its questions.
II. STATEMENT OF THE PROBLEM
Students, as proved by some of the available literature, prefer exam questions that are tailored to include low thinking-skills and correspond with lower cognitive domains; as a result, and as far as The researchers have known, some teachers tend to abide by this preference while; consciously or subconsciously; overlooking a wide risk margin of producing exam questions that are not anchored to a reliable criteria or a well-established cognitive taxonomy, and, eventually, passing unreal judgments in regard to the exam-takers' level of achievement, a misleading feedback about the learning continuum, and, then, deeply jeopardizing the reliability and validity maxims that any assessment process must go with, meet, and establish; breaching these maxims is totally risky as it yields an incomplete teaching and learning processes, fake judgments that do not mirror the reality of the teaching and learning processes, and, then, a hindering realm towards developing and elevating both processes. The status and the specifications of the questions of an achievement exams, in PTUK and, may be, in a considerable number of universities and colleges all over the world, and as far as this study proposes, are anchored to a set of professional traditions and heritage rather than a referenced taxonomies or specification matrixes; moreover, depending on the researchers' own experience and mere observation in (PTUK), the specifications of the questions of an achievement exam are thought about as an effort-consuming and as a secondary requirement that can be dealt with by individual teachers who normally refer to a set of professional heritage and imitation.
III. THE SIGNIFICANCE OF THIS STUDY
One aspect of the significance of this study is that it can offer, through its empirical approach, a wider sphere of understanding towards the essentiality of the specifications of the questions of an achievement exam to the educational process as a whole; this study, as far as the researchers have experienced in the field, can construct practical definitions, in a unified and connected matrix, to the specifications of the questions of an achievement exam which were, as far as the researchers have known, never surpassed by related research; Another facet of the significance of this study is that its investigative scope can shed more light on the status and the specifications of the questions of an achievement exam as perceived in the minds of the teaching-staff members of (PTUK) as this kind of perception, it is proposed, orients the process of producing and writing exam questions; considering the perceiving mentality, which acts behind the scene, is totally significant for further development and advancement of the teaching and learning processes.
IV. LITERATURE REVIEW
To support the argument, proposal, and questions of this study, the researchers scrutinized an adequate amount of the available related literature which was casted, in this section, in a relative chronological order; to go with the aim of this study, this literature review was divided into three sections: The first section presents the scope of importance and usage of questions in a communicative situation as well as the educational situations in the form of the questions of the achievement exam; the second section highlights the centrality of (Bloom's Taxonomy) as a reference matrix to the classification and the categorization of the questions of a given educational achievement exam; while the third section classifies some of the most common established specifications of the questions of an achievement exam which were rendered in a unified and more digestible exhibit that is somehow different from the scattered rending within the available previous literature; moreover, each section of the this literature review was summed up by a discretionary epitome to distill and unify the argument of each meant section.
a) The
Swart (2009, as cited in Jayakodi, Bandara, Perera & Meedeniya, 2016), confirm that "While questions can be given throughout the course, mid semester and the end semester exam questions often carry a considerable weight for the overall assessment." Swart (2009) also adds that "Final examination papers are used by academics to assess the retention and application skills of students."Demir & Eryaman (2012) confirm that "It is not surprising to discover that the evaluation of the students' learning with low cognitive level questions in primary and secondary schools as well as in higher education institutions is a common assessment strategy. "Demir & Eryaman, (2012) also add that "Both the teachers in primary and secondary schools and the university instructors tend to check whether the students memorize the decontextualized information by using semester exams and they do not force the student enough to critically analyze, synthesize and evaluate what they have learnt because of the low cognitive level questions in the exams. "Köksal & Ulum (2018) explain that "the questions presented on a paper determine whether the examination manages assessing the learners' performance or not. "Smith, Brown & Race (1996, as cited in Schneider, 2017), state that "Different types of test questions, per standard test design protocol, were used to allow students to have opportunities to express their differential test taking abilities. "McMillan (1997, as cited in Swart, 2009), concludes that "Questions are an essential component of effective instruction." Özden (1999, as cited in Demir & Eryaman, 2012), confirms that "People need questions in order to use one's life. "Sadker (2002, as cited in Demir & Eryaman, 2012) states that" Questions can and have been used for a wide variety of educational purposes: reviewing previously read or studied material; diagnosing student abilities, preferences, and attitudes; stimulating critical thinking; managing student behavior; probing student thought process; stirring creative thinking; personalizing the curriculum; motivating students; and assessing student knowledge. "Çepni (2003, as cited in Jayakodi, Bandara, Perera & Meedeniya, 2016), states that "Often the exam questions used to assess the level of the university students are at low cognitive levels. "Hussain (2003, as cited in Demir & Eryaman, 2012) concludes that "in the realm of teaching and learning, questions have been cited as not only the most often used, but also the single most important strategy used by instructors." Dillon (2004, as cited in Swart, 2009), points out that "Questions, effectively delivered, facilitate student learning and thinking, as they serve to motivate and focus student attention, provide opportunities for practice and rehearsal, and provide the opportunity for academics to assess how well students are mastering content. "Dillon (2004; Chin, 2004, as cited in Swart (2009) concludes that "The art of skillful questioning is a key to productive discussion by engaging students in higher order thinking. "Chin (2004, as cited in Swart, 2009), explains that there is a need for "significantly more complex thinking questions that can stimulate a student's mental activities. "Ali (2005) concludes that "The quality of asked questions on exams contributes developing creativeness of students and their criticism ability." Gürses, Bayrak, Bozoğlu, Açıkyıldız, Doğar & Özkan (2005, as cited in Demir & Eryaman, 2012) confirm that "questioning is the most essential step for the activity of thinking. "Gürses, Bayrak, Bozoğlu, Açıkyıldız, Doğar & Özkan (2005, as cited in Demir & Eryaman, 2012), also add that "Questioning can be admitted as a tactic that activates thinking. "Swart (2009) states that "Questions are used to obtain information, stimulate thinking, and redirect reasoning. Academics in higher education use questions on a daily basis to stimulate thinking and reasoning in students." Swart (2009) also adds that "Questions will remain as essential components of effective instruction. How these questions are formulated infi nal examination papers will depend much on the respective academics." Marquardt (2011, as cited in Swart, 2009), indicates that "Adults as well as children make use of questions to seek information or to gain a better understanding of the world in which they live. "Demir & Eryaman (2012) propose that "Whatever methodology is used, it is unquestionable the effect of the questions organized in fostering the critical thinking abilities. "Demir & Eryaman (2012) also add that "The exams including questions with a high level thinking skills can be used as well as an assessment tool and a teaching material. "Omar, Haris, Hassa, Arshad, Rahmat, Zainal & Zulkifli (2012), as cited in Köksal & Ulum, 2018), indicate that "A question is an element that is intertwined with the exam. Questions raised in exams play an important role to test the students' overall cognitive levels. "Chandio, Pandhiani & Iqbal (2016) indicate that" Examination system may be used as a powerful means of reforming teaching-learning process. "Jayakod, Bandara, Perera & Meedeniya (2016), state that "Exam questions are the main form of assessment used in learning. "Jayakod, Bandara, Perera & Meedeniya (2016) add that" Through the art of thoughtful questioning teachers can extract not only factual information, but also help learners in connecting concepts, making inferences, increasing awareness, encouraging creative and constructive thoughts."
To sum up, the abovementioned review considered questioning as a normal aspect of human life, and that asking questions is normally oriented towards meeting a human instinct and curiosity of trying to get more information and feedback which molds the modification and the accommodation processes of the communicative situation. The previous review also confirmed that questions, throughout the process of teaching and learning, occupy a spacious room of importance as questions, generally, constitute an inescapable means of communication between teachers at one end and students or learners at the other end; questions in this educational sense are inevitable when assessing and measuring: how much learning has occurred over a period of time, how much achievement has been reached, the type and quality of the educational performance, the desired skills, the comprehension magnitude, the knowledge depth, the levels of competency, and the clout in test-taking skills. The previous review considered that questions are normally connected with various educational purposes such as diagnosing problems within the learners' learning repertoire, reviewing the has been studied material, stimulating students' and learners' thinking, classroom management, activating classroom motivation and reaction, and orienting the learning processes. The abovementioned review connected questions, of a given achievement exam, with effective instruction methodologies as they are considered to be a very common strategy that is enhanced by teachers and educators so as to get better understanding of the learners' progress.
b) Bloom's Taxonomy and the classification and categorization of exam questions
Ali (2005) confirms that "The most common criteria used when analyzing the instructional objectives and questions is Bloom's taxonomy that is developed by Benjamin Bloom and known as by his name (Bloom taxonomy (BT))." Lord & Baviskar (2007) propose that "Developing questions based on Bloom's hierarchy would be a productive way of reversing the dangerous trend of graduating college students with a large number of misunderstandings in courses they have taken. Chang & Chung (2009, as cited in Abduljabbar & Omar, 2015)" also applied Bloom's taxonomy to evaluate and classify English question item's cognition level. "Omar, Haris, Hassan, Arshad, Rahmat, Zainal & Zulkifli (2012) indicate that "Normally, academicians would categorize a question according to the Bloom's cognitive level manually. "Abduljabbar & Omar (2015) concludes that "the Bloom's Taxonomy has become a common reference for the teaching and learning process used as a guide for the production of exam." Abduljabbar & Omar (2015) also states that "Many studies have sought to automatically classify exam questions based on Bloom's taxonomy. Furthermore, although limited, research has addressed the use of Natural Language Processing (NLP) techniques to resolve this problem. "Chandio, Pandhiani & Iqbal (2016); Stanny (2016), propose that" Bloom's taxonomy guides the development of test questions to assess higher-level thinking skills by drawing attention to what test questions and assessment prompts require students to do (retrieve facts, apply knowledge, make a prediction, solve a problem, or evaluate a theory)." Stanny (2016) further explains that "Bloom argued that teachers who write objective exam questions can use the taxonomy to determine whether questions require only a superficial knowledge. "Köksal & Ulum (2018) state that "Based on the findings, some assumptions have been made with the aim of suggesting how the exam papers which are being written or will be written should refer to Bloom's taxonomy. "van Hoeij, Haarhuis, Wierstra & van Beukelen (2004, as cited in Abduljabbar & Omar, 2015) "developed a classification-based tool that uses Bloom's anatomy to evaluate the cognitive level of short essay questions." Abduljabbar & Omar (2015), add that "To overcome the problem of exam question classification with a more effective solution, this study proposes a combination model which combines three machine learning approaches using a combination voting algorithm adopted to classify question items to agree with Bloom's cognitive levels." Yüksel (2007, as cited in Demir & Eryaman, 2012) classifies an "alternative categorizations based on Bloom's Taxonomy (1956) aims to formulate the Bloom's Taxonomy as truer and accurate. Some of the alternative categorizations propound against Bloom's Taxonomy are listed as follows: Categorization of Gerlach and Sullivan, Categorization of De Block, Categorization of Tuckman, Categorization of Williams, Categorization of Hannah and Michaelis, Categorization of Gagné and Briggs, Categorization of Stahl and Murphy, Categorization of Romizowski, Categorization of Quellmalz and Categorization of Haladayna. "Chang & Chung (2009, as cited in Yusof & Hui, 2010); Omar, Haris, Hassan, Arshad, Rahmat, Zainal & Zulkifli (2012) "presented an online test system to classify and analyze the cognitive level of Bloom's taxonomy to English questions. The system accepts the exam question as an input, which will then be segmented. This system has a database where various verbs of Bloom's taxonomy are stored. The database includes verbs with lower-case and capital letters. The system then compares all the verb tenses present in the questions. When a keyword is found in the test item, then the particular question belongs to the keyword. "Haris & Omar (2012, as cited in Abduljabbar & Omar, 2015)" employed a rule- based approach for question classification using Bloom's taxonomy in NLP. A rule-based approach evaluates and classifies written examination questions for computer subjects." recommended that "a study needs to be conducted to analyze the prevailing examination system through Bloom's Taxonomy." Jayakodi, Bandara, Perera & Meedeniya (2016) indicate that "Bloom's taxonomy of learning outcomes has been applied to classify the exam questions." Bloom (1956), as cited in Aviles (1999) indicates that "Knowledge is probably the most common level tested in higher education because instructors can simply use a textbook to determine what "knowledge" students must learn and create exams targeted to a textbooks' contents. "Chandio, Pandhiani & Iqbal, (2016) explain that all" The questions asked in these papers are classified and analyzed from the vintage point of Bloom's Taxonomy to determine whether the present assessment system focuses on the lower degrees of learning like remembering, understanding, applying or it transcends to the higher degrees such as analyzing, evaluating and creating." Chandio, Pandhiani & Iqbal, (2016) further explain that "It can be concluded that Pakistan's secondary boards need a paradigm shift where there is a dire need of expert and experienced examiners to induct more questions catering to the higher order thinking skills of Bloom's Taxonomy while setting examination papers. Also, more time should be given to the examiners and it should be made sure that the questions are not repeated. "Köksal & Ulum (2018) conclude that "the analyzed exam papers lacked the higher level cognitive skills contained in Bloom's Taxonomy." Köksal & Ulum (2018) also assert that "the exam questions include only knowledge and comprehension levels of Bloom's taxonomy. That's to say, according to Table 1, the exam questions are based on the lower order cognition levels of Bloom's taxonomy while they lack the higher order cognition levels. The percentage of knowledge level contained in the exam questions is while it is for the comprehension level." Lawson (1990, as cited in Lord & Baviskar, 2007), concludes that "thinking comes together as a continuum in the upper segments of Bloom's levels." Lawson (1990, as cited in Lord & Baviskar, 2007) further explains that "in bright individuals, analysis often serves to order and structure a problem. After this, synthesis is employed to generate solutions, and evaluation assesses the suggested solutions against the objectives identified in the analysis phase. "Jayakodi, Bandara, Perera & Meedeniya (2016) elaborate that" Developing questions based on Bloom's hierarchy would be a productive way of ensuring the expected quality of student learning achievement. "Anees, S. (2017: 10), concluded that " question was prepared without considering the cognitive levels of Bloom's Taxonomy which directly affect students' performance. Anees (2017) further explains that "teachers should use different software to find out their levels of questions after making question papers, to make a balanced question paper which evaluate the whole performance of students and contain on all cognitive levels of Bloom's Taxonomy." Azar (1998, as cited in Ali, 2005), concludes that "the teachers in secondary schools don't have experience of asking questions by considering the Bloom's Taxonomy." Ali (2005) accordingly adds that "teachers should have taken courses on measuring and assessing students' achievement by considering the cognitive levels of Bloom's Taxonomy." Ali (2005) also elaborates that "student teachers at education faculties should also take courses about what Bloom's Taxonomy is and how they should consider it while preparing exam questions."
To sum up, the above mentioned review acknowledged the prevalence, centrality, reliability, and dependability of Bloom's Taxonomy (1956) to calibrate the questions of an achievement exam; the cognitive domains of this taxonomy suit the varying levels of learners as each domain is subdivided into specific indicative behavioral verbs; the previous review stated that even those alternative models and taxonomies, which have been developed in the field, were referenced, in one way or another, to Bloom's taxonomy. The previous review proposed that exam questions, whenever prepared and written in accordance with Bloom's Taxonomy, they can lead to credibility, reliability, and validity in the learners' responses. The above mentioned review professedly revealed that teachers, generally, don't have enough experience in exploiting Bloom's Taxonomy when they usually prepare and write an educational exam questions; the mentioned review above also revealed that the domain of "knowledge", which represents the lowest cognitive level within Bloom's taxonomy, is the mostly referenced by teachers and educators when writing and preparing exam questions. As a result, the bulk of the previous review confirmed the need for training courses and workshops on how to make use of Bloom's Taxonomy when writing formidable, effective, valid, and reliable educational questions for a given achievement exam.
c) The Specifications of questions of an achievement exam
Smith, Brown & Race (1996, as cited in Schneider, 2017) explain that "More precisely, the completion questions used in the study were part of a larger examination that included short-answer and essay questions that were designed to elicit critical thinking, as well as true-false and multiple-choice questions." Blank-Libra (1997, cited in Gall 1984), as cited in Demir & Eryaman, 2012) "provides evidence to support the notion that higher-level questions will provoke higher-level responses from students. The same principle, of course, applies to lower-level questions" Brualdi Timmins (1998), as cited in Köksa & Ulum, (2018), proposed that "Instructors who prepare exams to improve students' high order cognitive skills promote interaction between themselves and their students." Çepni & Azar (1998, as cited in Ali, 2005) postulate that "students might be at difference cognitive levels." Aviles (1999) proposes that "Creating comprehension questions are more difficult than creating knowledge questions because words or phrases cannot simply be removed from a sentence and hidden among multiple-choices". Popham (1999, as cited in Swart, 2009) confirms that "questions must not be opaque and ambiguous by nature and must not contain complex syntax, difficult vocabulary, or unintended clues." Leeds (2000, as cited in Swart, 2009) indicate that "Effective questions include problem-solving or informational questions." (Leeds (2000); Black, Harrison & Lee (2003); Chin (2004); Jones, Harland, Reid & Bartlett (2009, as cited in Köksal, & Ulum, 2018), confirm that "Efficient exam questions should cover various difficulty levels to refer to the different capabilities of learners." Piaget (2001; Bruner 1960, as cited in Schneider, 2017) "argue that assist devices on exams facilitate the interaction of the test taker with key exam elements so that they may better construct their understanding of test questions. Arguably then, use of such devices should actually improve test validity." Hand, Prain & Wallace (2002, as cited in Ali, 2005) "showed that students prefer low-order questions and don't prefer questions which need to be thought on." Black, Harrison & Lee (2003, as cited in Swart, 2009) state that "Effectual questions must help to raise issues on which academics need feedback or about which the students need to think." Lundberg (2004, as cited in Swart, 2009) further explains that "Short answers or multiple-choice questions requiring mainly factual recall tend to elicit surface learning, while essays (or long-answer questions) are more likely to encourage deep learning. "Ali (2005) concludes that "Teachers are in need of preparing questions which develop students' scientific thinking." Ali (2005) further elaborates "that teachers should prepare questions together and they should pay attention for choosing questions from every step of cognitive levels." Andrade (2005, as cited in Balch, Blanck & Balch, 2016) concludes that "a rubric provides feedback which in turn provides clear and individually focused diagnostic feedback." Lord & Baviskar (2007) confirms that "It is generally believed by the test creator that, while short-answer and multiple-choice questions can be used efficiently to test the lower levels of learning behaviors, they are not sufficient to assess the higher levels." Thompson, Luxton-Reilly, Whalley & Robbins (2008) confirm that "During the analysis of the examinations, we found examples of questions that could be reworded in such a way that the cognitive level is altered." Jones, Harland, Reid & Bartlett (2009, as cited in Köksal & Ulum, 2018), propose that "A good assessment requires an exam paper that covers different cognitive levels to accommodate diverse capabilities of learners." Swart (2009) confirms that "academics must acquire the art of skillful questioning if they are to produce effective questions that will engage students in higher order cognitive processes such as problem-solving and critical thinking." Swart (2009, as cited in Jayakodi, Bandara, Perera, & Meedeniya, 2016) by the same token, adds that "When questions are prepared, there should be an effective balance between questions that assess the high level of learning and questions that assess the basic level of learning." Swart (2009) accordingly, further explains that "The number of multiple-choice questions (38% on average in the Knowledge objective) used in these examination papers further suggests that surface learning is being promoted. "Marquardt (2011, as cited in Swart, 2009) concludes that" the quality of the questions often depends on the nature of the topic. For example, children frequently ask questions that may merit a simple "yes" or "no" reply (closed-ended questions, according to Marquardt." Marquardt (2011, as cited in Swart, 2009) further explains that "Critical thinking is promoted through open-ended questions." Demir & Eryaman (2012) conclude that "It is necessary to ask high cognitive level questions to enable prospective student teachers to think in a multifaceted way. Therefore, they can avoid the tendency of superficial thinking that they get used to by answering cognitive level questions." Demir & Eryaman (2012) further adds that "The questions given in the exams by the instructors reflect the objectives, goals, outputs and the methodologies that the instructors apply in their teaching." Omar, Haris, Hassan, Arshad, Rahmat, Zainal & Zulkifli (2012, as cited in Köksal & Ulum, 2018) state that "Although a list of assessment types are available, a written exam is the most employed tool chosen by academic institutions." Freahat & Smadi (2014, as cited in Köksal & Ulum, 2018) confirm that "While low level cognitive questions increase the acquisition of the accurate knowledge and pave the way for acquiring high-cognitive skills, high level questions are practical tools for prompting thinking and improving other cognitive skills like problem solving and decision making." Paul, Naik & Pawar (2014, as cited in Köksal & Ulum, 2018) confirm that "choosing the right question is obviously the most difficult part of forming the exam paper, in addition to being the most time taking activity." Abduljabbar & Omar (2015), point out that "the process of questions writing is very challenging step for the lecturer. The situation is getting more challenging when lecturers try to produce good quality and fair questions to assess different level of cognitive." Abduljabbar & Omar (2015), further add that "the question must be provided in accordance with the subject content learned by students to fulfill learning objectives." Balch, Blanck & Balch (2016) reasoned that "as long as tools such as rubrics are incorporated, the student and the teacher will produce the optimum learning experience. The reward will be mutual." Chandio, Pandhiani & Iqbal (2016) conclude that "if questions are repeated in examinations, which is a very dangerous trend as it gives rise to rote learning. The reason being, that even if the questions belong to higher order thinking domain and are repeated, the repetition will cause students to memorize the answers to such questions." Jayakodi, Bandara, Perera, & Meedeniya (2016), state that "An exam question often falls into more than one level of assessment categories of a given taxonomy." Anees (2017) concludes that "The best ranked university should provide the good questions' criteria." Anees (2017), further explains that "the young teachers should be trained in preparing high order questions." Köksal & Ulum (2018) indicate that "While providing suitable exam questions at schools, composing the proper ones may be a problematic issue."
To sum up, the previous review presented a multi-towered consideration to the specifications of exam questions: it was agreed upon that composing good quality and effective exam questions is not an easy task and careful attention must be paid and exerted; those involved in this process should be trained on the procedure as it is both a skill as well as an art that can be acquired. The previous review stated that composing good quality exams should take into consideration the students' individual differences, and that students normally do not prefer questions that need higher- levels of thinking, and usually they – students - exhibit a tendency of preference towards lower-cognitive questions. As a result, those who are tasked to prepare exam questions should consider a set of maxims: first, questions should not be prepared individually, but,on the contrary, be prepared as a team product; second, questions should be prepared in accordance with a strict balance between questions that assess high-levels of learning outcomes and those that assess basic-levels of the learning outcomes; third, the process of preparing exam questions should incubate the orientation of developing students various cognitive skills and competences; fourth, the previous review acknowledged that rubrics play a double role as they provide precious feedback about students' comprehension skills in regard to the language and context of a given rubric, in addition to their traditional role of facilitating the process of comprehending the required response from a given question; as a result, incorporating clear and well-structured rubrics yields the optimal outcome of the learning experience; fifth, the stage of preparing exam questions should consider that composing fair and good quality exam questions, which can assess various cognitive levels, is difficult, problematic, demanding and challenging, time- consuming, and absolutely not an easy task. The previous review acknowledged that the best ranked universities provide academics with special training on the skills of preparing and composing exam questions.
As stated by the previous review, the process of preparing effective questions should be functioning within the scope of the following banners: first, it is generally believed that short-answer and multiple-choice questions can be most appropriate to assess the lower levels of learning behaviors, but, on the other hand, they are not sufficient to assess higher cognitive levels; second, the quality of the exam question correlates with and, at the same time, reflects the quality of the exam as a whole; third, it should be born that preparing exam questions should be steered towards elevating the exam takers' cognitive skills which will also result in promoting an atmosphere of interaction and some kind of a telling encounter between those exam takers and their teachers; fifth, composing multiple – choice questions is easier than composing comprehension questions, and that an exam with an all multiple-choice format confirms that surface learning is promoted and encouraged; sixth, exam questions should reflect the objectives, goals, outputs and the methodologies that the instructors apply in their teaching; seventh, the language and wording of an exam question should be carefully carried out so as not to alter the cognitive level of the outcome meant to be elicited from the question; eighth, exam questions should not be confined to just one level of assessment of the reference taxonomy, but on the contrary be distributed and balanced to engage students in higher order cognitive processes such as problem-solving and critical thinking; ninth, it is acknowledged that a written exam is the most enhanced method of assessment that is chosen by academic institutions; tenth, the previous review summarized some of the specifications that questions of a given exam should be attributed to:
Multiple-choice questions, which represent the knowledge domain, should not consume a wider or more percentage in comparison with other types of questions that represent other domains of the given taxonomy.
It is essential that the exam questions should cover different cognitive levels to integrate diverse capabilities of learners.
It is necessary to ask high cognitive level questions to enable prospective-student teachers to think in a multifaceted way.
Effectual questions must help to raise issues on which academics need feedback or about which the students need to think.
Effective questions include problem-solving or informational questions.
Efficient exam questions should cover various difficulty levels to refer to the different capabilities of learners.
Questions must not be opaque and ambiguous by nature and must not contain complex syntax, difficult vocabulary, or unintended clues.
Repeating the same questions in various sets of examinations is a very dangerous trend as it gives rise to rote learning, and that the repetition will cause students to memorize the answers to such questions.
Short answers or multiple-choice questions requiring mainly factual recall tend to elicit surface learning, while essays (or long-answer questions) are more likely to encourage deep learning.
Various Cognitive-level questions can immune students against the tendency of superficial thinking.
High level questions are practical tools for prompting thinking and improving other cognitive skills like problem solving and decision making.
V. METHODOLOGY AND DISCUSSION
To achieve the purpose of this study, the researchers followed a descriptive quantitative approach through which a 24-item questionnaire which was rewritten and recompiled in reference to the related literature and in accordance with the amendments asked for by some experts in the field; the reliability of the questionnaire was measured by calculating the internal consistency and the (Cronbach's alpha) factor which reached the value of (.936); this value confirmed that the questionnaire is valid as the instrument of this study; the questionnaire, then, was distributed to a sample of (140) subjects out the population of (355) teaching staff-members of (PTUK); table(1) reveals the attributes of the subjects of the sample:
| Percentage | Frequency | Variable Level | Variable |
| 55.7% | 78 | Male | Sex |
| 44.3% | 62 | Female | |
| 15.0% | 21 | Instructor | Academic Rank |
| 25.7% | 36 | Lecturer | |
| 40.0% | 56 | Assistant Professor | |
| 15.7% | 22 | Associate Professor | |
| 3.6% | 5 | Professor | |
| 32.1% | 31 | 1-5 years | Teaching Experience |
| 30.0% | 42 | 6-10 years | |
| 33.6% | 47 | 11-16 years | |
| 14.3% | 20 | More than 16 years | |
| 14.3% | 20 | Engineering and Technology | College |
| % 20.7 | 29 | Commerce | |
| % 18.6 | 26 | Arts and Educational Sciences. | |
| % 4.3 | 6 | Science and Agriculture Technology | |
| % 21.4 | 30 | Applied Sciences. | |
| % 20.7 | 29 | Technical College. |
| Mean | Response Scale |
| (4.2) and more | Very high. |
| (3.40-4.19) | High. |
| (2.60 – 3.39) | Medium. |
| (2.59 – 1.8) | Little. |
| Less than (1.8) | Very Little. |
To answer the main question of this study, the means and the standard deviation of the responses of the teaching staff-members of Palestine Technical University-Kadoorie (PTUK), for the questionnaire items, in regard to main question of this "the status and the specifications of questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK)."; table (3) shows the results:
| Deviation | Mean | Item | Questionnaire Numbering. |
| 0.83 | 4.15 | I compose the questions of the exam distributed between objective questions and essay questions. | 1 |
| 1.18 | 3.49 | I compose the questions of the exam while including all types of objective questions. | 2 |
| 1.14 | 3.77 | When I compose the questions of an exam, I make sure that objective questions do not exceed 25% | 3 |
| 1.30 | 3.15 | When I compose the questions of the exam, I consider the answering time for each question. | 4 |
| 1.24 | 3.76 | I realize that time duration for both the first and second exams equals the duration of a normal lecture for each one of them. | 5 |
| 1.24 | 3.46 | I take into consideration that the questions of the exam should cover the studying material as a whole, and in line with the objectives and the course outline | 6 |
| 1.39 | 3.54 | I tend to compose various questions which measure various levels of achievement with no kind of repetition. | 7 |
| 1.32 | 3.26 | When I compose the questions of the exam, I make sure not repeating duplicate questions from previous exams. | 8 |
| 1.38 | 3.36 | I produce two forms of the same questions while changing the order of the questions. | 9 |
| 1.35 | 3.65 | When I compose the questions of the exam, I use a simple language that goes with the language I use in teaching. | 10 |
| 1.33 | 3.37 | I avoid composing a question that indirectly implies an answer to another question. | 11 |
| 1.41 | 3.37 | When I compose the questions of the exam, I make sure they ascend from the easy to difficult. | 12 |
| 1.46 | 3.40 | When I compose the questions of the exam, I make sure they cover various cognitive domains. | 13 |
| 1.30 | 3.44 | When I compose the questions of the exam, I make sure they go with the individual differences. | 14 |
| 1.34 | 3.36 | When I compose the questions of the exam, I make sure they contain extra elective questions. | 15 |
| 1.39 | 3.36 | When I compose the questions of the exam, I make sure that punctuation marks are correctly enhanced to give the exact intended meaning. | 16 |
| 1.28 | 3.46 | When I compose the questions of the exam, I phrase them in a simple, but a well structured language without spelling mistakes or typing errors. | 17 |
| 1.41 | 3.06 | I tend to leave enough answer space for each question when I write the exam questions. | 18 |
| 1.28 | 3.49 | I tend to discuss the questions of the exam with my colleagues who teach the same course | 19 |
| 1.40 | 3.21 | I compose the questions of the exam the night before the date of the exam. | 20 |
| 1.37 | 3.24 | I do realize the exam does have an educational domain, and not just and assessing one. | 21 |
| 1.35 | 3.35 | I do realize that the student can learn from his mistakes in the exam if granted the chance to review the exam paper. | 22 |
| 1.34 | 3.07 | I prefer computer-based exams to avoid the manual correcting and marking. | 23 |
| 1.48 | 3.23 | When I compose the exam questions, I consider that questions go with the cognitive domains of Bloom's Taxonomy. | 24 |
| 0.37 | 3.42 | Total Degree |
The above mentioned table (3) shows that the highest item in rank, with the mean of (4.15), was for the questionnaire item, "I compose the questions of the exam distributed between objective questions and essay questions." While the second in rank, with the mean of (3.77), was the questionnaire item "When I compose the questions of an exam, I make sure that objective questions do not exceed 25%." This result, the researchers believe, mirrors the past experience, and may be, negative attitudes of those teaching staff-members of (PTUK) towards objective questions, in general, and multiple-choice question specifically which mostly prevailed, in an online home-sent examining, during the pandemic situation of the Covid-19; this result, the researchers believe, also reflects the teaching staff-members' hankering to traditional types of questions such as essay questions in testing and examination, especially, it should be added, that this study is carried out during the first semester of the academic year 2021-2022 which signals the return to the face-face teaching and paper-based examination and testing.
Table (3) also shows that the lowest ranked item, with the mean of (3.06) has been "I tend to leave enough answer space for each question when I write the exam questions." This result, the researchers believe, depicted the fact that the process of stringing questions of the achievement exam, from the part of the surveyed sample of teaching staff-members in (PTUK), is still propelled by a teaching mentality and not a learning mentality; in other words, the exam is seen as a complementary tool for the teaching process, and not as a precious tool that has an important learning dimension.
Table (3), additionally, reveals that the item which came exactly before the lowest item in rank with a mean of (3.07) has been "I prefer computer-based exams to avoid the manual correcting and marking." This result, the researchers believe, does not reflect a positive attitude towards the format of computer-based exams, but rather to its procedural labor-free toll; this result, the researchers believe, also mirrors the fact that the teaching staff-members of (PTUK) view the achievement exam as a formality and not as a necessity that can be exploited to steer the whole learning and teaching processes.
a) Results of the hypotheses of this study
i. The first null-hypothesis: There is no significant difference when (α= 0.05) in the means of the teaching staff-members' responses, in regard to the main question of this study "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK), that are attributed to the variable of "sex"; to test this hypothesis, a (t-test) was conducted and the results has been revealed in table (4) which indicates that the value of (t) was (0.25), and the value of (P) was (0.81); these values tell that there was no significant statistical difference when (α= 0.05) which leads to accepting the first null- hypothesis. This result means that both males and females of the teaching staff-members of (PTUK) share a unified approach and attitude towards the status and specifications of the questions of an achievement exam; the researchers believe that the absence of organizing regulations in regard to the specifications of the questions of the achievement exam widely opens the gates of imitation from the part of female teaching staff-members to their fellow male teaching staff-members, especially that males outnumber the females nearly within all faculties.
| Significance Level | (T) | Degree of freedom | Standard deviation | Mean | Number | Sex |
| 0.81 | 0.25 | 138 | 0.37 | 3.41 | 87 | Male |
| 0.36 | 3.42 | 62 | Female |
ii. The second null hypothesis: There is no significant difference when (α= 0.05) in the means of the teaching staff-members' responses, in regard to the main question of this study "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK), that is attributed to the variable of "college"; to test this hypothesis, one-way analysis of variance (ANOVA) was conducted and the results has been revealed in table(5) and table (6) consecutively: table (5)reveals that there are apparent differences in the responses of the subjects of the sample of this study in regard to "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK), that are attributed to the variable of (college). "Table (6) reveals the results of the one-way analysis of variance (ANOVA) which was conducted to identify whether these apparent differences carry any statistical significance or not; as shown in table (6), it was confirmed that these apparent differences carry statistical differences as the value of (F) was (3.61), and the value of (P) was (0.004); these values led to refusing the second null hypothesis.
| Mean | Number | College |
| 3.25 | 20 | Engineering and Technology |
| 3.39 | 29 | Commerce |
| 3.57 | 26 | Arts and Educational Sciences. |
| 3.18 | 6 | Science and Agriculture Technology |
| 3.55 | 30 | Applied Sciences. |
| 3.33 | 29 | Technical College. |
| Statistical Significance (P) | Value of (F) | Mean of Squares | Degree of Freedom | Sum of Squares | Source of Variance |
| 0.004 | 3.611 | 0.448 | 5 | 2.24 | In-between groups |
| 0.124 | 63 | 16.645 | In-group | ||
| 66 | 18.887 | Sum |
Table (7) further shows, depending (LSD) test for post comparisons, that there is a significant statistical difference in-between Faculty of Engineering and Technology, Faculty of Arts and Educational Sciences, and the Faculty of Applied Sciences in favor for the Faculty of Arts and Educational Sciences; table (7) also shows that there is a significant statistical difference in-between the Faculty of Arts and Educational Sciences, Faculty of Agricultural Sciences and Technology, and Palestine Technical College in favor for the Faculty of Arts and Educational Sciences; in addition table(7) reveals that there is a significant statistical difference in-between Faculty of the Agricultural Sciences and Technology and the Faculty of Applied Sciences in favor for the Faculty of Applied Sciences; table (7) also reveals that there is a significant statistical difference in-between the Faculty of Applied Sciences and The Palestine Technical college in favor for the Faculty of Applied Sciences. The researchers believe that these results constitute a natural outcome to the fact that most of the teaching staff-members of the Faculty of Arts and Educational Sciences do have some kind of guiding educational knowledge that they can exploit when writing and preparing the questions of an achievement exam; by the same token, the researchers attribute the significant statistical difference in favor of the Faculty of Applied Sciences to the fact that both of the Faculty of Arts and Educational Sciences and the Faculty of Applied Sciences had been one single faculty under the name of Faculty of Arts and Applied Sciences; this fact, the researchers believe, mirrors another reality that the teaching staff-members of both faculties were, jointly and relatively, able to orchestrate some aspects of examination formats and requirements, and, in addition they were able to share related educational experience.
| Faculty of Engineering and Technology | Faculty of Business and Economics | Faculty of Arts and Educational Sciences | Faculty of Agricultural Sciences and Technology | Faculty of Agricultural Sciences and Technology | Palestine Technical College | |
| Faculty of Engineering and Technology | 0.13 | 0.32 * | 0.07 | 0.29 * | 0.07 | |
| Faculty of Business and Economics | - | 0.18 | 0.20 | 0.16 | 0.05 | |
| Faculty of Arts and Educational Sciences | - | - | 0.39 * | 0.02 | 0.24 * | |
| Faculty of Agricultural Sciences and Technology. | - | - | - | 0.37 * | 0.15 | |
| Faculty of Applied Sciences | - | - | - | - | 0.22 * | |
| Palestine Technical College | - | - | - | - | - |
iii. The third null hypothesis: There is no significant difference when in the means of the teaching staff-members' responses, in regard to the main question of this study "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK), that are attributed to the variable of "academic rank." To test this hypothesis, one-way (ANOVA) was applied, and table (8) reveals that there are apparent significant differences when in the means of the teaching staff-members' responses, in regard to the main question of this study "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK)" that are attributed to the variable of (academic rank); to test whether these apparent differences carry statistical significance, one-way (ANOVA-test) was conducted: Table (9) shows that the differences in the means of the responses, which are attributed to the variable of academic rank, have no statistical differences because and the value of which leads to accepting the third null hypothesis. This result reinforces the belief that the teaching staff-members of (PTUK), with their different academic ranks, view the achievement exam as an end that can be achieved with any set of questions or format; the researchers also believe that the absence of any instructional training yields some kind of commonality of improvisational tendency while preparing and writing the questions of an achievement exam.
| Mean | Number | Academic Rank |
| 3.51 | 21 | Teacher |
| 3.43 | 36 | Lecturer |
| 3.39 | 56 | Assistant Professor |
| 3.38 | 22 | Associate Professor |
| 3.32 | 5 | Professor |
| Statistical Significance (P) | Value of (F) | Mean of squares | Degree of freedom | Sum of squares | Source of Variance |
| 0.71 | 0.54 | 0.075 | 4 | 0.298 | In-between groups |
| 0.138 | 135 | 18.589 | In-group | ||
| 139 | 18.887 | Sum |
iv. The fourth null hypothesis: There is no significant difference when (α= 0.05) in the means of the teaching staff-members' responses, in regard to the main question of this study "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK), that is attributed to the variable of the "teaching experience"; to test this hypothesis, To test this hypothesis, one-way (ANOVA) was applied, and table (8) reveals that there are apparent significant differences when (α= 0.05) in the means of the teaching staff-members' responses, in regard to the main question of this study" The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of Palestine Technical University (PTUK) "that are attributed to the variable of (teaching experience)" table (10) reveals that there are apparent differences in the responses of the subjects of the sample of this study in regard to "The status and the specifications of the questions of an achievement exam from the points of view of the teaching staff-members of
Palestine Technical University (PTUK), that are attributed to the variable of (teaching experience)." to test whether these apparent differences carry statistical significance, one-way (ANOVA-test) was conducted: Table (11) shows that the differences in the means of the responses, which are attributed to the variable of (teaching experience), have no statistical differences because and the value of which leads to accepting the third null hypothesis. The researchers believe that this result reinforces the fact that "teaching experience" cannot be counted for with the maxim of how many years spent in teaching, but rather how much of those years were exploited in deducing new outlooks by digesting the amount of feedback accumulated throughout those years spent in teaching; this result, the researchers believe, is a logical outcome of the absence of formidable educational anchors and well-established taxonomies; imitating the traditional heritage in preparing and writing exam questions yields a hindering factor towards exploiting years of the teaching experience.
| Means | Number | Teaching Experience |
| 3.45 | 31 | Less than (6) years |
| 3.41 | 42 | 10years – 6 |
| 3.43 | 47 | 15years – 11 |
| 3.36 | 20 | 20years – 16 |
| Statistical Significance (P) | Value of (F) | Mean of squares | Degree of freedom | Sum of squares | Source of Variance |
| 0.83 | 0.28 | 0.039 | 3 | 0.117 | In-between groups |
| 0.138 | 136 | 18.770 | In-group | ||
| 139 | 18.887 | Sum |
VI. FINDINGS AND CONCLUSIONS
In accordance with its objectives, findings of this study indicate that the specifications of the questions of an achievement exam are not much considered whenever the teaching staff-members of (PTUK) usually prepare and write exam questions, and that the questions of an achievement exam are mostly prepared and written from a teaching mentality, which mostly imitates and enhances a traditional practice that considers an achievement exam as a complementary tool for the teaching process, and not as an essential tool for the learning process; findings of this study indicate that an achievement exam is also viewed, in itself, as an end with little consideration to the fact that it can be a valuable source of feedback that can provide significant insights into the teaching and learning processes. Findings of this study also indicate that: first, computer-based exams are relatively preferred by the teaching staff-members of (PTUK) because of, the researchers believe, and in reference to previous related literature, their labor-free correcting toll, and not because of the specifications of their questions, especially the multiple-choice format; second, depending on their responses, both of the females and males of the teaching staff-members of (PTUK) relatively exhibit a very close and unified approach, attitude, and practice towards the status and specifications of the questions of an achievement exam, especially in the absence of any guiding and referential taxonomy; third, the teaching staff-members of the Faculty of Arts and Educational Sciences, in comparison with other staff-members of other faculties in (PTUK), are most likely to exploit some of their educational background when writing and preparing the questions of an achievement exam, but, once again, this advantageous and the would be exploitable educational background should be harmonized with a clear tabulated list of specifications and a referential taxonomy; fourth, "teaching experience", as has been found within the boundaries of this study, cannot be calibrated by how many years spent in the teaching profession, but rather how much of those years were exploited in deducing new outlooks through digesting the amount of feedback accumulated throughout those years spent in teaching;
fifth, findings of this study indicate that imitating the traditional heritage whenever preparing and writing questions of an achievement exam represents a hindering factor towards exploiting years of the teaching experience, and results in a progress cul-de-sac; by the same token, conclusions of this study indicate that: first, the pandemic situation of Covid-19 may have negatively impacted the status and the specifications of the questions of an achievement exam in the sense that the teaching staff-members of (PTUK) exhibited some kind of avoidance to multiple-choice questions which were heavily enhanced during the pandemic situation of Covid-19; second, this study confirmed, as a conclusion, that the absence of a tabulated list of specifications to the questions of an achievement exam will widely open the door to speculations, improvisation, and imitation to the traditional heritage in preparing and writing the questions of an exam; third, the teaching staff-members of (PTUK) view the achievement exam as a formality and not as a necessity that can be exploited to steer the whole learning and teaching processes, and that the questions of an achievement exam are still propelled by a teaching mentality rather than a learning mentality, and that those questions are rarely anchored to a specific taxonomy or a tabulated list of specifications; fourth, this study confirmed, as a conclusion, that the absence of any instructional training yields some kind of commonality of improvisational tendency while preparing and writing the questions of an achievement exam; this commonality of improvisational tendency, deductively, yields unreliable judgments, and mirrors fake reality of the teaching and learning processes; fifth, the teaching staff-members of (PTUK), with their different academic ranks, view an achievement exam as an end that can be reached with any set of questions or any exam format; sixth, this study concludes that the absence of formidable educational taxonomies, which the specifications of the questions of an achievement exam are anchored to, reinforces the practice of imitating the traditional heritage in preparing and writing exam questions.
VII. RECOMMENDATIONS
In reference to its findings and conclusions, this study procedurally recommends that the teaching staff- members, of (PTUK) and elsewhere, should pay more attention to the specifications of the questions of an achievement exam, and that such specifications should be anchored to a well-established educational taxonomy, i.e. (Bloom's Taxonomy). This study recommends that faculties of (PTUK), as well as faculties elsewhere, should offer training to the teaching staff-members on how to prepare and write questions of an achievement exam in parallel with educational specifications and well-referenced taxonomies; the Examination Management Department in (PTUK) should prepare and then circulate a well-established and a tabulated list of educational specifications that acts like a guiding resource whenever the teaching staff-members are to prepare and write the questions of an achievement exam, and, at the same time, meets the valid regulations and the educational mission and standards of (PTUK). This study recommends that all teaching staff-members in (PTUK) should view an achievement exam as not only a teaching means, and as an end itself, but also as a learning tool that can provide essential insights into the teaching and learning processes; this study, in accordance with the previous related research, confirms that specifications of the questions of an achievement exam act like parameters that, from one side, orient the amount of, type, quality, and usefulness of feedback, and, from the other side, correlatively, dictate the comprehension capacity as well as the level of achievement that can be reached by students, learners, and exam-takers; as a result, this study recommends that teaching staff-members of (PTUK) should fully exploit such feedback so as to be able to get more understanding and more capabilities to develop the whole process. Inquisitively, this study recommends further empirical research to investigate the effect of enhancing a taxonomy-referenced specifications, while preparing and writing questions of an achievement exam, on students and learners' competency, comprehension skills, and the level of achievement in various courses and different faculties in (PTUK) and faculties elsewhere; this study recommends more research to investigate the impact of the pandemic situation of Covid-19 on the status and the specifications of the questions of an achievement exam; this study recommends correlative studies to investigate the impact of the linguistic aspect and wording of questions, of a given exam, on the achievement of students, learners, and exam-takers in that given exam; This study recommends comparative research to investigate the relationship between taxonomy-referenced specifications of the questions of an exam and the type, quantity, quality, and usefulness of feedback that can be gained in return; this study recommends further studies to investigate the orienting clout of the specifications of the questions of an exam towards the teaching and learning processes; this study recommends further studies to investigate the reasons that motivate the teaching-staff members in (PTUK), as well as the teaching staff-members in various universities elsewhere, to normally resort to a cloud of specifications that mainly stemmed from the educational heritage whenever writing and preparing questions of an exam, rather than anchoring the whole process to a well-established educational taxonomy i.e. (Bloom's Taxonomy) of cognitive domains; this study recommends more studies to investigate the effect of the semantic and connotative dimensions of the verbs classified in Bloom's Taxonomy cognitive domains on the status and the specifications of the questions of an achievement exam.
ACKNOWLEDGEMENT
The authors of this article would like to praise the kind attention, encouragement, and professionalism of their fellow colleagues of the teaching staff-members of (PTUK) who bore the burden of completing and returning the questionnaire related to this study; the authors would also like to thank Palestine Technical University- Kadoorie (PTUK) for the academic patronage, encouragement, and liaison.