In this paper the methodology of the Discussion Boards language assessment in Content and Language Integrated Learning (CLIL) Advanced Business Russian course at Regent’s University London is discussed by drawing on the students’ responses to the teacher’s comments based on two business topics studied in class. In order to concretise our assessment, this paper suggests an assessment method based on Biggs and Collis’ SOLO Taxonomy for analysing how the students’ language learning outcomes matched the SOLO Taxonomy levels of understanding and the rubric scores of the Discussion Boards Assessment. The obtained data indicate that the language learning outcomes may be directly connected with the levels of students’ thinking and their ability to construct content. In this paper the problem of the difference between the Advanced Business Russian learning and teaching methods and those suggested by some researchers is analysed underscoring the fact that in the Advanced Business Russian course business academic language is used which is different from everyday language discourse. Drawing on the two questionnaires offered to the students before and after their participation in the Discussion Boards Assessment, it is also argued that their learning perspective changed as a result of their collective work on the assessment task. The paper concludes that this underscores the learner’s central role in the learning process and their ability to construct knowledge.

Keywords: content and language, teaching and assessment, advanced business russian course, data.

The aim of this paper ‘Discussion Boards Assessment in Content and Language Integrated Learning (CLIL): Prioritising language assessment in the Advanced Business Russian course at Regent’s University London’ is to demonstrate the importance of language assessment within the CLIL classroom based on the experience of teaching Business Russian at Regent’s University London. In the Advanced Business Russian course at Regent’s University London we are faced with the problem of teaching business through the medium of the Russian language so CLIL seems to be the most appropriate mode of instruction. Coyle et al. define CLIL as ‘a dual-focused educational approach in which an additional language is used for the learning and teaching of both content and language’,¹ and although this definition can be open to various interpretations in our context conceptualization of CLIL as an inclusive and holistic educational approach seems appropriate. It means that in our CLIL Advanced Business Russian course learners become active participants in the content creation, which goes hand in hand with the development of their language and thinking skills as it has been demonstrated by the students’ work on the Discussion Boards Assessment tasks. Given the relatively recent adoption of CLIL practices, assessment in CLIL, and particularly, assessment of language in CLIL have not been comprehensively researched and discussed. Barbero stated that CLIL assessment was ‘an area where there was incomplete and not systematic documentation’² while Coyle et al.¹ agreed that ‘the theme of assessment is a difficult and sometimes contentious area amongst CLIL teachers’.¹ In discussing the CLIL assessment, Coyle et al.¹ maintained that frequent corrections of language accuracy would impact the flow of communication and frustrate learners.¹ To counteract this, they suggested occasional ‘language clinics’ organised by the teacher ‘on demand’.¹ However, Advanced Business Russian teaching and assessment would not be significantly improved by conducting a ‘language clinic’; Russian language has a complex grammatical and morphological structure, which places considerable demands on the learner in terms of syntax and spelling. Curdie opted not to correct errors of spelling and grammar in her students’ discussion boards, explaining that ‘there would be no real discussion’ if this were done (Curdie, S., 208, para. 3).

There is some ambiguity in research literature on the question of language assessment in a CLIL classroom or the way it was aligned with learning, teaching and the intended learning outcomes of the course.³ Coyle et al.¹ reviewed several theories on CLIL language learning and teaching but concluded that ‘there was no single methodology for language learning and teaching, or set of prescribed technics”.¹ This paper focuses on language assessment in the CLIL Advanced Business Russian course at Regent’s University London to contribute to the debate by underlining the importance of language assessment in a CLIL classroom and its alignment with learning, teaching and the course ILOs.

In summary, the aims of this paper are

1. To define students’ progress in thinking and understanding with reference to their language development in a structured way drawing on Biggs and Collis’ SOLO Taxonomy
2. Identify descriptors of competence both for language and ‘subject literacy’ (Nikula, 2016) by defining assessment criteria for the CLIL Discussion Boards Assessment;
3. Create a scored rubric for the Discussion Boards Assessment with a description of the level of competence for each score and its potential weaknesses;
4. Create two questionnaires for the participants in the Discussion Boards Assessment to analyse their motivation both prior and post their participation in this assessment.

The proposed chapter has the following structure: Abstract, Introduction, Background, Context, Research Question and Data, Method and Results, Discussion, References.

It is important to bear in mind that the Discussion Boards Assessment is a formative assessment in that ‘its results are used for feedback during learning [because] ‘students and teachers both need to know how learning is proceeding’³ as well as an Assessment for Learning (AfL) in that it allows both learners and teachers to improve their performance. According to Biggs and Tang, the underlying principle of any assessment is that its tasks reflect the course intended learning outcomes and teaching and learning activities that happen in the classroom.³ This principle is also relevant to the language assessment in CLIL and there is no ambiguity among CLIL teachers and researches about it but, as it has been mentioned earlier, the question of how to teach, learn and assess language in CLIL still remains contentious. In the CLIL Advanced Business Russian course an academic, highly specific business-oriented language is used which is different from everyday conversational discourse. Quartapelle & Schameitat refer to Cummins who defined this type of language as the ability that ‘allows you to verbalize complex cognitive processes (Cognitive Academic Language Proficiency - (CALP)’.⁴ Indeed, as Quartapelle & Schameitat state.

‘In CLIL the subject content from the beginning requires a wide linguistic basis, characterized by a certain syntactic complexity and a specific vocabulary. In order to act and interact on specific topics, the student must be sufficiently ‘equipped’ from a linguistic point of view.’⁴ The question then arises: what kind of teaching and learning is required in order ‘to equip’ learners linguistically and how is it reflected in the assessment? Many researchers attempted to tackle this question but failed arrive at any conclusive decision. For example, Coyle at.al. offered several recipes for learning language in a CLIL classroom identifying three types of language acquisition: language of learning, language for learning and language through learning but argued that language teacher should ‘shift linguistic progression from dependency on grammatical levels of difficulty toward functional and notional level of difficulty demanded by the content’.¹ Expressing their view on how to teach the past tense, they postulated that learners should be facilitated in developing the concept of ‘pastness’ and ‘past markers’ and instead of learning paradigm of the verbs conjugated in the past tense, they should simply use ‘certain phrases’.¹ Unfortunately, this method cannot work for the Advanced Business Russian because by just ‘developing the concept of “pastnes’ or of any other grammatical phenomena, learners in the CLIL Advanced Business Russian class would not be able to use a complicated system of the verb conjugation in Russian or deal with the Russian participles that are a common feature of any Russian academic text, what ‘linguistic demands of the subject’ require.’¹ In other words, we need to concretise grammatical concepts and phenomena to make them usable in the Advanced Business Russian classroom. Many researchers of CLIL also place a great emphasis on communicative approach to learning a second language in CLIL. For example, observe that ‘approaches to language learning have moved [also] from orientations almost exclusively directed to grammar and translation to more eclectic approaches geared to learning how to communicate in a second or foreign language.’¹ Morton and Jakonen refer to Loewen’s recent characterisation of language learning in the context of the communicative language learning activity as a ‘brief attention, either planned or incidental, to problematic language items within a larger communicative context’.⁵ Cummins, nevertheless, seems to contradict this approach saying that ‘the use of academic language is typically less interactive and less contextualized’.⁴ Quartapelle, & Schameitat agree that a CALP learner ‘does not require a fluency in addition to that which refers to the subject itself, the language of the subjects and the symbolic ones are sufficient for the development of subject content.’⁴ Still, communication does occur in the CLIL Advanced Business Russian classroom but on a different level as compared to the modes of communication suggested by the researchers above. Quartapelle, & Schameitat emphasise the importance of the thinking process in an academic language discourse saying that ‘the specific discourse forms and terminology belonging to the different subjects and to the different aims of the lesson have a relation with language activity and thinking skills.’⁴ This paper attempts to focus on the language assessment or, to be more precise, on the assessment of the quality of the students’ second language (Russian) outcomes in the CLIL Advanced Business Russian class drawing on the students’ participation in the Discussion Boards forums and on Biggs and Collis’ SOLO Taxonomy in order to demonstrate what impact learners’ language learning outcomes have on their thinking development.

Speaking about CLIL context in the UK it is necessary to consider it in the light of a decrease in foreign language learning in this country. As a response to this situation there has been a growing awareness among UK language teachers and academics both at a school and university level that it is necessary to raise interest in language learning in UK. As a result of this concern Nuffield Inquiry (Nuffield Foundation, 2000) was launched in 2000 which analysed the state of the language teaching in UK in 1999-2000 and found it unsatisfactory as far as the second language acquisition was concerned. The recommendation of this inquiry was that the CLIL programmes should be introduced in the UK at the national level. The rationale for this recommendation, apart from the acknowledgment of the need for introducing more content into foreign language programmes, was that CLIL programmes will not only strengthen learners’ skills in foreign languages but also raise their interest in foreign language learning and their motivation. Apart from Nuffield Inquiry there was an extensive research conducted in the UK on the conceptualisation of CLIL in the UK context. The outcomes of this research directly indicated that CLIL was

1. A new approach to learning languages
2. A means to find an alternative way to a traditional language teaching
3. A tool to engage language learners at a higher cognitive level
4. A means to raise learners’ motivation and interest
5. A tool for knowledge construction through Language 2
6. A way to develop cross-cultural awareness
7. A means to encourage “a further understanding of content such as problem solving, making judgements, evaluation and reflection to name a few”.
8. A vehicle for practicing problem-based, learner-centred and collaborative learning.
9. These recommendations opened a new perspective for teaching and learning the second language both at school and university level.

Before analysing language assessment in the CLIL Advanced Business Russian course it is useful to describe the course ILO, its structure and content, bearing in mind that the CLIL Advanced Business Russian course design has been based on the principle of “constructive alignment” according to which the course intended learning outcomes (ILO), learning and teaching activities and assessment are aligned to form a continuous system of supportive learning.³ Conceptualising the principle of ‘constructive alignment’ Biggs says that, The ‘constructive’ aspect refers to what the learner does, which is to construct meaning through relevant learning activities. The ‘alignment’ aspect refers to what the teacher does, which is to set up a learning environment that supports the learning activities appropriate to achieving the desired learning outcomes. The key is that the components in the teaching system, especially the teaching methods used, and the assessment tasks are aligned to the learning activities assumed in the intended outcomes. The learner is ‘trapped’ and cannot escape without learning what is intended. Below a brief overview of the Advanced Business Russian course ILO, the course structure, methods of assessment, assessment tasks and the indicative content are introduced to substantiate our adherence to the principle of ‘constructive alignment’.

Course intended learning outcomes

On completion of this module, students should:

1. Have a detailed knowledge and effective understanding of grammatical, phonological and lexical structures, and of some registers of the target language, with emphasis on the way it is used in business contexts.
2. Have a broad knowledge and critical understanding of contemporary economic, socio-political and business issues in the countries where the target language is spoken.
3. Be able to use the target language competently and effectively as a medium for understanding, expression and communication in professional and social situations, demonstrating confidence in the use of receptive (reading and listening) and productive (speaking and writing) skills.
4. Communicate to an external academic/business audience.

Course structure and indicative content

The duration of the Advanced Business Russian course is 2Semesters with 12weeks in each.

Contact Time 72hours over 24weeks. Self-Study: 100hours over 24weeks

Assessment methods         

4 skills tests and various forms of continuous assessment (written work produced outside or inside the classroom, mini tests, oral presentations, case-study, etc.) over the whole year

End-of-module examination

Assessment is based on three elements, with the first and second elements weighting 30% each and the third element weighting 40%. Table 1 below, shows the Assessment Tasks and their weightings. Table 2 course content.

Rationale for using discussion boards for language assessment in the CLIL advanced business Russian course

The rationale behind the choice of the Discussion Boards for the students’ language assessment is quite simple - as an online virtual environment learning tool Discussion Boards allow users to collectively compose content, to share information and to comment on each other’s entries, encouraging their communication and cooperation.

Drawing on this principle, we acknowledge that the Discussion Boards language assessment in the CLIL Advanced Business Russian course was built on the principle of ‘learner centeredness’ in that students’ learning was characterised not ‘as a simple acquisition process based on teacher transmission but as a process whereby students actively construct their own knowledge and skills’.⁶ Conceptualising this idea, it is posited that the teaching theory of ‘constructivism’ which ‘emphasizes what students have to do to construct knowledge’³ was the foundation of the Discussion Boards Assessment in that students created knowledge ‘through a process of active construction’. The Discussion Boards Assessment encouraged students to construct knowledge by supporting problem-based, learner-centred and collaborative learning, previous -experience learning, cognitive skills-based learning and cognitive tools facilitated learning. Below it is explained how.

Problem-based, learner-centred and collaborative learning

The Discussion Boards Assessment supported the problem-based, learner-centred and collaborative learning in that its tasks were formulated in such a way as to present a problem, for example, ‘Why is franchising an unpopular business in Russia?’ or ‘Why do a lot of small businesses in Russia suffer bankruptcy in the first year of their operation?’ Students worked collaboratively commenting on the teacher’s statements and on each other’s entries, so collaborative learning was taking place.

Previous - experience based learning

The Discussion Boards Assessment tasks were also based on the learners’ previous experience, one of the main principles of constructivism, as the themes of the online discussion were directly connected with what the students had learned during the course. As research shows if learners see the online discussion as irrelevant to the content of the course, they are unwilling to participate in the Discussion Boards interactions.

Cognitive skills-based learning

The Discussion Board Assessment engaged students cognitively and supported strong cognitive presence in that it presented a problem which needed solving drawing on students’ understanding, thinking, evaluation, reflection, making judgments and conceptualisation.

Cognitive tools facilitated learning

Students were also encouraged to use various cognitive tools which can be both mental and computational devices aimed at supporting, guiding, and extending the thinking processes of their users.⁷ The cognitive tools that the learners in the Advanced Business Russian course used included:

1. Search engines such as Google, Bing, Yahoo!., Alta-Vista, Yandex, Rambler etc.
2. Class work before and after completion of the Discussion Boards Assessment etc.
3. Exchange of information relevant to the topic discussed etc.
4. Collective work on the relevant vocabulary.
5. Mind mapping.

Research questions, data and method

This paper attempts to address the following research question: How were the Advanced Business Russian language learning outcomes assessed in the CLIL Advanced Business Russian course by the Discussion Boards Assessment drawing on Biggs and Collis’ SOLO Taxonomy and The Discussion Boards Assessment scored rubric devised by the teacher? To address the research question, the following research instruments were used: Biggs and Collis’ SOLO Taxonomy, students’ entries in the Discussion Boards forums, the Discussion Boards assessment scored rubric, the descriptors of competence both for language and content and students’ questionnaires before and after their participation in the Discussion Boards Assessment.

Data on students’ participation in the discussion boards assessment

In order to analyse the students’ participation data, we drew on a collaborative work of the Advanced Business Russian course students during 2semesters on two Discussion Boards forums: ‘Problems facing small businesses in Russia’ and ‘Franchising: why is it an unpopular business in Russia?’ The assessment brief was to comment on both the teacher’s statements and the other participants’ entries. (See the detailed analysis of the Discussion Boards Assessment in Chapter 3 ‘Analysis and Results’)

Students’ collaborative work before they participated in the Discussion Boards Assessment involved:

1. Class discussion of the problems raised in the teacher’s statements and the ways of tackling them
2. creating a collective mind map with the theme vocabulary and the questions relevant to the topic discussed
3. Using computerised cognitive tools such as search engines to gain information on the topic discussed
4. Reviewing grammar which could be useful in writing their comments such as the degrees of comparison or participles.

On completion the Discussion Board Assessment students were engaged in a number of class activities, such as:

1. Class discussion of the difficulties encountered while completing the Discussion Boards Assessment;
2. Peer evaluation of the participants’ entries;
3. Class discussion of the grammar and vocabulary involved;
4. Class discussions of the best ways to prepare for this kind of assessment;
5. Class discussion of the advantages and the disadvantages of the Discussion Boards Assessment;
6. Contribution to the two questionnaires on their participation in the Discussion Boards Assessment.

Below, we present an example of a mind map, created by the students who participated in the Discussion Board Assessment on the topic of Franchising in Russia.

Useful vocabulary:

entrance fee

additional expenses

volume of sales

retraining

personnel selection

business environment

to depend on

user of the trade mark

to provide freedom

raw materials

to sell on credit

trademark

franchiser

franchisee

point of view

turnover of the capital

independent

royalty

public catering

Franchising in Russia

QUESTIONS:

What is franchising?

1. What are the benefits of franchising?
2. What are the drawbacks of franchising?
3. What assistance do franchisers offer their franchisees?
4. What obligations do franchisers and franchisees have towards each other?
5. Who is an ideal franchiser and an ideal franchisee?
6. Why is franchising an unpopular business in Russia?

Biggs and Collis’ SOLO taxonomy and the discussion boards assessment scored rubric as the instruments for measuring students’ progress in language learning reflected in their understanding and thinking processes

In previous sections we discussed the special features of the academic discourse as compared to everyday routine communication, focusing on the quality of the academic language which “allows you to verbalize complex cognitive processes (Cognitive Academic Language Proficiency − (CALP)”.⁴ It follows that CALP language has ‘a relation with language activity and thinking skills.’⁴ Biggs and Collis also suggest that ‘the evaluation of thought [from childhood to adulthood] gives an important clue as to quality, which concerns all educators’.⁸ But, as Biggs and Collis continue, teachers find it very difficult to define quality, ‘particularly in terms understandable to students’.⁸ Indeed, the quality of learning depends on many factors such as a learner-teacher relationship, learner’s motivation and intentions, learner’s aims and objectives that can be different from those of a teacher and many others. Biggs and Collis came up with a structural system⁸ which would allow teachers to define the quality of learning in an structured way and to ‘discriminate well learned from poorly learned material’,⁸ The SOLO Taxonomy (the Structure of the Observed Learning Outcomes) developed by Biggs and Collis in 1982, allows to define students’ learning outcomes in terms of their complexity, enabling teachers ‘to assess student’s work in terms of its quality’. (http://johnbiggs.com.au/academic/solo-taxonomy/).

Below is Biggs and Collis’ SOLO Taxonomy:⁸

The following five stages of SOLO Taxonomy classify students’ learning outcomes according to their complexity, categorise levels of learners’ understanding and show difference in quality of their responses to assessment tasks:

1. Prestructural: (student can’t express a single point of the argument, basically, can’t answer the question)
2. Unisrtuctural: (student uses only one point of view taken from a familiar source such as a textbook)
3. Multistructural: (student uses two or more concepts or points of view)
4. Relational: (student presents several points of view, can establish interrelation between concepts)
5. Extended abstract: (student demonstrates deep abstract and independent thinking, can produce a hypothesis).⁸

Although Biggs and Collis’ SOLO Taxonomy is mainly about assessment of quality of the students’ responses to the assessment tasks (Relational, Extended abstract levels) it also provides the structure for the quantitative assessment (Prestructural, Unistructural and Multistructural levels), because ‘this distinction between knowing more and restructuring parallels two major curriculum aims: to increase knowledge (quantitative: unistructural becoming increasingly multistructural); and to deepen understanding (qualitative: relational, then extended abstract).³ We do occasionally observe the quantitative phase responses (Unistructural and Multistructural) of the Advanced Business Russian course students, it usually happens when students were beginners in Russian when they started a Business Russian course or when their intention from the beginning was ‘to get the task out of the way with minimum trouble’³ or ‘only to achieve a minimal pass’.³

To examine the class data concerning both quantitative and qualitative assessment of the students’ language outcomes we draw on the students’ entries in the Discussion Boards forums in response to the Discussion Boards Assessment task (for a reference see the topics of the Discussion Boards Assessment in 2.1). The analysis of the quantitative aspect of the Discussion Boards Assessment (the amount of knowledge demonstrated by the students through language) and the qualitative aspect (the progress in their understanding demonstrated in their responses to the tasks assessments) builds on the classification of the students’ entries according to the SOLO Taxonomy levels and the Discussion Boards Assessment scored rubric. That is, we take examples of the students’ responses to the assessment tasks and distribute them according to the levels of Quantitative and Qualitative phases of the SOLO Taxonomy and the Discussion Boards Assessments rubric scores.

Below, we introduce the Discussion Boards criteria for content and language and the Discussion Boards Assessment scored rubric (Table 3) to match its scores with the Biggs and Collis’ SOLO Taxonomy levels of understanding. Because the focus of this Chapter is on the assessment of language in the CLIL Advanced Business Russian course we use this method to assess students’ progress in language learning reflected in their responses to the assessment tasks.

The discussion boards assessment criteria

Instructions: Comment both on the teacher’s two statements concerning small businesses in Russia and franchising, and on at least three student entries.

1. Level of understanding criteria: your posts should demonstrate that

1. You have read both the articles that were discussed in class and that you understood them,
2. Have not merely repeated the content of the articles but that you have worked with this material, analysing and interpreting it.

2. Content criteria: your posts should demonstrate

1. Critical thinking;
2. Originality;
3. Ability to develop an argument;
4. Ability to provide evidence to support your conclusions;
5. Ability to integrate and evaluate information and data from a variety of sources.

3. Language criteria: your language should:

1. Express the content as it is described in the content guidelines;
2. Be discipline-specific, i.e. It should demonstrate knowledge of the vocabulary, specific to the topics discussed;
3. Demonstrate coherence and cohesion to convey meaning and construct content.

The Discussion Boards rubric presented in Table 3 below concretises the Discussion Boards Assessment criteria by using scores to evaluate levels of students’ performance with respect to their content and language integrated competence and by matching them with the levels of understanding described by Biggs and Collis in their SOLO Taxonomy.

The scored rubric, presented here, is analytic and criterion-referenced² in that it assesses students’ performance ‘along several different. Dimensions’,² thus providing us with an insight of how the language “integrates with the content”.²

Classification of the students’ language outcomes in the Discussion Boards Assessment according to Biggs & Collis’ SOLO Taxonomy and the Discussion Boards scored rubric

Ten students of the Advanced Business Russian course participated in the Discussion Boards Assessment commenting on the teacher’s statements on two topics: ‘Problems facing small businesses in Russia’ and ‘Franchising: why is it an unpopular business in Russia?’ Both the teacher’s statements and the students’ responses are translated into English.

Here are the teacher’s statements:

1. Small businesses in Russia face a lot of problems, such as lack of state support, corruption of the government bureaucrats, difficulty of getting a bank loan for the development of a small business, bureaucratic barriers and a complicated system of taxation to name only a few. Please comment on the teacher’s statement and on other participants’ entries and express your opinion on the measures the government of Russia should take to improve the situation with the small businesses in Russia.
2. Franchising has a lot of advantages, but it has some disadvantages as well. The main advantages of franchising are: franchising maximises the capacity for fast business development, it allows a franchisee to work under a franchiser’s name using his technologies, equipment, raw materials etc. But franchising has a number of disadvantages too, for example, franchisee must enter into a formal agreement with his franchiser, franchiser dictates his franchisee how to run his business, where to buy raw materials, what supplies to use etc. which limits franchisee’s freedom. Please comment on the teacher’s statement and on the other participants’ entries and express your opinion on why franchising is not a popular business in Russia.

Because of the space limits in this Chapter we analyse here only the students’ responses to the teacher’s first statement. Excerpt (1) illustrates how three students’ (S1, S2, S3) responses to the first statement match the levels of Biggs and Collis’ SOLO Taxonomy, and the Discussion Boards Assessment scored rubric. They are first presented in their original form in Russian and then translated into English. The students’ mistakes both in grammar and spelling are preserved and put in bold both in Russian and in English.

1. S1: Yes, little businesses in Rassia has lot problems. I think that a very bed problem this is corruption. The government Rassia should give Rassian little businesses more helps, it should make simpl procedure how to get loan from bank for the divelopment little business and necessary a system of tax priveleges for little businesses in Rassia.
2. S2: I think that the reason for the problems faced by the Russian small businesses is that there is no legislation for the small businesses in Russia what must protect their owners’ rights. Also, the Russian government should take serious measures to fight corruption because to survive owners of the small businesses in Russia must pay protection money to the Russian mafia, it should also simplify the taxation system for the small businesses and help owners of the small businesses with financing at the first stages of their business development.
3. S3: I think that only a relevant and consistent policy both on the part of the state and the Russian banks aimed at giving support to the small businesses in Russia and facilitating development of the Russian small businesses will allow to improve the situation that exists in this segment of the Russian economy. The Russian state should create regional funds to give help to the small regional businesses in getting government and big industrial enterprises’ contracts. Besides, the Russian government must start a resolute fight with corruption and the bureaucracy of the government officials. Russian banks should develop a system of the small businesses financing, they should also simplify the procedure of allocating credits to the Russian small businesses owners. Small businesses are very important for Russia because they promote creation of the Russian middle class, the guarantor of stability and the democratic development of the country.

The first student simply repeated the teacher’s statements, so his response was tautological. He made a lot of grammatical and spelling mistakes but managed to create some sort of content by suggesting two measures for improving the situation of the small businesses in Russia (to simplify procedure of getting bank loans and to create a system of tax privileges for the small businesses owners). Matching his response to the levels of Biggs and Collis’ SOLO Taxonomy we can see that his language learning outcome and, accordingly, his level of understanding, is consistent with SOLO’s Unistructural Level of understanding which is corroborated by the Discussion Boards assessment rubric score of 40–42. The second student managed to explain the reason for a difficult situation of the small businesses in Russia (absence of legislation for the small businesses to protect their rights) and offered three measures the Russian government should take to improve the situation (to fight corruption, to simplify the system of taxation and provide financial help to the small businesses owners). He also explained the reason for the necessity of fighting corruption (to survive the owners of the small businesses in Russia must pay protection money to the Russian mafia). His language level was very good, he made only two grammatical mistakes ( used ‘what’ instead of ‘that’ and confused the gender of the modal verb ‘must’) so his language learning outcomes that agree with his level of understanding fit Multistructural level of Biggs and Collis’ SOLO Taxonomy and correspond to his rubric score of 55-57. The third student’s language learning outcomes helped him to hold his question open⁸ and present a well-structured argument for the necessity of government help to the small businesses in Russia. He also conceptualised his idea by explaining the importance of the small businesses for the creation of the middle class in Russia using sophisticated advanced business Russian language. He made no grammatical or spelling mistakes and his language proficiency helped him to integrate all aspects of his response into a coherent whole which matched the Relational level of Biggs and Collis’ SOLO Taxonomy both for language and understanding and corresponded to the Discussion Boards rubric score of 71–72.

Students’ questionnaires before and after their participation in the Discussion Boards Assessment as evidence of change in their learning perspective

As it was established earlier, the Discussion Boards Assessment is a criterion - referenced assessment; however, the question of performance evidence is a different matter in that it includes not only evidence of ILO and criteria fulfilment, but also other factors referred to by Coyle et al. as ‘affective evidence’.¹ This, according to them, examines student motivation and dedication. As they state further, ‘monitoring participants ’attitude towards CLIL and their motivational level should be a key element in an evaluation process’.¹ Taking the above into account, two research instruments – i.e. two questionnaires were used to investigate the Advanced Business Russian learners’ ‘affective dimension’.¹ These were offered to the students before they had participated in the Discussion Board Assessment, and after they had completed it. The students’ opinion about the Discussion Boards Assessment is extremely valuable, as often ‘teachers and students have rather different positions and points of view with regard to subject and language learning as well as with regard to teaching and the way it is structured’.⁹

Questionnaire 1 offered to the students before their participation in the Discussion Boards Assessment:

Do you think that participation in the Discussion Boards Assessment will enlarge your Business Russian vocabulary, consolidate your grammatical knowledge and strengthen your writing skills, resulting in your stronger ability to express your thoughts (i.e. to construct content) on two topics offered to you – ‘Problems of small businesses in Russia’ and ‘Franchising in Russia’?

Below, is a representative sample of the three students’ responses:

First student:

I am not sure whether Discussion Boards can help me with my Business Russian vocabulary, strengthen my grammar and improve my writing because to start writing something useful, you already need to know a lot of vocabulary and grammar. I would prefer simpler exercises.

Second student:

Maybe, but my writing is poor, and I don’t know anything about small businesses or franchising in Russia. I don’t understand the Russian grammar either, especially declensions. How can I express my opinion on these topics?

Third student:

Yes, I think so. I would like to try my hand at writing something in Russian, but I need to learn some vocabulary. I am interested in the problems of the small businesses in Russia as I was thinking of starting a small business in Russia or in my native country Bulgaria. But it is still difficult for me to express my opinion on these subjects because I know about them too little. The only thing that can help me is that there are the same problems with the small businesses in Bulgaria and franchising is a popular business in my country.

Questionnaire 2 offered to the students after their participation in the discussion boards assessment

Do you think that participation in the Discussion Boards Assessment has enlarged your Business Russian vocabulary, consolidated your grammatical knowledge and strengthened your writing skills, resulting in your stronger ability to express your thoughts (i.e. to construct content) on the two topics offered to you - ‘Problems of small businesses in Russia’ and ‘Franchising in Russia’?

The same students’ responses

First student:

To some extent, yes, because we studied both topics before participating in the Discussion Boards. The teacher also helped us to compile a vocabulary list for these two topics. I also liked the way we worked together composing a mind map which included the essential vocabulary and our thoughts about problems of small businesses and why franchising is not a popular business for Russians. I also liked how other students responded to my entries and I liked commenting on theirs. It was great fun!

Second student:

Yes, definitely! The best thing was that I could return to my work and correct it later after I looked up some words in the online dictionary, so I feel that my business vocabulary and my writing skills have improved. It was great working together on a mind map and composing questions relevant to the Discussion Boards topics!

Third student:

Yes, it was useful for me as I had never participated in the Discussion Boards forums before. I used an online dictionary which was very convenient and read some articles connected with the topics, unfortunately, not in Russian but in English! But the articles helped me to understand the problems of the Russian small businesses such as corruption and high taxes. When I couldn’t express my thoughts, I asked my Russian friend to help me and it was great. Besides, it was a collective work - we created a mind map where we compiled a list of business vocabulary and wrote relevant questions. I liked this work!

To summarise, we have demonstrated by the examples of the students’ entries in the Discussion Boards forum 1 that Biggs and Collis’ SOLO Taxonomy is a useful instrument for tracking students’ progress both in language learning and thinking which is also reflected in the Discussion Boards Assessment scored rubric. We argue that the alignment of the language assessment in the CLIL Advanced Business Russian course to language learning, teaching and the course ILO is a guarantee of the CLIL assessment validity as a criterion-referenced assessment and that the language assessment in CLIL is as important as the assessment of content. It can be also argued that the questionnaires responses above demonstrated students’ apprehension about the assessment usefulness and their ability to cope with its task before they had participated in it because, as Coyle et al. point out ‘in the early stages of a CLIL programme, enjoyment, motivation and self-esteem can be at risk as the students come to terms with the initial challenges of adapting to a CLIL methodology’,¹ but after they had completed the assessment they expressed satisfaction and acknowledged the fact that learning had taken place in the course of their work on the Discussion Boards Assessment.

In this chapter we attempted to throw a light on and get an insight into the problem of language assessment in a CLIL drawing on the Discussion Boards Language Assessment in the Advanced Business Russian course at Regent’s University London. Considering that language learning and assessment remain a contentious area in CLIL, our objective was to offer a method of assessing students’ achievement both in language learning and understanding with the help of Biggs and Collis’ SOLO Taxonomy and the Discussion Boards Assessment scored rubric. By doing so we attempted to clarify at least some uncertainties concerning assessment in a CLIL classroom. Below are some observations that can be made based on the results of our research:

1. Students’ progress in language learning depends on many variables such as their intentions for learning, their background, learning styles, motivation, learning environment and ability, to name only a few, but a teacher can change them by his or her teaching methods and fair, criterion-referenced assessment.
2. Students’ cognitive skills and thinking are directly related to their progress in language learning because ‘discourse does not just express meaning. Discourse creates meaning. As we acquire new areas of knowledge, we acquire new areas of language and meaning.’¹
3. The Discussion Boards Assessment was an assessment for learning ‘since it has the aim of informing the planning of future learning and teaching.²
4. The Discussion Boards Assessment demonstrated how students’ motivation, interest, dedication, self-esteem and satisfaction with the results of their work have grown.
5. The students’ personal perspectives of learning have changed during their work on the Discussion Boards Assessment because it

Contributed to the collaborative learning

1. Helped students to realise their potential
2. Gave them incentive to improve their language skills and construct their own
3. Knowledge
4. Helped them to strengthen their cognitive and writing skills
5. Stimulated their interest in, and their awareness of, developments in Russia.^10,11

None.

The authors declare that there is no conflict of interest.

None.

1. Coyle D, Hood Ph, Marsh D. Content and Language Integrated Learning.Cambridge: Cambridge University Press. 2010.
2. Barbero T. Assessment Tools and Practices in CLIL. In: Quartapelle, Editor. Assessment and Evaluation in CLIL. AECLIL‒EACEA. 2012;40‒61.
3. Biggs J, Tang C. Teaching for Quality Learning at University. What the student Does, 4th edition. UK: Maidenhead, Berkshire, Open University Press, McGraw‒ Hill Education. 2012.
4. Quartapelle F, Schameitat B. Teaching and Learning with CLIL. In: F Quartapelle, Editor. Assessment and Evaluation in CLIL. AECLIL‒EACEA. 2012;30‒40.
5. Morton T, Jakonen T. Integration of Language and Content Through Languaging in CLIL Classroom Interaction: A Conversation Analysis Perspective. In: Nikula T, Dafouz E, Moore P, Smit U, Editors. Conceptualising Integration in CLIL and Multilingual Education. Multilingual Matters. Bristol, UK: Multilingual Matters. 2016;171‒188.
6. Barr RB, Tagg J. From Teaching to Learning – A New Paradigm for Undergraduate Education. Heldref Publications. 1995;1‒12.
7. Derry SJ. Flexible cognitive tools for problem solving instruction. Paper presented at the annual meeting of the American Educational Research Association, Boston, MA. 1990;16‒­20.
8. Biggs J, Collis K. Evaluating the Quality of Learning ‒ the SOLO Taxonomy. New York: Academic Press. 1982.
9. Wegner A. Seeing the Bigger Picture: What Students and Teachers Think About CLIL. International CLIL Research Journal. 2012;1(4).
10. Biggs J. Aligning the curriculum to promote good learning. LTSN Generic Centre. 2002.
11. Kiely R. CLIL – The question of assessment. 2019.