A rehabilitation audit was performed comparing admission and discharge values of FIM and Barthel Index (self report) of the last 6 years. The results indicate substantial changes in FIM and BI thus justifying the rehabilitation process. Performing an audit of the used instruments we found a small group of patients (between 6% (post surgery patients) and 26.4% (rheumatological patients)) for whom no overall improvement could be indicated. These patients differ significantly from the other patients in reaching higher values on admission thus indicating a possible ceiling effect in FIM and in BI.

Current rehabilitation standards require evidence based rehabilitation procedures and a significant outcome improvement as a consequence of the implemented rehabilitation programmes. While the analyses on which medicine bases its evidence stem from prospective, control studies performed mostly by university centres with the highest methodological vigour, the routine outcome measure analyses have to be conducted using locally available resources. Additionally, it has been suggested that the demonstration of outcome improvement is complicated by many problems such as small numbers and heterogeneity in the study population.^1,2 There are strategies to overcome this particular problem. Multicentre trials or meta-analyses provide some possibilities; a long-term data collection offers another one.

Selecting an outcome measure for a rehabilitation centre a broad spectrum of patients, diagnoses and treatments should be considered in addition to validity, reliability and sensitivity to the relevant changes targeted by the intervention. The measures must also be feasible to use in routine practice and the team should find them useful.

Analysing data collected over a long time span as in our case the assumptions and decisions made at the time of the planning and implementation of the outcome measures must be to some degree adhered to. There has been a high increase in the last years in publication of relevant studies dealing with the issue of outcome measure in rehabilitation that added to our knowledge on outcome measures in a substantial way. In the early nineties when our system of outcome measures was designed the therapists constructed a net of global and of specific measures of which they believed would provide a comprehensive monitoring of their goal attainment in rehabilitation. The dimensions of independence, disability and resources play an important role in most rehabilitation goals. Several global measures of independence in a range of tasks gained in popularity in the last decades. A survey of rehabilitation units in the UK³ demonstrated that the Barthel Index (BI),⁴ the Functional Independence Measure (FIM)⁵ or the Functional Assessment Measure (FIM+FAM)⁶ are the most commonly used global measures. Sulter, Steen and De Keyser⁷ analysed all randomized trials between 1995 and 1998 in which the efficacy of thrombolytic, neuroprotective drugs or antithrombotic compounds was investigated and found that BI was the most often used outcome measure.

Although the BI and FIM were published a long time ago and gained wide acceptance,^8–10 a series of substantial controversies were carried out in professional journals indicating number of essential shortcomings,¹¹ such as floor and ceiling effects (BI), (see¹² for ceiling effect in amputee patients;¹³ for ceiling and floor effect in stroke rehabilitation,¹⁴ for floor effect in patients with severe brain damage, insensitivity of FIM to cognitive and psychosocial disability, insensitivity to change in BI, dependence of FIM on the health care system in the USA and many other.¹ It has particularly been proposed that the various modifications of BI^15–17 should be considered because of their psychometric qualities. However, some authors were not able to confirm its assumed improvement in sensitivity.¹⁸

A systematic analysis of ADL measures has been conducted by Law and Letts¹⁹ comparing the ADL measures available between 1960 and 1988, among them the BI. Although the authors report poor internal consistency for BI their overall recommendation is positive. BI with some other measures shows the best reliability (see also²⁰ and validity (see also²¹ evidence, is short, and appears to discriminate among levels of ADL function. It has demonstrated evidence of the ability to predict function against other standards. It also belongs to the group of instruments with the best potential for responsive measuring change in ADL function. Its concurrent and predictive validity has been well established against extent of motor loss,²² prognosis,²³ severity of stroke²⁴ and other criteria.

The BI has predictive validity in terms of survival,²⁵ functional recovery²⁶ and other criteria.²⁷ Walker and colleagues²⁸ measured significant outcome differences with BI in the group of stroke survivors who were not admitted to hospital and did vs. did not receive occupational therapy. Alarcon and colleagues²⁹ report that the only variable predictive of prolonged stay in hospital was a Barthel score of <45 on admission.

There are reports on reliability of BI indicating that test-retest reliability and inter-code reliability (22), inter-observer-reliability³⁰ and inter-method reliability³¹ are satisfactory. However, it has also been indicated that BI scored upon a physicians' interview with a patient is not reliable compared to scoring based upon nurses' observation.³²

As its sensitivity to small improvements in the rehabilitation process is relatively crude a modified index with a five-point scoring has been proposed.¹⁷ However, Wood-Dauphinee and co-workers²⁷ report that BI was the most efficient measure and therefore required the fewest subjects to identify a significant outcome effect in stroke rehabilitation. The assessment by BI can also be reliably conducted by a telephone interview by health professional and lay-persons alike.³³

The self-rating BI has been less extensively evaluated.^34–36 Collin and colleagues²⁰ found that Barthel Index as a self-report measure is highly reliable in a series of 25 consecutive admissions to an acute rehabilitation unit. However, they indicate that the most disagreement between self-report and observation was in the self-reports of cognitively impaired patients. Analysing the internal consistency they maintain that most of the disagreement was minor and was associated with more complex items such as transfers. Wade maintains that ADL indices should report actual performance in the relevant setting.³⁷ It has been shown that the self-report BI is also reliable as a postal questionnaire as all items except 'toilet' and 'mobility' had at least moderate reliability by the kappa coefficient.³⁶

The Functional Independence Measure (FIM)^5,38 is a well-established and widely used measure of functional status. The first 13 items of the FIM represent a measure of motor function and the last 5 items represent a measure of cognitive function. It has been indicated that FIM item difficulties vary across impairment groups.³⁹ The authors calculated item difficulties and report that 'feeding' and 'grooming' are the easiest motor items whereas 'stair climbing', 'tub/shower transfers' and 'locomotion' are the most difficult ones. Among the cognitive items 'comprehension' and 'expression' are the easiest items, 'problem solving' is the most difficult.

Chau and colleagues⁴⁰ studied inter rater agreement of the French version and report that all activities of self care, sphincter control, and of transfer, stair, expression and problems solving were rated with very good agreement (kappa coefficient higher than 0.80) and locomotion walk - wheelchair, comprehension, social interaction (kappa coefficient 0.7-0.8) and memory (kappa coefficient 0.63) with good agreement. They also found certain differences between the ratings by educators and by physio or occupational therapists.

There is a number of studies reporting using BI as well as FIM, comparing them and analysing their relationship. Van der Putten and colleagues⁴¹ compared the BI and FIM as outcome measures after inpatient rehabilitation and report that BI and FIM total, FIM motor and FIM cognitive scales were appropriate measures for the stroke patients, but that FIM cognitive scale has limited usefulness in progressive multiple sclerosis because of its notable ceiling effect. Nyein and McMichael⁴² and Turner-Stokes (1) demonstrated that BI can be derived from the FIM motor.

This article addresses the issue whether the ongoing measure with FIM and the self-report BI provides an adequate and satisfactory outcome measure in a community based rehabilitation centre and whether the patients improved in the course of rehabilitation

This investigation uses data collected as part of the routine outcome measures in the Rehabilitation Centre. The patients are screened at the point of entry and prior to their discharge. The FIM data were collected by the nurses, the BI was administered by the occupational therapists. As the FIM assessment is possible to perform independently of the patients' abilities the FIM screening covers nearly 100% if the patients. BI (self-rating), on the other hand, relies on the language and cognitive abilities of the patient and was, therefore, not completed as often as FIM. Consequently, the data set contains only those patients for whom both of the screening measures are available (N=1033).

The table 1 describes the characteristics of the patients with FIM and self-report BI. The diagnoses are summarized in 6 groups. The median age is between 49 (patients after brain injury) and 80 years (geriatric multiorgan illness). The proportion of female patients lies between 41% (brain injury) and 63% (geriatric multiorgan illness). The patients were referred mostly from an acute hospital (63% of brain surgery patients; 80% of post surgery patients) with the exception of patients with rheumatological illness (27%) and patients with other neurological illness (47%) (Table 1).

Barthel Index is a scale of activities of daily living that includes 10 fundamental items from 2- to 4-level scales and is one of the standard measuring devices of functional disabilities or resources. We used the modified version by Granger and colleagues^16,22,43 to measure each patient's ADL performance on admission and on discharge. Fifteen items related to self-care ability, continence and mobility are scored by determining whether the patient can perform them independently, with assistance or supervision, or not at all. The scores for each item are summed and the total ranges from 0 (complete dependence) to 100 (independence in terms of ADL). The BI can be divided to measure the discrete functions of self-care (0-53 points) or mobility (0-47 points). The Barthel Index was originally developed as a means of assessing the level of independence in patients with neuromuscular or musculoskeletal disorders.

The Functional Independence Measure (FIM) is an 18 item observational sheet and interview schedule. Each of the patient's activities (self care, sphincter management, transfer, locomotion, communication and social cognition) is rated between 1 (totally dependent) and 7 (fully independent). The FIM was filled out for every patient treated at the Rehabilitation Centre of the Medical Clinic, Buerger Hospital in Solothurn during 1995-2000. Additional personal and socio demographic data were also collected. This evaluation contains data of the observations made on admission and on discharge of the patients.

The setting

The Rehabilitation Centre is a 25 bed in patient hospital adjoined to an acute hospital in a small town in Switzerland. It cares for a broad range of patients although recently the rehabilitation of geriatric multi organ illnesses was outsourced to another centre. The rehabilitation team is active in implementing and optimising treatment informed by M. Johnstone's teaching and is in process of restructuring the rehabilitation organization according to the Total Quality Management principles and it reached good benchmarking results in rehabilitation outcome⁴⁴ and in time spent with patients.⁴⁵

Floor and ceiling effects of FIM and BI

As there were several indications and complaints about the floor and ceiling effects of the used instruments a simple floor and ceiling effect check was performed. The Table 2 summarizes the amount of patients in % who assessed or were assessed with the lowest degree of independence on the admission measure. There are some variation in the items and in the type of diagnosis the patients received. The items 'bathing' and 'washing' contain a large number of patients with the lowest rating thus indicating a possible floor effect. The group of post surgery patients contains the highest number of patients with the lowest rating in the highest number of items suggesting that in this group of patients some functions may not be sufficiently differentiated in the lower rating. 'Drinking' seems to posses the highest ceiling effect in BI. The patients with rheumatological illness reached the highest number of the patients with the highest rating in the most items. Consequently, an improvement in these patients could not be monitored by this measure. Using the total sum of all items we found less floor effects (between 0 and 7.4% of patients of a diagnosis group) and less ceiling effect (between 8 and 47%). The highest total sum on discharge as a measure of a ceiling effect suggests between 17 and 70% of patients who could have not scored higher because of the limitation of the scale. The BI self care scale showed a lower floor effect and higher ceiling effect that the mobility scale (Table 2).

Although there are more possibilities to differentiate in FIM because of the 1-7 value scales the floor and ceiling effects are also noticeable (Table 3). The items differ but the highest number of patients with the lowest assessment can be see in regard to 'climbing stairs' in all diagnosis groups (17-59%). The ceiling effect is indicated by a high number of patients reaching the highest rating on admission for 'bowel management ' and for 'eating'. However, the amount of patients reaching the highest rating in all 18 items on admission and thus not being able to respond with improvement is relatively low (between 0.3 to 3.7% with the exception of patients with rheumatological illness (14%) (Table 3).

This assessment of floor and ceiling effects is very crude as there are many items in BI with just a two value scale as compared to 7 point scale of the FIM.

However, it indicates substantial differences between items (eating, understanding: between 0 and 7.4% of patients indicate the lowest value; stairs climbing: 17-59% of patients indicate the lowest value), between diagnosis groups (rheumatologic illness: 17% and post surgery patients: 59% indicate the lowest value in one of the functions) and between measures (BI: 81% of the patients of one diagnosis group indicate in regard to one function the lowest value compared to FIM with 59 %).

Similarly, the ceiling effect is also stronger in BI (99.5% of the patients of one diagnosis group report the highest value in regard to one function) than in FIM (80.7%).

Factor structure of FIM

The widespread practice of adding all values of the 18 FIM items into an overall score has been occasionally questioned. Although the FIM items correlate highly among themselves and the reliability coefficient alpha is very high, the issue of the linearity, interchangeability, equal distances etc. is still open. Recently, Ravaud and colleagues⁴⁶ warned of using the FIM total score without precaution because of the multidimensionality of the FIM items. They recommend using various subscores. Consequently, before using FIM scale it is advisable to analyse the factorial structure of FIM as gained with the given sample.

We performed a factor analysis with the patients of each diagnostic group separately for the admission and discharge data. The admission FIM total data in patients with vascular brain damage, with brain injury and with other neurological illness were presented as being two dimensional separating clearly between FIM motor and FIM cognitive and accounting for 73.6%, 87.5% and 72.8% (Tables 4, 5). We obtained a 3 dimensional solution in post surgery patients (accounting for 68.7%), in patients with rheumatology illness (accounting for 78%) and in patients with geriatric multiorgan illness (accounting for 70.1%). The third factor comprises either of 'eating', 'grooming' and 'upper body dressing' as in post surgery patients or of 'comprehension' and 'expression' as in the other two patient groups (Table 4) (table 5).

Factor analysing the discharge FIM data we obtained two-factorial solution only in vascular brain diseases patients (accounting for 74.2%) and in patients with other neurological illness (accounting for 74%) (Table 6). The FIM data of patients with traumatic brain injury, rheumatological illness and geriatric multiorgan illness indicated three dimensions. The third dimension comprises of 'bladder management', 'bowel management' and 'transfer to toilet' in traumatic brain injury patients and of 'comprehension' and 'expression' in the other two groups of patients (Table 7).

Factor analysis of BI

There is a well-documented discussion on the dimensionality of BI. Factor analytical and various multidimensional scaling methods were applied and reported. In order to analyze the factorial structure of BI we calculated factor analysis for various diagnosis groups with admission and discharge data. In all except vascular brain damage and traumatic brain injury groups we obtained more than the two factors which are prescribed as subscales explaining accounting for between 56 and 80% (Table 8). The BI mobility factor crystallised in all diagnostic groups (Getting in/out of bed, going to the toilet, bathing, walking, stairs climbing) but in addition to this we found also 'lower body dressing, 'personal hygiene' and 'washing' in all except in the post surgery and geriatric multiorgan illness group as a part of the first factor. Further, the first factor in some diagnosis group also contained 'upper body dressing' (Table 9). The factorial structure of the vascular brain illness group consists of the firsts factor shared by other diagnostic groups, the second factor containing 'drinking', 'eating', 'bladder control' and 'bowel control'. The BI of the post surgery patients indicates a four factorial structure. First factor containing mobility items, second self care, third 'eating', 'drinking' and 'upper body dressing' and fourth 'bladder' and 'bowel control'. The rheumatological illness patients relate motor and self care items in factor 1, 'eating', 'upper body dressing', 'bowel' and 'bladder control' in factor 2 from which 'drinking' is separated in factor 3. The BI data of the geriatric multiorgan illness patients indicate a 4 factor structure with motor items (factor 1), self-care, 'dressing' and 'eating' (factor 2), 'bowel' and 'bladder control' (factor 3) and 'drinking' (factor 4). The factor analysis of the discharge data seems to fit to a similar factorial structure as the admission data. The vascular brain illness patients have a nearly identical factor 1. The bladder and bowel control build a separate factor in discharge data, as do the items 'drinking' and 'eating'. The traumatic brain injury patients fit to 3 dimensions instead of 2 as on admission. The motor and self-care items need two dimension and bowel and bladder control build one factor. The following picture could be found in the group 'other neurological illness' (some mobility items with self care (F1), bladder and bowel control (F2), eating and drinking (F3). In the geriatric multiorgan illness group the items 'bladder' and 'bowel control', together with 'going to the toilet' are related to each other building one factor in the discharge data, while the motor and self care items distributed over two factors. Thus, we obtain a 2-4 factorial structure, depending on diagnosis and on the fact whether they are admission or discharge data.

Reliability

The three Barthel scales (total, self care, mobility) reached satisfactory reliability in all patient groups on admission and on discharge (Table 10). Although the lowest reliability coefficient (0.60 in rheumatological illness, on discharge, self care scale) would call for improvement the next lowest reliability coefficients (0.69 in post surgery patients on admission, self care scale; other reaching 0.70) are acceptable. In most cases the reliability coefficients of admission data are very similar to those of discharge data within each of the patient groups.

The reliability coefficients of FIM scales (total, motor, cognitive) are generally higher (the lowest being 0.80 (in geriatric multiorgan illness patients on discharge, cognitive scale) (out of 42 reliability coefficients are only 9 between .80 and .89. The other are equal or higher than .90) (Table 10).

Comparison of scale means on admission and discharge

BI

Analysing whether the patients' BI scores improved between admission and discharge we calculated t test for dependent samples for each diagnosis group (Table 11). There is a highly significant improvement in the mean values of the BI total (sum of all items) in all groups (p<0.001) except in patients with brain injury (p<0.05). The mean values of the BI self care scale are significantly higher on discharge than on admission in all patients group (p<0.001) except in patients with traumatic brain injury (p<0.06). The differences in BI mobility scale between admission and discharge are also significant (traumatic brain injury patients: p<0.05; all other groups p<0.001) (Table 11).

FIM

A comparison of FIM total means on admission and discharge indicates a significant improvement of all patient groups (patients with traumatic brain injury: p<0.005; all other groups: p<0.001) (Table 12). The lowest FIM total mean was achieved by the vascular brain disease patients (83 points on admission) and the highest by the rheumatological patients (115 points on discharge). The smallest difference between admission and discharge FIM total can be find in rheumatological patients (5 points), the largest in patients with vascular brain disease (14 points) and in traumatic brain injury patients (15 points). The FIM motor scale indicates highly significant changes in all groups of patients (p<0.001) with the exception of traumatic brain injury patients (p=0.008) (Table 12). The FIM cognitive admission - discharge comparison shows significant changes in vascular brain disease, in post surgical patients and in patients with other neurological illness (p<0.001). Patients with rheumatological illness (p=0.013), patients with traumatic brain injury (p=0.057) and geriatric multiorgan illness patients (p=0.424) achieved less noticeable improvement (Table 12).

Changes in BI and FIM

The mean comparisons indicate mean changes in the whole groups of patients. However, the declared goal of a rehabilitation team is to help every patient. The information how many patients improved their total score on discharge is an important information on how many patients were judged as more independent on discharge and how many patients judged themselves as being more independent. The table 13 summarizes the data for each diagnostic group of patients. Well over three quarter of patients improved their independence as assessed by the nurses (FIM total) (minimum 64 maximum 87% of all patients of each diagnostic group; 80% of the whole sample). Some 10% did not change in their overall score (5-28%), 10% did not manage to maintain their admission degree of independence (7-17%). The BI scores indicate that patients assess their degree of improvement in their independence more sceptical than the nursing staff (61% patients vs. 80% of nurses indicate a change with discharge score higher than admission score). Less that a half of all brain injury patients and rheumatological patients report improvement in their independence (48% and 40%). Building a group of patients who did change neither in BI nor in FIM we obtained about 12% of patients (whole sample; ranging from 26% (rheumatological illness) to 6% (post surgery)) (Table 14). Who are these patients who did not improve their independence? Are they too severe cases (floor effect) or are they patients with a high admission score (ceiling effect)? (Table 13) (Table 14).

A comparison of means of the patients who improved and those who did not in regards to the BI total, BI self care, BI mobility, FIM total, FIM motor, FIM cognitive, indicates that patients who did not improve their independence were more independent on admission that the patents who improved their independence (Table 15). Although this result is significant in the total sample there are some difference in the diagnostic groups. The result is not significant in vascular brain damage patients and in brain injury patients, where it is due to the low number of patients (N=4). In addition, there are some differences between BI being significant in post surgery patients and FIM being significant in 'other neurological illness' group. The results of this comparison would support the thesis that non-improvement of the patients is mostly due to the high admission values, that is, to the ceiling effect of the measures. However, it should be noted that all patients indicate a better value minimally in one of the BI items on discharge, thought some of them also report a worse value thus reaching a total which is either negative or unchanged (Table 15).