The Measurement of Performance
in the Community and Voluntary Sectorby Brian Davey
Summary
This discussion paper covers current trends towards quantitative performance indicators which measure outputs and outcomes in the voluntary sector and the demand that groups demonstrate "continuous improvement" on these indicators. It argues that this model of evaluation is unrealistic for the community and voluntary sector and that 'good enough' standards would be more appropriate . There is an innapropriate transfer of measurement methods taking place from large scale formal organisations, which work with standardised processes and which operate for different motivations, to smaller scale ones, which are operating closely with people's private, domestic and informal lives - usually for altruistic and not for profit motives. The paper gives many reasons why the new measurements will not work and are practically destructive to the goals and values of the voluntary sector. Different methods of evaluation are needed for our sector - including a willingness to take into account how projects feel (their emotional temperature). The voluntary sector is now highly stressed and, crucial to improvement, is to take the pressure off. High stress/low trust relationships are invading the sector as it is being measuring up by our more powerful "partners" for continuous improvement in the pursuit of their agendas. A 'rational management' approach to evaluation, purely on the basis of paper based measures will not work. There is an intrinsic indefiniteness which does not permit useful measurment of performance in many intangible person to person voluntary sector contexts. 'Good enough standards' still contain an idea that there are minimums which it is unnaceptable to fall below. Support for whistleblowers has a role in protecting these minimums.
The main message is: by far the best way of improving the performance of the community and voluntary sector is to make groups happier and less stressed organisations for their paid workers, volunteers and users. If figures are to be collected then they should be, above all, to find out how we can pursue this purpose.
Measurement in the Community and Voluntary Sector - How Smart is it?
Introduction - Good Enough Standards or Continuous Improvement
This article is to examine two key assumptions underpinning current community and voluntary sector policy - (1) the assumption that quantifiying and measuring outputs and outcomes against targets will improve performance and efficiency and (2) an assumption that we should be pursuing "continuous improvement" in these output and outcome measures.
I explore many aspects of the 'mathematisation' of social policy and the way we are expected now to quantify our management of ourselves in the voluntary sector (but don't worry, there are no difficult maths here). It must be stressed that I am not arguing., as such, against all measurement, quantification and mathematics in the human relations and care work. I think it is worth, in appropriate contexts, exploring how to attain, and then maintain adequate performance standards, where desirable. I favour 'good enough' standards. ( I discovered while writing this that the good enough idea has recently been championed by Professor Andrew Samuel in a book called 'Politics on the Couch'. In fact, however, a colleague tells me that many voluntary sector organisations used to have a management manual on their shelves called "The Good Enough Manager' so the idea has obviously been around a long time). To emphasise the point: 'good enough standards' are not the same as the 'continuous improvement' of targeted measures.
One gets a flavour of the part of the argument by pointing out that to describe someone as "calculating" usually has negative connotations. One does not expect good quality emotional relationships with calculating people - and if one is receiving care from them one would hardly expect the care to be very warm. This might seem like a play on words but, in fact, as I shall show, there are very real concerns of this type.
Another serious problem lies in the fact that, for measurement and quantification to take place, there must be things (categories) that can be measured and quantified. The key words currently being imposed on the not for profit sector (like "outputs", "outcomes", "performance", "efficiency" "best value") are often very difficult to pin down. "How can they be applied to the intangible things we do?' is a puzzle for bewildered voluntary sector workers and volunteers.
I am sure that, to the policy makers, and to the unquestioning officials who are seeking to get these ideas and measures implimented, this bewilderment may seem like a failure of understanding on the part of the community and voluntary sector - but I think it can be shown that the failure of understanding lies with the policy makers and officials, not with us. In fact, they are wanting to apply these words and ideas, which were developed in one part of the economy and society, where they may perhaps have an application, to another part, where conditions are very different. Thus, while the policy goal of "continuous improvement of measurable outputs and outcomes" may seem to authors to have a clear meaning when expressed in their policy documents, this is far from the case in day to day work in health and social care, when it comes to implimenting them.
In recent years policy making has become separated from operational practice - and this has not helped. Officials who draft policy do not have to impliment their own policies, so they are protected from the frustration they would feel at having to apply their own ideas. The new insistence on "partnership" has not helped either. Partnership often really means the big organisations quasi-supervising the small ones - and their supervision is innapropriate because the staff who are working in the big organisations really operate in different conditions. When they force their management styles and values down our throats, as advice and methods we are pressured to accept, they do not realise how innapropriate to our conditions their work methods are. Policy making separated from operational practice, as well as 'partnership' which functions as indirect supervision, have both led to management and performance control styles developed in the commercial and production sphere of life, and in big formal organisations running on different values, being imposed on smaller, informal organisations, where people are not motivated in the same way. The methods do not work and are, in fact, destructive. However, in the absence of genunine partnership between equals, in the absence of operationally experienced officials, it is often difficult to get these officials to hear and listen.
Different Kinds of Organisations require different kinds of management styles
On a production line, or in the running of a power station, or when processing social security claims or taxation returns, continuous and repeated processes are used to make a more or less standardised product, result or service outcome. The work processes are maintained, through continuous routines. They can thus be regulated according to definable rules or procedures. In general, the more that products and work are standardised and repetitive, the less is the ability to exercise their own judgement that the workers who do the processing have - the more they must simply follow set procedures. When decisions are taken for you, as to how you should act in this procedure -bound way, one can describe one's actions as having an automatic quality. Of course, work procedures may have a considerable degree of complexity. It may be necessary to master knowledge of a great many rules to accomplish the processing task. Development of the knowledge of these rules represents a development of skills and may even be interesting (for a time). However while one is still following laid down rules one is, so to speak, an automaton as one does not have any discretion to make decisions and judgement. The output, outcome, performance and quality approaches currently being imposed on the voluntary sector, were largely developed for, and in, organisations of this type. Given the standardised procedures, activities and products it makes sense to think of these organisations as having outputs, outcomes and performance standards for workers (though the work may be alienating, that is another matter!).
In such large organisation it is only at the senior levels, where what might be described as decisions with a quality of freedom, are exercised. It is at these levels that discretion is exercised, and judgements made, as to what, and how many, products and services will be available; how the organisation itself will be organised to process the rules; how annoying anomalies, that were not thought of under the rules, will be best dealt with, in the spirit of the rules; and how the rules will be changed - perhaps in the light of anomalies and changing circumstances, perhaps in the light of changing polical priorities.
The Introduction of Computers
In recent years, it should be said, the introduction of computers has considerably influenced management styles in the private sector, if less so in the public and state sectors. When work activities can be quantified, it becomes possible for decision makers to get comprehensive numerical information very rapidly on the activities of their subordinates - without the mediation of many supervisors and managerial intermedaries. These intermediaries have often been cut out in recent years as a cost saving device ("rightsizing")- which means that, whatever support experienced intermediaries might have previously given to subordinates, is now lost. At the subordinate work level there is an important trend towards being sent instructions as to what to achieve but not how to achieve these things. ( 'The deregulation of the work place'.) The goals are still set at the higher level but, often, no longer the procedures. This is sometimes presented as 'empowering' the workers but, without support, the target setting and negotiations around acceptable self-set targets can leave subordinate workers floundering and unsupported. Nevertheless they remain under performance pressures. They are still judged by what they achieve in pursuing purposes still defined by senior managers. For such subordinates the absence of supervisors means there is nobody there with whom to negotiate difficulties, differences and dissonances. Senior management are, in effect, presiding over organisations so large, that they have abdicated their responsibilities and are no longer accountable for their actions. This can be described as a sort of absentee managerialism.
These type of work relationships for subordinates seem awfully like a lot of underfunded voluntary sector organisations to me. Here again there are performance expectations on undersupported workers who just have to get on with it, as best they can. They have no experienced managers who have done their jobs before to turn to. But they do have to send back their performance returns to the funders. These performance returns now demand continuous improvement and are often difficult to make any sense of. The methods for performance management designed in the large organisations are being foisted off onto us through a kind of external management which so called bigger partners like local authorities exercise when they regulate supposdely "independent" voluntary organisations. In the process the more realistic notions of a good enough performance, an idea that would allow people to be more honest, and be able to deal with problems without resorting to lying and spin, are getting lost.
Common characteristics of community and voluntary sector organisations
In fact there is a deep contrast between the ways in which voluntary organisations work and the public and private sectors' operations. The point about much, probably the majority, of community and voluntary activities is that they are an extension of people's private and domestic lives in local and small scale settings. They arise where the resources to be found inside the family (when people have a family), or at home, need to be supplemented by mutual and other support arrangements and other activities. Or they arise where people decide to make a contribution to the wider community - a free choice. And a defining characteristic of domestic activities, as opposed to employed activities, is that it is a place where one takes one's own decisions - about purposes and about the procedures to meet those purposes. One has a degree of personal discretion as to how one spends one time; as to how one organises the routines and procedures of everyday life; over what purposes one pursues and what procedures one uses to realise these personal purposes. In all of these things (purposes, daily routines etc), there is a huge variety from one person, and one family, to another. Also, although a lot of domestic life is routinised, it is also the case that, unlike automatised employed work, where many people do the standardised processes every day, much domestic activity is non-standard and non-routine. If 'variety is the spice of life' then people seek variety and "an escape from the routine" in their free time - which is also the time when they make their contribution to community and voluntary organisations.
The Voluntary character of the voluntary sector
The voluntary and community sector is just that: it is voluntary. It is not set up by the statute of the state - it is not there because someone else says that it must be. Many activities in neighbourhoods are extensions of domestic activities which are mostly informal, time limited and small scale. They are non-routine, they are non-standard and often non-continuous. Indeed, when people are socially excluded and have lots of social and economic problems they may find it quite different to stabilise their lives into routines - they, or their close relatives and friends, get ill; they have financial crises because they are close to the financial edge and dependent on state bureaucracies; they are often all over the place emotionally; they find managing homes and relationships without money more difficult. Vulnerable people also have less 'social capital' i.e.trustworthy relationships with other people who they can rely on in hard times. They are thus hit harder by those hard times. The chaotic lives of socially excluded people has a tendency to spill over into whatever they get involved in. It takes the form of a tendency to unreliability and instability, practical and emotional, which makes involvement in many self help, community and voluntary sector organisations a very bumpy and unpredictable ride. Creating some order and calm out of this chaos is the essence of the work - but it is not easy. Every experienced person in the community and voluntary sector knows of the sandcastle building experience - of developing a group with volunteers, and then the volunteers moving, getting a job, getting ill, so that one has to start to develop the organisation again. To call for 'continuous improvement' and to expect targetted development along a measured 'milestoned' path is a cruel way of torturing voluntary sector workers and volunteers. It is like insisting on planned and timetabled victories in a war zone.
The non standardised character of the voluntary sector
Of course, it is a simplifying generalisation to say that the voluntary and community sector is a place of non standardisation, with fewer routine activities and services, when compared to the large commercial and public sector organisations. Funded voluntary sector groups who get staff can usually ensure some routine timetabled activities - because paying and contracting staff gives some guarantee that things will happen at particular times. However, even when there are paid staff, voluntary sector organisations are often pretty chaotic because the staff tend to have to be Jacks (or Gills) of all trades in tiny organisations. They operate in unstable environments in terms of the availability and stability of their volunteers etc. This is not at all like the large organisations who can depend on staff who are paid to be there, and who can always call on the full range of in-house specialists - something which makes possible long term planning with a far greater degree of ease and assurance.
The Smaller You are the Less Future Control - and the less targeting is possible
Large organisations, working with established resources and tried procedures, have an ability to plan the future to a degree because they have resources. They can call upon contracted staff, who are paid to be there, to do standardised work tasks. They have (profit) resources to invest in productivity increases - R and D, new machinery etc. There is is a degree of future predictability to their operations. Measurement is possible in this context. Of course, things go wrong - as when, for example, 12 million people were predicted as attending the Dome and only 6.5 million turn up! Unexpected exchange rate changes, tax changes, new regulatory frameworks, the appearance of new competitors and products, all these may undermine the planning predictions of large organisations. In the voluntary and community sector the ability to predict is even more fragile because - processes are often experimental, volunteers and vulnerable helpers cannot always be relied upon, resources are available only short term. In short voluntary and community groups are more likely to be blown off course... They also do not have profit resources to generate continuous improvements of productivity.
These are the reason why community and voluntary organisations often find it so difficult to work to targets. It explains why "continuous improvement" is quite a different requirement It also explains why it is so difficult to describe what voluntary organisations do in terms of measured standardised "things": outputs; outcomes, performance. Such concepts, derived from large organisations, often cannot be used to describe what they do. Specifically:
Continuous improvement implies continuous standardised operation
You cannot measure anything in order to aim for "continuous improvement" (a goal taken from new managerial ideologies like Total Quality Management) if it is non-continuous and one off. Yet many community activities are just that: groups function through a series of unique one off activities. (A Planning for Real exercise, an outing, a christmas party, a summer festival, a harvest festival, a consultation meeting, a demonstration or picket). In many of these events no long term process is intended so there are no outputs or outcomes to be measured long term. Indeed, "success" may be measured by the speed with which an issue is taken up and resolved. Campaigning and advocacy work is often aimed at getting the faster possible outcome to resolve a problem on behalf of the people in the group. The last thing groups want is "continuous improvement" because they want to get a problem resolved so that they do not have to work on it any more. The only outcome to be "measured" here is whether another organisation stops or amends a particular practice.
Continuous improvement implies non contradictory motives - yet things aren't always so simple
'Continuous improvement' is hardly possible on a common measure when people get involved in community groups for different or mixed motives. This is very commonly the case in voluntary organisations because voluntary involvement is motivated in so many different ways.- e.g. a group may be considered "successful" even though its numbers fall because the group has helped some people improve their confidence and skills - so they go on to get a job. Yet the group may only be successful for others if it provides them with a social world to overcome their isolation. For the latter people, who are involved to get social contact, the moving on of the first group may involve the loss of regular contacts with friends, which is what they were looking for. So what is a success for one group leads to the organisation being a failure to another.
Another contradiction, which is very common in community groups, is between the role of a group as a confidence builder (and hence employment stepping stone for volunteers) and the role of the group as a service provider. Typically once the group "succeeds" by helping their best volunteer get a job it "fails" because it cannot provide the same level of service until it recruits and trains a new volunteers to replace them. In circumstances like this one might feel that, instead of "continuous improvement of outputs or outcomes" a more realistic, and less futile, goal would simply be "achieving and maintaining an adequate level of service (outputs and outcomes)". This is not at all the same thing.
When measurement of voluntary effort means looking a gift horse in the mouth
Another problem is that calculating and measuring management techniques raise serious questions for the values and the motivations which largely underpin a lot of care work and voluntary sector activity. I have already pointed out that much voluntary activity is discretionary and voluntary, made in people's own time. People do this voluntary activity for a variety of activities but they are not paid for it and that makes it gifted labour. Most grants to voluntary sector organisations and groups go to pay the expenses, office costs, support costs for paid support workers, but, nonetheless, there is a huge add-on of gifted time that is thereby made possible. To start measuring this activity up, and to demand 'continuous improvement' of it, is to 'look a gift horse in the mouth'. Indeed, it it to go further than looking a gift horse in the mouth - it is actually very sinister as it represents an exploitation of people's altruism in the service of greed and inequality.
Calculation - the exploitation of gifted labour
Earlier I made an apparent pun on different meanings of the word "calculating". There is a serious point here about different motivations. To have a "calculating mentality" is usually understood to mean being concerned to calculate what you can take out of a situation or process. It is about taking (e.g. taking the difference between receipts and costs of an enterprise, which are profits). This is the motivation and way of thinking found in the private sector - it is the profit motive. However, much care and voluntary sector work is about giving (e.g. labour for out of pocket expenses only). The voluntary sector is "not for profit". The giving and taking motivations cannot be put in the same framework for comparison purposes, to determine things like "best value", without very destructive and extremely exploitative effects on the givers. It creates a situation of a cheap option for care work. "Continuous improvement of the output and outcome measures" can then be seen to mean the cheaper and cheaper option of voluntary sector care - to keep taxes down and therefore taking the pressure off the rich......
To spell this out even more clearer: voluntary labour and slave labour both have in common that they are free labour. Clearly they are at opposite ends of a spectrum. However, under certain conditions, voluntary labour may take on more and more characteristics of slave labour and move along the spectrum. For example, Nottingham city council wants citizens to take on the role of managing its community centres as volunteers. However, it has recently explicitly denied that these volunteers have any rights to be consulted - when it decides it wants to sell such centres to the chums of senior officials and politicians. Having taken away the right for its volunteers to be consulted, the next step along the spectrum, in the direction of slave labour, is to start measuring up voluntary labour and insisting that it be submitted to evaluation techniques which will bring about "continual improvement" in relation to outcomes. (The final step would be for central government to insist that people work for tame and compliant 'voluntary sector organisations' and the director of leisure and community services, in exchange for rock bottom benefits and to find various other ways to bulldoze people into "voluntarism and involvement". Such things were rather common in Eastern Europe and the communist block, which is why people there are very hostile to voluntarism now, and why civil society and social capital is virtually non existent in the former eastern bloc).
To take a more local example - a colleague told me of a conversation with someone from the local branch of a national voluntary sector organisations which cares for elderly people. Both paid staff and volunteers work for this organisation and are finding that more and more is being asked of them in improved measured results. This has led to many volunteers walking away - which leaves the paid staff with the stress of having to achieve more with less helpers. This is a very good way to destroy the voluntary sector.
'Continuous improvement' - means it is never good enough already
The underlying assumption of continuous improvement is that no matter how good your performance is already there is always room for improvement. Dissatisfaction is built into the concept. The work ethic, that one can always, and should always try harder, is the message from the highly paid officials on their secure contracts to those who are on short term grant aid or working as volunteers for expenses. Naturally, trying harder will help you secure funding these officials assure their 'junior partners' - and then they tell us, that they need to do this because the local authority wants to reduce its grant budget in the future. Continuous improvement is necessary so local authorities and government can reduce their expenditure because, it seems, the rich are not rich enough and have to have lower tax rates - in order to have the incentives to produce the wealth of the economy. Hurrah!(It has never been proved that lower tax rates cause people to work harder - and you can equally argue that they help rich people 'work' less, by giving them the same take home pay for less effort....)
Measuring takes resources away from non measured things in favour of measured things
Yet another problem of measuring is that it takes resources and priority away from things to which no numbers and measurements are assigned and diverts these resources to the things which are measured. A very clear example can be seen in the National Health Service now. The insistence that waiting lists not be longer than a year means that people get operations for things which sometimes have less clinical priority and urgency than other operations, in order that one year targets are fulfilled.
As I said at the beginning I am not opposed, as such, to measuring and counting. There is a case for simple targets for attaining and maintaining adequate minimal standards. These might be called "good enough" standards - to call attention to the fluid nature and difficulty of maintaining coherence in any organisation. The problem comes when one has target setting combined with an idea of 'continuous improvement', and silly ideas like "the pursuit of excellence" or "best" value. That is because there will always be a need to take into account unknowns, in order to be ready to respond to variables that are not taken into account, and your gut feeling that something is going wrong.
When things are not taken "into a count" - the measurement of measures.
When we do not measure things we say that they are not "taken into account." They are not taken into a count. They do not count. The Greek word for measure is "medere" which is also where the word 'medicine' is derived from. A measure, it will be noticed, is something that is measured. But measured target setting, combined with an insistence on continuous improvement, creates conditions in which non-measured things deteriorate, out of view, in order to achieve the targets. A usually out of view non measured thing is the stress on volunteers and/or workers. Stress is not often seen to count and it is not often 'taken into a count'. Yet in the human care field the effects of stress can often be highly dangerous. A project may be meeting all its measured paper targets but, when you go there, it doesn't feel right. It feels deeply unhappy. Relationships are acrimonious. There is high absenteeism and high turnover. In fact, if you look at the statistics that are collected, it is a shocking fact that teachers, nurses, social workers, and voluntary sector employees are the four largest groups of callers to the UK National Workplace Bullying Advice Line. The limited evidence that exists suggests that the voluntary sector, although it may be meeting its targets, is now one of the most stressed sectors in the eonomy! But who is counting?
Measuring staff and volunteers - the self defeating effect of insufficient trust
Given this fact there are serious reasons to believe that measurement and target setting are undermining workers and volunteers in a way that is seriously undermining the sector. It is self defeating and unsustainable. When you crack the whip of competition over people, when you monitor their performance and exercise more and more tight supervision of them with ideas like 'continuous improvement', they become unhappy, angry, frustrated and frightened. Unhappy, angry, frightened people are not in fact more productive at all. In quite a short space of time they become ill; they no longer do the things they did before out of good will; they start to sabotage surreptiously what you are doing; they look for other jobs; they are distracted; everyone's time becomes more and more absorbed in trying to heal disputes and conflicts; they resent what is happening and they co-operate as little as possible. The idea is as old as human society. The chinese philosopher, Lao Tsu, writing several centuries before Christ, says in the Tao te Ching that "People are difficult to lead, Because those above interfere with them.."
Some of the earliest studies of work stress by the Survey Research Centre at the University of Michigan's Institute of Social Research found that "high performance groups were led by supervisors who were employee orientated rather than production orientated and who avoided close supervision". (Tim. Newton, Jocelyn Handy and Stephen Fineman "Managing" Stress. (Sage Publications 1995 ). p 33).
Why Measured performance appraisal doesn't work
The supervision that works is the minimal supervision concentrated on the people. So do the performance appraisal systems that are everywhere appearing improve things then? (Whatever "improve' means here). In fact they probably don't. For the private sector there is no evidence to suggest a link between the operation of a formal performance management system and improved organisational performance in private sector organisations. (according to Bevan,S. and Thompson, M. 1991 "Performance Management at the Crossroads" in Personnel Management, November: 33-36.) Also "appraisers and appraisees perceive that there are literally no consequences whatsoever for a good or bad appraisal" (Latham G.P. 1986 in CL Cooper and IT Robertson (eds) International Review of Industrial and Organisational Psychology. Chichester Wiley. (Isn't that what one observes when certain chums of the Director get a grant even though their annual report shows their organisation to be shambolic?). As these things can be shown in research to be verified then how come we are getting them forced down our throat in the voluntary sector and the public sector? Why should we believe that they will improve performance here either?
High Stress/Low Trust Organisations - where Measurement has gone too far
These types of supervisory appraisals imposed from the outside will probably make things worse. Recent research on Teacher Stress carried out by the Dr Geoff Troman from the School of Education at the Open University introduces what I think are very useful concepts. Although Troman's concepts are developed for schools I think they can be carried over to the voluntary sector. Troman speaks of teacher stress as a component of a "low trust society". Troman defines a low trust society as one in which teachers now find themselves in and in which they are therefore subjected to constant observation and criticism. This creates individually experienced stresses, that are socially produced, leading to a deterioration in relationships. There is a breakdown in trust because there is so much emphasis on accountability, regulation, audit and therefore an escalation of conflict. Just as with the Unversity of Michigan research, which found that close supervision around production targets was worse, so increasing supervision of school teachers is making things worse.
A very similar process is occurring in the voluntary sector. As Andrew Passey and Fran Tonkiss argue "a changing operating environment - marked by greater regulation, competition, professional fundraising, corporate sponsorship and a 'contract culture' - means that voluntary organisations are increasingly governed by mechanisms designed to develop and maintain confidence. For instance, relations with the state are shifting from a focus on wider outcomes (trust relations based on shared evaluations of social good) to specific outputs (confidence relations based on target-driven contracts). A growing number of cause-related marketing links with business, meanwhile, is blurring the boundaries between the profit-making and nonprofit sectors.
"Such shifts highlight the problematic and potentially contradictory relationship between the institutional nature of voluntary organisations, and their values base. We would argue that a primary means of understanding these processes is in terms of a movement away from trust to confidence relations between voluntary organisations and their various 'stakeholders', through institutional and contractual means. While a mode of trust linked to social values and to principles of voluntarism remains the foundation for voluntary organisation, its formalisation within institutions has the effect of transforming trust into a different set of relations - which, if necessary for the sector's 'efficiency', potentially undercut its resources of trust. ("Trust, confidence and voluntary organisations: between values and institutions" Fran Tonkiss and Andrew Passey, in Sociology May 1999)
As in schools one gets the development of low trust/high stress work. One of the deepest stresses, that undermines productivity and creativity, is that for many voluntary sector workers and volunteers, that one can less and less believe in what one is doing, because one is not treated with respect, because one is not able to work in accordance with one's own values, and one is checked up on by people who, as far as one knows, have less experience than oneself, but because they work for bigger organisations, suddenly become your big brother.
Measuring clients and users - its destructive effects on self esteem
What makes the whole thing even worse still, is the fact that measurement itself frequently has emotional effects on clients and users which undermine the goals of the activity being measured. To illustrate what I mean I have only to cite the personal experience of my project Ecoworks which, at one time, was grant aided by the Mental Health Foundation, to fund part of our work. The MHF insisted that it only wanted to fund work which was going to benefit people with a severe and enduring mental health problem. It was only interested in funding our work if we could collect measurable evidence on whether we were making progress or not to helping this group and improving their mental health and/or quality of life.
Now the essence of our approach was that people could be helped if we stopped relating to them as "clients" or "patients" and work alongside them as colleagues on an equal footing, integrating people who had a mental health record with those who did not. But the insistence that we collect quantitative evidence completely undermined our intended work approach. For it meant that we had to collect statistics on one group of people who were to be designated as those 'with severe and enduring mental health problems'. This meant marking them out as different - when our whole intended emphasis was on not treating them as different. We were supposed to sit down with people, almost from the beginning, with a form to fill in, which bore upon their clinical status. Yet it was the fact that everything with this group of individuals served to emphasise their clinical status that was, in our view, the larger part of their problem. As our worker put it, these individuals could not sneeze without reports being written on them. Indeed, these people with 'the long term and severe mental health problems' were highly sensitive to the Catch 22 they were in. Many were most reluctant to help in filling in the forms - as served to emphasise their status. In practice, some we knew were long term patients denied they were. I think it important to note that, in our view, our (partly fudged) method partly worked and because we them as little as possible! But we did not then have the full panoply of statistics to prove it. But then ordinary people do not have statistical studies made on them in ordinary relationships and activities, which we were trying to provide. Thus, at one stage, the MHF project officer, who never once visited the operational project, but would summon us to London, as if for a test, responded by delaying payment of grant aid, and creating severe difficulties. (She subsequently moved on and we developed a better relationship with the MHF - who developed a better understanding of us).
The point here is that measurement can undermine goals with users of services, clients and patients. You cannot measure outcomes for people with problems, without defining them as people with problems - and then measuring up how bad their problems are at the start of the research process. Only thereby can you measure later on how well you have done in resolving and taking away these problems. However, this is to shoot yourself in the foot right at the start. Your very first act is to undermine people's self esteem and their trust in you as an equal when it is the absence of real equal relationships that traps them in the first place!
The emotional temperature cannot be quantified but is real
The quality of human emotion, and simple honesty, is such an important feature in the care and human relations field that it will always be impossible to rely solely, or even mainly, on quantitative measurement in the voluntary and community sector. Considerations of the emotional quality and temperature in projects, i.e. whether relationships feel good or bad, cannot be made in an simple minded quantitative way from afar. There needs to be an on the spot assessment - by people who are themselves emotionally healthy and therefore receptive to what is going on. It is not possible to conceal, for long, from an on the spot observer, whether stress, anger, fear, frustation and acrimony are the dominant feelings. If they are then, whatever are the state of the performance measurements, something is very wrong. Most voluntary sector organisations have been set up to deal with the fact that people are living in stress, anger, fear and frustration which arise in their life conditions and relationships. If these feelings are present chronically in the organisations set up by and for them then things have probably gone wrong - no matter what the figures say. To address these feelings in an evolving way is the best method to improve many human focused organisations. This may entail collecting measures on how many people are unhappy and why they are unhappy - changing the focus of measurement onto feelings and their causes - seeing what should be done about the unhappiness and the source of it.
It is by assessing on the spot feelings that we improve things
Sympathetic operationally experienced officials becoming more familiar with projects day to day could help. So would giving the officials, and groups, discretion to deviate from already set target goals in the search for happier work places and projects. However I fear that this idea is probably not going to find much support from senior levels. It might be expensive. It might leave too much to the discretion of their senior's subordinates, and apparently also, to chance. If they are going to give money to other organisations senior management in grant giving organisations want to be able to sure that they can ensure quality control from outside, and from afar, so that they can prove value for the money that they have been given the responsibility to hand out. Senior officials cannot be everywhere at once and to rely on the on the spot judgements of operational officials would be rely on delegated power, leaving the discretion and judgement low in hierarchies. Yet if they do not, the senior officials will continue to be quite literally out of touch - with a spurious belief in paper based performance regulation which actually enourages spin, lying and hypocricy.
To forestall the criticism of laxity it should be pointed out that the notion of good enough targets and standards also contains within it the idea that standards should not fall below unacceptable minimums. Sometimes groups do fail and/or go of the rails. A culture of pretence and spin does not help reveal these cases but prevents them coming into the open. Nevertheless even in the best of regulatory regimes there will be failing projects and ethical lapses. For this the encouragement of internal responses from the voluntary sector itself is called for. It is the insiders who know what is going on. It is these insiders, whose values may be offended by colleagues who not working with adequate standards or adequate ethics who should be encouraged to fight their corner against the peers or superiors if standards to fall below the acceptable minimum. This requires an adequate supportive policy for whistleblowers and the trade unions who need to support them - as whistleblowing is a role that can be frightening and psychologically exhausting.
Why Pythagoras became a murderer - some things are intrinsically indefinite
As I have shown, attempts to measure and quantify in the field of human relations, care and in many aspects of health, are often self defeating in terms of objectives and runs up against a property of intrinsic quantifiable indefiniteness. What makes for these paradoxical, self deafeating and indefinite features are the emotional characteristics and the fact that, as extension of domestic activities, community activities are largely based on trust, informality and a gift relationship. This makes use of standard management techniques futile. When managements try to use them from outside the sector they end up creating confusion, demoralisation and undermining the values of the sector. Such managers and policy makers will have to live with the idea of indefinitness and 'good enough standards' - checking up on organisations by delegating more to staff to become more day to day familiar with projects to be sure that they know the feel of those projects.
Living with the slippery idea of intrinsic indefinitness and "good enough" measurement is not new in other fields - even in mathematics but there are those that this leaves highly uncomfortable. One may reflect for a moment on something perhaps remembered from school maths: the attempt to give a definite numerical value to the ratio of the circumference of a circle to its diameter, Pi. Of course, the reader may remember that Pi is approximately 3.141......but the point I am making here is that we can actually only ever give an approximate number for Pi, never a definite one. This fact so alarmed the Greek philosoher Pythagoras, that he murdered one of his students, Hipparsus of Tarentum, for revealing the awkward truth.
Measured unemotional rationality only takes you so far
Pi presumably upset Pythagoras so much because he wanted to prove that reality was essentially reduceable to mathematics - rather as managers like to believe that management can be solely based on rational techniques, on firm measures, and that messy things like human feelings, assessed on the spot, do not need to come into it. Yet this was not possible for Greek rationality nor is it possible for current management rationality.
Pi was termed an "irrational number" - because rationality was essentially about appropriate "ratios". If you look in the word "rational" you will find the words "ratio" and "ration" in it. To the Greeks, these were all concepts that ran together under the concept of reason. As I have already indicated the word 'medicine' also comes from the Greek word "medere" to measure. In our own language measure means 'to quantify' but 'measure' also means to take action in pursuit of a purpose. One can go on like this at length. Many of the other words that we use to organise our current day to day thinking about reason and rationality also have a mathematical flavour - and we should remember that reason and rationality is usually counterposed to the emotional side of life.(Especially male life!). For example the word "mean" means average. However, the word 'means' means 'to convey meaning' too! For Aristotle 'virtue' lay in the mean - in other words, virtue was the average between two extremes. Thus courage was, for Aristotle, the mean between timidity and rashness. So one can see that meaning, rationality and ethics, as guides to human action, were all closely connected as measuring and quantifying concepts and underpinned the whole structure of thought and reasoned judgement. Other words and related words liked "medium" and "mode" also underpin our whole way of thinking about rationality, ethics, policy, choices etc - model, modal, visual medium, causal medium, the media, mediate etc
It is therefore possible to see, how deeply unsettling to Greek thought was the discovery that some things just could not be calculated definitely and were "irrational". It mean that they had to live with crazy things!
Just because it's "irrational" doesn't mean it isn't real!
The science of the last century also found extremely important unsettling examples of very similar kinds of paradoxes, even in the so called hard sciences of physics and maths. It is not just Pi that is intrinsically indefinite or "irrational". The whole of Quantum Mechanics, as formulated in Heisenberg's Uncertainty principle, is based on the similarly unsettling notion that Uncertainty is an inherent feature of reality at the sub atomic level. If you treat light as a particle and measure its exact position, you get a fuzzy reading for its momentum (energy) but if you treat it as a wave, and measure its momentum, you get a fuzzy reading of its position. You cannot pin it down and have readings for both momentum and position. Nor can you pin down best value in the voluntary sector either!
It is worth noting here that Quantum Mechanics also had some radically important things to say about "measurement" which has other parallels in human affairs. The physicists who developed Quantum Mechanics noticed that the observer, and the measuring apparatus, could not be left out of the theorisation of what was going on. They had to be considered in their interrelationship with what was observed and measured. As we have seen, the act of measuring in human affairs, also has profound, and destructive, implications in human relations activities. R.D.Laing said much the same thing in psychiatry, much to the displeasure of the psychiatric establishment.
In mathematics and logic too Kurt Gödel dismayed many of his contemporaries with his famous theorem which proved something in mathematics, rather like the famous paradox where a Cretan says: "All Cretans are liars". (If so he is a liar himself - and what does that make of the statement?). Gödel in effect proved a "mathematical statement" which means "This statement is unprovable." According to Professor of Mathematics at Oxford University, Roger Penrose, Gödel's theorem undermines the notion that human consciousness can ever be considered only as computational activity. Indeed it cannot. It has an emotional dimension. Organic life forms developed cognition firstly, as a way of responding to their environment. These cognitions were experienced for countless millenia of evolution as feelings, not as mathematical calculations. Feelings, in their interpersonal dimensions, are emotions. (see my review of Fritjof Capra's Book "The Web of Life" on my web site on the nature of consciousness and how it evolved).
So intrinsic indefiniteness, and the need to take into consideration unquantifiable intangibles, is nothing new in the history of thought - however, it is rather inconvenient for current policy. For if it can be shown that it is inherently impossible to accurately quantify and measure the activities that currently happen in the public and voluntary sectors then what becomes of policies like "Best Value"? The notion of "best value" implies that it is possible to measure value in some objective way in order that one can say what is "the best". "Good enough' is more subjective, negotiable, intangible. If measurements can be shown to be intrinsically unmeasurable and/or if the attempt at measurement undermines the aims of measurement (the pursuit of "best performance" or "best value") because it makes performance worse, then, whether these things are enacted in law or not, they are still a nonsense.
In Conclusion
"Best possible" and "continuous improvement" should be replaced by "good enough"
Excessive measuring should be done away with - e.g. Nottingham city council asks for quarterly monitoring figures and returns whereas the county only asks for yearly figures. This is a trust question.
For many purposes the only voluntary sector figures that should be needed are attendance figures - including consistency of individual attendance and equal opportunity data. If people turn up voluntarily, and keep turning up, it proves they are getting something out of the activity or project.
Subjective evaluations of the mood and emotional quality of a project should be a valid way of evaluating it - this requires a team of sincere, on the spot, operationally experienced officials.
Where a project 'feels bad' it may be legitimate to collect new measures as to why it feels bad - e.g. of stress levels and why people feel bad or stressed. Improvement then becomes a more flexible evolving idea, focused on emotional quality.
Procedures for self management of quality will often be better improved by supporting whistleblowers - i.e. quality and ethical control from insiders, at the ground level, as much as from the top and from the outside.
Brian Davey MSF - Written in a personal capacity
E mail Brian@bdavey.freeserve.co.uk
January 2001
© BRIAN DAVEY