The EQF, Evidence and Innovation

After an interesting weekend, where the Twitter seemed to explode with a whole host of accusations towards the College of Policing, and by-proxy towards me (having previously worked there for 12 months), there are clearly some issues that need ironing out. There are many questions that need answering, some via the College, but others are about our culture and the way that we – as a collective – view and react towards change within our midst. Amongst many things, this blog will discuss what ‘evidence’ actually is in the context of change, and how it is used.  I can’t speak for the College and I won’t try, I think there is lots of work needed to connect with officers and that this weekend is a great example of why.

 

When writing these blogs I am trying to bridge a gap between academic writing and police blogging. It is no small feat and requires me to re-write and re-visit each paragraph on repeat, until I finally hit publish and await the ensuing discussion. I tread a fine line between not using purely research based writing to inform because it is dry and often difficult to read, and leaving it out and posting an opinion piece. For this reason I leave out academic referencing, and more often than not, the authors too. Within the first few paragraphs of my last blog here I actually discuss about 50+ studies. It is in shorthand and you can’t  see the individual studies, but I have written discussions on the same studies several times (including for my recent MSc). I discuss how ‘good’ the studies were, (with regards to their methods) and what they actually tell us when you gather them together. The evidence is evaluated using the Maryland Scale in terms of how reliable it is. The lowest standard of evidence is opinion, rising to RCT trials. I’m not totally sold on this model, but it provides a useful guide and is often used across Policing as way of evaluating studies.

cnmc-nlss-02-eng

So, what is it about academic study that everyone is shouting about? “It’s not policing and policing is a fast paced environment., where quick decisions are made, and we deal with disorder and high risk crisis incidents etc. etc.” This is a commonly held view in policing, and to put it very shortly, it’s total tosh. I’m not usually this direct, and I know the middle road is usually the best to tread, but this needs saying: It’s anti-intellectualism, it reveals stigma, and at its basest form it evidences prejudice.  Learning from the rigorous work of others is exactly what creating a ‘knowledge base’ for the profession is all about. It stops us making the same mistakes again and again and it ensures that we as a collective remain open to new learning. There’s actually a really good article called the ‘Dialogue of the Deaf’ (I can’t post it as it is subject to copyright) and it details the issues between the Police and Academia really well. The police say that the research is not operationally useful, and the academics conversely want to pursue knowledge for knowledge’s (and Academia’s) sake. This leads to an impasse, where the cops want useful knowledge that makes a difference, and academics pursue knowledge that will get them published. This impasse is dissolving, and we are seeing some great collaborative work that hits the best of both worlds.

The scientific method

I don’t want to go into too much detail as I can hear the snores already kicking in, but the methodology of academic study is the bit that creates its value. What do I mean by methodology? It is the rigorous questioning of the way that the data is gathered and assessed, that ensures its value later down the line. It isn’t about quick snapshot study, it’s about planning and executing, and then testing and re-testing, and then questioning your findings. This is a reflective cycle – this means that you are constantly checking and self-checking, and then peer checking your analysis. It means that you are testing out ideas (hypothesis) against data that actually means something, instead of collecting data that holds huge inherited bias. This is the difference between real study, and pseudoscience. Here’s a quick graphic that compares the two:

scivspseudosci3

I heard – during many exchanges on Twitter yesterday – that academic study has no place in policing. Well, it doesn’t if you want to be locked in the column on the right? If you were to take a selection of the way that we used to view police ‘performance’ during performance culture, you will see that the methods that the police used were firmly sat in the right column. But as Simon Guilfoyle has been discussing recently, that’s what happens when you don’t provide training in the methods used to manage properly (discussed in his latest blog here). There are huge risks if we carry on sitting in the right column, like a lack of institutional/personal learning, doing harm when we think we are doing good, and making decisions based on misunderstood data that ultimately harm our staff.

In shorthand, if you view ‘evidence’ as belonging in the right column, there are big issues with drawing meaningful conclusions. It doesn’t mean that the info isn’t useful, especially with regards to quick and dirty opinion polls. They are great for examining opinion, but opinion isn’t evidence of anything other than opinion, and not liking something isn’t evidence that it does/doesn’t work… This is really common in any workplace, and leads to Semelweiss type problems, where new things that work are rejected by professional communities because they are counter-culture. Some changes that are really positive can be rejected by the workforce because of the way that they are perceived/implemented – not because of whether they actually work (read anything on Change Mgmt. by Leandro Herrero/Peter Senge if you want studies on this).

Evidencing Innovation

This is the most common issue I have seen, and it is really dangerous. The misunderstanding around evidence based practice is causing a questioning of innovative changes. It’s right to question – and it should be encouraged at all times, but using the call for evidence to stifle new practice is totally counter-productive. Here is a break down of what Innovation can look like in the police, (follow the link) but if you want it shorthand, if you don’t take risks or have a supporting culture behind you, it makes it ridiculously difficult. I think the risk taking is happening more often now, but the supportive culture is absent. This is a result of many things and cannot be ‘blamed’ on anyone. It’s something we should all take ownership for though, and work to develop if we are ever going to see any positive change take place.

What is happening now in the policing community is interesting to see, as ‘Evidence Based Practice’ – previously debunked by the community as ‘academic interference’, has been taken on, bastardised, and it is now being used as a weapon against change.

‘Where is the evidence that Direct Entry at Inspector works?’

There isn’t any. How can there be? We’ve never tried it.

Where is the evidence that police need degrees?’

There isn’t any. How can there be? We’ve never tried it.

Where is the evidence that Police Now works?

There isn’t any. How can there be? We’ve never tried it.

I could go on. The above questions lead to speculation at best. If you were to take Direct Entry as a great example, I could tell you that it works in many, many places around the world. But I can’t tell you if it will work here. But that’s ok. Let’s be comfy with that and try it out. If it doesn’t work, we’ll evaluate it properly and we will see if it was worth trying. There will always be risk in trying something new, but there are enough checks and balances in place to mitigate that risk (like lots of other people to learn from and keep an eye/develop them in the workplace etc.).

Another discussion point: “Why are we rolling it out if we haven’t evaluated it properly?” Well, forces have asked to be part of the trial. This makes the trial bigger, and in turn this allows for a better evaluation to take place. Basing a country-wide full roll out on the experience of a small amount of people is pretty thin, so let’s widen that base out and make the evaluation more robust. The better the evaluation, the more learning we all take from it. This is essentially growing the sample that you use to draw conclusions from, and the greater the sample, the more representative the results.

And it could be that the schemes fail – and many people will love that (clearly), but early indications are that they aren’t the apocalyptic disgrace heralded by the doomsayers. Let’s wait on the full evaluations before we jump all over something that may be very positive for staff in the workplace.

The real evidence for change

By all accounts (and many staff surveys), workplace wellbeing in the police is pretty poor in many places in the UK, and we have just seen the largest proportion of senior officers ever under investigation for potential misconduct. On these issues alone, I would say that we have a solid evidence base for the reasons for some change to take place in the way that we see leadership.

Having just conducted a study myself on promotion and selection within the cops (specifically with relation to BME candidates and frontline perceptions), I can say that some officers’ trust in current police promotion systems in the UK is appalling. They do not believe that the systems select the best candidates consistently, they believe they are often unfair, and that they suffer lowered morale because of them. The information around the systems and the way that they are used can promote an environment where nepotism and networking are perceived to hold higher value than competence. This research will land shortly hopefully and it is currently changing practice in individual forces in the UK.

Finally: culture. There are many positive aspects to our culture, but there are many repeatedly evidenced issues with our culture (discussed here) that we really need to sort out. We are an internal looking beast, and the scepticism and cynicism with which we view ‘outsiders’, the suspicious and socially removed behaviours that we evidence every day on twitter, and the – let’s face it – downright nastiness (that verges on bullying) shown towards those with contrary views to the masses are evidence within themselves. They have been studied repeatedly by ethnographers and they are enduring behaviours that are just toxic. They don’t make for a healthy work environment, and they don’t make for an environment conducive to real and actual debate. I shall continue to bang this drum – and I don’t care if I get pilloried – as ‘openness’ is important for the police to develop a better relationship with the public, and we have to try and ‘open’ our views and test our assumptions as much as possible if we are ever to improve on this.

I suspect, that the way we view these changes has much to do with our own personal circumstances. During change, we have a tendency to immediately ask: “How does this affect me?” There are some great books on this by John Kotter that deal with coping with change in life/organisations, and this is an enduring (and researched) trait. It leads us to make assumptions like:

Direct Entry is bad because it affects my promotion chances.
Direct Entry is bad because I have had to work my way up there.
Direct Entry is bad because my knowledge is special and outsiders don’t have it.
Direct Entry is bad because they will be relatively poor decision makers.

Now these may all be personally held beliefs reinforced by experience. I can’t question them, they are the beholders’ and totally sacrosanct. I would however question the way that others’ views are seen. Do their contrary opinions hold equal weight? Are contrary ideas seen to be accepted and debated? Or are they attacked and seen as some sort of virus?

Are these questions above the right ones to be asking at all? Should they not be – and we are a public service – about the service we provide:

Does direct entry create different leaders that benefit the service?
Is it possible to do these jobs without the lengthy experience beforehand?
Is the knowledge really that special? Or are there lots of other ‘knowledges’ we can learn from?
Is it possible that new types of decision making are a good thing?

These can be explored during a pilot, but they can only be explored during a pilot. We can’t evidence them. Is it possible that direct entry may change the service for the better? Of course it is! Is it therefore something that can explored? Again, of course it is! The ‘openness’ I am discussing in the blog above isn’t about agreeing with the idea, it’s about having the mindset to explore new things with a critical – but tolerant – eye. Acceptance of difference (including difference of thought) is a choice.

I’m not sure the tolerance is there yet. Maybe we can all work on it some more?

 

Disclaimer: a) I never worked on Direct Entry (although many assumed I did), and I’m not totally sold on it operationally at Insp level. I think a lot of work has to go into designing the right learning prior to it happening, but I have faith and as an idea it is well worth a try. I think the reservations can be overcome. b) The EQF was not my idea and I am not ‘driving’ anything. I did some of the background research and I believe that improving education is always a positive thing. I don’t believe in total graduate entry – which is good, as the College is also looking at non-graduate apprenticeships. And I also believe the service is full of awesome people without any transferable qualifications – and that is wrong, we should be accrediting those skills. 

Unfolding some rumours about the EQF…

So, I have finally – after vowing I wouldn’t – decided to come and discuss the Educational Qualification Framework (EQF) for Policing. Why should I come into the debate? Good question really. I guess first, I’m a cop, so it affects me, and it affects the people that I am working with. I have also just finished working on the Leadership Review, part of which included some research on whether education was an important part of future leadership. Indeed, I also did a lot of the comparative work on the EQF, looking at how other jobs structured their professions from start to finish. It totally opened my eyes.

The reason I’ve decided to blog, is that the info that is out there is sparse. I can’t comment on other’s communications, or indeed other people’s opinion, as it is theirs. I also don’t contend how people ‘feel’ about this. It is their prerogative and totally individual. I can shine a light on some of the research that I found and why a lot of the myths/concerns around at the minute don’t stack up. It may help paint some foreground and some background into the debate, and context is always so important.

Brace yourself, it’s a long one…

Current research evidence and making an evidence based decision

The current research around this in the UK is very, very sparse. There have not been many studies on whether being a graduate increases the quality of the officer. People have assumed that this is a major driver for its introduction, and this is just untrue. Cops up and down the country are making difficult decisions in complicated circumstances all the time, and doing so with great compassion and skill. Indeed, within environments where we work with partners, cops are often the only ones without a qualification, yet their performance still causes them to become leaders in those fields.

So, if there’s no evidence in the UK that suggests cops need to be graduates, why should we consider it?

Research elsewhere does indicate that certain things do happen. There’s good evidence from the US (in research terms) that puts together many studies, that shows that graduate officers use less force than non-graduate officers – and before you say that’s because they avoid confrontation, that was ‘controlled for’ in the studies, and there is no difference in arrest rates. There is also good evidence that graduate officers receive less public complaints.

There is weaker evidence across a number of studies that only hit low levels on the Maryland Scale of Evidence (this means that they aren’t as rigorous). These indicate higher levels of empathy/emotional intelligence and higher uses of alternate disposals. I would be loath to rely on these, but it is an interesting area for further research.

And on that note, some commentators have mentioned that there is a lack of an evidence base to make this decision in the UK to move to a graduate only profession and that there should be further research before doing it. I think this would be totally the right thing to do, but right now, the cops are on a burning platform, and we need to move quickly before things start collapsing. A systematic review would take many years, and right now we do not have anything like the time needed to conduct it. If austerity carries on at the same rate as it is forecast to; by 2021 the overall level of officers will fall by half what it was 5 years ago. Half the amount of officers…

What does this mean? Well you may have caught commentators from the NPCC talking about how this will affect their basic ability to answer 999 calls, and if we can’t answer 999 calls, how are we going to train our cops properly? This is a serious consideration… As training becomes more complex and specialist, how are we to deliver it? In many forces the L&D function will almost cease to exist entirely, with only space for mandatory training sessions based on legislative change. The cops are currently one of the only professions that pay their recruits a full wage to train, without – in turn – asking them to work. This is an amazing thing and something that has been of immense value in the past, but when we can’t answer 999 calls, can we ethically keep this function? Keeping people safe has to be our priority, and if that means taking our training out of that ‘fully paid’ role, that is what may have to happen. Am I happy about this? No. Would I like for this not to be the case? Yes. Is it something that I think should be preserved? In principle, yes, but if it means less cop cars or less frontline officers when things are stretched – I’m sorry, the public come first.

People have seen the ‘shifting the burden of paying onto the individual’ comment and assumed that this decision is about saving money, but it isn’t, it’s about preserving core function. We are a public service, and we need to conduct our core duties that allow us to remain viable – this may mean that a lot of things that we hold dear may disappear. It’s a travesty, but saying or feeling that it is a travesty doesn’t stop it from happening. As a side note, if we begin to fail at core function like 999 Response, this paves the way for privatisation at a frightening rate – please bear this in mind…

So, in summary, there is some evidence that being a graduate changes the behaviour of officers. Is it enough to conduct wholesale change? No. Have we got time to conduct a proper research review of graduate officers in comparison to non-graduate officers? No – austerity is forcing change at a far faster rate than would allow it. And the final chunk of ‘evidence,’ based on current profiles and projections, the cuts are at a level that would remove most – if not all – of the internal learning and development function, how would we train our officers?

The Knowledge Base

People will have heard these terms being thrown around over the last few years: ‘The Knowledge Base,’ ‘the Evidence Base,’ or the ‘Profession’s Body of Knowledge.’ They all mean the same thing, but let me run them through with you, so this next bit makes sense.

When training to be a doctor or a lawyer, students will look at the great body of knowledge that the profession has built up over many years. They will leaf through books, journals, magazines, commentaries, and lecture notes. They will read about the failures of the past, the successes, the near-misses, and more importantly, they will learn about how more knowledge is generated. In medicine it is research, in law it is statute and case law (in the main), and they know that they will need to stay up to date with this knowledge throughout their career.

Where is the Policing Knowledge base? Good question, I wasn’t taught any policing history during initial training, there was no identified body of knowledge, and mainly I spent 5 months learning various bits of legislation rote. In fact, most transactional training since has been very similar. Many will be saying: So What? Good question, well here is what happens when you don’t have a knowledge base:

The profession makes the same mistakes again and again as learning from past mistakes does not take place. As a physical example, it takes huge mistakes like those that happened at Soham for system change to take place, but personally, I haven’t seen anything about Soham or what it meant for policing (in other words, we change the system and not the people).

Many wheels are re-invented. This means that pilot after pilot in new technology/practice takes place all over the country, and that knowledge is not shared. I remember testing body worn cameras over ten years ago. Ten years! The profession is only just now conducting research into whether they offer benefit. Even if we decide that they are amazing, it will probably take another 5-10 years before we have a uniform system in place, and even then officers won’t be able to access the lessons learned, because the Knowledge Base is currently so sparse. This is being developed now by the College of Policing, but think of the waste that has taken place over the preceding ten years – just on bodycams… Establishing a Knowledge Base makes policing a whole lot less expensive to the tax payer, because it stops us repeating the same problems repeatedly.

Lack of consistency. People mix stuff up around this bit. This isn’t about training clones, or even about training everyone in the same way. Localism and individual preference is really important, and quashing that is bad for a healthy profession. This is about getting core understanding out there, in a way that allows for basic decisions to be made in the right way. An example: If you are burgled, research has shown that you are far more likely to be burgled again within the next few weeks/months. It also shows that your neighbours are at far higher risk. I have personally seen many officers advise the opposite. Why? Because there is a lack of structured continued learning in the cops, and the Knowledge that we have was never taught during training.

What does extending the training towards a degree actually do? Well it compliments the transactional training (statute and legal requirement), and the craft that is learned from peers in the workplace. Let’s face it, most useful learning is done ‘in the job’ almost via campfire stories or visibly seeing others do the job. This is great if you think that the job is totally the same as it was a hundred years ago, but it has carried on changing over that time and many things have strayed behind – technology being only one example. It’s about introducing the ‘science’ to the ‘art’ and the ‘craft.’ It is the main difference between a trade, and a profession.

If you don’t make the move to make this a requirement, what happens? Well, you have people making decisions using common sense, plus the science and enhanced understanding, and you have people making decisions using common sense. Advising your burglary victim that it is highly unlikely they will be targeted again – common sense. Advising them to strengthen their security and pulling in crime prevention and providing extra awareness – common sense and science with enhanced understanding. Which would you want as a victim?

I need to add as a caveat here, that it has nothing to do with the quality of the person in the above scenario, indeed there is nothing that says a graduate does the second better than the first. What it does say though, is that the Knowledge Base will not make an impact unless it goes out as widely as possible.

And again, I move to the So What? There is a growing Knowledge Base that has to go out as widely as possible. Why? Because it helps us make less victims, and it helps us make better decisions. Right now, taking on this extra knowledge is optional. Some forces don’t like it, other don’t subscribe to it, and others love it and are taking it on quickly. What does this look like to the public? It looks like a service that may criminalise in one area, whilst another will not, it looks like a service that may make less victims in one area, whilst another will not, and it looks like a service that gives better victim care in one area, whilst another will not.

Core consistency is very, very important – and right now we don’t have the resources or the structure to make it happen.

Diversity

This is the rebuff that has been the most quoted. Let me ask something. There is no other public service that currently has the openness of the police. We ask for hardly any entry requirements (some forces do but they are the exception and not the rule), we train everyone and pay them whilst they train, we are currently actively seeking diversity and offer pro-active support to minorities, and we pay the same rate as other graduate professions. Right now, we are immensely attractive when you look at our barriers to entry.

And our current levels of diversity are awful. Let’s be honest, every other profession has higher levels of diversity, and they have far more barriers.

Here’s the question… Is it possible, that actually, our lack of barriers affects our status as a profession?

From personal experience, I know officers who are from a minority background who have received derision for their choice to join the police as their community holds the job in poor regard. Why? Because it isn’t a profession, and the level of structural learning is low-non-existent, and the culture is often perceived to be racist. I would hazard a guess, that tackling this status should be a priority if we are to recruit from diverse backgrounds. 23% of current under-graduates studying come from a diverse background, and they make up far less of the population as a proportion. Graduate entry could be attractive and actually help with recruitment.

Aside from that, the evidence in the EU and the US of bringing in graduate entry into policing affecting diversity indicates a slight rise in diversity following its introduction – as long as positive action and other entry schemes such as apprenticeships remain in place (which they will). Police Now (Graduate scheme) has also recruited (in its first intake) far higher levels of diversity than standard recruitment across the country…

Summary: Suggesting that it will affect diversity recruitment is an interesting suggestion, as our diversity recruitment is poor now and we are one of the most open public services. Using the available evidence, we can gain an indication that this issue won’t raise its head, in fact it may even go in the opposite direction. The truth will out in the future, if it happens, and we shall see. If it does, then it will need addressing.

Transferability

This is the one that I will likely receive most abuse for (yes, abuse, as I have seen pretty awful abuse directed at graduates across the service since this discussion started). The current structure of our job does amazing things for people. It provides security, it provides structure, and it provides stability. There are very few jobs that allow tenure, and yes we have paid for it by withdrawing our industrial rights. In the past, when the pensions were excellent, finding another job was almost seen as a compliment to your life following retirement. With the pension re-structure, and the tenure extensions, the chances of staying in the job for life are limited. I have spoken with many officers who say that they will likely move as they can’t foresee themselves on the frontline working 24 hour shifts at age 60 (and who could blame them?).

This is where the change in the ability to transfer out of the job kicks in. Because the service is well paid, and because most similar jobs (in terms of pay) are at graduate level, our cops may face a significant pay cut and cut in their quality of life when they leave. How is this fair? Cops are functioning at graduate level now. They make fast paced, high risk, complex decisions every day, in an environment filled with graduates from other services. That is one of the reasons why accredited prior learning is being introduced, to allow serving officers the option of accrediting their experience, so that they do not get trapped in a job that they would like to leave, without significantly affecting their quality of life.

Staying in a job that is affecting your health, with few alternatives available, without the option of pursuing higher level learning (such as Masters etc.) could be pretty toxic to the future workforce. And yes, this means that policing may become a 10 year profession for some,(or even shorter) but the changes from a 30 year profession to a much longer one was not of the police’s making. Ensuring that the service supports the new environment is pretty darn important for wellbeing, and for culture, and for putting our current awesome staff to the same level in the external environment that they are performing at internally.

To reiterate, this is optional. Do you want to stay the whole 38+ years and don’t want to accredit your learning? No problem. Totally your choice as a cop.

Finally

I have talked extensively about the risks above. What happens if we don’t do this? Here’s a pretty big one… The future of policing does not look like the policing of ten years ago. It will be fast, highly scrutinised, highly visible, and decision making will be far more complex. Making a decision to arrest a DV offender won’t be as simple as ‘positive action,’ it will require knowledge of the accepted professional practice, exposure to the changing Knowledge Base, common sense, and the ability to communicate.

Example: When you go to a doctor, they will know the drugs that they can prescribe through their exposure to the medical Knowledge Base, they will know their current area policy for prescription, and they will need common sense and communication to make that person comfortable and understand what is happening.

If those doctors were just using their common sense and their communication, and had no knowledge of the available research and made the wrong decision, there would be some pretty tough questions to be answered. And they aren’t under the microscope all the time, and verbally challenged constantly, and dealing with highly drunk/drugged people (in most cases), and probably being recorded via mobile phone.

The cop environment is tough now. As the research grows around What Works, officers and constabularies will have to keep up to speed with it. I’m not talking about reading academic journals, I’m talking about continuing professional development. We can’t train in two years, and then do defensive tactics/fitness test/public order until we retire. The world is changing faster than that, and as such so will our accountability.

If you know more, you are accountable for more, and having just seen millions of pounds go into developing the Knowledge Base, I would say that getting our future cops up to speed with it isn’t optional, because it protects them as much as possible.

There’s a perfect storm coming if we don’t get ahead of it, and protecting our cops in the midst of it requires the introduction of science/research understanding. This isn’t a slight on any serving cops, it’s not a barrier designed to keep people out, and it’s certainly not an attack on the level of service we provide now (which is exceptional in most cases). It’s about improving the levels of learning, whilst having no money to do so, and protecting the wellbeing of our cops with regards to future scrutiny and visibility, and their desire to leave the cops should they so wish (there are other reasons that I won’t touch as this is too long already).

As a final note, the slating of graduates as police officers has just been shocking. I have met some cops who have been poor who have been graduates, I’ve also met some cops who weren’t graduates, who were also poor. That is a recruitment/development issue and not an educational one. Tarring educated cops as ‘bag carriers’ is nothing but anti-intellectualism and serves no other purpose than to divide. Gone are the days when graduates entering the profession was rare, it’s just a pity that the same attitudes have stuck around despite the world moving on. This is a debate that we need to have, at least keep it civil and respectful – as befits the service which we represent.