IdealGovernment has wanted this for years. But – as he himself proves – it won’t be easy. Let me get some things off my chest. Then let the new era of civilised and mutually respectful dialogue commence.

I don’t agree with everything Mr Wills said, so I’ve taken his speech and commented in line. Attention conservation notice: this is quite a long post

Databases lie at the heart of this revolution. They offer the opportunity to improve dramatically the efficiency and responsiveness of public services. Take the Tell Us Once project, for example.

But the TellUsOnce problem won’t be solved by databases. If it was, you could have solved it by 2005 (as you promised you would in 2000).

OTOH a VRM platform would show this problem solved for so little money that the invoice wouldn’t even cross your desk – a fraction of the cost of yesterday’s “One Place” web site (which in turn you could have had for free a decade ago if you had practised respectful and courteous dialogue and listened to Stef).

The increasing sophistication of data management has sparked serious public concern about privacy and civil liberties

We’d be fine if the approach were sophisticated. It’s the crudeness, combined with raw power, that concerns us.

Go onto Google UK and search for ‘UK government big brother state’ and you get one and a half million entries.

WTF? Is this how we gauge if it’s of public concern? As a joke maybe, eight years ago. Oh look: go onto Google UK and search for: ‘Wills eejit or ****?’ and you get 1.21m entries. Does that make it the question everyone is asking?

He talks of ‘striking a balance’ between security and liberty. He’s no Benjamin Franklin. It’s not about striking a balance. It’s about getting it right because we need both privacy and security in the systems which underpin essential public services. Mr Wills claims that kids can’t get free school meals, that voters can’t vote without data sharing by public bodies. This is absurd. He says hardly any constituents complain about surveillance but they demand CCTV in their hundreds.

To reconcile all this we need, he says

“democratic discourse, rational and mutually respectful discourse, wary of anyone, on any side of the debate, who claims a monopoly of wisdom. These issues are complex and difficult and resolving them will require intellectual rigour, a willingness to learn from experience and to engage continually with alternative points of view. Only through such a democratic iterative process can we hope as a society to resolve this issue satisfactorily.”

Hurrah. This could be straight out of the Ideal Government guide to, well, ideal government. We like that.

Sadly, such a rational, respectful discourse, so essential to the creation of public policy on this crucial issue, has been largely absent in recent years, replaced all too often by reciprocal caricaturing and stereotyping, with understanding and respect all too seldom present. And this matters.

This is true too. I must say I welcome his recognition of it, and his acceptance that

Government must take its share of the blame for this failure of discourse.

Mr Wills’ spin doctors had leaked his speech to a class-obsessed tabloid journalist David Aa. I can see now that when I saw the next bit in that hostile context I misread it:

Too often, we have been overly defensive and dismissive of criticism. Government believes it is acting benignly and legally and has not adequately recognised the fears of those who believe this is not the case.

Mr Aa is paid to be annoying, and it works. In my irritation I took the extract to mean that government was defensive and should be more assertive, but Mr Wills’ point here (and the context makes it clear) is a more subtle one which I welcome.

He then says companies make mistakes too. True, but we withdraw our business from them when they are mediocre. They’re not entrusted with passing laws, taxing us, and trying to run public services. He goes on

Where government gets it wrong, we are learning to hold our hands up and take immediate steps to put matters right. The loss of Her Majesty’s Revenue and Customs disks triggered radical reforms of data security in government.

Her Majesty’s data was not on that disk. Mine was. Nevertheless, horrified, many of us welcomed that loss, because it finally started to show how bad things had really got.

Did it trigger radical reform? Well, it certainly triggered half a dozen reports from tame senior officials and management consultants. It triggered episodic bans on Blackberries and memory sticks, and the glueing up of USB ports. But we have yet to see the radical reform we need which is to restore control of personal data to the individual.

When we recognised that data sharing provisions in the Coroners and Justice Bill had been too widely drawn we immediately withdrew them.

It’s true. Eventually they did. This would be a good moment for Mr Wills to thank the selfless NGOs who campaigned so hard to rectify this awful mistake he was trying to make. But instead he speaks of

opponents…too quick to assume the worst of government without any evidence to support their assumptions, replacing argument with rhetoric.

But…but…but how then did the unpaid NGO volunteers successfully win the argument against your proposed policy and show it was so ill-conceived? Do you recall how hard you and your officials resisted?

Michael: you and your colleagues will find this cultural change you call for harder than you yet realise. But as long as you try, and appear sincere in your endeavour, we will support you every step of the way. Have we all taken on board the rules you call for? No rhetoric. OK. No rhetoric.

To reject all the benefits that databases offer the public, simply because a mistake might be made, is to strike the balance in the wrong place.

1. Name the benefits. 2. This isn’t about striking a balance, it’s about doing it the right way.

Should we really avoid trying to do all we can to prevent another Soham tragedy?

Oh, please leave the memory of those poor girls in peace! Why blame your poor personal data practices on them, and on the equally unfortunate Victoria Climbie? Michael: this is rhetoric.

Or stop doctors accessing vital medical records?

Doctors can have it with my permission. I choose not to share it with bureaucrats and the forces of law and order. That preference will not endanger society.

Or fetter the provision of welfare entitlements, such as free school meals, for the most vulnerable?

Just get your act together. Deliver the services to which people are entitled, without hoovering up their data when they are at their most vulnerable and brewing it up into HMG’s patent toxic soup.

Basic principles for protecting the use of data are that it should be proportionate and necessary

Indeed. Furthermore, that in the absence of a specific and lawful purpose there should be informed consent. Assuming that 12 year olds understand this and are cool about it is not sufficient. Allowing hundreds of thousands of officials access to childrens’ records or to identifiable medical data is unlawful, just as taking and retaining the DNA of innocents has proven to be.

But you acted all surprised when you lost the Marper case. And you’re behaving as if we had a proper implementation of the EU data protection directive, and are acting all surprised that Europe is taking enforcement action against the UK.

We don’t live in a database state as much as a database society.

Yes, but we’ll sort ourselves out with the Twitterverse. Government runs the state part of it. That’s what you’re screwing up, because of the history of groupthink and the poor dialogue you rightly point to. That’s what we want whoever wins the next election to sort out. Hey! It could be you! Maybe.

They deliver real benefits for the public and it skews debate about the challenge they pose to all of us if anyone ignores this or pretends otherwise.

Let’s see if we can find real evidence for these benefits, because there’s a strong case to be made that the database state is designed to serve an unholy alliance of administrative convenience and security fears.

Meanwhile it is becoming everywhere apparent that a wholly organisation-centric “CRM” approach to life is nothing like as advantageous to the public, or indeed to large organisations, as it was sold as being by the management consultants which Alistair Darling has just banned.

What’s it costing you per data subject to keep records up to date in WUYJ, MIAP, BusinessLink, ContactPoint, eCAF, CfH? How accurate are the data? How complete? And how much duplicated? How much is explicitly permissioned by the data subject? Is it proven to a legal standard that in every case government is holding only data which is necessary and proportionate in a democratic society? Can you demonstrate clear auditable and informed consent?

But, like all technologies, databases can do damage if misused. The issue is not whether to have them but how they can be deployed without damaging privacy.

Ah! Phew! Hooray!

It’s a question of balance and the challenge is how to strike it.

No it’s not! It’s not about balance, any more than climbing to the moon on a ladder is a question of balance. I don’t care how good your balance is: it’s the wrong way to go to the moon.

You get accurate data at lower cost and personalised services without privacy intrusion by putting people back in charge of their own data. The Internet works at both ends, you know. You just can’t have every part of public services grab every piece of data they can about everyone, take away the barriers to data sharing, then hope to create an accurate picture of everything such that you can eliminate fraud, keep the public safe, and provide personalised services. Try doing the maths.

You know all that great “Power of Information” work the government has done on opening up APIs and letting public data out? Sometime soon a Secretary of State or a PM will announce, in a second such major policy shift, that the really big prize is in how we work with personal data. Government will relinquish the desire to own and control it. You will open up government APIs and let people’s structured, scalable private data in, under thier control. You will leave people in charge of their own lives, which is how reality is because we have to put all the pieces together anyway.

I repeat: this is not a question of striking the right balance. It’s about creating a secure platform for personalised services and new value. Getting it wrong, which we described in our report Database State, wastes vast amounts of money and of people’s time; it fails to deliver good customer services and breaks the law. We the taxpayer are thus intruded upon, failed by public services, caused to waste endless time sorting it out. We’ll have to pay to get it wrong and then pay again to see it all put right.

But where we should have a constructive dialogue, we have all too often an impoverished discourse where slogans substitute for evidence.

Too true. I’ve seen so much of that from government politicians and officials. I’m so weary of it. So many of us are so weary of it. But it cheers me up when NGO-world gets creative and cheeky. I love the pertinent art, the campaigning. Not just because it’s often witty, but also because it’s based on a deeper truth, coupled with a better sense of human nature, than your officials have served you up with.

The Rowntree Report on what the authors called the Database State is a good example of how the public discourse is flawed.

It shows how bad things have got. It’s pretty shocking that the fact that much of what you’re constructing falls outside what it is legal to do in Europe should come as such a surprise. It has taken you a full NINE MONTHS to reply to it. Did you really not consider these questions before you embarked on Transformational Government, and mixed up the security agenda with the public-service agenda? Was that wise? Don’t shoot the messenger here.

This could have made an important contribution towards meeting the challenges of new technology. The subject matter was important and its academic authors have a distinguished provenance.

I think you’re building up to shooting the messenger.

However, a detailed reading of the report reveals it was riddled with factual errors and misunderstandings and reached conclusions without setting out the evidential base for doing so.

Argh! You just shot the messenger!

So opaque was its methodology that it has taken months to work through it to respond in detail.

Stop shooting! Remember, this is the new era of respectful dialogue!

The methodology is clear as day. We wrote up 46 large databases (with precious little help from government). We discussed them against European criteria for legality. During the course of the work the Marper judgement confirmed that the DNA database was indeed illegal. We gave all our references.

I’ll get on to the government’s response in due course. It has taken their however many dozens of staff nine months to reply, so working at the same pace I should be ready to reply by around 2030-2050.

I hope that all those who read the original report and provided publicity for it will do similarly for today’s response to it.

I’m sure they will if it’s interesting, credible and passes the Mandy Rice-Davies test.

It is important that we now move beyond rhetoric

I think I may find myself quoting this again, perhaps frequently We may need a new acronym to go alongside Wibbi: IMBR.

new and detailed dialogue between all concerned to ensure that we seize the opportunities of this new information age while protecting ourselves against its risks. So when government is considering how data might be used for the public good, the voices of users and practitioners can be heard. That requires an open, constructive approach on both sides.

Yup, we all sign up to that. High bloody time.

To that end, I am announcing today that the Ministry of Justice will host an event early in the New Year to consider how we approach the data sharing aspects of reforms to the electoral register.

Alright then. But not in Feb; I’m in Iran.

He goes on to talk about cross-referencing the electoral roll with the NI number. Well, you need to do something to make this process more secure. Not so sure about the NI number: it might suddenly look as if we have 80m registered voters in the UK.

Those identifiers will not be available to the public, for obvious reasons – they are solely for the electoral registration officer’s use.

Not obvious to me at all Michael – I think you have organisation-centric assumptions engrained so deeply you’re not even aware of them.

I also intend to jointly host an event with Delyth Morgan from the Department for Children, Schools and Families which will focus on ContactPoint.

Clearly it would have been helpful to have this dialogue before you specified the system and started spending hundreds of millions of pounds of taxpayers’ money on it with Cap Gemini. So when we say “better late than never” it is with quite a heavy sigh.

We can never be complacent about databases – the challenge in getting the balance right

No it’s not about balance! It’s about getting it right!

But we can only do this on the basis of a rational and mutually respectful dialogue between all concerned. I hope the measures I have announced today can be the start of such a dialogue

Indeed. IMBR. I think you, your speechwriters and spin doctors will find this a major cultural adjustment. You’re clearly not there yet. But we will support and uphold you in your attempt to do just that, and for my part (I cant speak for anyone else) I shall try to do the same.

Was this whole long rambling post worth doing? Is Mr Wills going to get his act together and sort this out? I checked out his currency with one of Westminster’s most effective and reliable lobbysists. The verdict: “I wouldn’t expend too much effort on trying to enlighten him.” I find that harsh. He’s still a Minister after all.

Let us give Mr Wills credit for officially ushering in – if not yet himself exemplifying – the era of mutually respectful and courteous dialogue about the right way for government to work with personal data in public services. From the moment I click on “Publish” on this post I undertake to exemplify it myself. Hurrah!

An entrepreneur whose fledgling business was ruined by a false entry in a court database has had his claim for compensation rejected by a High Court judge. The decision could set a broad and troubling precedent, because Mr Justice Bill Blair QC – brother of the former PM Tony Blair – ruled that the civil service cannot be found liable for the damage caused by its record keeping mistakes.

The case was brought by Andre Power, from Benfleet in Essex, against the Ministry of Justice, which is responsible for HM Courts Service. It was prompted by the failure of Chi-qi.com, a world music download service and social network Mr Power set up with a business partner in 2006.

After sinking tens of thousands of his own money into the venture, Mr Power fell three months – or about £5,000 – behind on his mortgage payments and was taken to Southend County Court by his lender in July 2007. He settled the debt in full before it came before a judge.

An unknown official at Southend County Court nevertheless registered a County Court Judgement (CCJ) against his name for £254,000, the full value of the mortgage.

Commenting on the error, Phil Booth, the national coordinator of the campaign group NO2ID, said: “In the database state, it’s the citizen who takes all the risk and suffers the consequences – even when he or she has done nothing wrong. My sympathies go to Mr Power. I fear he’s not the first and he definitely won’t be the last.”

*sigh* Grrrr. This is, we need hardly say, far form Ideal. It is the Kafkaesque as opposed to Orwellian face of joined-up government. Experian’s conniving role in this is beyond the pale.

Wibbi: government accepted liability for damages caused by inaccurate data. Experian too.

Wibbi subject access requests were free, and automated, so we could more easily see what garbage government and Experian are circulating about us.

Criminal Records Bureau errors lead to hundreds being branded criminals
More than 1,500 people have been wrongly branded as criminals or mistakenly given a clean record by the government agency set up to vet those workin g with children, The Daily Telegraph can disclose.

By Christopher Hope, Whitehall Editor
Published: 9:56PM BST 02 Aug 2009
The number of errors by the Criminal Records Bureau (CRB) has more than doubled in the past 12 months, despite intense pressure for it to improve its performance..The latest figures show that 1,570 people being checked by the CRB were wrongly given criminal records, mistakenly given a clean record or accused of more serious offences than they had actually committed in the year to March 31…

This is the Kafkaseque (as opposed to Orwellian) manifestation of the database state. I wonder if there’s a balanced scorecard anywhere of whether CRB on balance does a good job (taking into account people who have stopped volunttering to help out because they find the idea and cost of such a check offensive or simply not worth it).

Wibbi restitution procedures were designed in so there were an easy way to correct the inevitable cockups?

Er, perhaps Wibbi I took a balanced view and didn’t let the Telegraph wind me up first thing on a Monday morning….Does anyone actually have direct personal experience of CRB? If so please say what it’s like using the box below…

“Flawed scientific thinking” in the government’s proposed changes to the DNA database will leave it open to further challenges by the courts, experts have said, in a stark attack on Home Office plans to overhaul the current system.

The proposals, set out as part of a government consultation after the existing DNA database was found in breach of human rights last year, are based on exaggerated scientific claims and ignore the realities of persistent offending, two leading criminologists have said. “There is a flaw in the scientific evidence that sustains the government’s argument,” said Keith Soothill, emeritus professor of social research at Lancaster University.

Meanwhile, there’s an interesting supportive comment on Ben’s blog for the professor behind the Jill Dando work

one of the cleverest, most literate and numerate crime scientists in the world. He was asked to do a quick response to help the Met and HMG respond to a European Court judgement, but he couldn’t get access to the proper data, and did some back of the envelope stuff as a first-draft background. He knew it was a mish-mash and set about doing some proper research (which has now, separately, been submitted to a peer reviewed journal). The Jill Dando Institute was asked by the HO if his notes could be published
as an Appendix, and in the name of openness they said yes. He has been grievously upset ever since – long before Ben’s review. Hence the irony: this is not a story of a bad scientist but a good egg and just the sort of person who would read Bad Science with relish.

So, as part of a self-correcting blogosphere, I retract the comment about inappropriate shroud-waving. I was cross that the memory of that innocent victim be used to offer spurious validity. But what seems to be emerging is that this is (as per our usual category) simply official fibbing/bad stats from Ho Moffiss, and the the Institute is as innocent as porr old Jill Dando was.

In fact, this study from the Jill Dando Institute, attached to their consultation paper as an appendix, is possibly the most unclear and badly presented piece of research I have ever seen in a professional environment….This research was incomprehensible and unreadable. Anybody who claims to have been persuaded by the data quoted here is telling you, loudly and clearly in the subtitles, that they don’t need to understand a piece of research in order to find it compelling. Such people are not to be trusted, and if research of this callibre is what guides our policy on huge intrusions into the personal privacy of millions of innocent people, then they might as well be channeling spirits.

Buy Ben a pint and he might just tell you what he really thinks.

It seems to me the effects of VRM on public services will be of four sorts. It will improve public services. It will cut costs. It will de-tox the “database state”. And it is the perfect basis for co-creation of our public services. Instead of Transformational Government’s plan to pour concrete into the living heart of the state, VRM will allow a transformation that is living, human, and compassionate.

Is that how you see it? Have I missed anything big?

The parents of a girl who died suddenly have received a school letter demanding she improves her attendance. The letter, dated 16 March, said “students must have at least 92% attendance and Megan’s is currently 60.4%”.

With considerable restraint her Mum commented

“Megan doesn’t go to that school anymore. She’s been dead for two months now so it’s not surprising her attendance is low.” …A spokesman said Capita software SIMS (School Information Management Systems) maintained Megan’s details. “Megan’s name had been taken off the school roll when she died, and removed from the main school database,” the spokeswoman said. “However, unknown to the school, her details had remained in a different part of the computer system…”

NHS bosses leading the rollout of the Summary Care Record are refusing to take no for an answer from patients who say they want to opt out, Pulse can reveal.

GPs in the first wave of the rollout are being asked to send their PCT a list of patients opting out of the programme, despite fears the request may breach confidentiality.

Documents obtained under the Freedom of Information Act show NHS Lincolnshire, one of the first trusts to adopt the care record, wants GPs to hand over patients’ details, so they can be invited ‘to the surgery to discuss it further’.

The documents also reveal that when the PCT writes to patients, it will put the practice logo on letters so that they are sent ‘as if from the practice’.

My suregry wouldnt put up with this, They think it’s a diabolical idea. But they’re far more worried about the NHS as a whole….

To lower health care cost, cut medical errors, and improve care, we’ll computerize the nation’s health record in five years, saving billions of dollars in health care costs and countless lives.

Hey! Why dont we sell them NPfIT/CfH? On second thoughts…let’s just see how they go about it. This might turn out to be one of those regrettable “big promises”. But I suspect it will be a case of watch & learn.

Second up:

Instead of politicians doling out money behind a veil of secrecy, decisions about where we invest will be made public, and informed by independent experts whenever possible. We’ll launch an unprecedented effort to root out waste, inefficiency, and unnecessary spending in our government, and every American will be able to see how and where we spend taxpayer dollars by going to a new website called recovery.gov.

Why don’t we sell them the House of Lords’ lobbying system? On second thoughts…(repeat until discouraged).