Someone has finally written down what good AI in care looks like

The AI in Social Care Alliance launched this year, out of the second AI in Social Care Summit run by Digital Care Hub, the Institute for Ethics in AI and Casson Consulting. It is worth a read if you have not seen it.

Two things in it we think are exactly right.

The first is their warning. They say AI in care is not neutral, and that it can entrench poor practice or concentrate power in the hands of technology vendors. That is a real risk and it does not require anyone to behave badly. A system that decides what a good note looks like will slowly teach a team to write for the software rather than about the person. A model that flags concerns can be confidently wrong about someone who cannot argue back. Records end up inside a platform a provider cannot afford to leave. All of that happens simply because a supplier never thought about it.

The second is their definition of responsible AI. That it should support, not undermine, the values of care. Independence, choice, control, dignity.

We agree with both, so here is how that plays out in what we build.

People decide, always. Lola drafts the note and flags anything worth a second look. She does not decide what was true about someone's day. A support worker reads it, changes what is wrong, and saves it. That is not a safeguard added on top, it is the shape of the product. A system writing records nobody checks will gradually invent a version of a person, and six months of that and the record stops describing anyone real.

Time given back, not filled. Case notes go from around an hour a day to five minutes. That number is true but it is the least interesting part. What matters is where the other fifty five minutes go. If they get absorbed into more admin then nothing has been achieved except a better looking spreadsheet.

Built this way round from the start. Most systems in this market were designed before any of this existed and have added AI to the outside since. Ours was not. That is not a boast, it is the reason we can put people at the centre rather than fitting them in afterwards.

The Alliance is writing a constitution for AI in care, with clear lines nobody should cross. Some of those will land on things suppliers do now without thinking, and we do not assume we come out of it perfectly. We would rather be held to it than argue we are already fine.

Three questions worth asking any supplier

1. What happens to my data if I leave? Not whether they can export it. How long it takes, what format it arrives in, and whether it still makes sense when it lands.
2. Who decides what the system flags as important, and can I see how? If nobody can explain it without fetching an engineer, that tells you something.
3. When it gets something wrong about a person, how does my team overrule it? And does anyone at the supplier's end ever hear about it.

Care records outlive placements. A child in residential care might read theirs one day. Whatever writes them deserves some thought.

Details on getting involved with the Alliance are going out through the Digital Care Hub newsletter.