For the the third in our series of discovery workshops, we invited people working with FOI in public authorities to discuss how a network might support them — and we had more than 50 attendees joining us from a range of organisations and specialities. 

Discussion was lively and informative, with many expressing a thirst for community and knowledge-sharing in their roles.

We began with a group brainstorming session to discuss the challenges and obstacles people were facing in their work with FOI. 

From this, we pulled out the four major groupings below, so that smaller breakout groups could discuss what attempts had been made previously to mitigate these challenges, how effective these had been, and what an FOI Network could do to help.

Volume/complexity of requests

The increasing volume of FOI requests being received (a challenge which overlaps with that of the lack of resources, below) came up as a common issue, especially in conjunction with the increasing use of AI to generate requests. 

Here, there are two concerns: that AI is leading to more complex (if not necessarily more effective) requests; and that there is potential for a deliberate, malicious use of AI-generated FOI requests that might overwhelm an authority without their necessarily being aware of it. 

AI-generated/assisted requests are hard to formally recognise (although many are developing a ‘gut feeling’ around them), but also not inherently illegitimate. The group discussed tactics such as asking for clarification or ID, to flush out potentially inauthentic requests if suspected. A participant from the ICO also shared their recent AI guidance.

What could an FOI Network contribute here? Convening people was seen as useful in helping to understand patterns and themes between authorities, and shared approaches. This might take the form of directly organised networking activity, or supporting and promoting the informal networks that already exist. 

Building centralised resources might also help in creating tools for assessing thresholds of vexatiousness, while also providing better assistance to requesters on what good and bad degrees of complexity looks like in an FOI request. 

Locating information

Finding the information that is being requested is at the very heart of what an FOI officer does  — and can present a sizable challenge, especially where data is not collected or stored consistently. 

Discussion touched on issues around record keeping, proactive publication, resource and support from colleagues in sourcing/collecting  information: while the officer is the entry point for requests for information, they most likely hold little of it themselves. 

The amount of resource, support and priority is given to record keeping and FOI across the organisation affects how effective an officer can be. 

Information can only be easily accessed if it is stored well: some participants talked about requests for data that is not currently centrally held, but which requesters argue should be, leading to antagonistic interactions, despite the Information Officer not being to blame. 

Participants talked both about resourcing conflicts where other priorities were legitimately higher (eg “The information holders are clinical staff (NHS) and trying to get them to answer FOI requests when they are busy with patients is not reasonable”); but also situations where requests not taken seriously by senior teams, or other departments were slow to engage with them. 

As such, a key challenge for information officers is navigating both the formal and relational structures of their organisation, and a key challenge for an FOI Network is finding ways to support this role in developing a culture of transparency and good record keeping practice.

Part of this fits with our theory that good FOI statistics are an important factor in empowering information officers — because this visibility would make FOI performance between organisations more salient, and so a greater concern for senior decision-makers. 

Lack of resources

This group discussed the lack of resources, staffing and slashed funding — including elsewhere in the organisation, where diminished budgets can remove the institutional knowledge and capacity to effectively find information.

This was another area where there was great enthusiasm for better connections between officers across organisations, especially for the small, isolated teams. This would allow all to benefit from the knowledge of a wider group. 

For a longer-term fix to the lack of resources, a united network could lobby to central government. This isn’t just about “more money”, but the effective production of centralised resources that would help everyone (eg software, tools and licences). 

It was noted that redaction was a problem that was significantly time consuming, and available redaction tools (like Adobe Pro) were expensive and had limited licences. 

Proactive publication was also identified as a resourcing issue: in repeated requests for hot topics; but also in that publishing information can lead to more requests asking for specifics. 

From our point of view running WhatDoTheyKnow, this should still mean a greater public benefit from the information provided (people who wouldn’t ask for it have access to it), but does caution against an easy “publish more, request numbers decrease” approach — which does not align with the experience of practitioners. 

Working with requesters

This group discussed complaints resulting from a requester not knowing how to navigate the FOI system or complaints processes, and thoughts on more effective communication. A lack of requester awareness that the FOI route is not the same as the complaint route was identified, as well as public confusion between FOI and data subject rights. 

There is recognition that the requester doesn’t always know what information is available, which can lead to complex initial requests. But there was also a reported increase in adversarial/angry complaints, with a perception from request-makers that information was being denied when in didn’t exist. Where people are making requests across multiple authorities, getting refusals from some but not from others, can lead to this impression, while actually just reflecting differences in what data is collected. 

Better information and signposting about how to make a good FOI request was considered helpful, but within limits. Improved web forms can be helpful, but are not the only route in. At the same time, from a volume and overload point of view, a concern that greater awareness of the act might lead to more requests. This makes it important to define what we’re after as a network that can reconcile both a civil society “it’s good if more people are aware of and use their rights” with the practicalities of make that right real, which includes understanding of capacity.

In general, a lot of the potential in this area is around helping those making, and those answering requests to understand each other, or at least understanding more about how things work behind the scenes. 

 —

Finally, there was a more general discussion about forms a network can take, including the difficulty of convening both requesters and practitioners. Creating spaces for authority-side practitioners to talk helps with the smooth functioning of the FOI Act; these spaces would be more hesitant if always shared with civil society groups (and vice versa). 

We want to find ways to bridge these groups, while recognising that both individually can be constructive.  We need a set of layered discussions about how to make FOI work in practice, that can manage both communities of practice, also bridging both sides — recognising where common frustrations and collective goals can be served through better communication and coordination. This is inherently going to be complex, but will be so worthwhile to explore. 

—

Illustration: Alghozy

We continue with exploratory work that will help shape what the FOI Network will become, and how it will best serve specific types of those who use Freedom Of Information. Last week, in our second workshop of three, we convened journalists and specialist users to help us understand what the barriers are to their use of FOI, and how the Network will be best positioned to help.

More than 20 people joined us, from a range of organisations and specialities. 

A group brainstorming session surfaced the challenges and obstacles that attendees face when using FOI in their work. From these, we pulled out four priority themes; then we further discussed how best to mitigate these challenges; and what an FOI network could do to help.

Themes and conclusions

Passive pushback

The phrase “passive pushback” describes a culture within authorities not of active hostility, but in which FOI requests are not processed promptly, and where transparency is neither celebrated nor prioritised. 

Participants talked about different kinds of delays and obstructions, and strategies they’d attempted to get around this, including formal approaches such as referring to relevant ICO decision notices, or working with lawyers so correspondence included their letterhead. 

Could an FOI Network support, incentivise or celebrate a culture of openness and compliance within authorities? The group discussed the importance of both carrots and sticks (or shame and envy), rewarding and highlighting good practice; and potentially putting out comparative FOI statistics to make performance more visible and easy to benchmark. 

Active pushback

Active pushback covers more explicitly obstructive behaviour, including the use of rules and processes to slow down the progress of a request (for example, the use of public interest exemptions which may later be reconsidered and overturned), and coordination between authorities (where the intention is rarely to ensure everyone is being correctly open, but more likely to be working against the spirit of the ‘applicant blind’ principle that is written into the FOI Act). 

Participants shared the methods they’d attempted in the face of such stonewalling, including complaints to the ICO, “meta requests” asking authorities about their coordination mechanisms, and naming and shaming authorities for obstruction. 

As a network, we can provide peer support and resources around such tactics, and useful responses. 

But there is also an important advocacy angle, where building evidence and lobbying to change ICO/OSIC enforcement strategies can be an important collective rather than individual counter to obstructive approaches. 

A State of FOI report could cover both the positive “culture of openness” stories, but also dig more into patterns in obstructive responses. 

Writing/managing good requests

One important issue that came up in conversation was that it is a skill to write and manage FOI requests well — one that takes time to learn. It requires both an understanding of when legal approaches are helpful, but also a sense of what is possible through Freedom of Information (where a focus on specific information that already exists is required). 

Building this expertise can be a problem if FOI skills are not already well embedded in your organisation, and you are starting effectively from scratch. 

Here, community and training would be helpful. As a newcomer takes time to build up accumulated wisdom, peer support and mentor programmes could be helpful in walking through concrete examples of where FOI can and can’t be helpful. 

Going beyond that, we discussed the concept of training products and services that could help support increasing specialist use. These might include a custom newsletter featuring FOI tricks and tips, not aimed at first time users, but for journalists and specialists. The network already includes several organisations and individuals who could provide such training and expertise — and a willing audience. Here, the value of the network would be in joining people up, rather than trying to do everything itself.

The role of technology was also discussed. It can help with the challenge of managing requests  — one peron mentioned (unprompted!) WhatDoTheyKnow Pro as helpful for tracking and extracting data. 

Some are using AI to help them refine their requests, and we discussed the potential ways we could make AI assistance lean more towards “sharper, easier to process” requests, and away from the problems, already observed, of for example where AI hallucinates ICO notices and inserts them into correspondence. Here a network could help disseminate a ‘skill’ to bring more specialist knowledge towards shaping the AI’s actions; and a network that was also inclusive of FOI practitioners could help refine that from both sides. 

Understanding/using appeal processes

The final challenges we discussed were around understanding and using the appeal processes that are built in to the FOI system, noting that there’s a strong difference between Scottish FOI and the rest of the UK: legal appeals to OSIC are rare (and only possible on points of law). 

We again talked about the value of peer support; as well as building collections in response to common pushbacks. It was acknowledged, though, that taking things to tribunal is hard, and that one can be outgunned by the representation of the public authorities. 

Here it’s important that network activity is not just support for individuals, but lobbying and campaigning for improvements to the overall appeal system. 

This might be through pressing for improved data and information such as ICO timescales, and the tribunal returning to publishing current and upcoming cases; but also through lobbying for greater funding for regulators to be able to run an effective and timely system of appeal. 

Takeaways

Thanks to all who took part in the conversation, which has helped shape our understanding of what is useful activity to support journalists and specialist users.

Practical peer support for those in smaller organisations might be especially important — but dealing properly with the issues raised also involves wider campaigning activity around making sure the overall system is functional to be able to engage with. 

 —

In our third workshop, we move to a focus on those working in public authorities, where we want to explore more of these questions around supporting a culture of openness, and also our common interest in improving the quality and clarity of requests. If you are an Information Officer or practitioner, please do join.

 —

Image: GuerillaBuzz

The FOI Network is an informal coalition of civil society organisations, journalists and academics with an interest in Freedom of Information, convened by mySociety and State of the Future. Last week, we held our first meet-up, in the shape of an online workshop, to discuss and prioritise the ways in which we might strengthen and defend the right to information in the UK. With potential threats rumbling on the horizon, it was a timely conversation.

Attendees came from a range of organisations and specialities. We had a group brainstorming session to identify firstly, opportunities to strengthen or expand FOI; and secondly, key threats and challenges.

From these, we pulled out four themes, which we discussed from the angle of which activities would bring the most impact for the effort expended.

Thematic groupings

The four topics discussed were:

• The expansion of FOI to currently uncovered bodies/sectors
• AI as an opportunity and challenge, and proactive / better publication
• Practical difficulties/support for FOI within public bodies
• The government’s opposition/lukewarm attitude towards existing/greater transparency

FOI expansion

As  FOI’s reach is expanded, so is its utility to new groups, who would benefit from the Act in different ways. This is an approach that can benefit communities who have historically had low levels of power.

Expanding the FOI Act to new authorities would make possible new avenues of research and enquiry, providing access to useful data where it is not currently available. This includes important areas such as housing; or private contractors to government, where the case for increased transparency is easy to make.

We’re fortunate that we can look to Scotland, where Registered Social Landlords are subject to FOI requests, as an example: there is already a good evidence base for successful expansion. Equally, good arguments for expansion could be made by showing the types of essential questions that cannot currently be answered under the regime as it stands.

AI as opportunity and challenge, and proactive / better publication

In this area, the group decided that there are no highly effective actions that would also be easy to implement. Instead, we would be looking at a range of sensible small interventions around better guidance, training research, and more intensive technical work around proactive disclosure and unlocking the benefits of public data.

There is a wider problem around AI potentially overwhelming appeal mechanisms (for more on this, see the two Information Commissioners’ talks at FOI Fest). There is more to explore here, around triage methods, AI and increased volumes of both requests and appeals.

Practical difficulties/support for FOI within public bodies 

An effective FOI system requires information officers to be well-resourced and supported within their organisation.

Here, potential actions ranged from campaigns for better stats around FOI (making FOI more visible to decision-makers, as in Scotland); sharing and promoting the success stories of FOI to show the value of the work; better networking/surveys of the profession; campaigning for statutory FOI officers; and technical support on document management/search technologies. 

There was also some discussion around organisations where responding to requests pulls officers away from other work, affecting the prevailing attitudes towards FOI. The concept of statutory officers would have some bearing on this.

We will develop this segment in a further workshop, to which practitioners themselves will be invited. 

Government opposed to/has a lukewarm attitude towards transparency

A key concern is how we improve FOI, when some of the mood music coming out of government is in favour of greater restrictions. But at the same time, “government” is a wide term: while there will be some institutional reluctance to transparency, there will also be some pockets where it aligns with other stated objectives. 

We need a clearer map and understanding of these factions. We may need to be both defensive, pushing back against threats to transparency, while also building diverse institutional support. One benefit of an FOI network is that different parts of the coalition can do both at once.

From this follows a need for positive, public advocacy for the benefits of transparency, as well as a clear narrative of how it fits into wider government agendas around the redress of historic injustice, anti-corruption, value for money and so forth.

So that’s the summary of our discussion. We’ll keep you posted with progress reports from the FOI Network.

The next meet-up is about how the Network can support journalists and specialist users of FOI: if that’s of interest, sign up here.

—

Image: Mark Fletcher-Brown

Last Thursday, a diverse set of people with a common interest came together at Birkbeck College in London for FOI Fest — a one day conference examining the successes, challenges and future of the Freedom of Information Act. We heard from a wide range of perspectives: those who use FOI, those who process requests, those who research the workings and effects of the Act, and those who oversee its application.

We couldn’t have wished for a better day: there was so much to discuss, with relevant and timely presentations, and real engagement in the room. As this was a kick-off event for the incipient FOI Network, the enthusiasm was gratifying; and there seems to be consensus that this should be considered the inaugural FOI Fest, rather than a standalone event.

We heard from both FOI regulators, listened to expert discussion, learned from an excellent set of skills sessions where experts shared insights on how to use FOI well; and enjoyed a set of short lightning talks where users explained how they had brought change through FOI. We’ve added links to each video presentation so you can watch any that are of interest.

Gavin Freeguard, longtime mySociety associate and FOI expert, was our host and co-organiser, kicking off proceedings by pointing out that the introduction of the FOI Act represented a “fundamental and vital change in the relationship between the public and government”. Our rights under FOI have survived two decades, and in the process an ecosystem of activists, researchers, campaigners, journalists and commissioners has arisen.

Information Commissioner John Edwards couldn’t be with us in person, but did send a video address, talking frankly about how the body is equally as stretched as they know many authorities are too — partly thanks to the increased use of AI. This was a theme that would recur through the day, from different authorities, although, as John was keen to point out, more FOI requests means more people accessing their right to information, and that, in itself, is a real positive.

Warren Seddon, Director of FOI and Transparency at the Information Commissioner’s Office, provided our first keynote, picking up on the theme of the increase in requests needing their oversight (they’re on track to receive more than 10,000 FOI complaints this year alone). AI-generated requests tend to be longer, contain inaccuracies and are less easy to understand. However, there was good news too: the ICO are exploring setting up an external monitoring system for public authorities outside of central government; Warren told us to watch this space.

Warren Seddon

Next, investigative journalist Jenna Corderoy, CEO of the Campaign for FOI Maurice Frankel, and Transparency International’s Rose Whiffen joined Warren for a panel discussion to interrogate the question “What has 21 years of FOI changed?“. 

Maurice began with the observation that the ICO’s powers are not strongly enough applied, resulting in many authorities experiencing no penalties for bad faith or late responses, albeit he’s detected some improvement since John Edwards took post. 

Jenna pointed out that times have changed since the introduction of FOI, but the Act hasn’t kept up, as can be seen for example by the government’s use of WhatsApp for messages that can be deleted and therefore cannot be publicly released. There’s an attitude issue, too: many authorities still see FOI as burdensome or even malign, when really we should all be supportive of this essential mechanism for transparency, allowing the vital uncovering of corruption and wrongdoing. 

Rose asserted that FOI has shifted public expectations, and built momentum for better government transparency, “rebalancing the information asymmetry which is where corruption thrives” and “shifting the risk calculation around corruption”.

Warren added that the things politicians were afraid of when the Act was first introduced have, for the most part, not been realised; and that it’s a great thing that any one of us can find out information from our local school, hospital or council.

Andreas Pavlou, Eleanor Shaikh, Isaac Beevor and Alex Parsons

In the next session, we enjoyed three ‘lightning talks’ – five minute presentations. Eleanor Shaikh, whose work you can read about in more detail in our blog post, used her slot to explain how the Act had given her the tools to uncover fundamental aspects of the UK Post Office Horizon scandal, many of which made front page headlines. She stressed the importance of taking your time to avoid being labelled ‘vexatious’, spacing requests out.

Isaac Beevor, from our Scorecards partners Climate Emergency UK, explained their use of FOI requests to obtain standardised datasets around councils’ road expansion plans, EPC ratings in social housing, energy procurement and more, as detailed in this blog post. He included a shoutout for WhatDoTheyKnow Pro and its Projects functionality, which meant they could share the work across a team of volunteers.

In the Q&A, an audience member expressed how very inspiring these two examples were, showing how one person can use FOI to bring real change.

Finally, Andreas Pavlou, Lead at the Open Government Partnership Independent Reporting Mechanism, brought the good news of positive advances in Access to Information in countries such as the Netherlands and Brazil, banging the drum for one of our favourite watchwords, collaboration.

George Greenwood

Before breaking for lunch, we enjoyed three skills sessions.

George Greenwood, Investigations Reporter at the Times, shared ten tips from his long experience of using FOI, which you can see for yourself on our Bluesky thread.

Lucas Amin of Democracy For Sale shared his experience of using EIR (Environmental Information Regulation) to uncover big stories: he’s also shared these at one of our ATI Network online events, so you can rewatch that here if you’re interested. 

And Ben Worthy, senior lecturer in Politics at Birkbeck College, explained his research project which used FOI to dig more deeply into how FOI works, experimenting to see whether formal requests were more likely to get a response (they are).

David Hamilton

After lunch, we complemented the morning’s keynote with another, this time from David Hamilton, the Scottish Information Commissioner, who echoed his ICO counterpart’s observations about the rise in complaints — partly because of AI and partly because public recognition of FOI is “higher than it’s ever been” (partly due to the recent high profile inquiry into the First Minister). “If you see the rights, you’re happy to use the rights”, he said, noting that the Scottish Information Commissioner is on the front page of a newspaper on average every three days at the moment, and that transparency was going to be a massive issue in Scotland’s forthcoming May election.

David also ran through the FOI Reform Bill currently going through Scottish Parliament, noting the differences between this and the FOI Act (which covers England, Wales and Northern Ireland, plus UK-wide public authorities that are based in Scotland).

For our second panel of the day, asking “What next for FOI?“, David was joined by Director of the Campaign for Freedom of Information in Scotland Carole Ewart, Ben Worthy, and mySociety’s Alex Parsons and Louise Crow.

David Hamilton, Louise Crow, Carole Ewart, Ben Worthy and Alex Parsons

Louise began by reminding us that not every user of WhatDoTheyKnow is making a Freedom of Information request: in fact, each response is viewed on average 160 times, massively multiplying the benefit of the information contained within it. The publication of responses also means that authorities don’t have to deal with the same requests multiple times, as the information is free for all to see. She then highlighted the importance of FOI adapting to the current rise of AI systems, and the necessity of record-keeping so that transparency about how they work is not allowed to become opaque.

Ben shared his research on the use of AI in requests, observing that in the last year the number of authorities indicating that they are receiving requests composed by AI has risen from 30 to 70%. 

Carole was frank about the blocks to reform, saying that the attitudes towards FOI needed to change so that it was no longer seen as adversarial or aggressive — but she welcomed the changes on the horizon for Scotland.

Then, in our second skills session, Maurice Frankel of the Campaign for FOI shared tips for success with FOI requests, not least around keeping them within cost limits; and Martin Rosenbaum, author of Freedom of Information: A Practical Guidebook and for many years BBC News’ FOI expert, explained the importance both of persistence and of wording your requests carefully. Finally, mySociety’s own Julia Cushion and Gareth Rees explained what we’ve been doing on WhatDoTheyKnow and Alaveteli recently, from our beginners’ guide to FOI, to this page showcasing what FOI has done around the world. 

There was just time for three more lightning talks: reshowing this video from Gabriel Geiger, Investigative Reporter at Lighthouse Reports, on their award-winning Suspicion Machines investigation that showed how machine learning algorithms were disproportionately targeting individuals based on ethnicity and gender; followed by talks from Alex Homer, Senior Journalist at the BBC Shared Data Unit who shared a live investigation around the National Police Chiefs’ Council; and Carole Ewart, Director, Campaign for Freedom of Information in Scotland spoke about their work aiding NGOs in the use of FOI.

Finally Louise Crow wrapped up the day, and before we knew it, it was time to repair to the pub to carry on conversations in a more informal setting. An all-round success – and, we hope, the start of a very useful new network that we’ll be progressing over the next few weeks. Please fill in this form if you have an interest in being part of it.

Thanks to all our speakers, and to everyone who came and helped make the day a success. We’re also grateful to JRRT for supporting this exploration of the network, and eCase for additional sponsorship. Special thanks to Ben Worthy and Birkbeck for providing the excellent venue for the conference, and to Gavin Freeguard for compering the day, and all his work bringing the network together.

Gareth Rees, Julia Cushion, Martin Rosenbaum, Maurice Frankel and Gavin Freeguard

WhatDoTheyKnow Pro, our Freedom of Information service for users such as journalists, researchers and campaigners, now comes with Projects bundled in at no extra cost. That means that, as well as sending batch requests more easily, you can also bring in colleagues or volunteers to help you refine and analyse the data you receive in response.

How WhatDoTheyKnow Pro can help you

WhatDoTheyKnow Pro is a useful tool for those sending the same FOI request to multiple authorities: what we call ‘batch requesting’. Often, when embarking on an investigation, or gathering data for research or to inform a campaign, it’s helpful to gather data from many sources to create a full picture.

For example, in this recent blog post, Zarino described how he used Pro to ask every local authority in the UK how many safeguarding referrals they had received from schools that they manage. Climate Emergency UK have also used Pro to good effect, gathering data about councils’ climate action that wasn’t otherwise publicly available.

WhatDoTheyKnow Pro helps with two of the more difficult elements of bulk requesting:

• finding/compiling a list of all the relevant authorities’ email addresses; and
• keeping track of which authorities have responded, and which need following up.

What Projects adds

Now, the inclusion of Projects eases another big challenge of bulk requesting: sorting through the masses of responses to pull out the information you need.

Even when you frame your request to ask for data in a certain format, as permitted by the FOI Act, experience suggests that you’ll rarely receive responses that fit neatly into a spreadsheet for your instant analysis.

As Zarino noted in his follow-up post on requesting safeguarding data, much depends on how the authority are storing the information at their end: “We think, in reality, very few of the authorities held this data in a format structured enough to count as a ‘dataset’, but a few did send over their data in spreadsheet format, which was nice to see! Others, however, sent us tables in Word documents, in PDFs, SharePoint links, even ASCII-art tables in raw email text.”

These days, AI might be helpful with some data-refining tasks; but as we discovered with our WhoFundsThem project recently, sometimes humans are the best bet for combing through responses and pulling out the parts you need, in the format you need. Climate Emergency also took the time to train large cohorts of volunteers to ensure the assessments they were pulling out of their FOI responses for the Council Climate Action Scorecards were fair and accurate. Both projects made good use of WhatDoTheyKnow Pro and Projects.

What can Projects do for you? 

If you have one or more associate working with you, or if you can assemble a team of willing volunteers, you can share the work of going through the FOI responses as they come in. Projects makes collaboration easy. 

You can use Projects to give your team an online interface where you describe the aims of your investigation, and set out the questions you need answers to. Your helpers will then go through each response in turn and identify the parts you need, putting them into your standardised format. At the end, all their inputs are pulled into a nice, tidy spreadsheet that allows you to do the analysis you need.

Contributors don’t need a Pro membership themselves, so there’s no extra cost to you, and the only extra effort required is in setting out what data it is that you need to pull out from the responses — something it’s useful to have straight anyway!

No team to help you? Projects can also be used solo, and still helps you keep track of the information you’re pulling out — helpful if there are lots of data points.

Subscribe to WhatDoTheyKnow Pro (with Projects included), or see the Help page for more detail.

 —

Image: Kylie Haulk

At the beginning of the year, we set ourselves an ambitious goal: to help a group of small organisations working with marginalised communities to run Freedom of Information–based campaigns using WhatDoTheyKnow Pro’s batch-request and project features. We recruited groups working in areas as varied as domestic abuse, arts funding, youth health, SEND provision, parental leave, fuel poverty, and migrant justice. 

As the year draws to a close, we’re reflecting on the project and the lessons we’ve learned from it. It’s been a total privilege working closely with these organisations, because it gave us a front-row view of the real challenges of frontline campaigning and community support. 

What became clear early on was that the hardest part of a batch-request project isn’t actually pressing “send”.  Campaigners know their issues intimately, but FOI requires a specific kind of precision: pinning down exactly what data will answer their question, what format it should be in, and which public bodies actually hold it. Moving from “we want to understand this issue” to “we need these five questions answered from these 150 authorities” is a surprisingly big leap. 

Luckily, WhatDoTheyKnow’s knowledgeable  volunteers were able to help our groups go from vague policy areas to precise questions, and to understand what information was already out there. One of our groups didn’t end up submitting a big batch request, as in the course of their preparatory research they found an already-published dataset from an industry body they didn’t know existed. This is still a win — proactive publication by authorities makes everyone’s life easier.

In the cases where we had good questions and had identified the right authorities, we then still had to tackle the practical reality: for small teams already stretched thin, a large FOI project which asks a lot of questions requires capacity to deal with the answers. These can come in a diversity of forms: follow-ups, clarifications, refusals, delays, internal reviews. Our Projects tool helps to make dealing with the range of responses easier, but the scale of the challenge can still require serious commitment of time and resources. Zarino shared his experience of this on our blog back in October.

Just this week we had a moment that illustrates this: one of the groups we were supporting sent a batch FOI request to 133 universities on 5 July. As I write this in December, they are still receiving responses. The most recent one, a refusal, arrived five months after the original request! 

We’ve got two strands of thought here. On one hand, it’s good to be realistic. Although these moments are frustrating, they also teach us to be prepared for slow, unpredictable timelines, and that persistence is part of the craft. On the other hand, we feel strongly that citizens shouldn’t have to be quite so persistent, that pace shouldn’t be quite so slow, nor unpredictable. That’s why we’re advocating for upstream policy improvements, such as in our recent evidence to the Scottish Parliament, and in our upcoming FOI Fest conference.

Although it’s not always been straightforward, this year reinforced why FOI is worth the effort. A particularly strong example came from SCALP and Netpol’s From Scotland to Gaza report, which, with our help, used batch FOI requests to uncover policing practices around protests. Their methodical approach combined data from public bodies with testimonies to make a compelling case that has shaped media coverage and public debate. It’s a reminder that FOI doesn’t just extract information, it empowers communities to speak with confidence.

All of this left us with a clearer sense of what we can do in future to help make big FOI projects work. A few lessons stood out:

• Start smaller: a 10-authority pilot builds confidence and tests the strength of the question.
• Co-design the requests: working together on wording and structure reduces uncertainty: the organisations have expertise of their area, while our volunteers have second-to-none understanding of how to write a clear request.
• Prepare organisations for the long tail: follow-ups, delays, and refusals are, unfortunately, to be expected, not signs of failure of the project. 
• Volunteers can help with the volume of work: Climate Emergency UK have set the standard for how to train, empower and mobilise the cohorts they need to churn through large quantities of data.
• See FOI as a strategic, not administrative tool: it’s most useful when tied directly to campaign goals.

We fundamentally believe that every organisation can benefit from FOI; they just need the right scaffolding and resources. If you know what you’re in for, the whole process becomes far less intimidating.

What next? We’re refining our approach, watching what happens with our initial batch of projects, and constantly updating our guides and help pages to support our users in their big and small FOI projects. Every request is a small act of collective muscle-building. We’re excited to keep learning and keep improving the support that makes those acts possible.

Photo by Danist Soh on Unsplash

Around the world today, organisations and communities are recognising the 26th International Day for the Elimination of Violence Against Women. This is a moment to reflect on one of the most prevalent and pervasive human rights violations in the world – but it’s also a call to action.

As one small action in that continued effort, we’ve been working with the End Violence Against Women Coalition (EVAW) this year, to explore how something like our Local Intelligence Hub could help their members organise for change.

Little did we expect that, in building a prototype Data Hub for them to explore their needs, we’d discover a gaping hole in data collection about the safety of girls at schools in 16% of local authorities.

But first, what were we aiming to achieve?

In a post last month, I shared some of the goals of this work – such as using data to galvanise support from MPs, to monitor patterns that official bodies might miss, and to help EVAW’s members make the case for increased local funding to address violence against women and girls (VAWG).

I also shared how we were using the batch request tools in WhatDoTheyKnow Pro (our advanced Freedom Of Information service) to generate new public data on VAWG prevalence in schools.

And, of course, all of this work builds on top of the Local Intelligence Hub we designed and built with The Climate Coalition and Green Alliance – which has already proved its worth as a tool for community organising and public affairs, including through events like this Summer’s #ActNowChangeForever Mass Lobby, and The Climate Coalition’s Great Big Green Week.

Now it’s time for an update – how did we get on?

A replicable pipeline of brand new VAWG data

When we built the Local Intelligence Hub with The Climate Coalition (TCC), much of the data we included was already publicly available: MP information from Parliament, demographic data from the ONS, public opinion data shared by polling companies. Combined with TCC member organisations’ own data on their local support and activities, the Hub was able to present a nuanced picture of climate and nature are being protected across the whole country.

We knew we faced a different challenge with the VAWG data hub. As I explained last month, public data in this space is often incomplete, or missing entirely. We wanted to use this as an opportunity to test how WhatDoTheyKnow and the Local Intelligence Hub could work together to generate and then publish brand new datasets on VAWG prevalence or activity, made public through FOI requests to local authorities and policing bodies.

We chose school safeguarding referral figures as a suitably challenging example that was also indicative of levels of risk to children. When school staff fear a child may be in danger in any way, they are meant to refer it to the safeguarding team at their local authority. The UK government collects some information about these referrals as part of its Children In Need census, but the definition of a “child in need” is somewhat open to interpretation, and we and EVAW both suspected that, as a result, the official data was only telling part of the story. The census also only covers local authorities in England, leaving Scotland and Wales to collect their own, incompatible data (the CRCS census in Wales, and Children’s Services Plans in Scotland).

With the help of the WhatDoTheyKnow volunteers, we drafted an FOI request to be sent to every UK local authority with a responsibility for education, asking for three things:

1. The total number of safeguarding referrals made to them, by schools in their area – this is data that technically should be collected by the CIN census for English authorities, but we suspect is not
2. Any sort of categorical breakdown they held about those referrals, such as a breakdown of the genders of the children involved – this doesn’t currently appear in any public dataset that we know of
3. The total number of schoolchildren in their area

You can browse the requests and responses on WhatDoTheyKnow. Here are some key things we learned through the process:

No matter how much you research your request, something will slip through

Our background research and even our first pilot requests failed to reveal that the total number of schoolchildren is something that’s already published for England, Scotland, and Wales. Thankfully, many authorities simply pointed us to this data (with a “Section 21” refusal – “information already accessible”), but others continued to provide the data for each year we requested. Had we known in advance that the data was already available to us, we could have left it out of our requests to English, Scottish and Welsh authorities. We can only hope, since this is such basic information, the authorities who did go on to provide the data to us didn’t spend too long gathering it.

You will receive information in every format imaginable, and your data extraction process needs to handle that

We asked for responses to be provided in a “re-usable, machine-readable format” if the authority deemed the information to meet the FOI Act definition of a ‘dataset’. We think, in reality, very few of the authorities held this data in a format structured enough to count as a ‘dataset’, but a few did send over their data in spreadsheet format, which was nice to see! Others, however, sent us tables in Word documents, in PDFs, SharePoint links, even ASCII-art tables in raw email text.

We also knew authorities might hold the information by calendar, academic, or financial reporting year, so we gave them the freedom to provide it to us in whichever scheme they had. Unsurprisingly, we received responses across all three (57% calendar year, 35% financial year, 8% academic year).

Happily, the crowdsourcing interface in WhatDoTheyKnow Projects enabled us to make relatively quick work of extracting the data we needed, but we were ultimately only able to extract a fraction of the information some authorities provided and we found that some of the interpretation of the responses (ie: “is this a financial year, or an academic year?”) heavily relied on human intuition, which means we’ll need to think carefully about the way we structure future requests, if we want to process the data through any sort of automated pipeline.

Complex requests are a risk

The more information you request, the more useful it might be to you, but the more you risk the public authority refusing to answer it on “Section 12” cost grounds. WhatDoTheyKnow’s advice is to keep your request as short and focused as possible. But we knew that historical data, across a few metrics and a few years, would be most useful to the VAWG Data Hub’s users, so we asked for as much as we felt we could justify – and it mostly paid off.

70% of responses to our batch request contained both key pieces of data we wanted (the total referrals for multiple years, and the gender breakdown). Another 7% contained just the yearly totals, without any gender breakdown.

7% are still awaiting a response, even now, over a month after the statutory deadline. And 6% of authorities said they didn’t hold the information at all (because they, surprisingly, don’t record the referrals they receive). Which leaves 10% who refused our request on cost grounds. If our request had been simpler, this number of refusals would likely have been smaller.

However, this result is in itself interesting: at least 16% of local authorities responsible for handling safeguarding referrals either don’t record them, or record them in such a way that it would take more than 18 hours of officer time to report how many they received in a given year, or how many relate to girls.

If the government is serious about halving violence against women and girls within a decade, this is precisely the sort of data local authorities will need at their fingertips, in order to monitor progress and allocate resources. The fact that it’s effectively inaccessible to 16% of them right now is a worry.

Combining data for new patterns and new questions

Remember how I mentioned we were adapting the Local Intelligence Hub for EVAW’s needs?

With our FOI data extracted through WhatDoTheyKnow, we were able to very quickly load it into a prototype VAWG Data Hub. Alongside it, we loaded in a whole new area type to filter by—“Policing areas” or Police & Crime Commissioners—as well as some examples of crime prevalence data (the number of reported VAWG-related incidents, by policing area) and public policy guidelines data (the Council Of Europe’s recommended minimums for VAWG service provision).

Thanks to my colleague Alex’s improvements to our TheyWorkForYou Votes infrastructure, we were also able to make quick work of importing VAWG-related MP data into this new hub – including VAWG-related parliamentary groups that MPs might be sitting on, or relevant votes and motions they’d supported.

Plus, of course, there was all the usual MP and area data that campaigners and public affairs teams have already found so useful on the Local Intelligence Hub – things like election results, public attitudes polling, and income and poverty indicators.

Data in action

With the data in place, it was possible for us to give EVAW’s member organisations a demonstration of how they could use a data hub like this as part of their campaigning, fundraising, and policy influencing work. For example, to find the council areas with the most school safeguarding referrals for girls and also the highest overall deprivation:

Or to find the MPs with the strongest support for VAWG prevention, but in constituencies with high VAWG prevalence:

All of the data we demonstrated this Autumn is still a work in progress, but it was reassuring to see almost 70% of members on a recent demo call saying that a VAWG data hub like this would “definitely” be useful to them in their day-to-day work.

We look forward to honing the VAWG Data Hub further with EVAW and their members, to make sure we’re asking the right questions, and presenting an accurate picture of the VAWG landscape.

Header image: Khyati Trehan, for Google Deepmind – Pexels Free License.

Do you live somewhere that boasts a magnificent municipal clock — a timepiece that anyone passing by can look up to, and check that they’re nicely on time for their next appointment? A vast clockface on the side of the town hall, perhaps; or a golden clocktower standing tall above the shopping streets…a landmark under which to meet friends?

OK, good. The next question is: does that clock actually work? 

If its hands have come to a solid halt; or it’s running at a dogged twenty minutes behind time; or its rusted chimes, once mellifluous, now sound more like the rasping call of an imperilled frog, the chances are that it’s been logged on the Stopped Clocks website.

Here you can see which clocks near you have fallen into disrepair; or check the time, which is delivered to you along with an apposite poem.  

The site is the work of Alfie Dennen, who describes himself as “somewhere between a technologist and an activist — with a tendency towards action over academics”.

It’s not entirely out of character, as he explains: “I’ve been building things for the web that blend activism and tech since the late nineties: for example We’re Not Afraid was a project that spoke to London’s — certainly my — defiance after the bomb attacks of 7/7; and Bus Tops was a project for the 2012 Cultural Olympiad which aimed to democratise both access to and the creation of public art.”

Nor has the project sprung up overnight. It all began with Alfie’s realisation that he could become the clock-winder that kept his own local clocktower, in North London’s Caledonian Park, ticking.

“Restarting the Cally clock in 2012 cemented for me that both documenting, and hopefully one day restoring, public clocks was a worthwhile thing to spend some significant portion of my life doing.”

Now, this is a very mySociety-type project, blending coding, community, and a sense of shared responsibility. We probably would have written about it anyway. But also, in gathering the data he needed, Alfie made substantial use not just of Freedom of Information through our WhatDoTheyKnow Pro service, but also our MapIt points-to-boundaries software — so we have all the more reason to ask him all about it.

To begin with, how did he realise that FOI might be a good tool to help with the site?

As you’ll see if you click around, the project crowdsources information — so if you know of a local clock that hasn’t been included, you can add it yourself. To identify which council area they sit within (see the map page here), Alfie had been using MapIt to generate boundary information.That gave him a vague awareness of mySociety and our other services, including WhatDoTheyKnow.

“I’d never used FOI before, but I realised that it’d be a great way to get baseline data over and above the data I can gather about stopped clocks directly — given that walking every street in the UK is a bit out of my current comfort zone!

“When I went to look at WhatDoTheyKnow properly, I released that I could send FOI requests in a batch, and that got me super enthused. Suddenly the looming month-long period of finding spare time to do them one by one disappeared! WhatDoTheyKnow Pro’s batch process, and clear interface to manage status as they were responded to, has been such a useful tool for me.”

Great — so, having sent off lots of requests, has Alfie seen any responses yet?

“The deadline for responses was the 21 November for most of the requests, but there’s still hope that some more will come in. 

“So far, of the 308 councils I contacted, 107 have given substantive responses. 70 councils responded to say they held “no information” and 113 are delayed or still pending. Between them, they identified 231 council-managed clocks, of which 175 are working, 34 have stopped, and 22 have an unknown status. 

“So about 15% of council-managed clocks are stopped, which honestly is better than I expected. But here’s the thing that really stood out: when I cross-referenced this with my database of 243 stopped clocks, it turns out that only about 40 are actually council-managed. The vast majority (so far) — 84% — are outside of the scope of Freedom of Information as they are in private hands, or owned by churches, or other bodies that used to be public but aren’t any more. In this sense, privatisation has created a clear accountability gap.

“The responses have varied greatly, which is interesting in itself. Some councils sent back detailed spreadsheets detailing every clock they manage, with maintenance schedules, budgets  — the lot. Some look after lots of clocks, while others have none at all — apparently. Only 6.4% of councils could tell us what they spend on clock maintenance. Of these, the average maintenance cost was £2,929/year.

So, curiosity aside, how will all this data be put to use?

“A big part of why I’ve been doing this foundational research through FOI requests is to provide a backbone to a book I’m writing, which looks at the last 45 years of austerity in the UK through the somehow very human lens of stopped public clocks. 

“Yes, stopped clocks are a small thing in the round when we look at all the issues facing our communities, cities and civic spaces today.

“But they’re also the perfect way to talk to people about how they feel about their town, their neighbourhood, their city. Ultimately this is my main aim: reaching people where they are to talk about re-engaging with our civic space, coming together and understanding each other and our built spaces in ways we once did but have lost sight of.

“The civic infrastructure which once supported the maintenance of public clocks has been systematically stripped away through a combination of privatisation and austerity dating back to 1979. 

“But I wanted the data to be useful beyond just the book, so I’ve built it into the Stopped Clocks website as an interactive policy tab. You can see the map of all FOI-tagged clocks, filter by ownership type, and read through the timeline of disinvestment.

“More practically, it means when someone finds a stopped clock in their town and wonders, “who’s supposed to be looking after this?”, maybe they will find the answer on the site. And if they want to campaign to get it fixed, they’ve got evidence: council responses, maintenance data, the broader context of how we got here.”

Each clock boasts a number of tags, so for example you can see which data came through FOI requests, and which council area they’re in — that part is thanks to MapIt.

“I’m also tagging clocks with their ownership type, which is a somewhat manual process; and whether they’re on listed buildings, using the Historic England API. 

“Tagging lets us start seeing clearer patterns, like how Lottery-funded church restorations from the 90s are failing on a very predictable timeline, or how privatised civic buildings — former town halls, libraries — now in commercial hands are disproportionately neglected.”

And so finally, what plans does Alfie have for the project? Presumably he has a strong incentive to avoid the irony of its becoming an untended asset itself.

“I can see a path towards a Stopped Clocks charitable foundation that does two things at once: gets clocks running again, and uses that process to rebuild civic engagement at the local level. 

“Because here’s what happens when you try to fix a stopped clock: you immediately find out who owns it, who’s responsible for it, why it stopped, why nobody’s fixed it. And that leads you straight into conversations about council budgets, privatised buildings, who decides what gets maintained and what doesn’t.

“It’s a way into talking about austerity and privatisation that doesn’t feel abstract or preachy. It’s just there, on the town hall, stopped. 

“People care about these things; they notice them every day, they remember when they worked. That gives you something to organise around that’s tangible and achievable. 

“Fix one clock, learn how the system works — or doesn’t; build the relationships and knowledge to tackle the next one.”

Many thanks to Alfie for talking to us about this project: we hope it inspires others to think differently about the assets that make up our public domain — perhaps even to ask if you can be the person who winds your own local clock.

—

Image: Kelsey Todd

The government is making a significant investment into AI in public services, and systems are changing apace.

AI is increasingly being deployed in every department of government, both national and local, and often through systems procured from external contractors.

In a recent article for Public Technology, mySociety’s Chief Executive Louise Crow flags that we urgently need to update our transparency and accountability mechanisms to keep pace with the automation of state decision-making.

This rapid adoption needs scrutiny: not only because significant amounts of money are being spent; but also because we’re looking at a new generation of digital systems in which the rules of operation are, by their very nature, opaque.

To see Louise’s thoughts on what needs to change, and why, as this new technological era unfolds, read the full piece here.

If you find it of interest, you may also wish to watch this recent event at the Institute for Government, The Freedom of Information Act at 25, where Louise was one of six speakers reflecting on the future of transparency in the UK.

—

Image: Alex Socra

Mayoral expenses are a big topic in France just now, in a moment that’s reminiscent of our own MPs’ expenses scandal back in 2009.

Chandeliers, luxury TVs and a duck house

The UK’s Freedom of Information Act had only recently come into force when investigative reporter Heather Brooke lodged a request for details of MPs’ expenses. The ins and outs make for a long — and interesting — story, but suffice to say that, with the nation gripped, this may have been the moment when FOI entered the public consciousness. 

When the expenses information finally went public, it caused widespread outrage, and had a long-lasting effect on the nation’s trust in politicians. Today, the scandal is perhaps most often remembered for an MP’s infamous duck house, but the overreach in what had been claimed seemed endless, with payments for chandeliers, swimming pool heaters and luxury TVs all being recompensed. 

mySociety was part of the successful campaign to head off a subsequent attempt from MPs to have their expenses made exempt from FOI. Fortunately that idea was quashed. There’s still a need for scrutiny, though:16 years later with our WhoFundsThem project, we continue to push for better transparency and adherence to the rules around MPs’ sources of income. 

Designer clothing, false eyelashes and a rabbit-shaped pizza

Meanwhile, over in France, expenses are very much in the news. In their case, it’s mayoral use of public funds that has whipped up a frenzy, with FOI requests lodged on the French Alaveteli site MaDada providing the relevant documents. 

Le Parisien covered the story (in French, of course — but Google Translate is handy) and also put out a video (again, if your French isn’t up to scratch, use the translated subtitles): at the time of writing it’s been watched almost 200K times.

In short, Freedom of Information is helping to reveal which mayors have used the occupational expense account to pay for lavish dinners and designer clothing (as well as, quite the detail, a ‘pizza in the shape of a rabbit’) and which have confined themselves to more essential or modest job-related purchases such as train tickets and rainwear for protection when cycling between meetings.

But at the same time, the video shows a citizen being pleasantly surprised by his mayor’s lack of profligacy — FOI can reveal laudable behaviour as well as misconduct. 

Putting FOI into the public consciousness

The story has grown over time. MaDada has many requests about public officials’ expenses, dating back quite a few years. The topic hit TikTok — one mayor’s expenses included false eyelashes, cashmere sweaters, and apparently…fossils for her mother — and then the mainstream news.

In Le Parisien’s video, MaDada’s co-founder Laurent Savaëte explains that this public conversation has brought peaks in usage to the site, proving the throughline from a news story to an increased societal interest in accessing information. 

We admired the video’s clear explanation of the timeline of a response, and what happens if an authority refuses to provide the information requested: all useful intel for beginner request-makers. 

And the coverage continues, with France’s second-biggest regional paper delving into the contents of MaDada (and requesting documents where they weren’t to be found) for a story just this week.

With this level of detail in the mainstream news, as with MaDada’s request for the president’s payslip, the story is quietly introducing to the French public, perhaps even normalising, the act of making FOI requests. Or perhaps we mean the act of demanding transparency from our representatives. Either way, it’s all good stuff.

An international concern

Transparency around representatives’ expenditure is of importance everywhere, and a natural fit for FOI. A recent analysis of news stories generated from information requested across all Alaveteli sites brought up similar questions in Ukraine (where the mayor of Odessa is raising his own salary), Moldova (where people are wondering why a friend was contracted to make repairs to the mayor’s office) and Croatia (where funds designated for road repairs that do not appear to have been made are being scrutinised).

—

Image: Bartjan (CC BY-SA 4.0)