I recently completed a most useful, enjoyable, and productive three-month (paid!) secondment to the Cabinet Office (CO). This was arranged by the University of Southampton’s public policy engagement team – Public Policy|Southampton in combination with the CO’s Open Innovation Team (OIT). With a background in information management, I was asked to join the CO’s tiny ‘Better Information for Better Government’ (BI4BG) team.
BI4BG, was originally formed in response to the findings of the Allan Review of digital record strategy review to grapple with overlooked issues arising from the move from paper to digital and the shortening of document retention period, before public release, from 30 to 20 years. Sir Alex Allan questioned capabilities in areas such as: capturing and managing digital information; selecting, retaining and disposing of digital records; how sensitivity checks are handled; policy compliance; the tools, including technology, and resources needed: availability of the right skills and capacity
The CO’s records went paperless in the early 2000s but there are earlier digital records. The first of these are just reaching the gate for transfer to the The National Archives (TNA), approved retention or for disposal—i.e. destruction.
The country simply cannot save everything, even in digital form. The issue is not storage, though this would be a cost factor. Rather the issues are access, liability, and search. Older documents may be in a format that has become unreadable (without un-budgeted reformatting costs). The original data owner, has a responsibility not to release (or give access to) personal information or that which has national sensitivity beyond the normal release/disposal window. Previously 30 years it is now tapering to 20 years which only adds to the pressure for an effective release triage process.
Search is technology's other big tool but the cleverness of web searches is far more than just textual comparisons. There is much more algorithmic input based on very large datasets. The latter knowledge is not built overnight, and getting to algorithmic insights depends on the quality of data fed in. This is where the early digital days will bite is largely devoid of (externally) accessible metadata. Though brute force of an AI attack on data is possible it is the most expensive approach and the logic of outcomes can be hard to read. So, a bigger digital bucket to hold the burgeoning amount of information within organisations is not, itself, a solution.
Internal network search is not as fast or powerful as web search. With few workers trained in, or even with knowledge of, or even have knowledge of, the concept of metadata, identifying what should become part of the public record is challenging at present. Meanwhile, AI is being generally promoted as a technical panacea whist giving little guidance as to how to implement AI. Regulations do not allow government to simply ignore unresolved issues of data security, provenance, privacy, ethics, FoI requirements, etc. In fact, many aspects of the problem relate just as to organisations outside the public sector. Helping the BI4BG team resolve this practical challenge and move the organisation forward has been most interesting.
Although my PhD active research was already completed before the secondment, the secondment offered a valuable opportunity to reflect on my findings in the context of a real, busy, organisation. Indeed, it offered useful validation of some of my prior conclusions. During the time I was there, I was able to see the additional stresses and strains of potential EU exit being absorbed. It speaks well of the CO to see this extra work being handled diligently and without complaint. Given the CO is a posting civil servants work hard to achieve, it was a privilege to get to serve alongside them and have a brief opportunity to give something back to the community at large.
The secondment was also a refreshing break from the often isolated intellectual task of crafting a PhD thesis. Needing to think deeply about the work and hand, and thus something other than one’s thesis is quite a tonic. Whilst it is definitely a ‘shirt-tucked-in’ work environment (postgrad dress code, if any, is rather more relaxed), I was delighted to find the CO a much more diverse organisation than I had expected: albeit my last Whitehall experience was in the 1990s. It is a friendly, helpful and generally supportive workplace that looks after its people. So, if you are reading this and thinking “It’s not for me, I won’t fit in”, please don’t! I thoroughly recommend my PhD colleagues to think of taking up the opportunity to talk to Public Policy|Southampton and to previous secondees.
I do want to acknowledge and thank my fellow ‘BG’s Glyn, Ian and Rob; Chris’ OIT team; and also David, Galla, Dan and Sarah in the Digital Knowledge and Information Management team. Above all, my thanks to the Public Policy|Southampton, especially Yaryna whose cheerful and responsive approach helped iron out the inevitable admin glitches and made the whole experience better as a result.
Mark Anderson, PhD, Web Science Institute at the University of Southampton