Pilot Project development under way

I’m pleased to report that last week we received our first funding commitment – from OPSI – sufficient to proceed in earnest with the first phase of development of the FreeLegalWeb Pilot Project.

At the core of FLW will be what we call the Citator – essentially linked data tables identifying and connecting everything we know about the law: both primary resources such as legislation, cases and other official documents, and secondary law resources such as publications and articles. Using the Citator we will be able to request data both internally from FLW and externally from the web via an API which will be made publicly available so that any external website can leverage that information.

In the UK OPSI is the key facilitator of public sector information policy, setting standards and providing a practical framework of best practice for opening up and encouraging the use of PSI. It also has direct responsibility for the publication (inter alia) of legislation and is advanced in its development of a new legislation website which will provide an API enabling direct addressing of legislation data and resources. Via the Citator FLW will both exploit the facilities the new legislation website offers and also help contribute to OPSI’s PSI policy agenda. On this basis OPSI is part-financing the development of the Citator.

A big thanks to John Sheridan at OPSI who has been a leading FLW supporter from the outset. Indeed it was he who first encouraged me to go public with the initiative last autumn.

We have some way to go in securing sufficient funds to commit to development of the full Pilot Project but are nevertheless proceeding with other aspects of development in the expectation that some of our other current funding prospects will come good.

Digital Engagement Report

Gov 2.0 is now officially “Digital Engagement”. Guided by Minister Tom Watson, the Digital Engagement team at the Cabinet Office is now headed by Director Andrew Stott who is charged with taking forward the Power of Information agenda.

The Digital Engament team will “work across Government departments to encourage, support and challenge them in moving from communicating to citizens on the web to conversing and collaborating with them through digital technology.”

Its debut is the Government response to the Power of Information Task Force Report – available via the new Digital Engagement blog. The POI principles are substantailly accepted by government: they are “no longer just recommendations in a report but will be core to the future development and implementation of government policy.”

Progress

A small group has been actively progressing the FreeLegalWeb agenda. Here’s an update on our progress:

Active people:

• With me, Robert Casalis de Pury and Harry Metcalfe are now jointly managing the project.
• Francis Davey is actively contributing to the development of the spec and content.
• Rudi Moffit is helping us develop and deliver our funding pitch.
• On the government side, Richard Stirling (Cabinet Office) and John Sheridan (OPSI) are facilitating the project via the POI agenda and have provided Rudi’s services.
• Thanks to William Flack, Nearly Legal, Joe Ury and Stephen Moore for their input as to content.
• Thanks also to all others who have contributed comments here and on the Google Group.

Development status:

A set of about pages and some pre-prototype visuals have been prepared to illustrate and prove concepts and help us develop the spec. Citator development is under way.

Data and content provision:

• The OPSI legislation API is near final.
• BAILII will be providing the judgments metadata we require.
• Stephen Moore of CaseCheck has agreed in principle to us syndicating all case summaries from CaseCheck (which are licensed on a CC by-nc-sa basis). Currently CaseCheck publishes about 4,000 (Scottish/UK) case summaries; these are to be complemented in April by about another 5,000 (E&W/UK) sourced from Law Brief Update.
• Francis Davey, Nearly Legal and William Flack are already involved re content for the Housing pilot. I will be pursuing others in this and other areas once we’ve finalised our pitch.

Current priorities are:

• Incorporate a Community Interest Company
• Complete the Funding Proposal
• Complete the Functional Spec

We should in the next few weeks be in a position to seriously tout for some funding and move this on.

Legal informatics – a big world

Tom Bruce, co-founder and director of the Legal Information Institute at Cornell Law School posts an interesting piece on the big new world of legal informatics, concluding:

We need to make informed choices between inexpensive automated approaches that work by brute force and the hand-crafted, highly-accurate approaches of legal bibliography that are not always scalable or affordable. We need to recalibrate what we mean by “authority”, and begin to think about measures of quality and reliability for legal text that avoid the creation of unnatural monopolies in legal information.

As a step in that direction LII has started a guest blog, VoxPopuLII, which will include big ideas from “folks in all different tribes in all different places on the intellectual and global map”.

We’ll be in touch, Tom.

(Hat tip to Joe Ury)

POI Task Force report

The Power of Information Task Force has just published its final report describing the key actions that government can take in the short to medium term significantly to improve its use of digital technologies.

Of particular relevance to FreeLegalWeb is the section on modernising data publishing and reuse.

Improving access to legislation

I’ll refer again to Lord Justice Toulson’s comments (at [2008] EWCA Crim 2467) that:

It is a serious state of affairs when the relevant legislation is not accessible, the Government’s own public information website (OPSI) is incomplete and the [court is unintentionally misled] as to the relevant Regulations in force. [This is a] problem of substantial constitutional importance.

He was echoing – and specifically referred to – the Law Commission’s recommendations in its 2006 report on Post-Legislative Scrutiny (Law Com 302,  at para 4.11):

We are concerned when information so fundamental to a democracy is difficult to identify, obtain and understand, and is frequently out of date.

How do we fix this?

There are three ways to address the problem – and all three need to be pursued with equal vigour.

Firstly, let’s improve access to existing legislation. This is the task OPSI have and it is one which FreeLegalWeb is addressing. But this is applying sticking plaster to the wounds.

So, let’s improve legislation via new legislation! The Leader of the House of Commons is on the case, describing the several ways in which the Government is simplifying and consolidating legislation. But that does not address the huge corpus of remaining existing fragmented legislation.

The third way we can improve access to legislation will be by far the most effective and it does not appear yet to have attracted any public discussion – and that is to improve the legislative drafting process.

Legislation is drafted by the Office of Parliamentary Counsel and it is therefore to the OPC that we should look for the answer. To do their job in preparing amending legislation OPC need to work from a consolidated version of the current legislation. Why cannot that be made available to us? (Do LexisNexis own it?!) And if, in drafting amending legislation, they are updating existing legislation, why cannot they apply technology to automate the process?

In 2004 Tim Arnold-Moore of the Royal Melbourne Institute of Technology published an excellent paper on Automating consolidation of amendments to legislation in common law and civil jurisdictions which gives a full background to drafting practices in commonwealth jurisdictions and discusses the reasons for extending the traditional role of the official drafting office to the management of the legislative repository:

They have historically maintained consolidations of legislation for their own use to facilitate the drafting of amending legislation. Rather than duplicating the effort to consolidate the amendments, the public resources used to maintain this repository should be for the benefit of all users of legislative material. If these versions are considered authoritative enough to be used as the basis for amending legislation, they should be considered sufficiently reliable for the government to endorse as authoritative. The cost of delivering these consolidations in a timely fashion to the public is a small proportion of the cost of manually maintaining these consolidations and there is no excuse for governments failing in this respect. Providing older consolidations is simply a matter of retaining old versions and providing a suitable mechanism to search and browse them.

The article also describes how the “redlining” of amendments on the consolidated master version of the legislation can be used automatically to generate the amending legislation. This stuff is happening down under.

Tasmania, with its automated consolidation system, usually has consolidated legislation available on the day the amendments commence if not before. Contrast this with the vast resources of the US Congress where updated versions of the US Code are made available as long as 2 years after the amendment becomes law.

Sadly the UK lags even further behind, with the SLD only substantially up to date to 2002.

See also AustLII’s Point-in-Time legislation system: A generic PiT system for presenting legislation.

Permanent URIs – a small(ish) step, big dividends

Jeni Tennison has provided on her blog a very helpful explanation of URIs (Uniform Resource Identifiers) and why the simple step of establishing permanent identitifier URIs for public sector data resources will be so helpful in opening those resources up for public consumption.

Identifier URIs are what are really important when it comes to opening up your data. They shift the focus from the documents that you serve to the resources that they are about. By assigning URIs to resources, you enable other people to talk about them. Even if that’s all you do, you have done good.

Identifier URIs don’t identify particular documents or particular views or versions of the information; they identify the idea. They can then be used to direct the user to particular documents or views wherever they may be. So, no matter how much the structure of the website changes over time, the URI will remain a reliable, permanent means of addressing the information.

For a law resource a suitable identifier URI can, typically, be constructed from its citation. Jeni and others have recently been responsible for settling for OPSI a URI scheme for legislation and we can look forward to publication of that scheme very soon. So, for example, ukpga/2008/32 will be the identifier for (UK Public General Act) 2008 c. 32. That simple, reliable, permanent address can then be used by OPSI to direct us to a particular view of the information, for example the web document for the Act.

A small step, but a huge boon “for anyone to re-present, mash-up, analyse and generally do whatever they want to do”.

Now wouldn’t it be good if we could look forward to similar schemes being adopted by the courts? Unfortunately there are a host of problems there, not least that the High Court does not publish its own judgments. And although the systems of standard neutral citation adopted in recent years by the higher courts in all jurisdictions and in some other courts and tribunals provide an obvious basis for identifier URIs, settling sensible schemes for older judgments and in other courts requires a good deal more thought.

Although only the courts can settle URIs for their own resources, we must do what we can to inform and influence that process.

Meeting 15/01/2009

6 attended the meeting to address organisational issues on 15/01/2009 at the MoJ meeting room kindly provided by John S and in the pub afterwards.

Thanks to those who attended.

We agreed the next steps required which are broadly:

• Set up a charitable organisation
• Set up an executive
• Develop a strategic road map
• Develop a pitch proposal for potential contributors / sponsors
• Develop the spec
• Develop a prototype incorporating Housing content

Some progress has already been made on all these fronts.

A fuller report and related documents will be posted shortly to the Google Group and decisions will be taken at an open meeting of supporters on a date tba.

The future of lawyers

From my review of The End of Lawyers?:

Susskind sees improved access to justice being achieved with the following building blocks:

• the empowered citizen – using the web to recognise when they have a legal problem, select an appropriate service provider and obtain service
• streamlined law firms  – who have decomposed their work and multi-sourced as described above
• a healthy third sector
• entrepreneurial alternative providers – who will find new and improved ways of delivering conventional legal services (encouraged, particularly, by the Legal Services Act)
• accessible legal information systems, and
• enlightened public information policy

(cue FreeLegalWeb).

Ignorance of the law

I referred earlier to Lord Justice Toulson’s comments in R v Chambers [2008] EWCA Crim 2467 which I felt highlighted the need for FreeLegalWeb. His comments in full:

64.  … It is a maxim that ignorance of the law is no excuse, but it is profoundly unsatisfactory if the law itself is not practically accessible. To a worryingly large extent, statutory law is not practically accessible today, even to the courts whose constitutional duty it is to interpret and enforce it. There are four principal reasons.

65. First, the majority of legislation is secondary legislation.

66. Secondly, the volume of legislation has increased very greatly over the last 40 years …

67. Thirdly, on many subjects the legislation cannot be found in a single place, but in a patchwork of primary and secondary legislation.

68. Fourthly, there is no comprehensive statute law database with hyperlinks which would enable an intelligent person, by using a search engine, to find out all the legislation on a particular topic. This means that the courts are in many cases unable to discover what the law is, or was at the date with which the court is concerned, and are entirely dependent on the parties for being able to inform them what were the relevant statutory provisions which the court has to apply. This lamentable state of affairs has been raised by responsible bodies on many occasions, including the House of Lords Committee on the Merits of Secondary Legislation. In its Report on Post-Legislative Scrutiny BAILII: [2006] EWLC 302, under the heading “Access to legislation and consolidation”, the Law Commission stated:

“4.11. One theme related to delegated legislation, on which a number of consultees commented, was access to legislation. The joint response of the Children’s Legal Centre and National Children’s Bureau addressed the problem that despite their familiarity with the broader legal framework, they still found access to be a real problem:

‘The lack of access to statutes with appropriate links to the regulations and guidance which are currently in force must be a cause of serious inconvenience to anyone who does not have access to specialist services. We are concerned when information so fundamental to a democracy is difficult to identify, obtain and understand, and is frequently out of date. It is frequently the case that secondary legislation and guidance are overlooked in the process of scrutiny, although their impact on the day-to-day operation of the law is as significant as the primary statute.’

4.12. The joint response stated that experience of practice in childcare suggests that many injustices are the result not of failure to comply with the statute, but of failure to know about, understand or access secondary legislation.”

69. The problem is not confined to secondary legislation relating to childcare. It affects many other areas of law of great impact on the ordinary citizen, such as social security benefits.

70. The Law Commission concluded this section of its report as follows:

“4.15. It is also important that all related statutory provisions, whether primary or secondary, should be capable of being readily accessed together. We are aware of the work being undertaken on the Statute Law Database and recognise that public access to that resource is a step in the right direction. We recommend that steps should be taken to ensure that the related provisions of primary and secondary legislation should be capable of being accessed in a coherent fashion by a straightforward and freely available electronic search.”

71. The Government’s response to that recommendation was presented to Parliament in March 2008, CM 7320. It stated as follows:

“35. Her Majesty’s Stationery Office (HMSO) and the Statutory Publications Office (SPO), which produces the Statute Law Database, are to work together to create a single, powerful and free to access online legislation service. The launch of the SLD has been a milestone in government’s online legislation publishing.

36. Over the last two years HMSO, via the OPSI website (www.opsi.gov.uk) has embarked on wide ranging improvements to how legislation is published online, taking account of key usability features for layout and navigation. This work is being undertaken as part of ‘The Transforming Legislation Publishing Programme’. The aim has been to present legislation in the most accessible and usable way [my emphasis], whilst maintaining the traditional strengths of immediacy and accuracy. One of the benefits is that it affords the opportunity to provide links to related information. Initially these links will be to the Explanatory Note for Acts or the Explanatory Memorandum for Statutory Instruments. Alongside this is also published an ATOM feed for the piece of legislation. This provides visitors with an easy way to keep up to date with subsequent additions to the website, like the addition of Explanatory Notes for an Act, and also the enacting or making of other related legislation such as Commencement Orders or, longer term, amending legislation. In future HMSO will be adding explicit links to Commencement Orders, and where legislation implements an EU Directive, a link also to that Directive.

37. HMSO/OPSI and SPO will continue to work together and with government’s online legislation visitors, to improve the service and ensure that UK legislation is available in a high quality and straight forward terms, with a freely available and powerful search.”

72. The aim is laudable, indeed imperative, but there is a long way to go and meanwhile the volume of legislation advances apace. It is a serious state of affairs when the relevant legislation is not accessible, the Government’s own public information website (OPSI) is incomplete and the prosecution in an excise case unintentionally misleads the court as to the relevant Regulations in force. Although the problem has in this case arisen in an excise context, it is part of a wider problem of substantial constitutional importance.

Now you may say this is a problem for Parliament to resolve via legislative reform. I’d go with that, but it won’t happen soon enough and wouldn’t have retrospective effect. Or you may say it is for Government to resolve by extending the SLD consolidation work to cover secondary legislation. I say, pigs might fly: it’s not on the cards and, if it were, would optimistically take another ten years, or – more likely – never be finished. So, in practical terms, we are left with OPSI improving access to the relevant as-enacted secondary legislation. OPSI is working hard to improve matters. But it doesn’t have all the answers, and it know this. The Power of Information agenda acknowledges that government shouldn’t try to be Big Brother, but provide better access to government information which will encourage innovative services to be developed which support the government’s own efforts. In respect of better access to the law, FreeLegalWeb has stepped forward and our award in the ShowUsABetterWay competition is recognition that – with support and encouragement – we can make a difference. There is no magic bullet that will solve the burgeoning secondary legislation problem, but we’ll be working with OPSI and the POI Task Force to improve matters substantially.