BRERC species data and the NBN Gateway
Written by Tim Corner - Manager of BRERC - for NBN News Spring 2008
Introduction
Bristol Regional Environmental Records Centre (BRERC) is the central repository for biodiversity and geodiversity records covering Bristol, North Somerset, South Gloucestershire and Bath and North East Somerset. BRERC is also referred to as a Local Records Centre (LRC).
The National Biodiversity Network’s (NBN) Gateway is in effect a web-based portal and central repository of biodiversity records at the national level. It enables anyone to access data via the internet. The NBN Trust has been keen to include the records held by LRCs on the Gateway and has made efforts to encourage submission of them.
Until recently BRERC, along with other Local Records Centres, had not been confident about submitting records to the Gateway for various reasons, but in particular because of fears that doing so could undermine the current business models of some LRCs, including its own. The NBN Trust and Government bodies would like environmental data to be freely available to anyone, including those using the information for commercial purposes. However, LRCs are reliant on charges made to commercial organisations for a significant proportion of LRC funding streams. Therefore there are concerns that submitting data to the Gateway could result in commerce by-passing LRCs in favour of acquiring their data needs via the Gateway alone.
Additionally BRERC has not routinely interrogated the Gateway for data it doesn’t hold, for instance when conducting data searches.
BRERC wished to examine why it was not submitting data and why it wasn’t making better use of the Gateway for its own needs, in order to identify some of the obstacles that prevent other records centres from doing the same. It was felt that some of the obstacles could be based on perception rather than reality, and a consequence of historical development.
BRERC approached Natural England and the NBN Trust with its thoughts and agreed to carry out a pilot project to test some of the technical aspects and possible impacts on its business and role as an LRC. They in turn would use the data provided as a demonstration of LRC capabilities and consider new approaches to funding LRCs with regard to data collection and capture.
The work entailed:
• Preparing BRERC electronic species data and then submitting this to the Gateway team
• Testing the access controls
• Evaluating any obvious impacts on BRERC business
• Conducting an internal review to develop protocols for accessing the Gateway for routine data searching services carried out by BRERC
• Reporting its conclusions to the NBN Trust and other LRCs.
Preparing and submitting data
In order to prepare the data so that it can be imported and interrogated by the Gateway, BRERC produced a translation tool in MapBasic. This tool translates .dbf files into .txt files in the ideal format required by the Gateway. The tool is opened and operated within MapInfo Professional.
BRERC recently reached critical thresholds for the electronic file size of its Main Species Database and subsequently produced a second Main Database clone to house new records whilst developing a replacement database (still ongoing).
BRERC anticipated difficulty in preparing and resubmitting data to the Gateway. During the submission stage it decided to involve several members of staff to ensure the process was understood by as many as possible.
The tool produced in MapBasic has been designed specifically for BRERC with regard to how it manages data, although generic versions could be produced.
The chosen formats required include NBN Taxon Version Keys that identify species, but do not require the species name or other fields BRERC uses for in-house maintenance. This means there is a need to include Taxon Version Keys in its databases.
Testing Access Controls
In order to allow for different levels of access to sensitive data it is necessary to mark such records as being sensitive. BRERC does this by having a separate calculating yes/no field for ‘sensitive’. The calculation takes the information from its drop-down lists database for ‘Record type/Sex Stage/Abundance’. An example is where ‘Record Type’ is a sett then that is marked as sensitive in that database.
BRERC has chosen to make all of its records available via the Gateway at a one kilometre resolution. The dataset downloaded by Natural England did not include sensitive data, but BRERC granted downloadable rights at full resolution of all other records to them. Access is essentially granted to datasets and not to parts of datasets.
At BRERC, we feel that the Gateway needs to continue to improve filtering for access controls that would negate the need to split datasets.
Currently we are confident that sensitive data can be screened and can only be accessed by those granted the right by BRERC.
BRERC has had concerns about how effectively it could manage data it submits to the Gateway with the Gateway tools available. One of the main issues has been the time it takes to understand how the site works and how and what is possible with regard to managing the data.
Since 2004, to their credit, the NBN teams have made considerable progress to address many of the issues raised by LRCs and others. Setting access controls and other features are much simpler. The NBN Metadata Model and other guidance have been made much simpler and more practicable. The concerns LRCs have had regarding commercial enquirers, general or core funding and standards of data management have been acknowledged and are beginning to be included on national agendas set by the NBN Trust and Natural England.
Submission to the Gateway generates metadata, charts and analysis available on line. Whether this information is automatically generated or not it is a useful resource for the managers of BRERC as it provides some of the information the BRERC Steering Group and Joint Advisory Committee often request.
The promotion of the internet as a conduit for data mobilisation, and the proliferation of database programmes such as MapMate are thought to have resulted in the duplication of many records. In addition the universal drive for data flow may have been at the expense of standards regarding verification and validation.
***Key Point: LRCs need to incorporate internal protocols and systems that aid the processes for responding to Gateway generated requests and for granting detailed access where appropriate.***
***Key Point: LRCs, the NBN and national schemes and societies need to address the issues of duplicates and potential lack of properly validated and verified data being submitted to the Gateway.***
***Key Point: BRERC recommends that a downloadable and nationally ‘approved’ dictionary is produced as soon as is possible.***
Evaluating impacts on BRERC business
The main concern regarding BRERC’s business model has been that by making data available to the Gateway it may inadvertently undermine the model and that commerce and others would no longer seek a charged service from BRERC.
The provision of sensitive data could undermine relationships with the recording and community groups that are vital in the mobilisation of data, whether or not any inappropriate release of sensitive data was real or an incorrect perception.
BRERC therefore took a risk of alienating itself among the LRC and data custodianship community and importantly local users and recorders. It has had to place considerable good faith in the NBN Trust teams, but feel this has been well placed. All of the team and relevant Natural England officers have been approachable at any time and have acted quickly to try to resolve any issues raised. The successes BRERC has had with this Project can be attributable, in part, to the sincere and honest assistance that has been received from those within the NBN Trust team and Natural England.
BRERC has been approached by some commercial consultants for a charged service after they have seen its data on the Gateway. This equates to more charged work for BRERC.
There have also been approaches from some members of the public and community groups for more information about the records they come across on the Gateway. This equates to more uncharged work.
Attributable enquiries have been few and have not adversely affected BRERC’s business. As more become aware and make use of the Gateway we anticipate an increase in workload. This increase would probably occur whether we had submitted data to the Gateway or not as it is a natural progression of internet usage.
Internal Review
BRERC conducted an internal review to develop protocols for incorporating the Gateway into its routine data search services. We continue to make improvements to our systems and procedures.
We already use the species dictionary facility on a regular basis for validation and status assignment purposes. We have incorporated the NBN preferred names and Taxon Version Keys into our own dictionaries and databases to enable greater alignment and compatibility with national and international conventions. BRERC continues a dialogue with the NBN Trust team in relation to the species dictionary.
Conclusions
BRERC strongly recommends that other LRCs and other data custodians make their species data available to the NBN Gateway, ideally at one kilometre resolution for public access.
BRERC strongly recommends that data providers to the NBN Gateway be required to automatically grant LRCs access to the submitted data at full resolution (apart from exceptional circumstances or where commercial sensitivities arise). This would allow greater mobilisation of data and would help LRCs to incorporate Gateway held data in data searches, screening, research and project work. This might seem at odds with the fact that currently most or all LRCs do not allow full access to all. However this would be in recognition of the role LRCs have as data custodians and key nodes within the NBN and the professional standards that we apply in managing and disseminating data.
● BRERC is pleased to have had the opportunity to take part in this Project.
● We are confident that we can satisfactorily manage the BRERC data on the Gateway.
● We recognise that BRERC has been slow to develop protocols and systems in-house to exploit the resources of the Gateway.
● BRERC does not feel the submission of data to the Gateway has adversely affected its business.
● The submission of the BRERC data means the Gateway now holds over a million extra records for the ‘Avon’ area.
As soon as you make data widely available you receive admonishments from experts who will spot the one unlikely and possibly incorrectly entered record. That is good. They have told you something you may have missed. The sharing of data ensures the sharing of the responsibility for improving and making that data more accurate. The NBN does not want to populate the Gateway with inaccurate records. Local authorities and agencies will want to be able to make informed decisions using accurate records; experts want to use accurate data and the public do not want to be misled.
The Gateway is one of the primary sources for information used by national bodies when determining certain strategies and initiatives. For example, you may wish to ensure your records are included in the setting of environmental stewardship targets or in the assessment of a species’ national status and distribution. National officers will utilise the national portals for much of their information. If we insist all enquirers should always refer to each one of us around the country then that is just as much a denial of access to the data we hold, since realistically many will not do so.
I am confident that data held by BRERC is now much more widely available, is being used in many more ways and is also being maintained with higher standards. Which is nice since, after all, that is the remit of BRERC.