inbo / etn-occurrences

Acoustic telemetry data 🔊🐟

Acoustic telemetry datasets

This repository was used to discuss the open data publication of a number of acoustic telemetry datasets. These datasets are exported from the European Tracking Network (ETN) database and contain detections of tagged fish that were picked up by the Permanent Belgian Acoustic Receiver Network.

The repository also contains a proof of concept to standardize these data to Darwin Core.

Workflow

1. Assess, discuss and improve quality of data related to an animal project in the ETN database
2. Download data using the ETN R package function download_acoustic_dataset()
3. Deposit the data on the Marine Data Archive
4. Document the dataset on IMIS
5. Publish the dataset and assign a DOI

Datasets

Shortname	Publisher	DOI	DataCite
2010_PHD_REUBENS	Ghent University	https://doi.org/10.14284/437	DataCite
2011_RIVIERPRIK	INBO	https://doi.org/10.14284/429	DataCite
2012_LEOPOLDKANAAL	INBO	https://doi.org/10.14284/428	DataCite
2013_ALBERTKANAAL	INBO	https://doi.org/10.14284/431	DataCite
2014_DEMER	INBO	https://doi.org/10.14284/432	DataCite
2015_DIJLE	INBO	https://doi.org/10.14284/430	DataCite
2015_HOMARUS	VLIZ	https://doi.org/10.14284/433	DataCite
2015_PHD_VERHELST_COD	Ghent University	https://doi.org/10.14284/435	DataCite
2015_PHD_VERHELST_EEL	Ghent University	https://doi.org/10.14284/434	DataCite

Contributors

List of contributors

License

MIT License

etn-occurrences's Issues

Correct scientific name "salmo salar" for 29 animals in 2013 Albertkanaal

29 animal/tags in 2013 Albertkanaal have scientific name salmo salar rather than Salmo salar. That causes them to be shown separately in overviews. Who has access to this project in ETN and can update their scientific name?

A69-1601-64485
A69-1601-64486
A69-1601-64487
A69-1601-64488
A69-1601-64489
A69-1601-64490
A69-1601-64491
A69-1601-64492
A69-1601-64493
A69-1601-64494
A69-1601-64495
A69-1601-64496
A69-1601-64497
A69-1601-64498
A69-1601-64499
A69-1601-64500
A69-1601-64501
A69-1601-64502
A69-1601-64503
A69-1601-64504
A69-1601-64505
A69-1601-64506
A69-1601-64507
A69-1601-64508
A69-1601-64509
A69-1601-64510
A69-1601-64511
A69-1601-64512
A69-1601-64513

dwc_animals.Rmd: structure and term overview

The focus of dwc_animals.Rmd is to map the animal metadata, i.e. data related to capture, release and surgery of the fish. The structure of the mapping is rather complex and involves mapping to the Event Core, Occurrence Extension and extended Measurement of Facts Extension (eMoF).
In this issue I give an overview of

• How I think the Core and extension file relate
• Which terms can be assigned to the core or extension file

The occurrenceID's and eventID's are composed of:

• projectname: name of the animal project
• id_pk, which is the unique identifier for a specific animal (within and across animal projects)
• occurrence or event
• capture, surgery or release

Note:

the release Event record does not represent a distinct Occurrence record from the capture event. Holding periods, movement of the animal while captured, and other confounding circumstances preclude this record from being considered a natural occurrence of the animal, see this page

Legend:
In blue, red or yellow: the Darwin Core terms, belonging to respectively the event core, occurrence extension or eMoF extension, in grey the column names from the raw data file.

Use vocabulary for wildOrHatchery

The field wildOrHatchery expects:

Enter "wild", "hatchery" or "unknown".

Currently the following values are used:

current value	controlled value	animal project	number of animals	remarks
hatched	hatchery	2013 Albertkanaal	62	corrected
hatchery	hatchery	2013 Albertkanaal	29	ok
wild	wild	2011 Rivierprik	39	ok
wild	wild	2012 Leopoldkanaal	104	ok
wild	wild	2013 Albertkanaal	100	ok
wild	wild	2014 Demer	16	ok
wild	wild	2015 Dijle	26	ok
wild	wild	2015 Fint	39	ok
wild	wild	2015 PhD Verhelst	229	ok
wild	wild	Homarus	2	ok
wild	wild	PhD Jan Reubens	41	ok
Wild	wild	2015 PhD Verhelst	13	corrected
WILD	wild	2013 Albertkanaal	60	corrected
(empty)	(to look up)	2013 Albertkanaal	45	corrected
(empty)	(to look up)	2015 Fint	5	corrected

Salmo salar: weight without unit

44 animals from 2013 Albertkanaal have a weight without unit (e.g. A69-1601-34437). This should be corrected.

Lacking values for person_id

Some of the records in the dataset have no person_id assigned to the capture/release/surgery event.
In the subset prepared for publication, this concerns 86 animals, all belonging to the animal project 2013 Albertkanaal. Can this information be supplied?

We will map person_id to the Darwin Core term recordedBy. The specification for this field is:

A list (concatenated and separated) of names of people, groups, or organizations responsible for recording the original Occurrence. The primary collector or observer, especially one who applies a personal identifier (recordNumber), should be listed first.

Ideally, we want the full name of the person here. Is this an option?

Use controlled value for captureMethod

The values for capture_method is currently free text, but it would be better if we used a controlled vocabulary. I would suggest to use lowercase English names, e.g. line fishing as in suggested in the ETN application. This would then be consistent with lenghtType(#13) values.

My suggestion:

captureMethod	controlled value	animals	remarks
Ankerkuil	stow nets	4	corrected
fuik	double fyke nets	61	corrected
Fyke nets	double fyke nets	1	corrected
FYKE NETS	double fyke nets	265	@fwaumans
HAND - DIVING	capture by hand while diving	2	@fwaumans
hand line	line fishing	7	corrected
Hook and line	line fishing	8	corrected
Line fishing	line fishing	22	@fwaumans
LINE FISHING	line fishing	101	@fwaumans
(empty)	can these be populated?	339

@PieterjanVerhelst @jreubens @IPauwels: suggestions welcome (especially where there are question marks)

Why are catches of the same animal 2 records in the animal table/view?

In the animal table, it appears that six individuals (indicated in individual here) have two records instead of one. Five of these individuals (10 records, first table) differ mainly in their dates of capture, surgery and release and also differ in their length. For each of these individuals, one record has a date of recapture provided, which suggests that these double records are a consequence of the recapture of the animal. A sixth individual only differs in its length and weight values (second table).
The consequence of these double records is that id_pk can't be used as a unique identifier for an animal as these animals get assigned two identifiers.

individual	projectname	tag_full_id	id_pk	length	catched_date_time	date_of_surgery	utc_release_date_time	recapture_date
1	PhD Jan Reubens	A69-1303-20695	674	34.6	2011-07-27 00:00:00	2011-07-27	2011-07-27 11:30:00	NA
1	PhD Jan Reubens	A69-1303-20695	673	34.6	2010-08-09 13:00:00	2010-09-08	2010-09-08 13:00:00	2011-05-19
2	PhD Jan Reubens	A69-1303-20698	678	40.0	2011-07-27 00:00:00	2011-07-27	2011-07-27 11:30:00	NA
2	PhD Jan Reubens	A69-1303-20698	677	37.5	2010-10-01 00:00:00	2010-01-10	2010-10-01 13:00:00	2010-12-05
3	2012 Leopoldkanaal	A69-1601-29920	388	57.4	2012-08-21 10:16:00	2012-08-21	2012-08-21 00:00:00	2013-09-03
3	2013 Albertkanaal	A69-1601-29920	389	89.0	2014-10-09 00:00:00	NA	2014-10-09 00:00:00	NA
4	2012 Leopoldkanaal	A69-1601-29925	393	66.0	2012-08-21 13:57:00	2012-08-21	2012-08-21 00:00:00	2012-12-10
4	2012 Leopoldkanaal	A69-1601-29925	394	77.0	2012-12-14 13:00:00	2012-12-14	2012-12-14 00:00:00	NA
5	2013 Albertkanaal	A69-1601-34503	542	32.7	2015-04-22 00:00:00	NA	2015-04-22 00:00:00	2015-04-26
5	2013 Albertkanaal	A69-1601-34503	543	30.4	2015-04-27 00:00:00	NA	2015-04-27 00:00:00	NA

individual	projectname	tag_full_id	id_pk	length	weight
6	2013 Albertkanaal	A69-1601-34486	524	28.4	254.0
6	2013 Albertkanaal	A69-1601-34486	525	29.6	329.8

Cleanded content for surgery locations

The current values in surgery_location should be cleaned somewhat before mapping to the DwC term locality. I suggest the following actions (see also #34 for cleaning of capture_location and #41 for cleaning of release_location)

1. Give more information for some of the locations

surgery_location	surgery_location_clean
tss D5 en D6	between wind turbine D5 and D6 C-Power
tussen D5 en D6	between wind turbine D5 and D6 C-Power
D5	wind turbine D5 C-Power
D6	wind turbine D6 C-Power

High max for lengths & weight

See this Google spreadsheet (sheet: "Lengths") and table below: the range (or rather the maximum) for the different types of lengths + weight is sometimes unrealistically high.

@jreubens @PieterjanVerhelst, can you tell me which ones you consider ok?

The rest will have to be corrected in the database, but I don't know what source info to use for that.

Checklist

• Alosa fallax total length: cm: looks ok
• Alosa fallax weight: g: wide range
• Anguilla anguilla: horizontal eye diameter: mm: very high max resolved by using .
• Anguilla anguilla: pectoral fin: mm: very high max resolved by using .
• Anguilla anguilla: total length: cm: very high max corrected
• Anguilla anguilla: total length: mm: why not cm? Some researchers add data in cm, others in mm. Flixibility is allowed in this field.
• Anguilla anguilla: vertical eye diameter: mm: very high max resolved by using .
• Anguilla anguilla: weight: g: wide range
• Anguilla anguilla: add length type corrected to horizontal eye diameter
• Cyprinus carpio: total length: cm: wide range
• Cyprinus carpio: weight: g: wide range
• Gadus morhua: total length: cm: very high max resolved by using .
• Lampetra fluviatilis: total length: cm: looks ok
• Lampetra fluviatilis: weight: g: looks ok
• Petromyzon marinus: total length: cm: looks ok
• Petromyzon marinus: weight: g: looks ok
• Platichthys flesus: total length: cm: looks ok
• Platichthys flesus: weight: g: wide range
• Rutilus rutilus: total length: cm: looks ok
• Rutilus rutilus: weight: g: wide range
• Salmo salar: fork length: mm: looks ok
• Salmo salar: total length: cm: wide range
• Salmo salar: weight: g: very low min
• Salmo salar: add length type: corrected to fork length
• Silurus glanis: total length: cm: very high max, can grow up to 2.5m
• Silurus glanis: weight: g: very high max
• Squalius cephalus: total length: cm: looks ok
• Squalius cephalus: weight: g: looks ok

Can tags be assigned to multiple network projects

I noticed in the data that A69-1601-34473 is assigned to two network projects:

• Dijle
• Zeeschelde

Is this to be expected?

Tag information in surgery event

As described in the mapping scheme in #31, tag information will be related to the capture event.
While deciding which tag properties to include, I encountered several problems/issues:

Which tag information is required?

Tag information can be found in two different metadata overviews generated by the etn-package:

• Animal overview
• Transmitter overview

Below I made an attempt to extract field names with potentially interesting information to include (for an overview of all available terms see attached images) :

• Animals overview:
  • tag_code_space
  • end_date_tag
  • animal_tag_release_id_pk
  • thelma_converted_code
  • tag_full_id

• Transmitter overview:
  • serial number
  • type
  • model
  • idCode
  • estimated_lifetime
  • acoustic_tag_type
  • frequency

Suggestions/comments are welcome

Confusing and lacking field definitions

It is very difficult for me to figure out what information to include as some of the field names in the overviews generated by the etn-package are not included in the field definitions (e.g. animal_tag_release_id), or included under a different name (tag_full_id, which is tag in the field definitions file) or included but not defined (e.g. thelma_converted_code). This is a general problem and I will make a different issue for this later.

Include DATE_OF_SURGERY in table and download

Cf. #22. @fwaumans can you add the field DATE_OF_SURGERY in the list of available fields to show in the animals table:

Can you add it to the data download as well?

Provide deployment information / status of network

As a test I've made a deployment (+ receiver) extraction from the data.

https://github.com/inbo/fish-tracking-data-publication/blob/master/data/output/temp/deployment.csv

Each record is basically a deployment (file) that was uploaded to the database. I wonder if that information will be relevant for the open data/external users ?

1. Since the coordinates of the receivers are going to be in the detections, they already know where receivers are deployed.
2. The information that is not in the detections is the from when till when a receiver was deployed, which might give an indication of where animals could be detected, but weren't (some kind of absence data). Is that kind of use to be trusted? Do any of you use this as such?
3. Also, I don't actually have the time window in the data: deployment.deploy_data_time is present, but deployment.recover_date_time is not. Is it in the database?
4. There is also quite a lot of coordinates information attached to a deployment: deployment.deploy_lat/long, deployment.intended_lat/long and deployment_recover_lat/long, but only deployment.deploy_lat/long is consistently populated, so I've used that as the location of the fish in the detections. Are the other coordinates used?

Salmo salar: length without length_unit

44 Salmo salar have a length, but no length_type or length_unit. This should be corrected.

Use controlled value for sex

The guideline for the field sex is:

Sex of animal. "M" or "F". Can enter unknown ("U").

This is not the case for all:

sex	controlled value	animals
f	female	60
F	female	367
Female	female	1
M	male	49
(empty)	(empty)	333

333 have no value. Should we be explicit and set these to U?

Tag status - difference between active and available?

Could I get more information relating to the field tag_status? Now this status is specified as ACTIVE, AVAILABLE and ENDED. There's no explanation in the field defenitions.
Although the content is quite intuitive, I don't really grab the difference between active and available. And does ended imply that the tag no longer transmits a signal?

Complete surgery latitude/longitude

In line with the discussion in #33, please complete the missing values for surgery_latitude and surgery_longitude using the following criteria:

1. Only complete surgery_latitude and surgery_longitude for records with a value for date_of_surgery. When no date_of_surgery is provided, leave these fields empty.

2. In most of these cases: surgery_latitude = release_latitude and surgery_longitude = release_longitude

3. In two cases (id_pk = 458 and id_pk = 459) however surgery_latitude = capture_latitude and surgery_longitude = capture_longitude:

id_pk	surgery_location	capture_latitude	capture_longitude
458	LEOPOLDKANAAL (STENENSCHUURBRUG)	51.26664	3.661414
459	LEOPOLDKANAAL (STENENSCHUURBRUG)	51.26664	3.661414

What are the relevant fields in Detections for data publication?

To be discussed on Tuesday, here's a first review of the fields in Detections and which ones we should map to Darwin Core.

• x1: index column, ignore
• receiver: now in dynamicProperties, maybe locationID
• transmitter: organismID
• transmitter_name: is NA for all, ignore?
• transmitter_serial: is NA for all, ignore?
• sensor_value: is NA for all, ignore?
• sensor_unit: is NA for all, ignore?
• sensor2_value: is NA for all, ignore?
• sensor2_unit: is NA for all, ignore?
• station_name: now in locationID
• datetime: eventDate
• id_pk: maybe use as occurrenceID?
• qc_flag: is NA for all, ignore?
• file: input file, not relevant
• latitude: decimalLatitude
• longitude:
• deployment_fk: no concept of deployments in DwC, ignore?
• scientific_name: scientificName
• location_name: now in locality
• deployment_station_name: no concept of deployments in DwC, ignore?
• deploy_date_time: no concept of deployments in DwC, ignore?
• animal_project: relevant?
• animal_project_name: relevant?
• animal_project_code: relevant?
• animal_moratorium: use as filter
• network_project: relevant?
• network_project_name: relevant?
• network_project_code: relevant?
• network_moratorium: use as filter?
• signal_to_noise_ratio: relevant?
• detection_file_id: not relevant

Include tag in download

@fwaumans currently the field tag is not included in the animals download, even though the field is included in the table:

Can this be added as the first field? That would make it easier to reference records. Note that some other tag related fields are already included, such as tag_end_date.

Required fields for publication of DwC occurrence data

For the publication of DwC occurrence data, the following fields are required at a detection view level:

Already in detection_view

• detections_view.id_pk: is this a stable identifier for the detection that can be made public?
• detections_view.datetime
• detections_view.transmitter (= tags.tag_code_space)
• detections_view.receiver (= receivers.receiver)
• detections_view.deployment_fk (= deployments_view.id_pk)
• detections_view.scientific_name (= animals_view.scientific_name)
• detections_view.deployment_station_name (= deployments_view.station_name): contains the most stable names for locations
• detections_view.animal_project_name (= projects_view.name): assuming this is the field with stable names
• detections_view.animal_moratorium (= projects_view.moratorium)
• detections_view.network_project_name (= projects_view.name): assuming this is the field with stable names
• detections_view.network_moratorium (= projects_view.moratorium)

Not yet available in detections_view

• tags.owner_organization
• animals_view.id_pk: decided we will use this as the stable ID for the animal
• animals_view.common_name
• animals_view.animal_sex
• deployments_view.deploy_lat: for fixed receivers and something else for wave-gliders
• deployments_view.deploy_long: for fixed receivers and something else for wave-gliders

Post operation holding period

In line with #45

post_op_holding_period is provided for (only one!) value in the dataset (within the scope of publication) : 1 hour.

Three options here:

1. Separate post_op_holding_period into is value and unit in the mapping script.
2. Create two new fields post_op_holding_period_value and post_op_holding_period_unit in the database
3. Use ISO 8601 format for durations: T[n]H[n]M[n]S, with [n] replaced by the value for each time element that follows the [n], T the time designator that precedes the time components of the representation, H the hour designator that follows the value for the number of hours, etc. To make clear that his is an ISO format I would add ISO8601: as a prefix. @peterdesmet is this interpretation correct?

I prefer option 3 here as this is a standardized format. I'm not not sure however whether this suggested change is allowed in the database.

NB: (but outside the scope of the data publication):
This field contains more values when you consider all animal project:
1'59", 6'53" 3'58", 1 hour and 0.1 This issue is applicable to these values as well.

Make length + weight doubles

The fields:

• LENGTH
• LENGTH2
• LENGTH3
• LENGTH4
• WEIGHT

Are indicated as being format: double:

One can enter text strings in those fields though. These leads to errors, e.g. , and . being used as decimal separators (caused #24).

@fwaumans can those fields be set to double + returning an error if a value does not comply.

Uniform use of capitals in type field

Type information related to the tags (field telemetry_type in ETN application) is always Acoustic. In 598 cases, this word is capitalized, in 74 cases, it is not:

projectname	number of lowercase values for "acoustic"
2015 PhD Verhelst	45
2013 Albertkanaal	29

I changed this manually in the ETN application. Now all values for telemetry_type are capitalized

Eels with incomplete measurements

@PieterjanVerhelst Of the 402 eels there are 11 that do not have all 4 measurements. Can this information be completed (especially for the first 2-3 that have the other measurements)?

tag	project	length_type	length2_type	length3_type	length4_type
A69-1601-31899	2012 Leopoldkanaal	total length	horizontal eye diameter
A69-1601-47322	2013 Albertkanaal	total length	pectoral fin	horizontal eye diameter
A69-1601-28261	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-28262	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-28263	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-34461	2015 Dijle	Anguilla anguilla	total length
A69-1601-26524	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-26525	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-26528	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-28259	2013 Albertkanaal	Anguilla anguilla	total length
A69-1601-28260	2013 Albertkanaal	Anguilla anguilla	total length

Figure with detection days by year and species

As discussed with @IPauwels, for taxonomic coverage, create a figure with:

• Rows: species
• Columns: year
• One column with total number of public tags
• Values: number of published detection days (cumulative)
• Colour cells on values (per species, see figure below)
• Indicate scientific name and English vernacular name
• Indicate years with data that are under moratorium: e.g. rather than having number of detections, indicate with *
• Avoid listing species that are completely under moratorium
• Maybe group species on habitat: obligatory marine, etc. (or extra column)

Example for inspiration (from this gist)

Cleanded content for release locations

NOTE: This table is updated with the changes discussed further in this issue

The current values in release_location should be cleaned somewhat before mapping to the DwC term locality. I suggest the following actions (see also #34 for cleaning of capture_location and #42 for cleaning of surgery_location)

1. Complete missing values for release_location, release_latitude and release_longitude

projectname	id_pk
2013 Albertkanaal	2353
2013 Albertkanaal	2348
2013 Albertkanaal	2354
2013 Albertkanaal	2350
2013 Albertkanaal	2352
2013 Albertkanaal	2349
2013 Albertkanaal	2347
2013 Albertkanaal	2345
2013 Albertkanaal	2355
2013 Albertkanaal	2356
2013 Albertkanaal	2357
2013 Albertkanaal	2331
2013 Albertkanaal	2332
2013 Albertkanaal	2333
2013 Albertkanaal	2334
2013 Albertkanaal	2343
2013 Albertkanaal	2337
2013 Albertkanaal	2344
2013 Albertkanaal	2338
2013 Albertkanaal	2335
2013 Albertkanaal	2336
2013 Albertkanaal	2342
2013 Albertkanaal	2339
2013 Albertkanaal	2341
2013 Albertkanaal	2346
2013 Albertkanaal	2330
2013 Albertkanaal	2340
2013 Albertkanaal	2351
2013 Albertkanaal	2358

2. More information requested:

release_location	records	release_location_clean
HH5	86	?
M-4-1 (LO)	17	?
M-4a-1 (RO)	2	?
M-4 (LO)	7	?

3. Release locations of Genk, Diepenbeek and Hasselt

(see #41 (comment))

For the release locations of Genk and Diepenbeek, the names “Genk”, “Diepenbeek” and “Hasselt” should be changed to: “Upstream shipping lock Genk”/ “Upstream shipping lock Diepenbeek”/ “Upstream shipping lock Hasselt”, respectively.

Anaesthetic concentration

The content of the field anaesthetic_concentration looks like this:

anaesthetic_concentration	records
NA	379
0.02 ml/L	80
0.2 ml/L	11
0.2ml/L	1
0.3ml/L	189

Two options here:

1. Clean up the content by adding space between the value and its unit (which we can use as a separator later to map measurementValue and measurementUnit)
2. Create two new fileds anaesthetic_concentration_value and anaesthetic_concentration_unit and assign accordingly.

I prefer option 2 which is in line with other measurement data such as length and weight. This would also imply that we can eliminate one data processing step in the mapping process. I'm not not sure however whether this suggested change is allowed in the database.

Define which network projects to include

The data paper covers detections from the following networks:

Network project	IMIS link
2017_Fremur	?
albert	http://www.vliz.be/en/imis?module=dataset&dasid=5876
Ars	?
bovenschelde	http://www.vliz.be/en/imis?module=dataset&dasid=5877
bpns	http://www.vliz.be/en/imis?module=dataset&dasid=5843
cpodnetwork	?
demer	http://www.vliz.be/en/imis?module=dataset&dasid=5879
dijle	http://www.vliz.be/en/imis?module=dataset&dasid=5880
leopold	http://www.vliz.be/en/imis?module=dataset&dasid=5855
maas	?
pc4c	?
saeftinghe	http://www.vliz.be/en/imis?module=dataset&dasid=5857
SPAWNSEIS	?
testvr2ar	?
thornton	http://www.vliz.be/en/imis?module=dataset&dasid=5847
ws1	http://www.vliz.be/en/imis?module=dataset&dasid=5844
ws2	http://www.vliz.be/en/imis?module=dataset&dasid=5845
ws3	http://www.vliz.be/en/imis?module=dataset&dasid=5854
zeeschelde	http://www.vliz.be/en/imis?module=dataset&dasid=5860

I think we should:

• Add the IMIS dataset ID to ETN if that isn't already done yet.
• Remove LifeWatch (listed in ETN twice) from ETN
• Check and add the IMIS link
• Check and update the dataset information in IMIS. E.g. it would be good to have the shortname 2013 Albertkanaal included. Who has access to this?

Cleanded content for capture locations

The current values in capture_location should be cleaned somewhat before mapping to the DwC term locality. I suggest the following actions (see also #41 for cleaning of release_location and #42 for cleaning of surgery_location)

1. Restrict the use of capital letters (only when required)

capture_location	records	capture_location_clean	OK
ABANDONNED FYKES (?)	5	abandonned fykes (?)
BOEREKREEK	7	Boerekreek	OK
DIJLE (STUW. MECHELEN NEKKERSPOEL)	9	Dijle (weir Mechelen Nekkerspoel)	OK
FISH WAY ASPER	1	fishway Asper	OK
FISHWAY ASPER	5	fishway Asper	OK
FISHWAY MERELBEKE	1	fishway Merelbeke	OK
FISHWAY OUDENAARDE	2	fishway Oudenaarde	OK
GROTE GEUL	10	Grote Geul	OK
ISABELLAGEMAAL	28	Isabellagemaal	OK
KLEINE KIL	9	Kleine Kil	OK
LEGEMEERSBEEK	6	Legemeersbeek	OK
LEOPOLDKANAAL (FRED'S CAFE)	8	Leopoldkanaal (Fred's café)	OK
LEOPOLDKANAAL (OESTERPUT)	3	Leopoldkanaal (Oesterput)	OK
LEOPOLDKANAAL (STENENSCHUURBRUG)	15	Leopoldkanaal (Stenenschuurbrug)	OK
LEOPOLDKANAAL (WATERVLIET)	8	Leopoldkanaal (Watervliet)	OK
sluis genk	36	shipping lock Genk	OK
WEIR ASPER	2	weir Asper	OK
WEIR GUTTER ASPER	5	weir gutter Asper	OK
WEIR MERELBEKE	8	weir Merelbeke	OK
ZEESCHELDE (BERGENMEERSEN. WICHELEN)	4	Zeeschelde (Bergenmeersen, Wichelen)	OK
ZEESCHELDE (KILLEWEG. DENDERMONDE)	2	Zeeschelde (Killeweg, Dendermonde)	OK
ZEESCHELDE (LEPELSTRAAT. RUPELMONDE)	2	Zeeschelde (Lepelstraat, Rupelmonde)	OK
ZEESCHELDE (RINGVAART. MERELBEKE)	111	Zeeschelde (ringvaart, Merelbeke)	OK
ZEESCHELDE (VEERBOOT. KASTEL)	7	Zeeschelde (veerboot, Kastel)	OK
ZWARTESLUISBEEK	10	Zwartesluisbeek	OK

2. Use correct encoding and shorten lengthy location names

capture_location	records	capture_location_clean	OK
Erezée: Salmoniculture du Service Public Wallon (SPW) située à Erezée.	44	hatchery Erezée	OK
Erez�e: Salmoniculture du Service Public Wallon (SPW) situ�e � Erez�e. Les poissons proviennent de la production de smolts 2013.	56	hatchery Erezée	OK
kwekerij Erezee	29	hatchery Erezée	OK
Lorc� Ambleve	2	Lorcé Amblève	OK
Lorc� Ambl�ve	1	Lorcé Amblève	OK
Lorcé Amblève	1	Lorcé Amblève	OK
Lorcé� Amblève	1	Lorcé Amblève	OK
M�ry : Pi�ge de d�valaison sur l'Ourthe (affluent de la Meuse) situ� sur le barrage de M�ry � 12km en amont de la confluence avec la Meuse	6	upstream the Méry dam in the River Ourthe	OK

3. Provide full names instead of abbreviations for stroomopwaarts and stroomafwaarts

capture_location	records	capture_location_clean	OK
fuik brug Demer verder SO steunbeer	1	Bridge River Demer upstream artificial construction Steunbeer	OK
sa stuw Mechelen	1	Downstream weir Mechelen	OK
SA stuw Mechelen	2	Downstream weir Mechelen	OK
SO Diepenbeek afvoerriool	3	Upstream draining channel Diepenbeek	OK
SO Genk afvoerriool	25	Upstream draining channel Genk	OK
SO Hasselt afvoerriool	10	Upstream draining channel Hasselt	OK
so steunbeer	1	Upstream artificial construction Steunbeer	OK
SO stuw Mechelen	21	Upstream weir Mechelen	OK
verder so steunbeer	3	Further upstream artificial construction Steunbeer	OK

4. Give more information for some of the locations

capture_location	records	capture_location_clean	OK
D1	3	wind turbine D1 C-Power	OK
D3	2	wind turbine D2 C-Power	OK
D5	15	wind turbine D5 C-Power	OK
D6	21	wind turbine D6 C-Power	OK
VLVS - Zone 2.2	1	Drowned Land of Saeftinghe	OK

JS error in ETN app

There is an error loading the Javascript libraries (and some of the css?) in the ETN app:

app.js?1.0:42 Uncaught Error: Module build failed: ModuleBuildError: Module build failed: Error: `sass-loader` requires `node-sass` >=4. Please install a compatible version.
    at Object.sassLoader (Z:\symfony\etn\node_modules\sass-loader\lib\loader.js:31:19)
    at runLoaders (Z:\symfony\etn\node_modules\webpack\lib\NormalModule.js:195:19)
    at Z:\symfony\etn\node_modules\loader-runner\lib\LoaderRunner.js:364:11
    at Z:\symfony\etn\node_modules\loader-runner\lib\LoaderRunner.js:230:18
    at runSyncOrAsync (Z:\symfony\etn\node_modules\loader-runner\lib\LoaderRunner.js:143:3)
    at iterateNormalLoaders (Z:\symfony\etn\node_modules\loader-runner\lib\LoaderRunner.js:229:2)
    at Z:\symfony\etn\node_modules\loader-runner\lib\LoaderRunner.js:202:4
    at Z:\symfony\etn\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:70:14
    at process.internalTickCallback (internal/process/next_tick.js:70:11)
    at Object.QFsf (http://www.lifewatch.be/etn/build/app.js?1.0:42:83486)
    at e (http://www.lifewatch.be/etn/build/app.js?1.0:1:101)
    at Object.012l (http://www.lifewatch.be/etn/build/app.js?1.0:1:524)
    at e (http://www.lifewatch.be/etn/build/app.js?1.0:1:101)
    at http://www.lifewatch.be/etn/build/app.js?1.0:1:462
    at http://www.lifewatch.be/etn/build/app.js?1.0:1:476
QFsf @ app.js?1.0:42
e @ app.js?1.0:1
012l @ app.js?1.0:1
e @ app.js?1.0:1
(anonymous) @ app.js?1.0:1
(anonymous) @ app.js?1.0:1
animals:655 Uncaught ReferenceError: $ is not defined
    at animals:655
(anonymous) @ animals:655
animals:1272 Uncaught ReferenceError: $ is not defined
    at animals:1272

The page currently looks like this for me:

Add missing capture and release latitude/longitude

Update: this text and table is an updated with the most recent information, based on the comments in this usse

It is important that the capture event and release event have coordinates. This is already the case for 574 animals, but for 236 those coordinates are missing. I understand it might be difficult to retroactively add those coordinates, but maybe it is possible based on the captureLocation and releaseLocation?

The table below shows how many animals per project have/don't have capture and release coordinates. Where FALSE they should be added.

projectName	capture_coordinates	release_coordinates	number of animals
2013 Albertkanaal	FALSE	TRUE	112
2013 Albertkanaal	FALSE	FALSE	29

Multiple tags for single animal

Some animals have multiple tags, e.g.:

"A69-9006-905;A69-9006-906,"2015 PhD Verhelst","2018-02-13 11:00","Gadus morhua"

We will have to figure out if those tags are added in sequence or at the same time and how the surgery date relates to these (first/latest).

@PieterjanVerhelst any insights on multiple tags for single animals?

Update length type suggestions

In light of #13, please update the as in examples in the application to their lowercase variants:

length_type

Change to as in: total length

lenght2_type

Change to as in: horizontal eye diameter

length3_type

Change to as in: vertical eye diameter

length4_type

Change to as in: pectoral fin length

geodeticDatum and coordinate uncertainty

I assume that the geodeticDatum upon which the coordinate system is based is WGS84?

With respect to the coordinate uncertainty:

The horizontal distance (in meters) from the given decimalLatitude and decimalLongitude describing the smallest circle containing the whole of the Location. Leave the value empty if the uncertainty is unknown, cannot be estimated, or is not applicable (because there are no coordinates). Zero is not a valid value for this term.

Best practise is to populate this field. For Darwin Core, 30 is the suggested number:

30 (reasonable lower limit of a GPS reading under good conditions if the actual precision was not recorded at the time).

Is this reasonable?

Fish without length information

The following individuals have no length information but do have length_type provided. Can these values be supplied?

For VLIZ:
@jreubens

projectname	id_pk	length	length_type
Homarus	251	NA	total length
Homarus	252	NA	total length
PhD Jan Reubens	674	NA	total length

For INBO:
@PieterjanVerhelst

projectname	scientific_name	id_pk	length2	length2_type
2012 Leopoldkanaal	Anguilla anguilla	483	NA	horizontal eye diameter

Missing surgery date and locations

I'm repeating the comment from a previous issue here (#33 (comment)) as it is quite important for the mapping process:

There's still the problem that date_of_surgery is empty for about 57% of the events and surgery_latitude and surgery_longitude are always empty (for the data within the publication scope)

I found out that surgery_latitude and surgery_longitude can be derived by comparing surgery_location with capture_location or release_location, but there's still no way to figure out whether surgery even occured for the remaining 57% of data with no information whatsoever relating to the surgery event. So my question is:

1. Did 57% of the animals have no surgery at all?
2. Did surgery occur but are these records missing from the database?

If 1 is true, then I can just do the mapping for the 43% that did have surgery. In case 2 is true, then I have to do the mapping for all data, but at this points most records will be empty.

Why is there surgery location/lat/long if there is no surgery date?

@PieterjanVerhelst @jreubens

I noticed that it is possible to define a surgery location (+ latitude and longitude), but not a surgery date. That means you cannot define a separate surgery event.

If the surgery date is assumed to be the capture date, can't we assume the same for surgery location then? That would reduce the amount of metadata that needs to be managed. It looks like it is used as such anyway: it is populated for 47% of the records, but is almost always the same as the capture location. Here are the records where it is different:

captureLocation	surgeryLocation	animals
D1	D5	3
D3	D5	2
D6	D5	2
D6	tss D5 en D6	1
D6	tussen D5 en D6	1
KLEINE KIL	LEOPOLDKANAAL (NOORDSTRAAT)	9
SS Auretta	SS Auretta (wrak 224)	1
Wrak Bruno Heinemann	Bruno Heinemann	1

I would suggest to either:

• Have surgery location AND surgery date
• Have neither (closest to the current use and my preference)

Release date before capture date

For some animals, the date of release is before the date of capture.
For the following cases, no time has been provided (didn't check them all):

projectname	id_pk	utc_release_date_time	catched_date_time	utc_release_date_time_new
2012 Leopoldkanaal	438	2012-07-20 00:00:00	2012-07-20 15:20:33	2012-07-20 15:50:33
2012 Leopoldkanaal	439	2012-07-24 00:00:00	2012-07-24 14:56:09	2012-07-24 15:26:09
2012 Leopoldkanaal	440	2012-07-24 00:00:00	2012-07-24 15:29:03	2012-07-24 15:59:03
2012 Leopoldkanaal	441	2012-07-24 00:00:00	2012-07-24 16:15:43	2012-07-24 16:45:43
2012 Leopoldkanaal	442	2012-07-24 00:00:00	2012-07-24 16:26:30	2012-07-24 16:56:30
2012 Leopoldkanaal	443	2012-07-19 00:00:00	2012-07-19 11:36:47	2012-07-19 12:06:47
2012 Leopoldkanaal	444	2012-07-19 00:00:00	2012-07-19 14:06:19	2012-07-19 14:36:19
2012 Leopoldkanaal	445	2012-07-19 00:00:00	2012-07-19 14:36:30	2012-07-19 15:06:30
2012 Leopoldkanaal	446	2012-07-20 00:00:00	2012-07-20 14:19:21	2012-07-20 14:49:21
2012 Leopoldkanaal	447	2012-07-20 00:00:00	2012-07-20 14:59:33	2012-07-20 15:29:33
2012 Leopoldkanaal	448	2012-07-17 00:00:00	2012-07-17 16:52:19	2012-07-17 17:22:19
2012 Leopoldkanaal	449	2012-07-18 00:00:00	2012-07-18 10:52:49	2012-07-18 11:22:49
2012 Leopoldkanaal	450	2012-07-18 00:00:00	2012-07-18 11:53:46	2012-07-18 12:23:46
2012 Leopoldkanaal	451	2012-07-18 00:00:00	2012-07-18 14:32:49	2012-07-18 15:02:49
2012 Leopoldkanaal	452	2012-07-19 00:00:00	2012-07-19 10:43:20	2012-07-19 11:13:20
2012 Leopoldkanaal	453	2012-07-17 00:00:00	2012-07-17 13:33:20	2012-07-17 14:03:20
2012 Leopoldkanaal	454	2012-07-17 00:00:00	2012-07-17 13:47:50	2012-07-17 14:17:50
2012 Leopoldkanaal	455	2012-07-17 00:00:00	2012-07-17 14:01:14	2012-07-17 14:31:14
2012 Leopoldkanaal	456	2012-07-17 00:00:00	2012-07-17 16:07:54	2012-07-17 16:37:54
2012 Leopoldkanaal	457	2012-07-17 00:00:00	2012-07-17 16:15:52	2012-07-17 16:45:52
2012 Leopoldkanaal	458	2012-07-06 00:00:00	2012-07-06 15:56:45	2012-07-06 16:26:45
2012 Leopoldkanaal	459	2012-07-06 00:00:00	2012-07-06 16:10:22	2012-07-06 16:40:22
2012 Leopoldkanaal	460	2012-07-17 00:00:00	2012-07-17 11:43:37	2012-07-17 12:13:37
2012 Leopoldkanaal	461	2012-07-17 00:00:00	2012-07-17 12:11:58	2012-07-17 12:41:58
2012 Leopoldkanaal	462	2012-07-17 00:00:00	2012-07-17 11:53:54	2012-07-17 12:23:54
2012 Leopoldkanaal	463	2012-07-05 00:00:00	2012-07-05 13:03:49	2012-07-05 13:33:49
2012 Leopoldkanaal	464	2012-07-06 00:00:00	2012-07-06 15:25:50	2012-07-06 15:55:50
2012 Leopoldkanaal	465	2012-07-06 00:00:00	2012-07-06 15:37:46	2012-07-06 16:07:46
2012 Leopoldkanaal	466	2012-07-05 00:00:00	2012-07-05 11:47:11	2012-07-05 12:17:11
2012 Leopoldkanaal	467	2012-07-05 00:00:00	2012-07-05 12:53:47	2012-07-05 13:23:47
2012 Leopoldkanaal	468	2012-07-05 00:00:00	2012-07-05 11:34:51	2012-07-05 12:04:51
2012 Leopoldkanaal	469	2012-07-05 00:00:00	2012-07-05 11:17:40	2012-07-05 11:47:40
2012 Leopoldkanaal	470	2012-07-05 00:00:00	2012-07-05 11:02:59	2012-07-05 11:32:59
2012 Leopoldkanaal	471	2012-07-04 00:00:00	2012-07-04 12:03:54	2012-07-04 12:33:54
2012 Leopoldkanaal	472	2012-07-04 00:00:00	2012-07-04 11:42:00	2012-07-04 12:12:00
2012 Leopoldkanaal	473	2012-07-03 00:00:00	2012-07-03 10:38:35	2012-07-03 11:08:35
2012 Leopoldkanaal	474	2012-07-04 00:00:00	2012-07-04 10:24:25	2012-07-04 10:54:25
2012 Leopoldkanaal	475	2012-07-03 00:00:00	2012-07-03 12:45:00	2012-07-03 13:15:00
2012 Leopoldkanaal	476	2012-07-03 00:00:00	2012-07-03 12:31:20	2012-07-03 13:01:20
2012 Leopoldkanaal	477	2012-07-03 00:00:00	2012-07-03 11:00:27	2012-07-03 11:30:27
2012 Leopoldkanaal	478	2011-10-20 00:00:00	2011-10-20 10:00:00	2011-10-20 10:30:00
2012 Leopoldkanaal	479	2012-07-03 00:00:00	2012-07-03 13:01:05	2012-07-03 13:31:05
2012 Leopoldkanaal	480	2012-07-03 00:00:00	2012-07-03 12:37:21	2012-07-03 13:07:21
2012 Leopoldkanaal	481	2012-07-03 00:00:00	2012-07-03 12:21:21	2012-07-03 12:51:21
2012 Leopoldkanaal	482	2012-07-03 00:00:00	2012-07-03 10:51:02	2012-07-03 11:21:02
2012 Leopoldkanaal	483	2011-12-04 00:00:00	2011-12-04 10:00:00	2011-12-04 10:30:00
2012 Leopoldkanaal	484	2012-07-04 00:00:00	2012-07-04 10:35:18	2012-07-04 11:05:18
2012 Leopoldkanaal	485	2011-12-04 00:00:00	2011-12-04 10:00:00	2011-12-04 10:30:00
2012 Leopoldkanaal	486	2011-10-20 00:00:00	2011-10-20 10:00:00	2011-10-20 10:30:00
2012 Leopoldkanaal	487	2011-10-20 00:00:00	2011-10-20 10:00:00	2011-10-20 10:30:00
2012 Leopoldkanaal	229	2012-10-16 00:00:00	2012-10-16 10:39:07	2012-10-16 11:09:07
2012 Leopoldkanaal	230	2012-10-16 00:00:00	2012-10-16 10:57:49	2012-10-16 11:27:49
2012 Leopoldkanaal	231	2012-10-16 00:00:00	2012-10-16 10:20:00	2012-10-16 10:50:00
2012 Leopoldkanaal	232	2012-10-16 00:00:00	2012-10-16 12:50:34	2012-10-16 13:20:34
2012 Leopoldkanaal	387	2012-08-21 00:00:00	2012-08-21 10:22:50	2012-08-21 10:52:50
2012 Leopoldkanaal	388	2012-08-21 00:00:00	2012-08-21 10:16:00	2012-08-21 10:46:00
2012 Leopoldkanaal	390	2012-08-14 00:00:00	2012-08-14 15:49:41	2012-08-14 16:19:41
2012 Leopoldkanaal	391	2012-08-14 00:00:00	2012-08-14 15:49:10	2012-08-14 16:19:10
2012 Leopoldkanaal	392	2012-08-21 00:00:00	2012-08-21 10:55:23	2012-08-21 11:25:23
2012 Leopoldkanaal	393	2012-08-21 00:00:00	2012-08-21 13:57:00	2012-08-21 14:27:00
2012 Leopoldkanaal	394	2012-12-14 00:00:00	2012-12-14 13:00:00	2012-12-14 13:30:00
2012 Leopoldkanaal	395	2012-08-21 00:00:00	2012-08-21 13:46:47	2012-08-21 14:16:47
2012 Leopoldkanaal	396	2012-08-21 00:00:00	2012-08-21 10:47:15	2012-08-21 11:17:15
2012 Leopoldkanaal	397	2012-08-14 00:00:00	2012-08-14 15:48:03	2012-08-14 16:18:03
2012 Leopoldkanaal	398	2012-08-21 00:00:00	2012-08-21 14:21:39	2012-08-21 14:51:39
2012 Leopoldkanaal	399	2012-08-22 00:00:00	2012-08-22 15:11:05	2012-08-22 15:41:05
2012 Leopoldkanaal	400	2012-08-21 00:00:00	2012-08-21 14:49:55	2012-08-21 15:19:55
2012 Leopoldkanaal	401	2012-08-21 00:00:00	2012-08-21 12:50:32	2012-08-21 13:20:32
2012 Leopoldkanaal	402	2012-08-21 00:00:00	2012-08-21 13:30:47	2012-08-21 14:00:47
2012 Leopoldkanaal	403	2012-08-08 00:00:00	2012-08-08 15:12:46	2012-08-08 15:42:46
2012 Leopoldkanaal	404	2012-08-21 00:00:00	2012-08-21 10:39:07	2012-08-21 11:09:07
2012 Leopoldkanaal	405	2012-08-22 00:00:00	2012-08-22 14:54:08	2012-08-22 15:24:08
2012 Leopoldkanaal	406	2012-08-21 00:00:00	2012-08-21 13:08:24	2012-08-21 13:38:24
2012 Leopoldkanaal	407	2012-08-21 00:00:00	2012-08-21 14:33:32	2012-08-21 15:03:32
2012 Leopoldkanaal	408	2012-08-08 00:00:00	2012-08-08 12:32:41	2012-08-08 13:02:41
2012 Leopoldkanaal	409	2012-08-08 00:00:00	2012-08-08 13:37:40	2012-08-08 14:07:40
2012 Leopoldkanaal	410	2012-08-08 00:00:00	2012-08-08 14:09:39	2012-08-08 14:39:39
2012 Leopoldkanaal	411	2012-08-08 00:00:00	2012-08-08 14:54:11	2012-08-08 15:24:11
2012 Leopoldkanaal	412	2012-08-21 00:00:00	2012-08-21 11:05:52	2012-08-21 11:35:52
2012 Leopoldkanaal	413	2012-08-09 00:00:00	2012-08-09 10:16:56	2012-08-09 10:46:56
2012 Leopoldkanaal	414	2012-08-09 00:00:00	2012-08-09 10:48:19	2012-08-09 11:18:19
2012 Leopoldkanaal	415	2012-08-09 00:00:00	2012-08-09 11:13:55	2012-08-09 11:43:55
2012 Leopoldkanaal	416	2012-08-09 00:00:00	2012-08-09 11:22:20	2012-08-09 11:52:20
2012 Leopoldkanaal	417	2012-08-08 00:00:00	2012-08-08 12:05:39	2012-08-08 12:35:39
2012 Leopoldkanaal	418	2012-10-08 00:00:00	2012-10-08 10:40:38	2012-10-08 11:10:38
2012 Leopoldkanaal	419	2012-10-05 00:00:00	2012-10-05 12:01:21	2012-10-05 12:31:21
2012 Leopoldkanaal	420	2012-10-05 00:00:00	2012-10-05 11:44:05	2012-10-05 12:14:05
2012 Leopoldkanaal	421	2012-10-05 00:00:00	2012-10-05 11:16:06	2012-10-05 11:46:06
2012 Leopoldkanaal	422	2012-10-05 00:00:00	2012-10-05 10:48:20	2012-10-05 11:18:20
2012 Leopoldkanaal	423	2012-10-16 00:00:00	2012-10-16 11:49:46	2012-10-16 12:19:46
2012 Leopoldkanaal	424	2012-10-16 00:00:00	2012-10-16 11:20:01	2012-10-16 11:50:01
2012 Leopoldkanaal	425	2012-10-10 00:00:00	2012-10-10 10:00:00	2012-10-10 10:30:00
2012 Leopoldkanaal	426	2012-10-10 00:00:00	2012-10-10 10:00:00	2012-10-10 10:30:00
2012 Leopoldkanaal	427	2012-10-10 00:00:00	2012-10-10 10:00:00	2012-10-10 10:30:00
2012 Leopoldkanaal	428	2012-10-08 00:00:00	2012-10-08 13:49:32	2012-10-08 14:19:32
2012 Leopoldkanaal	429	2012-10-08 00:00:00	2012-10-08 14:11:27	2012-10-08 14:41:27
2012 Leopoldkanaal	430	2012-10-08 00:00:00	2012-10-08 14:30:12	2012-10-08 15:00:12
2012 Leopoldkanaal	431	2012-10-08 00:00:00	2012-10-08 14:47:57	2012-10-08 15:17:57
2012 Leopoldkanaal	432	2012-10-08 00:00:00	2012-10-08 15:00:11	2012-10-08 15:30:11
2012 Leopoldkanaal	433	2012-10-08 00:00:00	2012-10-08 13:34:48	2012-10-08 14:04:48
2012 Leopoldkanaal	434	2012-10-08 00:00:00	2012-10-08 13:15:17	2012-10-08 13:45:17
2012 Leopoldkanaal	435	2012-10-08 00:00:00	2012-10-08 12:55:04	2012-10-08 13:25:04
2012 Leopoldkanaal	436	2012-10-08 00:00:00	2012-10-08 12:00:23	2012-10-08 12:30:23
2012 Leopoldkanaal	437	2012-10-08 00:00:00	2012-10-08 10:49:30	2012-10-08 11:19:30
2015 PhD Verhelst	638	2015-05-22 00:00:00	2015-05-22 11:02:59	2015-05-22 11:32:59
2015 PhD Verhelst	639	2015-05-22 00:00:00	2015-05-22 09:15:58	2015-05-22 09:45:58
2015 PhD Verhelst	640	2015-05-22 00:00:00	2015-05-22 09:10:56	2015-05-22 09:40:56
2015 PhD Verhelst	642	2015-05-22 00:00:00	2015-05-22 10:46:50	2015-05-22 11:16:50
2015 PhD Verhelst	643	2015-05-22 00:00:00	2015-05-22 09:39:39	2015-05-22 10:09:39
2015 PhD Verhelst	644	2015-05-21 00:00:00	2015-05-21 12:42:17	2015-05-21 13:12:17
2015 PhD Verhelst	646	2015-05-22 00:00:00	2015-05-22 10:23:23	2015-05-22 10:53:23
2015 PhD Verhelst	647	2015-05-22 00:00:00	2015-05-22 09:58:42	2015-05-22 10:28:42
2015 PhD Verhelst	648	2015-05-21 00:00:00	2015-05-21 12:30:41	2015-05-21 13:00:41
2015 PhD Verhelst	613	2015-10-21 00:00:00	2015-10-21 12:58:20	2015-10-21 13:28:20
2015 PhD Verhelst	614	2015-10-21 00:00:00	2015-10-21 13:46:04	2015-10-21 14:16:04
2015 PhD Verhelst	615	2015-10-21 00:00:00	2015-10-21 14:02:32	2015-10-21 14:32:32
2015 PhD Verhelst	616	2015-10-21 00:00:00	2015-10-21 14:31:44	2015-10-21 15:01:44
2015 PhD Verhelst	617	2015-10-22 00:00:00	2015-10-22 12:58:07	2015-10-22 13:28:07
2015 PhD Verhelst	618	2015-10-22 00:00:00	2015-10-22 13:36:18	2015-10-22 14:06:18
2015 PhD Verhelst	612	2015-09-17 00:00:00	2015-09-17 12:34:59	2015-09-17 13:04:59
2015 PhD Verhelst	619	2015-09-18 00:00:00	2015-09-18 12:11:49	2015-09-18 12:41:49
2015 PhD Verhelst	620	2015-09-18 00:00:00	2015-09-18 12:36:55	2015-09-18 13:06:55
2015 PhD Verhelst	380	2015-06-06 00:00:00	2015-06-06 16:05:38	2015-06-06 16:35:38
2015 PhD Verhelst	379	2015-04-17 00:00:00	2015-04-17 13:28:48	2015-04-17 13:58:48
2015 PhD Verhelst	378	2015-04-17 00:00:00	2015-04-17 12:49:10	2015-04-17 13:19:10
2015 PhD Verhelst	381	2015-06-06 00:00:00	2015-06-06 16:55:07	2015-06-06 17:25:07
2015 PhD Verhelst	598	2015-08-18 00:00:00	2015-08-18 15:20:33	2015-08-18 15:50:33
2015 PhD Verhelst	599	2015-08-18 00:00:00	2015-08-18 14:55:11	2015-08-18 15:25:11
2015 PhD Verhelst	600	2015-08-18 00:00:00	2015-08-18 15:06:37	2015-08-18 15:36:37
2015 PhD Verhelst	601	2015-08-19 00:00:00	2015-08-19 15:53:09	2015-08-19 16:23:09
2015 PhD Verhelst	602	2015-08-20 00:00:00	2015-08-20 15:51:18	2015-08-20 16:21:18
2015 PhD Verhelst	603	2015-08-20 00:00:00	2015-08-20 17:07:09	2015-08-20 17:37:09
2015 PhD Verhelst	604	2015-08-20 00:00:00	2015-08-20 17:29:29	2015-08-20 17:59:29
2015 PhD Verhelst	625	2015-08-12 00:00:00	2015-08-12 14:08:44	2015-08-12 14:38:44
2015 PhD Verhelst	595	2015-08-12 00:00:00	2015-08-12 13:16:18	2015-08-12 13:46:18
2015 PhD Verhelst	596	2015-08-12 00:00:00	2015-08-12 13:21:18	2015-08-12 13:51:18
2015 PhD Verhelst	597	2015-08-12 00:00:00	2015-08-12 13:39:35	2015-08-12 14:09:35
2015 PhD Verhelst	605	2015-09-16 00:00:00	2015-09-16 12:08:53	2015-09-16 12:38:53
2015 PhD Verhelst	606	2015-09-16 00:00:00	2015-09-16 12:12:24	2015-09-16 12:42:24
2015 PhD Verhelst	607	2015-09-16 00:00:00	2015-09-16 12:39:50	2015-09-16 13:09:50
2015 PhD Verhelst	608	2015-09-16 00:00:00	2015-09-16 12:47:53	2015-09-16 13:17:53
2015 PhD Verhelst	609	2015-09-16 00:00:00	2015-09-16 12:58:01	2015-09-16 13:28:01
2015 PhD Verhelst	610	2015-09-16 00:00:00	2015-09-16 13:09:06	2015-09-16 13:39:06
2015 PhD Verhelst	611	2015-09-17 00:00:00	2015-09-17 12:17:22	2015-09-17 12:47:22
2015 PhD Verhelst	621	2015-10-20 00:00:00	2015-10-20 12:13:34	2015-10-20 12:43:34
2015 PhD Verhelst	622	2015-10-20 00:00:00	2015-10-20 12:34:24	2015-10-20 13:04:24
2015 PhD Verhelst	623	2015-10-20 00:00:00	2015-10-20 13:16:23	2015-10-20 13:46:23
2015 PhD Verhelst	624	2015-10-20 00:00:00	2015-10-20 14:16:58	2015-10-20 14:46:58

Get IMIS dataset IDs for animal projects

The network projects are listed in IMIS (see #19), but it seems the animal projects (#7) are as well. See for example:

http://www.vliz.be/en/imis?module=dataset&dasid=5850
European eel acoustic telemetry dataset to assess eel movement behaviour

Which could be 2015 PhD Verhelst.

@jreubens Is the IMIS dataset ID stored in ETN? If not, I think we should link the two, as it would be really useful for the data paper. The overview of child datasets at http://www.vliz.be/en/imis?module=dataset&dasid=5250 should get us started.

Animal project	imis
2011 Rivierprik	5867
2012 Leopoldkanaal	5873
2013 Albertkanaal	5868
2014 Demer	5871
2015 Dijle	5872
(2015 Fint)	5858
2015 PhD Verhelst	5850
Homarus	-
PhD Jan Reubens	5846

Incorrect release date

19 animals in Albertkanaal 2013 have the comment:

uitzetdatum moet nog aangepast worden; 30 maart is niet juist

This should be corrected:

A69-1601-64495
A69-1601-64496
A69-1601-64497
A69-1601-64498
A69-1601-64499
A69-1601-64500
A69-1601-64501
A69-1601-64502
A69-1601-64503
A69-1601-64504
A69-1601-64505
A69-1601-64506
A69-1601-64507
A69-1601-64508
A69-1601-64509
A69-1601-64510
A69-1601-64511
A69-1601-64512
A69-1601-64513

Use controlled value for lengthType

The values for length_type, length2_type, length3_type, length4_type are currently free text, but it would be better if these used a controlled vocabulary. I would suggest to use lowercase, e.g. total length, which is already the case for most animals.

Table with current values and how to update these (including fixing some typos):

current value	controlled value	projects	animals	remarks
fork lenght	fork length	2013 Albertkanaal	29	corrected
horizontal eye diameter	horizontal eye diameter	2015 PhD Verhelst, 2012 Leopoldkanaal, 2013 Albertkanaal	392	ok
pectoral fin	pectoral fin length	2013 Albertkanaal, 2015 PhD Verhelst, 2012 Leopoldkanaal	392	@fwaumans
total length	total length	2013 Albertkanaal, 2015 PhD Verhelst, 2015 Fint, 2011 Rivierprik, 2012 Leopoldkanaal, PhD Jan Reubens, 2014 Demer, 2015 Dijle, Homarus	712	ok
Total length	total length	2015 PhD Verhelst	21	corrected
Total Length	total length	2015 Fint	4	corrected
vertical eye diameter	vertical eye diameter	2015 PhD Verhelst, 2012 Leopoldkanaal, 2013 Albertkanaal	346	ok
verticale eye diameter	vertical eye diameter	2013 Albertkanaal	45	corrected

Figure with tags by year and species

As discussed with @IPauwels, for sampling coverage, create a figure with:

• Rows: species
• Columns: year
• Values: number of deployed tags (including not yet published tags)
• Colour cells on values (per species, see figure below)
• Indicate scientific name and English vernacular name
• Maybe group species on habitat: obligatory marine, etc. (or extra column)
• Maybe add column with projects in which these species are tagged

Example for inspiration (from this gist)

This figure needs to be created from the non-public dataset, as it contains tags under moratorium.

Use consistent common name

Only 11 of the 810 animals have a common name:

• 7 Atlantic cod (2015 PhD Verhelst)
• 4 Fint (2015 Fint)

The definition of the field is The species' common English name, so Fint is incorrect. I would argue to either always populate the common English name or never.

Empty values for sensor type

Values for the field tag_sensor_type are:

tag_sensor_type	number of records
P	10
A	10
NA	652

This means that for 97% of the records, no sensor type is provided. @PieterjanVerhelst , @jreubens and @IPauwels can we complete this information?
These empty records occur for the following projects x tag model combinations:

projectcode	model	records
2011_rivierprik	V7-4x	32
2011_rivierprik	V8-4x	7
2012_leopoldkanaal	V13-1x	103
2012_leopoldkanaal	V7-4x	4
2013_albertkanaal	V13-1x	131
2013_albertkanaal	V7-2x	114
2013_albertkanaal	V7-4x	25
2013_albertkanaal	V9-2x	11
2014_demer	V13-1x	13
2014_demer	V7-4x	3
2015_dijle	V13-1x	3
2015_dijle	V7-4x	23
2015_phd_verhelst	V13-1x	96
2015_phd_verhelst	V13-1x-069k-1	40
homarus	V13-1x	2
phd_reubens	V9-1x	45

Add id_pk information to field summary

In a search for a unique identifier for each animal, I encountered the field id_pk.
There's no definition available for this field, so please update this in the appropriate csv file.

Move durif index info to age + age_unit

590 animals mention durif_index in comments, e.g.:

; durif_index : 
; durif_index : 100.1274
; durif_index : 100.130445
without calliper ; durif_index : 104.4134	

What is durif_index? Is it important? Can we structure this information better, e.g. by creating an extra field in ETN or by using one of the length fields:

lenght4: 100.1274
lenght4_type: durif_index

Projects with durif_index in the comments:

projectName	animals
2011 Rivierprik	39
2012 Leopoldkanaal	104
2013 Albertkanaal	162
2014 Demer	16
2015 Dijle	26
2015 Fint	8
2015 PhD Verhelst	193
Homarus	2
PhD Jan Reubens	40

Projects to include for data publication

Overview of the projects that will be included in the dataset we will publish:

animal project	scientific name	indiv	detections	include in dataset
2011 Rivierprik	Lampetra fluviatilis	39	114.605	yes
2012 Leopoldkanaal	Anguilla anguilla	104	2.215.830	yes
2013 Albertkanaal	Anguilla anguilla	161	4.784.914	yes
2013 Albertkanaal	Salmo salar	135	1.375.687	yes
2013 Albertkanaal	Sync tag	13	16M	no, not biological
2014 Demer	Petromyzon marinus	2	42	yes
2014 Demer	Rutilus rutilus	2	11.030	yes
2014 Demer	Silurus glanis	9	86.973	yes
2014 Demer	Squalius cephalus	3	139.013	yes
2015 Dijle	Anguilla anguilla	1	41.798	yes
2015 Dijle	Cyprinus carpio	2	4.944	yes
2015 Dijle	Platichthys flesus	8	101.488	yes
2015 Dijle	Rutilus rutilus	4	7.870	yes
2015 Dijle	Silurus glanis	11	78.501	yes
2015 Fint	Alosa fallax	44	241.530	no, not yet
2015 PhD Verhelst	Anguilla anguilla	136	1.696.255	yes
2015 PhD Verhelst	Gadus morhua	106	1.526.911	no, not yet
Homarus	Homarus gammarus	2	22.265	yes
PhD Jan Reubens	Gadus morhua	41	709.912	yes
PhD Jan Reubens	Sentinel	27	594.301	no, not biological

Update example for field sex

See #18: the recommended values are now:

male
female
unknown

Can the example in the help text be updated?

Definition
Sex of the animal. "male", "female" or "unknown".

Units/format example
as in: female

Unique identifiers for animals

The field animal_id appears to be to best field for unique identifiers:

Identification code that uniquely identifies each animal, as specified by the researcher.

However, this field is often empty, which makes it useless as a unique identifier.
Decided to take id_pk instead, but see #29

Include AGE and AGE_UNIT in table and download

@fwaumans can you add the fields AGE and AGE_UNIT in the list of available fields to show in the animals table:

Can you add them to the data download as well?