MergeNav/EDOffline Version 2.0

The MergeNav/EDoffline an off-line application was originally written to merge navigation with SEGY seismic data that had been converted into JPEG 2000 format. The application gradually morphed into a framework for showing the GSC's Marine survey data coverage.

This application relies on two technologies. All marine survey navigation is extracted from departmental databases and stored in compressed XML format; this extracted package is included with this program

This compressed XML format is read by this application during startup; therefore, the entire navigational database for surveys is loaded into memory when we run this application.

If you want to display track data, you would like a base map for reference. Rather than distributing base maps with this application, it was decided to fetch the base maps interactively from network-hosted WMS servers.

This program also provides the ability to query expedition navigation to find physical record holdings that correspond to different segments along the expedition track. In addition, navigation data from this program can be exported in shape files for import into GIS systems.

As it stands, this program is self-contained in that the entire GSC navigation database is included with this package, and the program may be run with no network connection. Since the base maps are derived from WMS servers, no base map will appear in this instance.

New features of this program ( not including many bug fixes) include :

Auto-detection of new program and data base files on the NRCan ftp server.

Akima spline interpolation of navigation data.

The ability to import navigation data from other sources and merge new fixes in JPEG2000 encoded SEGY data. These new data can be saved to form part of a supplementary navigational database.

Intellectual and Property Rights

This program is the intellectual property of the Government of Canada. All rights to modify and distribute this software are retained by the Government of Canada.

"Intellectual Property" means all rights in respect of Confidential Information, any know‑how, inventions and improvements related to Confidential Information, including without limitation, patents, copyrights, trade secrets, trade‑marks, registered industrial designs, any applications for same and all rights therein.

"Confidential Information" means any information of a scientific or technical nature disclosed to, generated or acquired by the Student in the course of carrying out the Student activities defined in this Agreement, whether oral or recorded in any form or medium and whether or not protectable by copyright.

We allow this program to be redistributed freely in its original form. However, please send me an email if you have a copy – I’ll ensure you get on a mailing list for upgrades.

Contact the author, Bob Courtney, Natural Resources Canada, 1 Challenger Drive , Dartmouth Nova Scotia B2Y 4A2 , to obtain the most recent version.

This program contains ocode from other sources ( open source code Math.NET (www.mathdotnet.com) and from the following commercial sources (IP Works and Steema TeeChart ) in compliance with their copyright restrictions. Source code can be obtained from the author at no charge.

Installation:

This application can be installed on Windows XP platforms with at least 2 GB of memory. It relies on Microsoft's .net framework 2.0 , and it is assumed that the user has already installed this library.

If a user is upgrading this software, the previous version has to manually remove using the add or remove application option in the control panel.

Users inside the NRCan firewall:

This application can be installed from the network using this link.

Users outside the NRCan firewall:

A zip file containing the distribution package can be downloaded from the network using this link.

Update navigation data

This program uses data that is periodically extracted from divisional databases and packaged in compressed XML. This packaged file should be periodically updated to include recent additions to her data holdings.

Users within the firewall will be automatically prompted to download a new version. This file is typically around 200 MB, so depending on your network connection the time to your system will vary.

Users outside the firewall will have to download this data package using FTP. See below.

Startup:

Double click on the EDOffline icon that is located under the program menu: All Programs=>NRCan=>EDOffline.

If the WMS server is online, then the base map should appear quite quickly. Sometimes WMS servers offer multiple layers within a given service, and the lower right listbox will list these layers. You can highlight one or more layers of the time, and pushed the Refresh Map Button to refresh the base map.

The first time the program is started, the compressed XML-based navigational data structure is downloaded from NRCan servers.. Be patient, depending on your internet connection, this stage may take a few minutes to connect and start downloading. You should see a window like this :

You can manually download this file from ftp://ftp.nrcan.gc.ca/gsc/courtney/EdOffline_MergeNav/GSCnav.xmz. On 64 bit machines, the file will be placed in C:\Program Files (x86)\NRCan\MergeNav and, on 32 bit machines, in C:\Program Files\NRCan\MergeNav

It will take some time to load the compressed navigation file into memory ( a few minutes) :

After the navigation file is loaded into memory, a list of expedition names within the currently defined area will be loaded into the coincident cruises list box:

Zoom and pan

Zoom into the image by “rubberbanding” a box on the screen using the left mouse button pressed down from the upper left of the box to the lower right.

Zoom mode of the image by “rubberbanding” a box from the lower right to the upper left corner.

The image can be panned by holding the right mouse button down and dragging to the new position.

Within the geographic boundaries of the viewport are changed, a new background map is fetched from the WMS server and the navigational data is queried and the only the expeditions that cross the current area are listed in the upper right listbox.

Note : The program is NOT a GIS and background maps are drawn in a simple lat/lon projection. The corners of the main window should be manually readjusted to achieve a reasonable looking aspect ratio for viewing.

Query expedition holdings

Physical records (e.g., expedition logs, field records of seismic data, digital tape or disc media containing segy information) that are housed within the GSC, correspomding to a selected expedition , can be queried by clicking once on one of the expedition/cruise names in the upperright listbox.

With a single click on the entry, the navigation track for the selected expedition is plotted within the current window. This track is plotted in black. A small red locator circle is position on the track; this position can be changed by moving the mouse over to other sections of the navigation track.

When the left mouse button is clicked, the text box located in the upper right corner will display the time and position of the selected position. The data types available at the selected point will be displayed in the centre-right list box.

If one clicks on a point along the expedition track, a physical record (PAD) window is also popped-up to display the in-house physical records that are time-tagged include data over the chosen point.

Some entries correspond to the whole expedition (such as the expedition log) , while others are more specific to the chosen point (such as a seismic section that crosses the point in question ) . In future versions, this pop-up will be augmented to include the availability of digital seismic sections that will be downloaded directly from the web.

If the expedition name in the upper right is double-clicked, then the viewport is changed to span only the area that bounds the navigation of the selected expedition. This track is plotted in red:

The track lines of other expeditions across this area may be over-plotted in black by single clicking on the entries in the upper right listbox.

Options

Graphics Toolbar

The toolbar may be used to edit, print, copied to the clipboard, or save the current view.

The other entries in the toolbar should be avoided, especially the 3-D option. Toggle the 3-D button to restore the original configuration if it is pressed by mistake.

The graphics are generated through a third-party plug-in called TeeChart (www.steema.com) and more information on this graphics options can be found on the manufacturers website.

Settings

The settings menu option will pop up a window that will allow users within the firewall to specify the location of the compressed XML navigation file.

The lower entry box allows the user to specify a preferred WMS server or to choose from a number of prescribed entries in the pulldown box.

Export

The export option can be used to export navigation in shape file format for import into GIS systems or in DBF format for input into MS Excel or a similar spreadsheet program.

An expedition name is chosen from the top pull-down selection box. A status line with the year, expedition vehicle, and geographic data for that expedition is updated.

The shape file will be exported with either a NAD83 (or equivalently WGS84) or NAD27 datum projection as specified with the radio buttons. This program does not convert between datums so the user should enter the datum that is displayed in the status line.

Please note that the DBF format has changed from previous versions. When the DBF file is exported as a text files in Microsoft Excel, the following format is followed :

Year Julian Day Hour Minute Second Latitude Longitude

1974 136 21 1 0 47.19417200 -60.41867100

1974 136 21 2 0 47.19667100 -60.42073100

1974 136 21 3 0 47.19916900 -60.42279100

1974 136 21 4 0 47.20167200 -60.42485000

1974 136 21 5 0 47.20417000 -60.42691000

1974 136 21 6 0 47.20666900 -60.42897000

Navigation Updates

Each time the program is operated, it will check the NRCan ftp server for updates to the navigation data base. If new versions are available, the label for the top menu strip button will change to New Navigation Available. Click on the menu strip button to download the new version of the navigation.

Program Update

Similarly, the NRCan file server is checked automatically to see if a new version of the program is available. If so, the label of the menu strip button will change to New Program Available. Click on this label to download the new program and follow the instructions given.

Merge EDNav in SGYJP2

This program was originally developed to merge navigation derived from the NRCAN Expedition Database (ED) into JPEG2000-encoded SEGY files. The tab page title "Merge EdNav in SGYJP2" is where this task can be accomplished :

Use the "Load Files" button to choose the SGYJP2 files into which to merge navigation data. It is good practice to only use files from one expedition at a time, although those files may contain different data types ( e.g., seismic and/or sidescan).

Each SGYJP2 file contains metadata that should specify the expedition id and that id will be used, by default, to choose the corresponding navigation from the ED database. In the case that the id is incorrect or has not been set, then the pull-down combo box can be used to specify another navigation source

N.B. Navigation from third party sources that has been imported into the supplementary data base will also be displayed in the pull-down combo-box. See the following section "Merge Nav from External File" .

A constant layback can be applied to the navigation to adjust for tow fish position.

The program will interpolate between navigational fixes using an Akima spline. (Hiroshi Akima, "A Method of Bivariate Interpolation and Smooth Surface Fitting for Irregularly Distributed Data Points", ACM Transactions on Mathematical Software, Vol. 4, No. 2, June 1978, pp. 148-159. Copyright 1978, Association for Computing Machinery, Inc).

This choice of spline reduces overshoot in interpolation.

Click the button labelled "Merge Navigation" to insert navigation into the SGYJP2 headers. The tracks on the merged files will be displayed, in 10 second increments, on the main navigation page.

Click on the triangle icon to identify and/or edit the line segments.

Merge Nav from External File

Navigation can be imported from third party sources using the procedures on the third tab page:

The external navigation file must be inASCII with the following format :

year julian_day hour minute seconds decimal_latitude decimal_longitude

with no column titles, for example :

1974 136 21 1 0 47.19417200 -60.41867100

1974 136 21 2 0 47.19667100 -60.42073100

1974 136 21 3 0 47.19916900 -60.42279100

1974 136 21 4 0 47.20167200 -60.42485000

1974 136 21 5 0 47.20417000 -60.42691000

1974 136 21 6 0 47.20666900 -60.42897000

(Note : the same format as the DBF export option with the titles deleted)

A second format is supported :

trace_number decimal_latitude decimal_longitude

This second option has not been well tested so contact me if you have problems with this feature.

Load Navigation from Disk

Click the "Load Navigation from Disk" button to choose an input navigation file. Once loaded, a map vie of teh data will appear in the right plotting pane. The file name will be used as the Expedition ID, but this can be overwritten by the user.

Save Navigation

This new navigation can be saved in a supplementary database ( C:\Program Files\NRCan\MergeNav\Supplementary.xmz) with has the same schema structure as the ED navigational data( See Appendix I). An unlimited number of navigation files can be stored in this database.

Once stored in the supplementary database, these navigational data can be viewed and queried on the main navigation display page :

Merging Navigation into SGYJP2 files

The navigation can be merged into SGYJP2 files using the buttons on the bottom of the third tab panel ( "Load SGYJP2 files from Disk", "Insert Nav into SGYJP2 Files").

The merged segments will be displayed in the right hand plotting area as well as in the main page navigational display area.

Appendix I – XML Schema for Navigational Data Interchange

Navigational data extracted from GSC marine databases is distributed along with the program and can be found in the Program Files/NRCan/MergeNav directory under the name GSCnav.xmz.

This file is in gzip compressed format and contains XML described by the attached schema. At the moment the XML does not contain sample information (e.g., grain size, radiocarbon dates, etc), but the structure has been sketched out. The schema, navigation.xsd, is included in Program Files/NRCan/MergeNav.

Main Tree

Expedition Table

Navigation Data

Bounding Box Information

Physical Record Data (PAD)

Station Data (Not populated at the moment)

Text Representation :

?xml version="1.0" encoding="utf-8"?>

<xs:schema targetNamespace="http://tempuri.org/XMLSchema.xsd" elementFormDefault="qualified" xmlns="http://tempuri.org/XMLSchema.xsd" xmlns:mstns="http://tempuri.org/XMLSchema.xsd" xmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:element name="Main">

<xs:complexType>

<xs:sequence>

<xs:element name="expeditions" minOccurs="0" maxOccurs="unbounded" type="Expedition">

</xs:element>

<xs:element name="Created_x0020_on" type="xs:dateTime" />

</xs:sequence>

</xs:complexType>

</xs:element>

<xs:complexType name="PAD">

<xs:sequence>

<xs:element name="Box_x0020_Num" type="xs:string" />

<xs:element name="Record_x0020_Num" type="xs:string" />

<xs:element name="Item_x0020_Num" type="xs:unsignedInt" />

<xs:element name="Data_x0020_Type" type="xs:string" />

<xs:element name="Instrument_x0020_Type" type="xs:string" />

<xs:element name="Start_x0020_Time" type="xs:dateTime" />

<xs:element name="End_x0020_Time" type="xs:dateTime" />

<xs:element name="Line_x0020_or_x0020_Fix_x0020_Num" type="xs:string" minOccurs="0" maxOccurs="0" nillable="true" />

<xs:element name="Medium" type="xs:string" nillable="true" />

<xs:element name="Combined_x0020_Record" type="xs:string" />

<xs:element name="Public_x0020_Access" type="xs:string" />

<xs:element name="Comments" type="xs:string" minOccurs="0" maxOccurs="0" nillable="true" />

<xs:element name="Scanned_x0020_Record_x0020_Online" type="xs:boolean" minOccurs="0" maxOccurs="0" />

<xs:element name="Internal_x0020_Url" type="xs:string" minOccurs="0" maxOccurs="0" />

<xs:element name="External_x0020_Url" type="xs:string" minOccurs="0" maxOccurs="0" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Local_x0020_BBOX">

<xs:sequence>

<xs:element name="Start_x0020_Time" type="xs:dateTime" />

<xs:element name="End_x0020_Time" type="xs:dateTime" />

<xs:element name="minimum_x0020_latitude" type="xs:double" />

<xs:element name="maximum_x0020_latitude" type="xs:double" />

<xs:element name="minimum_x0020_longitude" type="xs:double" />

<xs:element name="maximum_x0020_longitude" type="xs:double" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Global_x0020_BBOX">

<xs:sequence>

<xs:element name="Start_x0020_Time" type="xs:dateTime" />

<xs:element name="End_x0020_Time" type="xs:dateTime" />

<xs:element name="minimum_x0020_latitude" type="xs:double" />

<xs:element name="maximum_x0020_latitude" type="xs:double" />

<xs:element name="minimum_x0020_longitude" type="xs:double" />

<xs:element name="maximum_x0020_longitude" type="xs:double" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Lines">

<xs:sequence>

<xs:element name="start_x0020_time" type="xs:dateTime" />

<xs:element name="end_x0020_time" type="xs:dateTime" />

<xs:element name="geophys_x0020_type" type="xs:string" />

<xs:element name="title" type="xs:string" minOccurs="0" maxOccurs="1" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Navigation">

<xs:sequence>

<xs:element name="latitude" type="xs:double" />

<xs:element name="longitude" type="xs:double" />

<xs:element name="time" type="xs:dateTime" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Expedition">

<xs:sequence>

<xs:element name="EXPED_CD" type="xs:string" />

<xs:element name="EXPED_YEAR" type="xs:int" />

<xs:element name="EXPED_VEHICLE_NAME" type="xs:string" />

<xs:element name="DATUM" type="xs:string" />

<xs:element name="NAVIGATION" maxOccurs="unbounded" minOccurs="0" type="Navigation">

</xs:element>

<xs:element name="LINES" minOccurs="0" maxOccurs="unbounded" type="Lines">

</xs:element>

<xs:element name="Global_x0020_Bounding_x0020_Box" minOccurs="0" maxOccurs="1" type="Global_x0020_BBOX">

</xs:element>

<xs:element minOccurs="0" maxOccurs="unbounded" name="Granular_x0020_Bounding_x0020_Boxes" type="Local_x0020_BBOX">

</xs:element>

<xs:element name="PAD_x0020_holdings" minOccurs="0" maxOccurs="unbounded" type="PAD">

</xs:element>

<xs:element name="Stations" type="Station" minOccurs="0" maxOccurs="unbounded" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Station">

<xs:sequence>

<xs:element name="latitude" type="xs:double" />

<xs:element name="longitude" type="xs:double" />

<xs:element name="station_x0020_number" type="xs:string" />

<xs:element name="time" type="xs:dateTime" />

<xs:element name="station_x0020_type" type="xs:string" />

<xs:element name="water_x0020_depth" type="xs:double" />

<xs:element name="comments" type="xs:string" />

<xs:element name="photos" type="Photo" minOccurs="0" maxOccurs="unbounded" />

<xs:element name="grain_x0020_size" type="GrainSize" minOccurs="0" maxOccurs="unbounded" />

<xs:element name="radio_x0020_carbon" type="RadioCarbon" minOccurs="0" maxOccurs="unbounded" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="Photo">

<xs:sequence>

<xs:element name="photo_x0020_number" type="xs:int" />

<xs:element name="url" type="xs:string" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="GrainSize">

<xs:sequence>

<xs:element name="SUBSAMPLE_TOP" type="xs:string" />

<xs:element name="SUBSAMPLE_BOTTOM" type="xs:string" />

<xs:element name="GRAVEL" type="xs:string" />

<xs:element name="SAND" type="xs:string" />

<xs:element name="SILT" type="xs:string" />

<xs:element name="CLAY" type="xs:string" />

<xs:element name="MUD" type="xs:string" />

<xs:element name="MEAN" type="xs:double" />

<xs:element name="Standard_x0020_Deviation" type="xs:double" />

<xs:element name="KURTOSIS" type="xs:double" />

<xs:element name="SKEWNESS" type="xs:double" />

</xs:sequence>

</xs:complexType>

<xs:complexType name="RadioCarbon">

<xs:sequence>

<xs:element name="SAMPLE_TYPE" type="xs:string" nillable="true" />

<xs:element name="SAMPLE_COMMENTS" type="xs:string" nillable="true" />

<xs:element name="SUBSAMPLE_NUM" type="xs:int" nillable="true" />

<xs:element name="SUBSAMPLE_TOP" type="xs:float" nillable="true" />

<xs:element name="SUBSAMPLE_BOTTOM" type="xs:float" nillable="true" />

<xs:element name="RADIOCARBON_DATING_METHOD" type="xs:string" nillable="true" />

<xs:element name="NOT_DATED_BECAUSE" type="xs:string" nillable="true" />

<xs:element name="SUBSAMPLE_MATERIAL_TYPE" type="xs:string" nillable="true" />

<xs:element name="SUBSAMPLE_MATERIAL_TYPE_LEGACY" type="xs:string" nillable="true" />

<xs:element name="SUBSAMPLE_SPECIES" type="xs:string" nillable="true" />

<xs:element name="SUBSAMPLE_WEIGHT" type="xs:double" nillable="true" />

<xs:element name="PRIMARY_TEST_FACILITY" type="xs:string" nillable="true" />

<xs:element name="SECONDARY_TEST_FACILITY" type="xs:string" nillable="true" />

<xs:element name="CARBON_ISOTOPE_RATIO" type="xs:string" nillable="true" />

<xs:element name="MEASURED_RADIOCARBON_AGE" type="xs:float" nillable="true" />

<xs:element name="MEASURED_RADIOCARBON_S_DEV" type="xs:float" nillable="true" />

<xs:element name="CONVENTIONAL_RADIOCARBON_AGE" type="xs:float" nillable="true" />

<xs:element name="CONVENTIONAL_RADIOCARBON_S_DEV" type="xs:float" nillable="true" />

<xs:element name="RESERVOIR_CORRECTED_AGE" type="xs:float" nillable="true" />

<xs:element name="RESERVOIR_CORRECTED_S_DEV" type="xs:float" nillable="true" />

<xs:element name="SUBMITTING_SCIENTIST" type="xs:string" nillable="true" />

<xs:element name="ANALYSIS_RESULTS_RETURNED_DATE" type="xs:dateTime" nillable="true" />

<xs:element name="STRATIGRAPHIC_CONTEXT" type="xs:string" nillable="true" />

<xs:element name="SCIENTIFIC_COMMENTARY" type="xs:string" nillable="true" />

<xs:element name="GENERAL_COMMENTS_LEGACY" type="xs:string" nillable="true" />

</xs:sequence>

</xs:complexType>

</xs:schema>