![]() ![]() |
Natural Resources Canada | Earth Science Sector | Geological Survey of Canada
![]() ![]() |
GlobalDB System Release 3.7 |
GlobalDB System Site Map | Presentation | Requirements | Software and Scripts | Aliases | Using GShell | Using GQuery | Using Documentor
USING GQUERY
This page:
GQuery is a standalone application designed especially for selecting and extracting data from databases built with the GlobalDB schema. Its purpose is to help users filter a database and either prepare spreadsheets of data from selected records or generate "web sites" where one can browse through the entire data content of the selected records. GQuery was originally offered in two versions: GQueryADO and GQueryClient. GQueryADO is designed primarly to work with desktop databases in MS-Access format via an ADO alias. GQueryClient, which was developed to directly access the databases on a central server, is not included with this Open File.
During the development of GQuery, it was found that performance was poor if three derived tables (views) needed by the application were generated dynamically by the program. Two of these are the summary listings that appearing on the main screen of GQuery, and a third table is a record count listing. The solution was to generate the three tables each time a database is updated in one of two ways: (1) using a utility in GShell called Build Summary Tables under Tools menu of the entry form; or (2) by running the standalone program GQ_ADO_XtraTables included in the GlobalDB System package. Its usage is described below.
We suggest that you start GQuery and open a sample database, in order to explore its several screens and controls while you read this page, switching between your browser and GQuery itself. GQuery does not write or change anything in your databases and it is safe to experiment with.
To use GQuery, the user must have unzipped the Open File and copied over GlobalDBSystem321 and Data321 as described in Software and Scripts, and must have configured at least one database alias as described in Defining database aliases. When the program is started, the user is asked to enter his/her initials and click the OK button. If the user does not enter any initials or clicks the Cancel button, the program displays a message and stops. This not meant to be an annoyance; the initials are used when users create reusable custom spreadsheet templates as described below. Next, an information window appears in the centre of the screen on a single screen computer configuration or elsewhere on a multi-screen system. Once its OK key is clicked, the program's main form is displayed with an Exit button in the upper right hand corner and a row of six buttons immediately underneath. Each button corresponds to a separate function in GQuery: Select Database, Select Records, Prepare Template, Generate Output, Generate HTML, and Reset All. All but Select Database and Exit will be greyed out (disabled) until a database is selected.
SELECTING A DATABASE AND AN ENTITY
Once you have opened GQuery (above), a form will open with a selection of six buttons along the top, all but the leftmost Select Database button and the Exit button in the upper right hand corner will be greyed out. Clicking on Select Database will immediately bring up a list of databases database selection form for which you have created aliases. Click on the database you wish to use, and you will be prompted to choose between Deposits or Deposit Groups. If there are no Deposit Groups in the database, only Deposits will be listed for selection. GQuery will then display the summary data table on the grid in its main form. The bottom status bar displays the database name and its authors/compilers, which entity is being shown, as well as the total number of records in the set, the number of records in the current selection and the current position within the selected set your cursor is on (main form with sample data). The Select Database button will now be greyed out and the other five buttons along the top of the form will be enabled.
The right hand Reset All button will close the database and disable all buttons except for Select Database and Exit. Reset All should be used when switching to the other entity or to a different database.
The Select Records button brings up the record selection form, which is used to filter the database. One or several conditions can be defined by the user. As each condition is built, the conditional expression appears in the grid, together with the number of entity records selected (see remark below). By selecting a condition (clicking on the grid or selecting a condition number in the rightmost combo box of the top panel) and clicking the Main Screen button, the main form is redisplayed, and only the selected records will show on its grid (for example, selected record set where Country = Albania). Refinements to the selection can be made by going back to the record selection form and selecting another or defining a new condition.
From left to right, the topmost panel of the record selection form GDBS - Select Records shows two labeled combo boxes and two buttons. The Tables And Colums combo box lets the user choose to work with actual table and column names or with English captions. The Current Selection combo box displays the condition number, such as 'Q01' or 'Select all records", that indicates which of the conditions is currently selected and can be used to choose another condition. This current selection can also be made of changed by selecting a condition on the condition grid with the mouse. The Main Screen button brings back GQuery's main form with the currently selected record set (using the selected condition as a filter). The Cancel button brings back GQuery's main form with all records.
The panel beneath the top panel shows six buttons. The Build Condition button starts the construction of a new condition in the currently selected row of the condition grid. A warning is issued if that row already contains a condition. Combine Conditions starts the construction of a new condition based on one or several conditions already defined. Clear Condition clears the condition in the currently selected row of the condition grid. Clear All Conditions clears the entire condition grid. The two small buttons at the right hand side will display or hide a small window showing information about the currently selected condition: its key fields, the tables that were joined in building that condition, the table captions and the "where" part of the SQL statement of that condition (the "Conditional Expression" in the condition grid). The use of this information becomes more obvious when one builds complex conditions.
The condition grid occupies the main part of the record selection form. It displays the condition identifier (used in the Current Selection combo box), a conditional expression and the number of entity records selected. Remark: this number (number of selected records from the main entity) may differ from the number of hits indicated in a value list showing values from a detail table (number of records with a given value), simply because the entity record is at the "one" end of a "one to many" relationship.
In building each simple condition (as opposed to a combination of conditions), the user is prompted to select a data table and one or several data field(s). To help with this, GQuery will display the database schema in the form of a hierarchy with a tool called Database Tree View (DBTree). The plus sign to the left of the top entity can be used to expand the tree. The top window of DBTree shows all the tables linked to the main entity table (at the top), labeled with either their English caption or their real database table name (as selected above), and the number of records in each of these tables. The detail and the sub-detail tables correspond to the tabbed pages of GShell. Once a table is selected in the top window, the fields of that table available for filtering are displayed in the grid beneath the tree. If the window is too short for these fields to show up, drag the base of the data frame down with the cursor. One or several fields can be selected (holding down the [CTRL] key while clicking). Clicking the Next >> button will close DBTree and trigger one or several prompts for testing the values of the selected field(s).
Then, for each selected field and depending on the type of data of that field, the user is prompted to select one or several value(s) from a list (such as commodities or country), or enter a string to search in a long text field, or do a numeric test (with standard logical operators) or select an age range. The resulting condition appears in the grid.
The first three tools are self-explanatory but the age range selection form deserves an explanation. In the top left panel, the user can test the age of a specific feature or any one of them. The left center panel defines inclusive or exclusive age range. In the bottom left panel are the two edit boxes containing the numeric values of age (positive number, in millions of years) that define the test age range. These values will be used in making the selection. On the right hand side of the form, a grid displays the age ranges corresponding to each geological age, as recorded in the database. This is provided as an aid in entering the age range values. Once a row of that grid is selected, clicking the Start Age button will place the oldest age value in the Start Age edit box in the lower left panel; the End Age button will place the youngest age value in the End Age edit box in the lower left panel. Of course, the Start Age and End Age can be taken from different rows and the user is free to adjust the resulting numeric values. Clicking the Next >> button completes the operation.
Complex conditions are built by combining simple conditions, using the Combine Conditions button. For example, a condition could be "Country = 'Canada'" and another one be "Commodity = 'Pt'". The combine conditions form allows the user to specify that all conditions must be satisfied (logical AND), that at least one condition must be satisfied (logical OR), that a single condition must not be satisfied (logical NOT) or that a pair of conditions be compared using any one of the logical operators. Simple conditions to be combined are selected using checkboxes in the left part of the form; checking the radio buttons selects the type of comparison to be made. When more than two conditions are checked, only the first two radio buttons (AND and OR) are enabled. Join depth is explained below. The Next >> button completes the operation. The resulting condition now appears in the condition grid (record selection form with sample conditions) and can in turn be used in further combinations.
Behind each condition is a table in the temporary database, made of the key values of the tables involved in building the condition. For example, selecting by Country generates a table of records whose fields are DEPNO and RANK, DEPNO is from dpgeol and RANK is from dpctry. The small window referred to earlier displays that information. If two simple conditions have been defined, one with Country = Canada and the other with Commodity = Pt, and one wishes to combine these two conditions, as above, the query is made by comparing keys from those temporary tables and not values from the main database. The first field (DEPNO) of both simple conditions comes from the same table dpgeol but the second field (RANK) comes from different tables; here, join depth = 1 (a single key is involved in the overlap). Let us assume that the main entity selected is Deposit Groups and that the same two simple conditions have been created, one on Country of a Deposit and the other one on Commodity of a Deposit. The two corresponding temporary tables have three keys (DGNO, DEPNO and RANK), the first two of them overlapping. If one tries to combine the two conditions, join depth = 2 and this will make the join depth form appear. If one selects Deposit group, the resulting condition returns all deposit groups having at least one linked deposit with Country = Canada and at least one linked deposit with Commodity = Pt, which may or may not be in Canada. If Deposit is selected, the resulting conditions returns all deposit groups having at least one linked deposit with both Country = Canada and Commodity = Pt, a much more restrictive condition. Again, this becomes easier with practice.
From the main form, the Prepare Template button brings up the prepare template form which is used to define the columns of a custom output matrix, or spreadsheet. This GQuery function may be the one that takes the most practice to use, but the nice thing is that once you have prepared a template that suits your needs you can save it and reload it for another session (as long as you are consistent about entering your initials when starting the program).
The opening window of the Prepare Template form has a top panel containing a text combo box from which the user can select either table and column names or English captions, plus three buttons. The Load and Save buttons are used to load a previously saved template (a dialog box with template descriptions will appear) and save a new template (with a short description), respectively. The Main Screen button allows the user to return to the main screen. The main area in this form contains a grid with rows that display cryptic details about how each column of the spreadsheet will be defined. The row on the form above the grid and below the topmost panel contains buttons that allow the user to Build Column, up and down arrows to move a column up or down, a Clear Column button to clear a single column highlighted with the cursor, and a Clear All Columns button to clear the grid. To build each column of output, a build output form is used (a simple example and a complex sample which illustrates most features).
In the upper left area of the form, the user must enter a column heading for the output, and may enter a prefix and a suffix strings that can be used to respectively precede and/or follow each set of output fields (not used in these examples). If the source data table is a detail table, the output matrix will have as many columns of the current type as the maximum value of RANK in the database for that table (hence the proper use of the Renumber button in GShell); alternatively, the user can decide to concatenate the set of values of the selected fields from the database into a single output column. For this, a concatenation separator can be entered in the upper right area of the form.
In the middle area, the Select Table and Fields button brings up the Database Tree View (DBTree) tool with which to select a table and then one or several fields; the name of the selected data table will be displayed on the left and the names of the fields will be in the fields box in the center of the form. The Add and Remove buttons will respectively add or remove fields from the list. It is not possible to mix fields from several tables in a given output column. If the user selects a different table in DBTree, all the values in the form will be reset. To surround a field with parentheses (or some other pair of characters), a field is highlighted in the fields box and the enclosing characters can be selected in the enclosers combo box. The Up and Down arrows will move the fields and their enclosers up or down in the list. To the right of the fields box are the separators, only useful when several fields are selected. Underscore characters can be entered instead of spaces as they are move visible on the screen. There will be one less separator than there are output fields and each separator can be as long as one wishes (it is a character string). By default, multiple fields will be separated by the pipe character ("|"). The long edit box at the bottom of the form is an illustration the resulting output. The template can be saved for later use.
In this example of the prepare template form, the first output column will have all names concatenated in a single column; the second output column will have all commodities concatenated in a single column; the third output column will have a group of fields from the geological age table arranged with enclosers, repeated as many times (in several columns) as the maximum number of occurences of this type of record within the selected record set. This is easier to do than to describe!
The Generate Output button of the main form can be used once the record set has been defined by using the various record selection tools as described earlier; keeping all records is of course a valid option. A small form offers the user to generate the output file from the custom output template built or loaded as described above or directly from the summary table (no choice of output fields). The usual MS-Windows Save File dialog box will show up. The resulting output file, a simple tab delimited ASCII text file, can be opened by MS-Excel or imported in MS-Access, or in any program that can read such files. It is then easy to use the generated record sets with report generators or with GIS software.
The Generate HTML button will trigger the generation of a complete web site by first displaying the generate HTML output form. Two options, shown as tab pages, are offered: generate a complete HTML page set or generate an HTML index page only. The latter option could be used if a complete set of HTML pages has already been generated somewhere and the user only wants to create an index page, or table of contents to the current record selection.
The left tab page is used in generating a complete HTML page set:
In the first edit box, a title (mandatory, maximum 40 characters) must be entered. This title will appear in the top bar of the browser. Immediately underneath, the Root Directory box will capture the full path (mandatory) of the directory to which you wish the new web site to be anchored; clicking on the right end of the box opens a dialog box which will enable you to browse to this root directory. You must open that target directory in order to select it. For your convenience, the GlobalDB System installation package creates a directory called htdocs that may be used for that purpose. Still going down, the next edit box captures the name of the new directory (mandatory) that you designate for your web pages; you cannot specify the name of an existing directory. The first checkbox at the bottom left indicates whether or not web pages should be generated for linked entities. The second checkbox will make table borders visible on the output HTML pages. The third checkbox will trigger the inclusion of the Resource Disclaimer that NRCan lawyers have insisted on including in any reports that contain resource figures. The large box on the lower right may contain free text which will be displayed on the table of contents page. The text boxes above that allow users to create headings.
An HTML page is built for each record of the selected record set, displaying the entire database content for that record, including formatted references, resource and production data, and at the user's option, supplementary pages for all entities linked to that record (Deposits can be linked to Deposit Groups, Mines can be linked to Deposits). The index.html page serves as a table of contents for the web site and lists selected fields from the summary table, with hot links to each individual web page. Finally, another file called index.txt is a plain tab delimited ASCII file, containing the entity key value, the latitude and longitude, the country, the name(s) and the clan of the entities, the full URL of each corresponding web page and finally, the database name, its authors/compilers and release date. This file can be used directly with GIS software, such as ArcView or MapInfo, where the data points can be displayed and hotlinked to their corresponding web page.
The right tab page is used to generate an HTML index page only:
This is a quick method to create a web site for a selection set when a full HTML page set has already been generated. For example, one could generate a full set for deposits located in Canada, and then make a series of more restrictive selections, all within Canada. For these subsequent runs, it is only necessary to generate the HTML index page (with its accompanying style sheet and ASCII file), which is very quick.
As for the left tab page, the first edit box contains the title (mandatory, maximum 40 characters). This title will appear in the top bar of the browser. Immediately underneath, the URL or Full Path of HTML Page Set captures the full path (mandatory) of the directory that contains the HTML pages; clicking on the right end of the box opens a dialog box which will enable you to browse to this directory. You must open that directory in order to select it. Alternatively, if the full set of pages is on an HTML server, its full URL (http://.....) must be entered in the edit box. Still going down, the next edit box captures the name of the existing directory where your output will be saved; again, you must navigate and open the directory in order to select it. The last edit box contains the name of the index, without extension. For example, entering "CanadaSilver" will generate files CanadaSilver.html and CanadaSilver.txt. The large box on the right may contain free text which will be displayed on the table of contents page. Any number of table of contents pages can be stored in the same directory as their names are provided by the user.
The Generate button triggers the process, Clear All does what it says and Main Screen returns to the main screen.
The time required to generate the web site is a function of the number of records in the selected record set, the quantity and complexity of data available for each of these records, the number and complexity of linked entity records, the speed of the computer and of the hard disk. For example, it took around 8 minutes to generate a complete set of HTML pages for 1307 deposits and 68 associated mines in an MS-Access 95 database on a computer running at 3.0 GHz, with 1024 Mb of RAM running GQueryADO, a process that involved tens of thousands of queries to be run in the background. Using the same computer, it took 19 seconds for GQueryClient to generate an index page for 1105 deposit records in an InterBase database located on a Linux server.
The generated HTML pages are named as XXNNNNNN.html where XX is the entity code (such as dp for deposits, dg for deposit groups or mn for mines), and NNNNNN is the primary key value of DEPNO, DGNO or MINENO. Examples: dp23.html, mn138.html, dg4233.html. If those names are changed (it has been done...), the table of contents page and the ASCII file will no longer work.
REGENERATING THE THREE EXTRA TABLES
The GQ_ADO_XtraTables program displays a small form with only two buttons: Add Tables and Exit. Clicking the Add Tables button will open a standard Windows Open File dialog box from which to select the UDL file pointing to the target database (the ADO alias). Depending on your current directory, you might have to navigate to the WMGDP directory to find the UDL files. Once the file is selected and the Open button is clicked, the extra tables are generated. It is that simple!
GlobalDB System Site Map | Presentation | Requirements | Software and Scripts | Aliases | Using GShell | Using GQuery | Using Documentor
Important notices and disclaimers | 2014-09-26 |