tutorial 3-2 importing historical chemistry - esdat · (version 5 software updates available from )...

(version 5 software updates available from www.esdat.com.au)

Tutorial 3-2Importing Historical

Chemistry

For use with ESDAT version 5

Tutorial 3-2 Importing Historical Chemistry www.esdat.com.au

Copyright @EarthScience Information Systems P a g e | 1

Tutorial 3-2 Importing Historical Chemistry

Table of Contents

Summary ....................................................................................................................................2

Pre-requisites .............................................................................................................................2

Key Learning...............................................................................................................................2

Introduction ...............................................................................................................................3

Exercises.....................................................................................................................................4

Exercise One: Importing Location Information......................................................................4

Exercise Two: Importing Historical Chemistry – CrossTab Format ........................................8

Exercise Three: Importing Historical Chemistry – Chem Column Format ...........................14

Exercise Four: Editing and Deleting Historical Chemistry Data ...........................................16



SummaryThis tutorial:

details the method for importing historical chemistry data which have been stored inspread sheets (i.e. data not presented in the ESDAT Electronic Lab Data Format,ELDF)

is performed using the Sample Hydrogeology and Contam dataset in Access mode(and therefore can be completed using the demonstration version of ESdat)

applies to both Access and SQL Server versions of ESdat

requires approximately 90 minutes to complete

Pre-requisitesIt is assumed the user has previously completed:

Key LearningFollowing completion of this tutorial, the user will have developed skills in the following:

preparing ESdat to receive imported data

formatting historical chemistry data in preparation for importing into ESdat

importing historic chemistry data into ESdat

editing and deleting historical data from within ESdat

Tutorial 2 – Introducing ESdat

Tutorials 3-1 and 3-3 should be completed in conjunction with this tutorial.

For a full list of the Tutorials available for use with ESdat, please see the Tutorial List.

http://www.esdat.net/esdathelp/index.html?tutorials.htm



IntroductionMany ESdat users will have (or be presented with) historical analytical data that needs to beinterrogated efficiently and effectively. Historical data can be imported into ESdat for suchpurposes.

ESdat features a number of pre-formatted Excel sheets (Import Templates) that assist witheasy and accurate importing of historical data into ESdat.

For the purposes of this tutorial, it is assumed the data have been transferred intoelectronic form, in Excel spreadsheets (thereby allowing copy/cut and paste).

This tutorial will demonstrate the process of importing this data into ESdat, and tosubsequently manipulate the data within ESdat to produce an output table.

This tutorial applies the ESdat standard import tool(s) to bring data in ESdat.

Demonstration data used for this tutorial can be downloaded from here (theTutorial_3_2_import_Historical_Chemistry.xlsx spreadsheet).

www.esdat.net/downloads/Tutorials/Tutorial_3_2_import_Historical_Chemistry.xlsx

This tutorial features four exercises, which are summarised as follows:

Exercise 1

Creation of a new database and importation of some basic data.

Exercise 2

Importing historical chemistry using CrossTab Format.

Exercise 3

Importing historical chemistry using Chem Column Format.

Exercise 4

Edit or delete a record from chemistry table

http://www.esdat.net/downloads/Tutorials/Tutorial_3_2_import_Historical_Chemistry.xlsx



Exercises

Exercise One: Importing Location and Well InformationBefore importing laboratory analytical chemistry data into ESdat, information regardingsample locations must be imported. In this exercise, groundwater well constructioninformation will also be imported, as the laboratory analytical data is analysis ofgroundwater samples.

Following import of the location and well information, the chemistry data is imported,which is reconciled with the sample location information.

Open the Tutorial_3_2_import_Historical_Chemistry.xlsx spread sheet in Excel andsave a copy to your hard drive.

Review the data in the three different worksheets:

Locations Wells Historical Water Chemistry Import



Importing Locations Select Import from the View Type tool bar (the top tool bar, as shown in the figure

below)

Click on the Location button to automatically create and open a Location data ImportTemplate (will open in Excel)

Copy and paste the Location data from theTutorial_3_2_import_Historical_Chemistry.xlsx spreadsheet into the ImportTemplate that is generated, arranging data in the correct columns.

The populated Location Import Template should look similar to the figure presented below.

Go to the Add Ins tab at the top of the page and click on this to open the CustomToolbar (as shown in the figure below)



Click the ‘Upload’ button to upload data into ESdat.

The Upload Data Import Options window will open.

Click the Import button

Close the Excel spreadsheet

Importing Wells Return to the Import view by selecting Import from the View Type tool bar

Create a Wells data Import Template by clicking the Well Construction button (asshown in the figure below)

Transfer the wells information from the Well work sheet within the ImportChemistry.xls spread sheet into the Wells data Import Template

The Wells data Import template will look like the figure below.



Go to the Add Ins tab at the top of the page and click on this to open the CustomToolbar

Click the Upload button to upload data into ESdat

Close the Excel template (without saving)

Now ESdat is ready to import historical chemistry data.

In the next exercise historical chemistry data will be imported into ESdat.



Exercise Two: Importing Historical Chemistry – CrossTab Format

Return to Tutorial_3_2_import_Historical_Chemistry.xlsx spread sheet, that is openwithin Excel

Select the Historical Water Chem Import tab and review the data

Return to ESdat and select the Import view

From the Blank Import Templates area, click Historical Chem

Review the Import Template, noting there are two tabs in the Import template:

1. Chem CrossTab Format

2. Chem Column Format

The CrossTab Format presents data with individual analytes organised in columns, allowingmultiple analytes per row.

The Chem Column Format presents individual analytes in rows (i.e. one analytical result forsay arsenic, occupies a row).

The Chem Column Format is a more comprehensive format than the CrossTab format, butthe CrossTab format is more suitable for many historical chemistry spreadsheets.

Click on CrossTab Format tab and review the template (as shown in the figurebelow).



It is vital that the template has sufficient column heading available to accommodate the fullsuite of analytical data. For importing water chemistry data from Tutorial 3_2 ImportChemistry.xls, the following column headings are required:

o LocCode

o Sample_Date_Time

o Matrix_Type

o WellCode; and

o Multiple analyte columns (Analyte1, Analyte2 etc)….

Close the Import Template

Changes to the format of the Import Template can be made from within ESdat, as follows:

Return to ESdat

Go to Setup - Chemistry - Sample Fields Shown (as shown below)

The Default Fields for Chemistry Import dialogue box will open, as shown below.



Select Historical Chemistry option in the dialogue box

Select name of Fields (column headings) required to appear in the Import Template

From the Blank Import Templates area, click Historical Chem to re-open theHistorical Chemistry Import Template

The data within the Tutorial_3_2_import_Historical_Chemistry.xlsx spread sheet requiresreformatting to fit in the Import Template:

Select the Historical Water Chem Import worksheet (tab) inTutorial_3_2_import_Historical_Chemistry.xlsx

Select and copy all cells with entries

Open a new blank worksheet in Excel

Transpose Paste the data (right mouse click on the destination cell and select pastespecial and select Transpose, as shown in the figure)



Copy and paste those data to the Import Template so that they look like following:

Recall from earlier tutorials that the SampleCode field will be calculated based on a formulawritten in cell A6.

Copy the formula in A6 to whole column (extending to all rows with data)

Click Add-Ins

Click the Get Chem Codes button (as shown in the figure below)

The Associate Chem Codes dialogue box will open.

Note that the majority of ChemCodes are automatically assigned (in the column labelledAssign) against the ‘Original Chemical Name’, which is from the Tutorial 3_2 ImportChemistry.xls data, as shown in the figure below.

Note that some ChemCodes are missing (Carbonate, Bicarbonate and Sulfate)



Click on Carbonate Alkalinity as CaCO3 in the Original Chemical Name column

Find a matching chemical name (ChemName) from the Chemistry Lookup Table (thepanel at the right, as indicated by the heading)

It should be noted that names can vary slightly. Carbonate Alkalinity as CaCO3 appears asAlkalinity (Carbonate as CaCO3) in the Chemistry Lookup Table.

Click Send <= button to assign the ChemCode

Do the same for all missing ChemCodes

Click Update to send all ChemCode to Import Table

The ChemCodes are imported to Excel Import Table

Click the Upload button to import data into ESdat.

Check that all entries have been successfully imported (cells turn green if imported)

The data imported into ESdat can now be reviewed.

Return to ESdat and select the Data Views

Click on Water Chemistry and view LChem1_Chemistry

The data should appear consistent with the figure below:



Exercise Three: Importing Historical Chemistry – Chem Column FormatThe Chem Column format is used when receiving data from another relational database.

In this exercise data will be imported from the Chem Column Format sheet in the ImportTemplate.

Import Location & Wells data from Tutorial 3_2 Import Chemistry.xls to ESdat

Return to ESdat and the Import view

Click Historical Chem in Chemistry Import Area to open a new Import template (inExcel)

Click on Chem Column Format tab at the bottom of the spreadsheet

Copy the first few Location Codes (e.g. MW01A – MW03) from the Historical WaterChem Import work sheet within Tutorial 3_2 Import Chemistry.xls to the ImportTemplate

Transfer the ‘<’ sign present in front of any non-detects (results less than (<) thelaboratory limit of reporting, or LOR), to the Prefix column. The numerical value (e.g.the 100 from <100) should remain in the result column

The Import Template should look like the figure below:

Click the Upload button to upload data into ESdat Two additional tabs will be created in the Import Template spreadsheet

Chemistry_Samples and Chemistry_Results Import the Chemistry_Samples data Change to the Chemistry_Results sheet. Click Get Chem Codes and follow the same steps to populate ChemCode in the

Import Template as per in Exercise Two. Fill the mandatory Total_or_Filtered column with F for filtered and the rest with T



Fill the mandatory Result_Type column with Reg as default (unless a specific resulttype is identified)

Fill the mandatory Method_Name with NA as default (unless a specific method nameis identified)

The completed Chemistry Results tab within the Import Template should be appearconsistent with the figure below:

Click the Upload button to upload data into ESdat.



Exercise Four: Editing and Deleting Historical Chemistry DataWhen importing Historical Chemistry it is recommended that the Import Templatesgenerated are retained for future reference. Then, if edits are needed it is a simple matterto open the Import Template, edit as required and re-import using “Add New” or“Overwrite” as appropriate. Data tables in ESdat (such as Chemistry_Samples) have aoptional Data Source field, which is optimally populated with the Import Templatespreadsheet name for future reference.

The following describes how to edit historical chemistry data directly in ESdat if the above isnot available.

To view the imported water chemistry data, click on Water Chemistry data view.

All imported data will be displayed from the Data Panel.

Data cannot be edited from the Water Chemistry data view as calculations are applied tothis Output View (such as unit conversions).

Attempting to edit data from the Water Chemistry data view will open a dialogue boxindicating ‘The data cannot be edited here….’ (or similar).

Editing of raw data contained within ESdat can be undertaken by accessing the Data Tables.

A white button labelled Data Tables is located at the top right of the ESdat page, as shownin the figure below:

Hovering the cursor over this table prompts a drop down menu. The contents of the menucomprise all data tables that are used in combination to produce the various data views,(such as Locations, Boreholes/Wells, Geology etc.).



In order to edit chemistry data, select Chemistry_Results table from the Data Tables dropdown list.

However, under Data Tables, in the Chemistry_Results table, location codes are notdisplayed, making it difficult to identify the correct data to edit.

For this reason, editing of chemical results data should be completed via the Chem Trackingdata view.

Hover the cursor above the Chem Tracking button and a drop down list will appear Select ChemQA2_Chemistry_Samples_with_Raw_Results (as shown in the figure

below)

As calculations have not been applied to the data here, the raw data can be viewed andedited.

Assume an errors exists in the historical data that has been imported, specifically copperresults for BH03.

In order to correct the data:

Prepare a filter (via the Filter Panel) as follows (and shown on the figure overleaf):

OriginalChemName = 'Copper' And

LocCode In( 'MW03' )

click Apply Filter

Return to the Data Panel, where only the copper result for BH03 will be displayed.

Click on the Edit tool (which will turn red to indicate edit mode is on) Change the data for MW03 total copper to be 0.002 (from 0.003), selecting yes when

asked to edit the data



To edit a large number of records (and to save time), an alternative approach is to exportthe data to Excel, edit the data and reimport using the Update Existing or Overwrite Existingoptions rather than editing individually using Edit tool.



Deleting RecordsTo delete data, it is more efficient to filter for the data to be deleted first.

Once the desired data has been identified click on the Edit button to commenceEdit mode

Select the rows to be deleted, and right click and select Delete Row(s)

A prompt will check that the user wishes to continue with the deletion, as this action cannotbe undone.

For more information see the Importing, Editing and Deleting Tutorial.

tutorial 3-2 importing historical chemistry - esdat · (version 5 software updates available from )...

Documents