{"id":476,"date":"2018-01-10T09:04:31","date_gmt":"2018-01-09T22:04:31","guid":{"rendered":"http:\/\/cosima.org.au\/?p=476"},"modified":"2018-01-10T09:06:23","modified_gmt":"2018-01-09T22:06:23","slug":"technical-working-group-meeting-december-2017","status":"publish","type":"post","link":"https:\/\/cosima.org.au\/index.php\/2018\/01\/10\/technical-working-group-meeting-december-2017\/","title":{"rendered":"Technical Working Group Meeting, December 2017"},"content":{"rendered":"<h2>Minutes<\/h2>\n<p>Date: 14th December\u00a02017<br \/>\nAttendees:<\/p>\n<ul>\n<li>Marshall Ward (Chair) (NCI)<\/li>\n<li>Aidan Heerdegen, Andrew Kiss (ARCCSS\/ARCClEx ANU)<\/li>\n<li>Fanghua Wu (National Climate Center, China Meteorological Administration, Visitor ANU)<\/li>\n<li>James Munroe (Memorial University of Newfoundland, Visitor ANU)<\/li>\n<li>Russ Fiedler (CSIRO Hobart)<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<div>\n<h3>Output file metadata indexing<\/h3>\n<\/div>\n<ul>\n<li>MAS database at NCI. POSIX info. ncdump blob. nodal style. Can put index on netcdf files and search by them.<\/li>\n<li>James did similar\u00a0thing for COSIMA cookbook running in user space. James has had no action on MAS DB so far.<\/li>\n<li>Currently NCI is POSIX crawling hh5. James: need to switch on netcdf for certain directories.<\/li>\n<li>James: Ben pitched MAS as a great innovation. Maybe Andy needs to formally ask Ben for this?<\/li>\n<li>What can MAS deliver that existing DB cannot? James: stopped developing DB because of MAS.\u00a0SQLite was 40-50K vars\/files. Spinup of 1 deg model have 1M+ variables\/metadata. SQLite already 1-2GB. Only scales to 1M rows. Can\u2019t deploy postgres without admin access. Could host one, but should live on NCI resource. Makes sense to MAS.<\/li>\n<li>James: just a user role in DB and switching on netcdf indexing \u2014 should be fine. Marshall will follow up with NCI MAS bods to make sure this happens soon.<\/li>\n<li>Andrew\u00a0was concerned that\u00a0this will have on-going support. Use of MAS in other high profile projects (geoscience australia for example) means this is a critical piece\u00a0of infrastructure.<\/li>\n<li>James: can we just access their schema? Want to open source, not sure how. James: NCI has confluence, do they have bitbucket license? Marshall: no.<\/li>\n<li>Need mom.out copied to hh5 also to be able index important info with f90nml. Russ: logfile has just namelist info.<\/li>\n<li>Andrew: any equivalent for CICE and MATM? Maybe not? Andrew: need to make CICE and MATM print out namelists.<\/li>\n<li>Marshall: get Ben\/Andy to endorse official use of MAS by CoE.<\/li>\n<li>Marshall: do we need to add attributes to files to accomodate this? James: does the executable spit out a version string? No. Marshall his build script puts a version string. Russ: version part of FMS? Russ: Marshall took version out when moved to oom version of FMS. See Issue #31 on GitHub (can&#8217;t find issue Russ refers to). Russ: already have a version.c<\/li>\n<li>James: CSIRO wants some of the automated processing for decadal prediction. Can we apply to both?<\/li>\n<li>James: make a MOM module? Marshall: make codebase a submodule of payu<\/li>\n<li>Aidan talks about reproducible builds using <a href=\"https:\/\/spack.readthedocs.io\/en\/latest\/\">spack<\/a>. Reproducible builds require a package manager so that it can find and know about all the components of the build.<\/li>\n<li>Marshall will put hashes in executable in MOM.<\/li>\n<\/ul>\n<div>\n<h3>COSIMA Models<\/h3>\n<\/div>\n<ul>\n<li>Andrew tenth degree runs: salinity crashes in the arctic. Recent crash: MPI Abort error code 111. Resubmit? Use broadwell<\/li>\n<li>Andrew: has added regional runoff caps. Tighter caps in arctic rivers.<\/li>\n<li>Paul Spence issue with regional outputs, had incorrect bounds. Might affect in future. High temporal resolution in small regions.<\/li>\n<li>Russ: weird happened a while ago. Mixing velocity and tracer grids in a single file? At least for regional output. Mixing u and t grids? \u2014 Aidan look into it.<\/li>\n<li>Migrating to FMS submodule. When Marshall updated to oom one of the open boundary cases broke. Took 2-3 weeks of scientific coding to fix.<\/li>\n<li>Russ looking at CM2.5 and new FMS. AM4 has been released.<\/li>\n<li>Marshall: will make FMS a submodule. \u00a0This works for\u00a0decadal prediction people who will need this work done in any case.<\/li>\n<li>COSIMA will do JRA55 IAF tenth run.<\/li>\n<\/ul>\n<div><\/div>\n<div>Wednesday meetings next year. 11.30am.<\/div>\n<div><\/div>\n<div>\n<h3>Actions<\/h3>\n<p>New:<\/p>\n<ul>\n<li>CICE and MATM need to output namelists for metadata crawling (no-one assigned)<\/li>\n<li>Get Ben\/Andy to endorse provision of MAS to CoE (no-one assigned)<\/li>\n<li>Make MOM (and other models) emit GitHub version hash (Marshall)<\/li>\n<li>Collation errors on regional outputs (Aidan)<\/li>\n<li>Move FMS to submodule of MOM5 github repo (Marshall). Liase with Nic on implementation?<\/li>\n<li>Follow up with NCI MAS people (Marshall)<\/li>\n<\/ul>\n<p>Existing:<\/p>\n<ul>\n<li>Send link to spinup diagnostics spreadsheet to Russ (Andrew Kiss)<\/li>\n<li>Nic add MPI barrier before ice halo updates timer to check if slow timing issues are just ice load imbalances that appear as longer times due to synchronisation.<\/li>\n<li>Test Andy&#8217;s 5 year config with different netcdf library versions to check MATM error is not a just a library issue (Aidan)<\/li>\n<li>Check current sea surface salinity restoring smoothing (Aidan)<\/li>\n<li>Russ to add all his ocean bathymetry code to OceansAus repo.<\/li>\n<li>Nic to help Peter get his MOM repo up to date with MOM5 master branch, and then merge changes<\/li>\n<li>Look into OpenDAP\/THREDDS for use with MOM on raijin (Aidan, Nic, Marshall)<\/li>\n<li>Nic to present MATM code re-write proposal to TWG for feedback before sign-off. Will then be presented to Andy Hogg for approval.<\/li>\n<li>Nic create a discussion document (on COSIMA?) to document current approaches and strategies for future<\/li>\n<li>Work up test cases to cover the nudging code (Justin, Mirko) and supply them to Nic.<\/li>\n<li>Add new test cases to Jenkins test suite (Nic).<\/li>\n<li>Start a new google doc about coupler issues and MATM (Marshall)<\/li>\n<li>Ask Dale Roberts about effects of OpenMP for Roger (Marshall)<\/li>\n<li>Make a proper plan for model release \u2014 discuss at COSIMA meeting. Ask students\/researchers what they need to get started with a model (Marshall and TWG)<\/li>\n<li>Blog post around issues with high core count jobs and mxm mtl (Nic)<\/li>\n<li>Create document outlining options for configuration sharing (?)<\/li>\n<\/ul>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Minutes Date: 14th December\u00a02017 Attendees: Marshall Ward (Chair) (NCI) Aidan Heerdegen, Andrew Kiss (ARCCSS\/ARCClEx ANU) Fanghua Wu (National Climate Center, China Meteorological Administration, Visitor ANU) James Munroe (Memorial University of Newfoundland, Visitor ANU) Russ Fiedler (CSIRO Hobart) &nbsp; Output file metadata indexing MAS database at NCI. POSIX info. ncdump blob. nodal style. Can put index&hellip;<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[4,3],"_links":{"self":[{"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/posts\/476"}],"collection":[{"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/comments?post=476"}],"version-history":[{"count":2,"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/posts\/476\/revisions"}],"predecessor-version":[{"id":478,"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/posts\/476\/revisions\/478"}],"wp:attachment":[{"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/media?parent=476"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/categories?post=476"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cosima.org.au\/index.php\/wp-json\/wp\/v2\/tags?post=476"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}