netcdf_tests.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# activate crawler env 1st!\n",
    "import netCDF4 as nc\n",
    "from netCDF4 import Dataset\n",
    "import glob\n",
    "import thredds_crawler\n",
    "from thredds_crawler.crawl import Crawl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Example of getting metadata with explicit reference to a file on TDS- this works\n",
    "#after the files were repaired, they work here\n",
    "fname='http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/FI14/9922v-a.nc'\n",
    "#fname='http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/MVCO_14/10103wh-a.nc'\n",
    "#fname='http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/MVCO_15/10571whV-a.nc'\n",
    "#fname='http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/SANDWICH2016/10631whVp-cal.nc'\n",
    "fnc=nc.Dataset(fname,mode='r+') # don't use 'w' will delete existing file!\n",
    "m_att=fnc.getncattr('MOORING')\n",
    "print(fname)\n",
    "print(\" \".join([\": MOORING att is: \", m_att]))\n",
    "fnc.close()     # when using an openDAP URL, you don't need to do fnc.close"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We found several files that work with most netCDF clients, but that break the library rules uesd by the netcdf4 package. \n",
    "\n",
    "Once Identified, the files to be fixed go in Data_tests, are repaired (usually with Matlab, since it's not as picky then once validated go into their \"Data\" directories on silt.\n",
    "\n",
    "Some example test paths\n",
    "#fpath=\"http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/FI14/catalog.xml\"\n",
    "#fpath=\"http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/MVCO_15/catalog.xml\"\n",
    "\n",
    "these were actually verified:\n",
    "use gamone instead of geoport because the TDS on gamone is newer\n",
    "#fpath=\"https://stellwagen.er.usgs.gov/thredds/dodsC/TSdata/SANDWICH2017/catalog.xml\"\n",
    "#fpath=\"http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_15/catalog.xml\"\n",
    "#fpath=\"http://gamone.whoi.edu/thredds/silt/usgs/Projects/stellwagen/Data/MVCO_14/catalog.xml\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#test if the files in a catalog pass the netCDF library tests\n",
    "fpath=\"http://gamone.whoi.edu/thredds/silt/usgs/Projects/stellwagen/Data/MVCO_14/catalog.xml\"\n",
    "c = Crawl(fpath, select=[\".*nc\"])\n",
    "urls = [s.get(\"url\") for d in c.datasets for s in d.services if s.get(\"service\").lower() == \"opendap\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101010mc-a.nc',\n",
       " 'http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101011hffan_raw.nc',\n",
       " 'http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10102sc-a.nc']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "urls[0:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101010mc-a.nc\n",
      "mooring  1010 : there is a  Seabird mc 37 -SM ; SN  mc-2538\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101011hffan_raw.nc\n",
      "mooring  1010 : there is a  Imagenex Model 881 Digital tilt adjusting imaging sonar ; SN  1751\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10102sc-a.nc\n",
      "mooring  1010 : there is a  Seabird sc 16 plus ; SN  sc-6384\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10103wh-a.nc\n",
      "mooring  1010 : there is a  Workhorse ADCP ; SN  767.0\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10103whp-a.nc\n",
      "mooring  1010 : there is a  RD Instruments ADCP ; SN  767\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10103whp-cal.nc\n",
      "mooring  1010 : there is a  RD Instruments ADCP ; SN  767.0\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10104aqd-a.nc\n",
      "mooring  1010 : there is a  Nortek Aquadopp Profiler ; SN  AQD11275\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10109advb-cal.nc\n",
      "mooring  1010 : there is a  Sontek ADV ; SN  B245H\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10109advs-cal.nc\n",
      "mooring  1010 : there is a  Sontek ADV ; SN  B245H\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101110ecn-a.nc\n",
      "mooring  1011 : there is a  WETLabs ECO BBSB ; SN  287\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101111Avecb-cal.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101111Bvecb-cal.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101111Cvecb-cal.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101111vecs-a.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101112Avecb-cal.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101112Bvecb-cal.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101112Cvecb-cal.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101112vecs-a.nc\n",
      "mooring  1011 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/101113mc-a.nc\n",
      "mooring  1011 : there is a  Seabird mc 37 -SM ; SN  mc-3575\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10112sc-a.nc\n",
      "mooring  1011 : there is a  Seabird sc 16 plus ; SN  sc-6283\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10115aqd-a.nc\n",
      "mooring  1011 : there is a  Nortek Aquadopp Profiler ; SN  AQD 1829\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10119fan_raw.nc\n",
      "mooring  1011 : there is a  Imagenex Model 881 Digital tilt adjusting imaging sonar ; SN  1751\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10122Avecb-cal.nc\n",
      "mooring  1012 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10122Bvecb-cal.nc\n",
      "mooring  1012 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10122Cvecb-cal.nc\n",
      "mooring  1012 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10122vecs-cal.nc\n",
      "mooring  1012 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10123aqd-a.nc\n",
      "mooring  1012 : there is a  Nortek Aquadopp Profiler ; SN  AQD 6254\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10131Avecb-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10131Bvecb-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10131Cvecb-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10131vecs-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10132Avecb-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9688\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10132Bvecb-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9688\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10132Cvecb-cal.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9688\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10132vecs-a.nc\n",
      "mooring  1013 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9688\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10133aqd-a.nc\n",
      "mooring  1013 : there is a  Nortek Aquadopp Profiler ; SN  AQD 6200\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10321vecb-cal.nc\n",
      "mooring  1032 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10321vecs-a.nc\n",
      "mooring  1032 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10322vecb-cal.nc\n",
      "mooring  1032 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11208\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10322vecs-a.nc\n",
      "mooring  1032 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11208\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10323aqd-a.nc\n",
      "mooring  1032 : there is a  Nortek Aquadopp Profiler ; SN  AQD 6254\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10324solot-a.nc\n",
      "mooring  1032 : there is a  RBR SoloT ; SN  75965\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10331vecb-cal.nc\n",
      "mooring  1033 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10331vecs-a.nc\n",
      "mooring  1033 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10332vecb-cal.nc\n",
      "mooring  1033 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9688\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10332vecs-a.nc\n",
      "mooring  1033 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9688\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10333aqd-a.nc\n",
      "mooring  1033 : there is a  Nortek Aquadopp Profiler ; SN  AQD 6200\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10334solot-a.nc\n",
      "mooring  1033 : there is a  RBR SoloT ; SN  75968\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103510mc-a.nc\n",
      "mooring  1035 : there is a  Seabird mc 37 -SM ; SN  mc-2538\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103511hffan_raw.nc\n",
      "mooring  1035 : there is a  Imagenex Model 881 Digital tilt adjusting imaging sonar ; SN  1751\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103512solot-a.nc\n",
      "mooring  1035 : there is a  RBR SoloT ; SN  75972\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10351solot-a.nc\n",
      "mooring  1035 : there is a  RBR SoloT ; SN  75975\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10352sc-a.nc\n",
      "mooring  1035 : there is a  Seabird sc 16 plus ; SN  sc-6384\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10353wh-a.nc\n",
      "mooring  1035 : there is a  RD Instruments ADCP ; SN  9896.0\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10353whp-cal.nc\n",
      "mooring  1035 : there is a  RD Instruments ADCP ; SN  9896\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10354solot-a.nc\n",
      "mooring  1035 : there is a  RBR SoloT ; SN  75970\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10355aqd-a.nc\n",
      "mooring  1035 : there is a  Nortek Aquadopp Profiler ; SN  AQD11275\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10358advb-cal.nc\n",
      "mooring  1035 : there is a  Sontek ADV ; SN  B372H\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10358advs-cal.nc\n",
      "mooring  1035 : there is a  Sontek ADV ; SN  B372H\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10359advb-cal.nc\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mooring  1035 : there is a  Sontek ADV ; SN  B413B\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10359advs-cal.nc\n",
      "mooring  1035 : there is a  Sontek ADV ; SN  B413B\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103610ecn-a.nc\n",
      "mooring  1036 : there is a  WETLabs ECO BBSB ; SN  287\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103611Avecb-cal.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103611Bvecb-cal.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103611Cvecb-cal.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103611vecs-a.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11705\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103612Avecb-cal.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103612Bvecb-cal.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103612Cvecb-cal.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103612vecs-a.nc\n",
      "mooring  1036 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11716\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103613mc-a..nc\n",
      "mooring  1036 : there is a  Seabird mc 37 -SM ; SN  mc-3575\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/103614solot-a.nc\n",
      "mooring  1036 : there is a  RBR SoloT ; SN  75976\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10361solot-a.nc\n",
      "mooring  1036 : there is a  RBR SoloT ; SN  75966\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10362sc-a.nc\n",
      "mooring  1036 : there is a  Seabird sc 16 plus ; SN  sc-6283\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10363sgs-cal.nc\n",
      "mooring  1036 : there is a  Seabird Seagauge model SBE 26 ; SN  45\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10365aqd-a.nc\n",
      "mooring  1036 : there is a  Nortek Aquadopp Profiler ; SN  AQD 1829\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10366solot-a.nc\n",
      "mooring  1036 : there is a  RBR SoloT ; SN  75971\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10369fan_raw.nc\n",
      "mooring  1036 : there is a  Imagenex Model 881 Digital tilt adjusting imaging sonar ; SN  1751\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10371Avecb-cal.nc\n",
      "mooring  1037 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10371Bvecb-cal.nc\n",
      "mooring  1037 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10371Cvecb-cal.nc\n",
      "mooring  1037 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10371vecs-a.nc\n",
      "mooring  1037 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC 9691\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10373aqd-a.nc\n",
      "mooring  1037 : there is a  Nortek Aquadopp Profiler ; SN  AQD 6200\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10374solot-a.nc\n",
      "mooring  1037 : there is a  RBR SoloT ; SN  75968\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10381Avecb-cal.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10381Bvecb-cal.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10381Cvecb-cal.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10381vecs-a.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11189\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10382Avecb-cal.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11208\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10382Bvecb-cal.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11208\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10382vecs-a.nc\n",
      "mooring  1038 : there is a  Nortek Vector acoustic Doppler velocimeter ; SN  VEC11208\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10383aqd-a.nc\n",
      "mooring  1038 : there is a  Nortek Aquadopp Profiler ; SN  AQD 6254\n",
      "http://gamone.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data/MVCO_14/10384solot-a.nc\n",
      "mooring  1038 : there is a  RBR SoloT ; SN  75965\n"
     ]
    }
   ],
   "source": [
    "for fn in urls:\n",
    "    print(fn)\n",
    "    fnc=nc.Dataset(fn) # don't use 'w' will delete existing file!\n",
    "    all_atts=fnc.__dict__  # get all the global atts and their values as a dict\n",
    "    print('mooring ', all_atts['MOORING'],': there is a ', all_atts['INST_TYPE'],'; SN ',all_atts['serial_number'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " this file doesn't work - says it can't find \"MOORING\", but it's there\n",
    "!ncdump -h http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/V_fv/SANDWICH2016/10631whVp-cal.nc\n",
    "but it has ADCP_serial_number instead of serial_number, which may be the problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:crawler]",
   "language": "python",
   "name": "conda-env-crawler-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}