ismn package

Submodules

ismn.base module

class ismn.base.IsmnRoot(path)

Bases: object

Connection to the zip resp. extracted zip archive downloaded from the ismn website. This class only handles file access / requests made by the readers, lists files in path and can extract files to temp folders for safe reading.

path

Data directory

Type

Path

clean_subpath(subpath) → Union[pathlib.Path, pathlib.PurePosixPath]

Check if subpath is a valid path and adapt to archive format and os

close()

property cont

Get cont of object, or scan to create cont.

extract_dir(*args, **kwargs)

extract_file(*args, **kwargs)

find_files(subpath=None, fn_templ='*.csv')

List files in archive or a subdirectory of the archive that match the passed filename pattern.

Parameters

• subpath (str, optional (default: None)) – Use linux slashes ‘/’ (and no leading ‘/’) to define a subpath in the zip file. If None is selected, the whole archive is searched.

• fn_templ (str, optional (default: ‘*.csv’)) – Filename template for files that are searched in the passed dir.

Returns

files – Found files that match the passed template.

Return type

list[str]

property isopen: bool

open()

property root_dir: pathlib.Path

scan(station_subdirs=True) → collections.OrderedDict

Go through archive (zip or dir) and group station folders

Parameters

station_subdirs (bool, optional (default: True)) – Include the station dir as a subdir of network dir. If False is selected, station dir is included directly.

Returns

cont – Archive content, station dirs grouped by network dirs

Return type

OrderedDict

ismn.base.dir(func)

ismn.base.zip(func)

ismn.components module

class ismn.components.IsmnComponent

Bases: object

class ismn.components.Network(name, stations=None)

Bases: ismn.components.IsmnComponent

A network is described by a distinct name and can be composed of multiple ISMN stations.

name

Network name.

Type

str

stations

Stations belonging to the network. If a string is passed, an empty Station is added.

Type

OrderedDict[name, Station]

add_station(name, lon, lat, elev)

Add a station to the network.

Parameters

• name (str) – Station name.

• lon (float) – Longitude coordinate.

• lat (float) – Latitude coordinate.

• elev (float) – Elevation.

property coords: (<class 'list'>, <class 'list'>)

Get lists of lats and lons for all stations in the Network

get_citations()

Return reference(s) for this network. Users of ISMN should cite the networks they are using in a publication. This information can also be found on the ISMN website.

Returns

references – A list of references / citations / acknowledgements for this Network.

Return type

list

property grid: pygeogrids.grids.BasicGrid

Get grid for all Stations in Network

iter_sensors(**filter_kwargs)

Get all sensors in all stations in the network that comply with the passed filtering parameters.

Parameters

• arguments are used to evaluate the sensors (Keyword) –

• see –

:param ismn.components.Sensor.eval():

Yields

• station (Station) – Station that contains Sensor.

• sensor (Sensor) – Sensor at Station that matches to the passed filtering conditions.

iter_stations(**filter_kwargs)

Get all stations having at least one sensor observing a specific variable and/or sensing depth.

Parameters

• are used to check all sensors at all stations (Parameters) –

• stations (only) –

• have at least one matching sensor are returned. (that) –

• a description of possible filter kwargs (For) –

• see –

:param ismn.components.Sensor.eval():

Yields

station (Station) – Stations that contain at least one sensor that matches to the passed conditions.

property n_stations: int

Number of Stations in this Network.

remove_station(name)

Remove Station from Network.

Parameters

name (str) – Station name.

class ismn.components.NetworkCollection(networks)

Bases: ismn.components.IsmnComponent

A NetworkCollection holds multiple networks and provides functionality to perform access to components from multiple networks. A grid is added that contains all stations to perform spatial searches.

networks

Collection of network names and Networks

Type

OrderedDict

grid

Grid that contains one point for each station in all networks.

Type

BasicGrid

export_citations(out_file=None)

Returns the references for all networks in the collection. Optionally, they are also written to file. Information on how to correctly cite ISMN networks can be found on the ISMN website.

Parameters

out_file (str, optional (default: None)) – If a path is passed here, a new file will be generated with all references for the current collection.

Returns

references – Network names as keys and network references as values

Return type

OrderedDict

get_nearest_station(lon, lat, max_dist=inf)

Get nearest station for given longitude/latitude coordinates.

Parameters

• lon (float or list[float]) – Longitude coordinate(s).

• lat (float or list[float]) – Latitude coordinate(s).

• max_dist (float, optional (default: np.Inf)) – Maximum search distance.

Returns

• station (Station or list[Station]) – The nearest Station(s) to the passed coordinates.

• dist (float or list[float]) – Distance in meter between the passed coordinates and the actual location of the station.

iter_networks() → ismn.components.Network

Iterate through all networks in the Collection.

iter_sensors(**filter_kwargs) -> (<class 'ismn.components.Network'>, <class 'ismn.components.Station'>, <class 'ismn.components.Sensor'>)

Iterate through Networks in the Collection and get (all/filtered) Stations and Sensors at each Station.

iter_stations(**filter_kwargs) -> (<class 'ismn.components.Network'>, <class 'ismn.components.Station'>)

Iterate through Networks in the Collection and get (all/filtered) Stations.

station4gpi(gpi)

Get the Station for the passed gpi in the grid.

Parameters

gpi (int or list[int]) – Point index or multiple indices in self.grid.

Returns

station – Station(s) at gpi(s).

Return type

Station or list[Station]

class ismn.components.Sensor(instrument, variable, depth, name=None, filehandler=None, keep_loaded_data=False)

Bases: ismn.components.IsmnComponent

A Sensor with insitu observations.

instrument

Instrument name.

Type

str

variable

Observed variable.

Type

str

depth

Sensing depth.

Type

Depth

name

Name of the sensor.

Type

str

filehandler

File handler object to read data.

Type

IsmnFile

keep_loaded_data

Keep data in memory after loading.

Type

bool

data

Container for data in memory (if it is being kept)

Type

pandas.DataFrame

eval(variable=None, depth=None, filter_meta_dict=None, check_only_sensor_depth_from=False)

Evaluate whether the sensor complies with the passed metadata requirements.

Parameters

• variable (str or list[str], optional (default: None)) – Check if the variable name matches, e.g. soil_moisture. One or multiple of ismn.const.VARIABLE_LUT

• depth (Depth or list or tuple, optional (default: None)) – Check if the passed depth encloses the sensor depth. A list/tuple must contain 2 values where the first is the depth start and the second is the end. Start must be closer to 0 than end (or equal). A negative depth range is above the surface.

• filter_meta_dict (dict, optional (default: None)) –

  Additional metadata keys and values for which the file list is filtered e.g. {‘lc_2010’: [10, 130]} or

  {‘climate_KG’: ‘Dwa’, ‘lc_2010’: [10, 130] }

  to filter for a multiple landcover classes and a climate class.

• check_only_sensor_depth_from (bool, optional (default: False)) – Ignores the sensors depth_to value and only checks if depth_from of the sensor is in the passed depth (e.g. for cosmic ray probes).

Returns

flag – Indicates whether metadata for this Sensor matches with the passed requirements.

Return type

bool

property metadata: ismn.meta.MetaData

read_data()

Load data from filehandler for this Sensor by calling ismn.filehandlers.DataFile.read_data().

Returns

data – Insitu time series for this sensor, loaded from file or memory (if it was loaded and kept before).

Return type

pandas.DataFrame

class ismn.components.Station(name, lon, lat, elev)

Bases: ismn.components.IsmnComponent

A station is described by a distinct name and location. Multiple sensors at various depths can be part of a station.

name

Station name.

Type

str

lon

Longitude coordinate of station and all sensors at station.

Type

float

lat

Latitude coordinate of station and all sensors at station.

Type

float

elev

Elevation information of station.

Type

float

sensors

Collection of Sensors and their names.

Type

collections.OrderedDict

add_sensor(instrument, variable, depth, filehandler=None, name=None, keep_loaded_data=False)

Add a new Sensor to this Station.

Parameters

• instrument (str) – Instrument name. e.g. ThetaProbe-ML2X

• variable (str) – Observed variable. e.g. soil_moisture

• depth (Depth) – Sensing depth. e.g. Depth(0, 0.1)

• filehandler (DataFile, optional (default: None)) – File handler object that allows access to observation data and sensor metadata via its read_data() function (default: None).

• name (str or int, optional (default: None)) – A name or id for the sensor. If None is passed, one is generated automatically from other properties.

• keep_loaded_data (bool, optional (default: False)) – Keep data for the sensor in memory once it is loaded. This makes subsequent reading of the same data faster but can fill up memory if stations / networks are loaded.

get_depths(variable=None)

Get depths of sensors measuring at station.

Parameters

variable (str, optional (default: None)) – Only consider sensors measuring this variable.

Returns

depths – List of depths of all sensors that measure the passed variable.

Return type

list

get_min_max_obs_timestamp(variable='soil moisture', min_depth=None, max_depth=None)

Goes through the sensors associated with this station and checks the metadata to get and approximate time coverage of the station. This is just an overview. If holes have to be detected the complete file must be read.

Parameters

• variable (str, optional (default: 'soil_moisture')) – name of the variable, only sensors measuring that variable are used.

• min_depth (float, optional (default: None)) – depth_from of variable has to be >= min_depth in order to be included.

• max_depth (float, optional (default: None)) – depth_to of variable has to be <= max_depth in order to be included.

Returns

• start_date (datetime.datetime) – Earliest date observed by any sensor at the station after filtering for the passed requirements.

• end_date (datetime.datetime) – Latest date observed by any sensor at the station after filtering for the passed requirements.

get_sensors(variable, depth_from, depth_to)

get the sensors at which the variable was measured at the given depth

Parameters

• variable (str) – variable abbreviation

• depth_from (float) – shallower depth of layer the variable was measured at

• depth_to (float) – deeper depth of layer the variable was measured at

Returns

sensors – array of sensors found for the given combination of variable and depths

Return type

numpy.ndarray

get_variables()

Get variables measured by all sensors at station.

Returns

variables – List of variables that are observed.

Return type

list

iter_sensors(**filter_kwargs)

Iterates over all sensors in this station and yields those that comply with the passed filter settings (or all).

Parameters

arguments are used to check all sensors at all stations (Keyword) –

:param : :param only stations that have at least one matching sensor are returned.: :param For a description of possible filter kwargs: :param see: :param ismn.components.Sensor.eval():

Yields

sensors (Sensor) – (Filtered) Sensors at the Station.

property metadata: ismn.meta.MetaData

Collect the metadata from all sensors at station.

property n_sensors: int

Number of Sensors at this Station.

remove_sensor(name)

Remove sensor from station.

Parameters

name (str) – Sensor name.

ismn.const module

exception ismn.const.DepthError

Bases: ValueError

exception ismn.const.ISMNError

Bases: Exception

exception ismn.const.IsmnFileError

Bases: OSError

exception ismn.const.MetadataError

Bases: OSError

ismn.const.deprecated(func)

ismn.filecollection module

class ismn.filecollection.IsmnFileCollection(root, filelist, temp_root='/tmp')

Bases: object

The IsmnFileCollection class contains a list of file handlers to access data in the given data directory. The file list can be loaded from a previously stored csv file, or built by iterating over all files in the data root. This class also contains function to load filehandlers for certain networks only.

root

Root object where data is stored.

Type

IsmnRoot

filelist

A collection of filehandlers and network names

Type

collections.OrderedDict

temp_root

Temporary root dir.

Type

Path

classmethod build_from_scratch(data_root, parallel=True, log_path=None, temp_root='/tmp')

Parameters

• data_root (IsmnRoot or str or Path) – Root path of ISMN files or path to metadata pkl file. i.e. path to the downloaded zip file or the extracted zip directory (faster) or a file list that contains these infos already.

• parallel (bool, optional (default: True)) – Speed up metadata collecting with multiple processes.

• log_path (str or Path, optional (default: None)) – Path where the log file is created. If None is set, no log file will be written.

• temp_root (str or Path, (default: gettempdir())) – Temporary folder where extracted data is copied during reading from zip archive.

close()

classmethod from_metadata_csv(data_root, meta_csv_file, network=None, temp_root='/tmp')

Load a previously created and stored filelist from ismn.filecollection.IsmnFileCollection.to_metadata_csv()

Parameters

• data_root (IsmnRoot or str or Path) – Path where the ismn data is stored, can also be a zip file

• meta_csv_file (str or Path) – Csv file where the metadata is stored.

• network (list, optional (default: None)) – List of networks that are considered. Filehandlers for other networks are set to None.

• temp_root (str or Path, optional (default: gettempdir())) – Temporary folder where extracted data is copied during reading from zip archive.

get_filehandler(idx)

Get the nth filehandler in a list of all filehandlers for all networks. e.g. if there are 2 networks, with 3 filehandlers/sensors each, idx=4 will return the first filehandler of the second network.

Parameters

idx (int) – Index of filehandler to read.

Returns

filehandler – nth filehandler of all filehandlers in the sorted list.

Return type

DataFile

iter_filehandlers(networks=None)

Iterator over files for networks

Parameters

networks (list, optional (default: None)) – Name of networks to get files for, or None to use all networks.

Yields

file (DataFile) – Filehandler with metadata

to_metadata_csv(meta_csv_file)

Write filehandle metadata from filelist to metdata csv that contains ALL metadata / variables of the filehander. Can be read back in as filelist with filehandlers using ismn.filecollection.IsmnFileCollection.from_metadata_csv().

Parameters

meta_csv_file (Path or str, optional (default: None)) – Directory where the csv file with the correct name is crated

ismn.filehandlers module

class ismn.filehandlers.DataFile(root, file_path, load_metadata=True, temp_root='/tmp')

Bases: ismn.filehandlers.IsmnFile

IsmnFile class represents a single ISMN data file. This represents only .stm data files not metadata csv files.

See :class:`ismn.filehandlers.IsmnFile`

file_type

File type information (e.g. ceop).

Type

str

get_elements_from_file(delim='_', only_basename_elements=False)

Read first line of file and split filename. Information is used to collect metadata information for all ISMN formats.

Parameters

• delim (str, optional (default: '_')) – File basename delimiter.

• only_basename_elements (bool, optional (default: False)) – Parse only the filename and not the file contents.

Returns

• headr (list[str] or None) – First line of file split into list, None if only_filename is True

• secnd (list[str] or None) – Second line of file split into list, None if only_filename is True

• last (list[str] or None) – Last non empty line elements, None if only_filename is True

• file_basename_elements (list[str], None if only_filename is True) – File basename without path split by ‘delim’

get_metadata_ceop_sep(elements=None)

Get metadata in the file format called CEOP in separate files.

Parameters

elements (dict, optional (default: None)) – Previously loaded elements can be passed here to avoid reading the file again.

Returns

• metadata (MetaData) – Metadata information.

• depth (Depth) – Sensor Depth, generated from file name

get_metadata_header_values(elements=None)

Get metadata file in the format called Header Values.

Parameters

elements (dict, optional (default: None)) – Previously loaded elements can be passed here to avoid reading the file again.

Returns

• metadata (MetaData) – Metadata information.

• depth (Depth) – Sensor Depth, generated from file name

read_data() → pandas.core.frame.DataFrame

Read data in file. Load file if necessary.

Returns

data – File content.

Return type

pd.DataFrame

read_metadata(best_meta_for_sensor=True) → ismn.meta.MetaData

Read metadata from file name and first line of file.

Parameters

best_meta_for_sensor (bool, optional (default: True)) – Compare the sensor depth to metadata that is available in multiple depth layers (e.g. static metadata variables). Find the variable for which the depth matches best with the sensor depth.

class ismn.filehandlers.IsmnFile(root, file_path, temp_root='/tmp')

Bases: object

General base class for data and static metadata files (station csv file) in ismn archive.

root

Data access object

Type

IsmnRoot

file_path

File subpath in root archive

Type

Path

temp_root

Temporary directory

Type

Path

metadata

File MetaData collection

Type

MetaData

check_metadata(variable=None, allowed_depth=None, filter_meta_dict=None, check_only_sensor_depth_from=False) → bool

Evaluate whether the file complies with the passed metadata requirements

Parameters

• variable (str or list[str], optional (default: None)) – Name of the required variable(s) measured, e.g. soil_moisture

• allowed_depth (Depth, optional (default: None)) – Depth range that is allowed, depth in metadata must be within this range.

• filter_meta_dict (dict, optional (default: None)) – Additional metadata keys and values for which the file list is filtered e.g. {‘station’: ‘stationname’} to filter for a station name.

• check_only_sensor_depth_from (bool, optional (default: False)) – Ignores the sensors depth_to value and only checks if depth_from of the sensor is in the passed depth (e.g. for cosmic ray probes).

Returns

valid – Whether the metadata complies with the passed conditions or not.

Return type

bool

close()

open()

class ismn.filehandlers.StaticMetaFile(root, file_path, load_metadata=True, temp_root='/tmp')

Bases: ismn.filehandlers.IsmnFile

Represents a csv file containing site specific static variables. These attributes shall be assigned to all sensors at that site.

See Parent Class (IsmnFile)

read_metadata() → ismn.meta.MetaData

Read csv file containing static variables into data frame.

Returns

metadata – Static metadata read from csv file.

Return type

MetaData

ismn.interface module

class ismn.interface.ISMN_Interface(data_path, meta_path=None, network=None, parallel=False, keep_loaded_data=False, temp_root='/tmp')

Bases: object

Class provides interface to ISMN data downloaded from the ISMN website https://ismn.earth. Upon initialization it collects metadata from all files in path_to_data and saves metadata information in a csv file into the folder python_metadata in meta_path (or data_path if no meta_path is defined). First initialization can take some time if all ISMN data is present in data_path and will start multiple processes.

Parameters

• data_path (str or Path) – Path to ISMN data to read, either to a zip archive or to the extracted directory that contains the network folders. Download data from https://ismn.earth after registration.

• meta_path (str or Path) – Path where the metadata csv file(s) is / are stored. The actual filename is defined by the name of data_path and will be generated automatically if it does not yet exist.

• network (str or list, optional (default: None)) – Name(s) of network(s) to load. Other data in the data_path will be ignored. By default or if None is passed, all networks are activated. If an empty list is passed no networks are activated.

• parallel (bool, optional (default: False)) – Activate parallel processes to speed up metadata generation. All available CPUs will be used.

• keep_loaded_data (bool, optional (default: False)) – Keep data for a file in memory once it is loaded. This makes subsequent calls of data faster (if e.g. a station is accessed multiple times) but can fill up memory if multiple networks are loaded.

Raises

ISMNError – if given, network was not found ISMN_Interface.data_path

climate

All Climate classes and their descriptions.

Type

collections.OrderedDict

collection

Contains all loaded networks with stations and sensors.

Type

NetworkCollection

keep_loaded_data

Switch to keep data in memory after loading (not recommended).

Type

bool

metadata

Metadata for active networks, with idx that could also be passed to ismn.interface.read_metadata()

Type

pandas.DataFrame

landcover

All Landcover classes and their descriptions.

Type

collections.OrderedDict

parallel

Switch to activate parallel processing where possible.

Type

bool

root

ISMN data folder or .zip access

Type

IsmnRoot

networksOrderedDict

Access Networks container from collection directly.

gridpygeogrids.grid.BasicGrid

Grid from collection that contains all station lats and lons

activate_network(network: Optional[Union[list, str]] = None, meta_path: Optional[str] = None, temp_root: str = '/tmp')

Load (file) collection for specific networks.

close_files()

find_nearest_station(lon, lat, return_distance=False, max_dist=inf)

Finds the nearest station to passed coordinates available in downloaded data.

Parameters

• lon (float) – Longitude of point

• lat (float) – Latitude of point

• return_distance (bool, optional (default: False)) – if True also distance is returned

• max_dist (float, optional (default: np.inf)) – Maximum distance allowed. If no station is within this distance None is returned.

Returns

• station (ismn.components.Station) – Nearest station object that was found in within the selected distance

• distance (float, optional) – distance to station in meters, measured in cartesian coordinates and not on a great circle. Should be OK for small distances

get_climate_types(variable: str = 'soil_moisture', min_depth: float = 0, max_depth: float = 10, climate: str = 'climate_KG') → dict

See ismn.interface.ISMN_Interface.get_static_var_vals()

get_dataset_ids(variable, min_depth=0, max_depth=0.1, filter_meta_dict=None, check_only_sensor_depth_from=False, groupby=None)

Yield all sensors for a specific network and/or station and/or variable and/or depth. The id is defined by the position of the filehandler in the filelist.

Parameters

• variable (str or list[str] or None) – Variable(s) to filer out, None to allow all variables.

• min_depth (float, optional (default: 0)) – Min depth of sensors to search

• max_depth (float, optional (default: 0.1)) – Max depth of sensors to search

• filter_meta_dict (dict, optional (default: None)) – Additional metadata keys and values for which the file list is filtered e.g. {‘lc_2010’: 10} to filter for a landcover class. if there are multiple conditions, ALL have to be fulfilled. e.g. {‘lc_2010’: 10’, ‘climate_KG’: ‘Dfc’})

• check_only_sensor_depth_from (bool, optional (default: False)) – Ignores the sensors depth_to value and only checks if depth_from of the sensor is in the passed depth (e.g. for cosmic ray probes).

• groupby (str, optional (default: None)) – A metadata field name that is used to group sensors, e.g. network

get_landcover_types(variable: str = 'soil_moisture', min_depth: float = 0, max_depth: float = 10, landcover: str = 'lc_2010') → dict

See ismn.interface.ISMN_Interface.get_static_var_vals()

get_min_max_obs_timestamps(variable='soil_moisture', min_depth=- inf, max_depth=inf, filter_meta_dict=None)

Filter the active file list and return the min/max time stamp from ALL time series that match the passed criteria. This time period does NOT apply to all time series in the collection but is the OVERALL earliest and latest timestamp found.

Parameters

• variable (str, optional (default: 'soil_moisture')) – One of those in ismn.const.VARIABLE_LUT or returned by ismn.interface.ISMN_Interface.get_variables(): ‘soil_moisture’, ‘soil_temperature’, ‘soil_suction’, ‘precipitation’, ‘air_temperature’, ‘field_capacity’, ‘permanent_wilting_point’, ‘plant_available_water’, ‘potential_plant_available_water’, ‘saturation’, ‘silt_fraction’, ‘snow_depth’, ‘sand_fraction’, ‘clay_fraction’, ‘organic_carbon’, ‘snow_water_equivalent’, ‘surface_temperature’, ‘surface_temperature_quality_flag_original’

• min_depth (float, optional (default: -np.inf)) – Only sensors in this depth are considered.

• max_depth (float, optional (default: np.inf)) – Only sensors in this depth are considered.

• filter_meta_dict (dict, optional (default: None)) – Additional metadata keys and values for which the file list is filtered e.g. {‘lc_2010’: 10} to filter for a landcover class. if there are multiple conditions, ALL have to be fulfilled. e.g. {‘lc_2010’: 10’, ‘climate_KG’: ‘Dfc’})

Returns

• start_date (datetime.datetime) – Earliest time stamp found in all sensors that fulfill the passed requirements.

• end_date (datetime.datetime) – Latest time stamp found in all sensors that fulfill the passed requirements.

get_static_var_vals(variable='soil_moisture', min_depth=- inf, max_depth=inf, static_var_name='lc_2010') → dict

Get unique meta values for the selected static variable in the active networks.

Parameters

• variable (str, optional (default: 'soil_moisture')) – One of those in ismn.const.VARIABLE_LUT or returned by ismn.interface.ISMN_Interface.get_variables(): ‘soil_moisture’, ‘soil_temperature’, ‘soil_suction’, ‘precipitation’, ‘air_temperature’, ‘field_capacity’, ‘permanent_wilting_point’, ‘plant_available_water’, ‘potential_plant_available_water’, ‘saturation’, ‘silt_fraction’, ‘snow_depth’, ‘sand_fraction’, ‘clay_fraction’, ‘organic_carbon’, ‘snow_water_equivalent’, ‘surface_temperature’, ‘surface_temperature_quality_flag_original’

• min_depth (float, optional (default: -np.inf)) – Only sensors in this depth are considered.

• max_depth (float, optional (default: np.inf)) – Only sensors in this depth are considered.

• static_var_name (str, optional (default: 'lc_2010')) – One of: ‘lc_2000’, ‘lc_2005’, ‘lc_2010’, ‘lc_insitu’, ‘climate_KG’, ‘climate_insitu’

Returns

vals – Unique values found in static meta and their meanings.

Return type

dict

get_variables() → numpy.ndarray

get a list of variables available in the data

property grid

list_networks() → numpy.ndarray

list_sensors(network: str = None, station: str = None) → numpy.ndarray

list_stations(network: str = None) → numpy.ndarray

network_for_station(stationname, name_only=True)

Find networks that contain a station of the passed name.

Parameters

• stationname (str) – Station name to search in the active networks.

• name_only (bool, optional (default: True)) – Returns only the name of the network and not the Network.

Returns

network_names – Network that contains a station of that name, or None if no such network exists. Prints are warning and uses the FIRST found network name if there are multiple stations with the same name in different networks.

Return type

str or Network or None

property networks

plot_station_locations(variable=None, min_depth=- inf, max_depth=inf, stats_text=True, check_only_sensor_depth_from=False, markersize=1, text_scalefactor=1, dpi=300, filename=None, ax=None)

Plots available stations on a world map in robinson projection.

Parameters

• variable (str or list[str], optional (default: None)) – Show only stations that measure this/these variable(s), e.g. soil_moisture If None is passed, no filtering for variable is performed.

• min_depth (float, optional (default: -np.inf)) – Minimum depth, only stations that have a valid sensor measuring the passed variable (if one is selected) in this depth range are included.

• max_depth (float, optional (default: -np.inf)) – See description of min_depth. This is the bottom threshold for the allowed depth.

• stats_text (bool, optianal (default: False)) – Include text of net/stat/sens counts in plot.

• check_only_sensor_depth_from (bool, optional (default: False)) – Ignores the sensors depth_to value and only checks if depth_from of the sensor is in the passed depth_range (e.g. for cosmic ray probes).

• markersize (int, optional (default: 1)) – Size of the marker, might depend on the amount of stations you plot.

• text_scalefactor (float, optional (default: 1)) – Scale factor that is applied to header and legend.

• dpi (float, optional (default: 300)) – Only applies when figure is saved to file. Resolution of the output figure.

• filename (str or Path, optional (default: None)) – Filename where image is stored. If None is passed, no file is created.

• ax (plt.axes) – Axes object that can be used by cartopy (projection assigned).

Returns

• fig (matplotlib.Figure) – created figure instance. If axes was given this will be None.

• ax (matplitlib.Axes) – used axes instance, can be added to another figure for example.

• count (dict) – Number of valid sensors and stations that contain at least one valid sensor and networks that contain at least one valid station.

print_climate_dict() → None

print all classes provided by the Koeppen-Geiger climate Classification

print_landcover_dict() → None

print all classes provided by the CCI Landcover Classification

read(*args, **kwargs)

read_metadata(idx, format='pandas')

Read only metadata by id as pd.DataFrame.

Parameters

• idx (int or list) – id of sensor to read, best one of those returned from ismn.interface.get_dataset_ids() or one in ISMN_Interface.metadata.

• format (str, optional (default: 'pandas')) –

  This only affects the return value when a SINGLE idx is passed. If multiple indices or None is passed, a DataFrame is returned.

  • pandas : return metadata as dataframe (Default)

  • dict : return metadata as dict (only for single idx)

  • obj : return metadata as MetaData object (only for single idx)

Returns

metadata – Metadata for the passed index.

Return type

pd.DataFrame or dict or MetaData

read_ts(idx, return_meta=False)

Read a time series directly by the filehandler id.

Parameters

• idx (int) – id of filehandler to read, best one of those returned by ismn.interface.ISMN_Interface.get_dataset_ids()

• return_meta (bool, optional (default: False)) – Also return the metadata for this sensor (as a second return value)

Returns

• timeseries (pd.DataFrame) – Observation time series

• metadata (pd.DataFrame, optional) – All available metadata for that sensor. Only returned when return_meta=False

stations_that_measure(variable, **filter_kwargs)

Goes through all stations and returns those that measure the specified variable

Parameters

• variable (str) –

  variable name, one of:

  • soil_moisture

  • soil_temperature

  • soil_suction

  • precipitation

  • air_temperature

  • field_capacity

  • permanent_wilting_point

  • plant_available_water

  • potential_plant_available_water

  • saturation

  • silt_fraction

  • snow_depth

  • sand_fraction

  • clay_fraction

  • organic_carbon

  • snow_water_equivalent

  • surface_temperature

  • surface_temperature_quality_flag_original

• filter_kwargs – Parameters are used to check all sensors at all stations, only stations that have at least one matching sensor are returned. For a description of possible filter kwargs, see ismn.components.Sensor.eval()

Yields

ISMN_station (Station)

ismn.meta module

class ismn.meta.Depth(start, end)

Bases: object

A class representing a depth range. For depth range start and end:

0: surface >0: Below surface <0: Above surface

start

Depth start. Upper boundary of a layer.

Type

float

end

Depth end. Lower boundary of a layer.

Type

float

extend

Layer range in metres.

Type

float

across0

Depth range across surface layer

Type

bool

property across0: bool

enclosed(other)

Test if other Depth encloses this Depth. Reverse of ismn.meta.Depth.encloses().

Parameters

other (Depth) – Check if self is enclosed by other.

Returns

flag – True if other depth surrounds given depth, False otherwise.

Return type

bool

encloses(other)

Test if this Depth encloses other Depth. Reverse of ismn.meta.Depth.enclosed().

Parameters

other (Depth) – Check if other is enclosed by self.

Returns

flag – True if other depth is surrounded by given depth, False otherwise.

Return type

bool

property is_profile: bool

overlap(other, return_perc=False)

Check if two depths overlap, (if the start of one depth is the same as the end of the other, they would also overlap), e.g. Depth(0, 0.1) and Depth(0.1, 0.2) do overlap.

Parameters

• other (Depth) – Other Depth

• return_perc (bool, optional (Default: False)) – Returns how much the depths overlap. See func: ismn.meta.Depth.perc_overlap()

Returns

• overlap (bool) – True if Depths overlap

• perc_overlap (float, optional) – Normalised overlap.

perc_overlap(other)

Estimate how much 2 depths correspond. - 1 means that the are the same - 0 means that they have an infinitely small correspondence

(e.g. a single layer within a range, or 2 adjacent depths).

• -1 means that they don’t overlap.

Parameters

other (Depth) – Second depth, overlap with this depth is calculated.

Returns

p – Normalised overlap range <0 = no overlap, 0 = adjacent, >0 = overlap, 1 = equal

Return type

float

class ismn.meta.MetaData(vars: Optional[List[ismn.meta.MetaVar]] = None)

Bases: object

MetaData contains multiple MetaVars as a list (there can be multiple vars with the same name, e.g. for different depths)

add(name, val, depth=None)

Create a new MetaVar and add it to this collection.

Parameters

• name (str) – Name of the variable

• val (Any) – Value of the variable

• depth (Depth, optional (default: None)) – A depth that is assigned to the variable.

best_meta_for_depth(depth)

For meta variables that have a depth assigned, find the ones that match best (see ismn.depth.Depth.perc_overlap()) to the passed depth.

Parameters

depth (Depth) – Reference depth, e.g. the depth of a sensor.

Returns

best_vars – A dict of variable names and a single variable for each name that was found to match best to the passed depth. Any variables that have a depth assigned which does not overlap with the passed depth are excluded here!

Return type

MetaData

keys() → list

merge(other, inplace=False, exclude_empty=True)

Merge two or more metadata sets, i.e. take all variables from other(s) that are not in this metadata, and add them.

Parameters

• other (MetaData or list[MetaData]) – Other MetaData Collection or a list of MetaData, e.g. from multiple sensors.

• inplace (bool, optional (default: False)) – Replace self.metadata with the merged meteadata, if False then the merged metadata is returned

• exclude_empty (bool, optional (default: True)) – Variables where the value is NaN are ignored during merging.

Returns

merged_meta – The merged metadata (if inplace is False)

Return type

MetaData or None

replace(name, val, depth=None)

Replace the value of a MetaVar in the initialized class

Parameters

• name (str) – Name of the MetaVar.

• val (Any) – New value of the MetaVar.

• depth (Depth, optional (default: None)) – New Depth of the variable.

to_dict()

Convert metadata to dictionary.

Returns

meta – Variable name as key, value and depth as values

Return type

dict

to_pd(transpose=False, dropna=True)

Convert metadata to a pandas DataFrame.

Parameters

• transpose (bool, optional (default: False)) – Organise variables in columns instead of rows.

• dropna (bool, optional (default: True)) – Drop NaNs, e.g. when no depth is assigned or a variable is empty the corresponding rows/cols will be excluded from the returned data frame.

Returns

df – Metadata collection as a data frame.

Return type

pd.DataFrame

values() → list

class ismn.meta.MetaVar(name: str, val: Any, depth: Optional[ismn.meta.Depth] = None)

Bases: object

MetaVar is a simple combination of a name, a value and a depth range (optional).

property empty: bool

classmethod from_tuple(args: tuple)

Create Metadata for a list of arguments.

Parameters

args (tuple) – 2 or 4 elements. 2: name and value 4: name, value, depth_from, depth_to

Module contents