Custom Validators/Extending the IRATE Format

The IRATE format includes functionality for providing custom validators to allow more specific checking of a file than just for conformance to the IRATE format (outlined in IRATE Format Specification). This is done by subclassing the irate.validate.Validator class to make objects that test whether or not a given file conforms to the extended format.

The validator structure is contained within irate.validate. The most straightforward way to provide a set of validators for a given file format (called “validator types” in the IRATE documentation and scripts) is to use the custom type directory. This directory is typically $HOME/.irate, but for the exact directory for your platform and machine, call the irate.validate.find_custom_validator_dir() function, which will return the directory to place validator type scripts.

Any file that ends in ‘.py’ in the directory will be executed and searched for Validator subclasses. Those classes will be automatically added to a type taken from the name of the file (e.g. if the script is myformat.py, the validator type will be myformat). Normally, you will want to include all the default validators (these ensure the IRATE format is being followed), but if not, add a global variable to the file named includedefaults and set it to False.

Once this file has been created (or given to you by someone else and placed in the correct directory), the function irate.validate.activate_validator_type() can be called to activator that validator type, or the iratevalidate script can be used with the -t option (see Command Line Scripts and Conversion Tools).

Writing Validator Subclasses

Writing validators is quite straightforward and the full reference is in the docstrings for the irate.validate.Validator class. A basic validator simply looks like:

from irate.validate import Validator)

class MyGroupNameValidator(Validator):

    groupname = 'MyGroupName'

    def validate(self,grp):
        if 'SpecialData' not in grp:
            self.invalid('SpecialData dataset not present')

Validating will then fail for any group that has “MyGroupName” (case-sensitive) as part of its name that does not have a “SpecialData” element (i.e. Group or Dataset). The validator will automatically call the appropriate validator on child groups - all that’s necessary is to check if the particular group being validated here meets whatever standards are set for the group. If the group fails, a call to self.invalid('a message here about the problem') will properly report the format error.

An additional useful utility is the irate.validate.Validator.check_units() method. This can be called as self.check_units(grp['somedatasetname') to check that a dataset has units. Alternatively, it can be called as self.check_units(grp['otherdataset'],'unitname) to check that a particular dataset has units with a particular name. Note that this should be used instead of directly checking attributes on the dataset because the IRATE format supports storing units on groups above the dataset in the HDF5 heirarchy (see Unit Information for details on how units are stored in the IRATE format).