MetadataTranslator

class astro_metadata_translator.MetadataTranslator(header: Mapping[str, Any], filename: str | None = None)

Bases: object

Per-instrument metadata translation support.

Parameters:

headerdict-like

Representation of an instrument header that can be manipulated as if it was a dict.

filenamestr, optional

Name of the file whose header is being translated. For some datasets with missing header information this can sometimes allow for some fixups in translations.

Attributes Summary

all_properties	All the valid properties for this translator including extensions.
default_resource_package	Module name to use to locate the correction resources.
default_resource_root	Default package resource path root to use to locate header correction files within the default_resource_package package.
default_search_path	Default search path to use to locate header correction files.
extensions	Extension properties (str: PropertyDefinition)
name	The declared name of the translator.
supported_instrument	Name of instrument understood by this translation class.
translators	All registered metadata translation classes.

Methods Summary

are_keys_ok(keywords)	Are the supplied keys all present and defined?.
can_translate(header[, filename])	Indicate whether this translation class can translate the supplied header.
can_translate_with_options(header, options)	Determine if a header can be translated with different criteria.
cards_used()	Cards used during metadata extraction.
defined_in_this_class(name)	Report if the specified class attribute is defined specifically in this class.
determine_translatable_headers(filename[, ...])	Given a file return all the headers usable for metadata translation.
determine_translator(header[, filename])	Determine a translation class by examining the header.
fix_header(header, instrument, obsid[, filename])	Apply global fixes to a supplied header.
is_key_ok(keyword)	Return True if the value associated with the named keyword is present in this header and defined.
is_keyword_defined(header, keyword)	Return True if the value associated with the named keyword is present in the supplied header and defined.
observing_date_to_observing_day(...)	Return the YYYYMMDD integer corresponding to the observing day.
observing_date_to_offset(observing_date)	Calculate the observing day offset to apply for a given observation.
quantity_from_card(keywords, unit[, ...])	Calculate a Astropy Quantity from a header card and a unit.
resource_root()	Return package resource to use to locate correction resources within an installed package.
search_paths()	Search paths to use when searching for header fix up correction files.
to_altaz_begin()	Return value of altaz_begin from headers.
to_boresight_airmass()	Return value of boresight_airmass from headers.
to_boresight_rotation_angle()	Return value of boresight_rotation_angle from headers.
to_boresight_rotation_coord()	Return value of boresight_rotation_coord from headers.
to_can_see_sky()	Return whether the observation can see the sky or not.
to_dark_time()	Return value of dark_time from headers.
to_datetime_begin()	Return value of datetime_begin from headers.
to_datetime_end()	Return value of datetime_end from headers.
to_detector_exposure_id()	Return value of detector_exposure_id from headers.
to_detector_group()	Return value of detector_group from headers.
to_detector_name()	Return value of detector_name from headers.
to_detector_num()	Return value of detector_num from headers.
to_detector_serial()	Return value of detector_serial from headers.
to_detector_unique_name()	Return a unique name for the detector.
to_exposure_group()	Return the group label associated with this exposure.
to_exposure_id()	Return value of exposure_id from headers.
to_exposure_time()	Return value of exposure_time from headers.
to_focus_z()	Return a default defocal distance of 0.0 mm if there is no keyword for defocal distance in the header.
to_group_counter_end()	Return the observation counter of the observation that ends this group.
to_group_counter_start()	Return the observation counter of the observation that began this group.
to_has_simulated_content()	Return a boolean indicating whether any part of the observation was simulated.
to_instrument()	Return value of instrument from headers.
to_location()	Return value of location from headers.
to_object()	Return value of object from headers.
to_observation_counter()	Return an integer corresponding to how this observation relates to other observations.
to_observation_id()	Return value of observation_id from headers.
to_observation_reason()	Return the reason this observation was taken.
to_observation_type()	Return value of observation_type from headers.
to_observing_day()	Return the YYYYMMDD integer corresponding to the observing day.
to_observing_day_offset()	Return the offset required to calculate observing day.
to_physical_filter()	Return value of physical_filter from headers.
to_pressure()	Return value of pressure from headers.
to_relative_humidity()	Return value of relative_humidity from headers.
to_science_program()	Return value of science_program from headers.
to_telescope()	Return value of telescope from headers.
to_temperature()	Return value of temperature from headers.
to_tracking_radec()	Return value of tracking_radec from headers.
to_visit_id()	Return value of visit_id from headers.
translator_version()	Return the version string for this translator class.
validate_value(value, default[, minimum, ...])	Validate the supplied value, returning a new value if out of range.

Attributes Documentation

all_properties: dict[str, astro_metadata_translator.properties.PropertyDefinition] = {}

All the valid properties for this translator including extensions.

default_resource_package = 'astro_metadata_translator'

Module name to use to locate the correction resources.

default_resource_root: str | None = None

Default package resource path root to use to locate header correction files within the default_resource_package package.

default_search_path: Sequence[str] | None = None

Default search path to use to locate header correction files.

extensions: dict[str, astro_metadata_translator.properties.PropertyDefinition] = {}

Extension properties (str: PropertyDefinition)

Some instruments have important properties beyond the standard set; this is the place to declare that they exist, and they will be treated in the same way as the standard set, except that their names will everywhere be prefixed with ext_.

Each property is indexed by name (str), with a corresponding PropertyDefinition.

name: str | None = None

The declared name of the translator.

supported_instrument: str | None = None

Name of instrument understood by this translation class.

translators: dict[str, type[astro_metadata_translator.translator.MetadataTranslator]] = {'DECam': <class 'astro_metadata_translator.translators.decam.DecamTranslator'>, 'HSC': <class 'astro_metadata_translator.translators.hsc.HscTranslator'>, 'MegaPrime': <class 'astro_metadata_translator.translators.megaprime.MegaPrimeTranslator'>, 'SDSS': <class 'astro_metadata_translator.translators.sdss.SdssTranslator'>, 'SuprimeCam': <class 'astro_metadata_translator.translators.suprimecam.SuprimeCamTranslator'>}

All registered metadata translation classes.

Methods Documentation

are_keys_ok(keywords: Iterable[str]) → bool

Are the supplied keys all present and defined?.

Parameters:

keywordsiterable of str

Keywords to test.

Returns:

all_okbool

True if all supplied keys are present and defined.

abstract classmethod can_translate(header: Mapping[str, Any], filename: str | None = None) → bool

Indicate whether this translation class can translate the supplied header.

Parameters:

headerdict-like

Header to convert to standardized form.

filenamestr, optional

Name of file being translated.

Returns:

canbool

True if the header is recognized by this class. False otherwise.

classmethod can_translate_with_options(header: Mapping[str, Any], options: dict[str, Any], filename: str | None = None) → bool

Determine if a header can be translated with different criteria.

Parameters:

headerdict-like

Header to convert to standardized form.

optionsdict

Headers to try to determine whether this header can be translated by this class. If a card is found it will be compared with the expected value and will return that comparison. Each card will be tried in turn until one is found.

filenamestr, optional

Name of file being translated.

Returns:

canbool

True if the header is recognized by this class. False otherwise.

Notes

Intended to be used from within can_translate implementations for specific translators. Is not intended to be called directly from determine_translator.

cards_used() → frozenset[str]

Cards used during metadata extraction.

Returns:

usedfrozenset of str

Cards used when extracting metadata.

classmethod defined_in_this_class(name: str) → bool | None

Report if the specified class attribute is defined specifically in this class.

Parameters:

namestr

Name of the attribute to test.

Returns:

in_classbool

True if there is a attribute of that name defined in this specific subclass. False if the method is not defined in this specific subclass but is defined in a parent class. Returns None if the attribute is not defined anywhere in the class hierarchy (which can happen if translators have typos in their mapping tables).

Notes

Retrieves the attribute associated with the given name. Then looks in all the parent classes to determine whether that attribute comes from a parent class or from the current class. Attributes are compared using id().

classmethod determine_translatable_headers(filename: str, primary: MutableMapping[str, Any] | None = None) → Iterator[MutableMapping[str, Any]]

Given a file return all the headers usable for metadata translation.

This method can optionally be given a header from the file. This header will generally be the primary header or a merge of the first two headers.

In the base class implementation it is assumed that this supplied header is the only useful header for metadata translation and it will be returned unchanged if given. This can avoid unnecessarily re-opening the file and re-reading the header when the content is already known.

If no header is supplied, a header will be read from the supplied file using read_basic_metadata_from_file, allowing it to merge the primary and secondary header of a multi-extension FITS file. Subclasses can read the header from the data file using whatever technique is best for that instrument.

Subclasses can return multiple headers and ignore the externally supplied header. They can also merge it with another header and return a new derived header if that is required by the particular data file. There is no requirement for the supplied header to be used.

Parameters:

filenamestr

Path to a file in a format understood by this translator.

primarydict-like, optional

The primary header obtained by the caller. This is sometimes already known, for example if a system is trying to bootstrap without already knowing what data is in the file. For many instruments where the primary header is the only relevant header, the primary header will be returned with no further action.

Yields:

headersiterator of dict-like

A header usable for metadata translation. For this base implementation it will be either the supplied primary header or a header read from the file. This implementation will only ever yield a single header.

Notes

Each translator class can have code specifically tailored to its own file format. It is important not to call this method with an incorrect translator class. The normal paradigm is for the caller to have read the first header and then called determine_translator() on the result to work out which translator class to then call to obtain the real headers to be used for translation.

classmethod determine_translator(header: Mapping[str, Any], filename: str | None = None) → type[astro_metadata_translator.translator.MetadataTranslator]

Determine a translation class by examining the header.

Parameters:

headerdict-like

Representation of a header.

filenamestr, optional

Name of file being translated.

Returns:

translatorMetadataTranslator

Translation class that knows how to extract metadata from the supplied header.

Raises:

ValueError

None of the registered translation classes understood the supplied header.

classmethod fix_header(header: MutableMapping[str, Any], instrument: str, obsid: str, filename: str | None = None) → bool

Apply global fixes to a supplied header.

Parameters:

headerdict

The header to correct. Correction is in place.

instrumentstr

The name of the instrument.

obsidstr

Unique observation identifier associated with this header. Will always be provided.

filenamestr, optional

Filename associated with this header. May not be set since headers can be fixed independently of any filename being known.

Returns:

modifiedbool

True if a correction was applied.

Notes

This method is intended to support major discrepancies in headers such as:

• Periods of time where headers are known to be incorrect in some way that can be fixed either by deriving the correct value from the existing value or understanding the that correction is static for the given time. This requires that the date header is known.

• The presence of a certain value is always wrong and should be corrected with a new static value regardless of date.

It is assumed that one off problems with headers have been applied before this method is called using the per-obsid correction system.

Usually called from astro_metadata_translator.fix_header.

For log messages, do not assume that the filename will be present. Always write log messages to fall back on using the obsid if filename is None.

is_key_ok(keyword: str | None) → bool

Return True if the value associated with the named keyword is present in this header and defined.

Parameters:

keywordstr

Keyword to check against header.

Returns:

is_okbool

True if the header is present and not-None. False otherwise.

static is_keyword_defined(header: Mapping[str, Any], keyword: str | None) → bool

Return True if the value associated with the named keyword is present in the supplied header and defined.

Parameters:

headerdict-lik

Header to use as reference.

keywordstr

Keyword to check against header.

Returns:

is_definedbool

True if the header is present and not-None. False otherwise.

classmethod observing_date_to_observing_day(observing_date: Time, offset: TimeDelta | int | None) → int

Return the YYYYMMDD integer corresponding to the observing day.

The offset is subtracted from the time of observation before calculating the year, month and day.

Parameters:

observing_dateastropy.time.Time

The observation date.

offsetastropy.time.TimeDelta | numbers.Real | None

The offset to subtract from the observing date when calculating the observing day. If a plain number is given it is taken to be in units of seconds. If None no offset is applied.

Returns:

dayint

The observing day as an integer of form YYYYMMDD.

Notes

For example, if the offset is +12 hours both 2023-07-06T13:00 and 2023-07-07T11:00 will return an observing day of 20230706 because the observing day goes from 2023-07-06T12:00 to 2023-07-07T12:00.

classmethod observing_date_to_offset(observing_date: Time) → TimeDelta | None

Calculate the observing day offset to apply for a given observation.

In some cases the definition of the observing day offset has changed during the lifetime of the instrument. For example lab data might have a different offset to that when the instrument is on the telescope.

Parameters:

observing_dateastropy.time.Time

The observation date.

Returns:

offsetastropy.time.TimeDelta or None

The offset to apply when calculating the observing day for a specific time of observation. None implies the offset is not known for that date.

quantity_from_card(keywords: str | Sequence[str], unit: Unit, default: float | None = None, minimum: float | None = None, maximum: float | None = None, checker: Callable | None = None) → Quantity

Calculate a Astropy Quantity from a header card and a unit.

Parameters:

keywordsstr or list of str

Keyword to use from header. If a list each keyword will be tried in turn until one matches.

unitastropy.units.UnitBase

Unit of the item in the header.

defaultfloat, optional

Default value to use if the header value is invalid. Assumed to be in the same units as the value expected in the header. If None, no default value is used.

minimumfloat, optional

Minimum possible valid value, optional. If the calculated value is below this value, the default value will be used.

maximumfloat, optional

Maximum possible valid value, optional. If the calculated value is above this value, the default value will be used.

checkerCallable, optional

Callback function to be used by the translator method in case the keyword is not present. Function will be executed as if it is a method of the translator class. Running without raising an exception will allow the default to be used. Should usually raise KeyError.

Returns:

qastropy.units.Quantity

Quantity representing the header value.

Raises:

KeyError

The supplied header key is not present.

resource_root() → tuple[str | None, str | None]

Return package resource to use to locate correction resources within an installed package.

Returns:

resource_packagestr

Package resource name. None if no package resource are to be used.

resource_rootstr

The name of the resource root. None if no package resources are to be used.

search_paths() → list[str]

Search paths to use when searching for header fix up correction files.

Returns:

pathslist

Directory paths to search. Can be an empty list if no special directories are defined.

Notes

Uses the classes default_search_path property if defined.

abstract to_altaz_begin() → None

Return value of altaz_begin from headers.

Telescope boresight azimuth and elevation at start of observation.

Returns:

altaz_beginastropy.coordinates.AltAz

The translated property.

abstract to_boresight_airmass() → None

Return value of boresight_airmass from headers.

Airmass of the boresight of the telescope.

Returns:

boresight_airmassfloat

The translated property.

abstract to_boresight_rotation_angle() → None

Return value of boresight_rotation_angle from headers.

Angle of the instrument in boresight_rotation_coord frame.

Returns:

boresight_rotation_angleastropy.coordinates.Angle

The translated property.

abstract to_boresight_rotation_coord() → None

Return value of boresight_rotation_coord from headers.

Coordinate frame of the instrument rotation angle (options: sky, unknown).

Returns:

boresight_rotation_coordstr

The translated property.

to_can_see_sky() → Any

Return whether the observation can see the sky or not.

Returns:

can_see_skybool or None

True if the detector is receiving photons from the sky. False if the sky is not visible to the detector. None if the metadata translator does not know one way or the other.

Notes

The base class translator uses a simple heuristic of returning True if the observation type is “science” or “object” and False if the observation type is “bias” or “dark”. For all other cases it will return None.

abstract to_dark_time() → None

Return value of dark_time from headers.

Duration of the exposure with shutter closed (seconds).

Returns:

dark_timeastropy.units.Quantity

The translated property.

abstract to_datetime_begin() → None

Return value of datetime_begin from headers.

Time of the start of the observation.

Returns:

datetime_beginastropy.time.Time

The translated property.

abstract to_datetime_end() → None

Return value of datetime_end from headers.

Time of the end of the observation.

Returns:

datetime_endastropy.time.Time

The translated property.

abstract to_detector_exposure_id() → None

Return value of detector_exposure_id from headers.

Unique integer identifier for this detector in this exposure.

Returns:

detector_exposure_idint

The translated property.

abstract to_detector_group() → None

Return value of detector_group from headers.

Collection name of which this detector is a part. Can be None if there are no detector groupings.

Returns:

detector_groupstr

The translated property.

abstract to_detector_name() → None

Return value of detector_name from headers.

Name of the detector within the instrument (might not be unique if there are detector groups).

Returns:

detector_namestr

The translated property.

abstract to_detector_num() → None

Return value of detector_num from headers.

Unique (for instrument) integer identifier for the sensor.

Returns:

detector_numint

The translated property.

abstract to_detector_serial() → None

Return value of detector_serial from headers.

Serial number/string associated with this detector.

Returns:

detector_serialstr

The translated property.

to_detector_unique_name() → Any

Return a unique name for the detector.

Base class implementation attempts to combine detector_name with detector_group. Group is only used if not None.

Can be over-ridden by specialist translator class.

Returns:

namestr

detector_group``_``detector_name if detector_group is defined, else the detector_name is assumed to be unique. If neither return a valid value an exception is raised.

Raises:

NotImplementedError

Raised if neither detector_name nor detector_group is defined.

to_exposure_group() → Any

Return the group label associated with this exposure.

Base class implementation returns the exposure_id in string form. A subclass may do something different.

Returns:

namestr

The exposure_id converted to a string.

abstract to_exposure_id() → None

Return value of exposure_id from headers.

Unique (with instrument) integer identifier for this observation.

Returns:

exposure_idint

The translated property.

abstract to_exposure_time() → None

Return value of exposure_time from headers.

Duration of the exposure with shutter open (seconds).

Returns:

exposure_timeastropy.units.Quantity

The translated property.

to_focus_z() → Any

Return a default defocal distance of 0.0 mm if there is no keyword for defocal distance in the header. The default keyword for defocal distance is FOCUSZ.

Returns:

focus_z: astropy.units.Quantity

The defocal distance from header or the 0.0mm default.

to_group_counter_end() → Any

Return the observation counter of the observation that ends this group.

The definition of the relevant group is up to the metadata translator. It can be the last observation in the exposure_group or the last observation in the visit, but must be derivable from the metadata of this observation. It is of course possible that the last observation in the group does not exist if a sequence of observations was not completed.

Returns:

counterint

The observation counter for the end of the relevant group. Default implementation always returns the observation counter of this observation.

to_group_counter_start() → Any

Return the observation counter of the observation that began this group.

The definition of the relevant group is up to the metadata translator. It can be the first observation in the exposure_group or the first observation in the visit, but must be derivable from the metadata of this observation.

Returns:

counterint

The observation counter for the start of the relevant group. Default implementation always returns the observation counter of this observation.

to_has_simulated_content() → Any

Return a boolean indicating whether any part of the observation was simulated.

Returns:

is_simulatedbool

True if this exposure has simulated content. This can be if some parts of the metadata or data were simulated. Default implementation always returns False.

abstract to_instrument() → None

Return value of instrument from headers.

The instrument used to observe the exposure.

Returns:

instrumentstr

The translated property.

abstract to_location() → None

Return value of location from headers.

Location of the observatory.

Returns:

locationastropy.coordinates.EarthLocation

The translated property.

abstract to_object() → None

Return value of object from headers.

Object of interest or field name.

Returns:

objectstr

The translated property.

to_observation_counter() → Any

Return an integer corresponding to how this observation relates to other observations.

Base class implementation returns 0 to indicate that it is not known how an observatory will define a counter. Some observatories may not use the concept, others may use a counter that increases for every observation taken for that instrument, and others may define it to be a counter within an observing day.

Returns:

sequenceint

The observation counter. Always 0 for this implementation.

abstract to_observation_id() → None

Return value of observation_id from headers.

Label uniquely identifying this observation (can be related to ‘exposure_id’).

Returns:

observation_idstr

The translated property.

to_observation_reason() → Any

Return the reason this observation was taken.

Base class implementation returns the science if the observation_type is science, else unknown. A subclass may do something different.

Returns:

namestr

The reason for this observation.

abstract to_observation_type() → None

Return value of observation_type from headers.

Type of observation (currently: science, dark, flat, bias, focus).

Returns:

observation_typestr

The translated property.

to_observing_day() → Any

Return the YYYYMMDD integer corresponding to the observing day.

Base class implementation uses the TAI date of the start of the observation corrected by the observing day offset. If that offset is None no offset will be applied.

The offset is subtracted from the time of observation before calculating the year, month and day.

Returns:

dayint

The observing day as an integer of form YYYYMMDD. If the header is broken and is unable to obtain a date of observation, 0 is returned and the assumption is made that the problem will be caught elsewhere.

Notes

For example, if the offset is +12 hours both 2023-07-06T13:00 and 2023-07-07T11:00 will return an observing day of 20230706 because the observing day goes from 2023-07-06T12:00 to 2023-07-07T12:00.

to_observing_day_offset() → Any

Return the offset required to calculate observing day.

Base class implementation returns None.

Returns:

offsetastropy.time.TimeDelta or None

The offset to apply. Returns None if the offset is not defined.

Notes

This offset must be subtracted from a time of observation to calculate the observing day. This offset must be added to the YYYYMMDDT00:00 observing day to calculate the time span coverage of the observing day.

abstract to_physical_filter() → None

Return value of physical_filter from headers.

The bandpass filter used for this observation.

Returns:

physical_filterstr

The translated property.

abstract to_pressure() → None

Return value of pressure from headers.

Atmospheric pressure outside the dome.

Returns:

pressureastropy.units.Quantity

The translated property.

abstract to_relative_humidity() → None

Return value of relative_humidity from headers.

Relative humidity outside the dome.

Returns:

relative_humidityfloat

The translated property.

abstract to_science_program() → None

Return value of science_program from headers.

Observing program (survey or proposal) identifier.

Returns:

science_programstr

The translated property.

abstract to_telescope() → None

Return value of telescope from headers.

Full name of the telescope.

Returns:

telescopestr

The translated property.

abstract to_temperature() → None

Return value of temperature from headers.

Temperature outside the dome.

Returns:

temperatureastropy.units.Quantity

The translated property.

abstract to_tracking_radec() → None

Return value of tracking_radec from headers.

Requested RA/Dec to track.

Returns:

tracking_radecastropy.coordinates.SkyCoord

The translated property.

abstract to_visit_id() → None

Return value of visit_id from headers.

ID of the Visit this Exposure is associated with.

Science observations should essentially always be associated with a visit, but calibration observations may not be.

Returns:

visit_idint

The translated property.

classmethod translator_version() → str

Return the version string for this translator class.

Returns:

versionstr

String identifying the version of this translator.

Notes

Assumes that the version is available from the __version__ variable in the parent module. If this is not the case a translator should subclass this method.

static validate_value(value: float, default: float, minimum: float | None = None, maximum: float | None = None) → float

Validate the supplied value, returning a new value if out of range.

Parameters:

valuefloat

Value to be validated.

defaultfloat

Default value to use if supplied value is invalid or out of range. Assumed to be in the same units as the value expected in the header.

minimumfloat

Minimum possible valid value, optional. If the calculated value is below this value, the default value will be used.

maximumfloat

Maximum possible valid value, optional. If the calculated value is above this value, the default value will be used.

Returns:

valuefloat

Either the supplied value, or a default value.