NuRadioReco.modules.io.LOFAR.readLOFARData module

Reader module for LOFAR data

This module contains the reader class readLOFARData for LOFAR data (similar to the eventReader). This class converts raw TBB (.h5) data to the NuRadioReco Event structure.

NuRadioReco.modules.io.LOFAR.readLOFARData.get_metadata(filenames, metadata_dir)

Get metadata from TBB file.

Parameters:

filenameslist[str]

List of TBB file paths to read in

metadata_dirstr

Path to the TBB metadata directory

Returns:

station_namestr

antenna_setstr

time_unixint

time_nsfloat

clock_frequencyfloat

positionsndarray

dipole_idslist

calibration_delaysndarray

NuRadioReco.modules.io.LOFAR.readLOFARData.lora_timestamp_to_blocknumber(lora_seconds, lora_nanoseconds, start_time, sample_number, clock_offset=10000.0, block_size=65536, sampling_frequency=0.2)

Calculates block number corresponding to LORA timestamp and the sample number within that block

Parameters:

lora_secondsint

LORA timestamp in seconds (UNIX timestamp, second after 1st January 1970)

lora_nanosecondsint

The number of nanoseconds after lora_seconds at which LORA triggered

start_timeint

LOFAR TBB timestamp

sample_numberint

Sample number in the block where the trace starts

clock_offsetfloat, default=1e4 * units.ns

Clock offset between LORA and LOFAR

block_sizeint, default=2**16

Block size of the LOFAR data

sampling_frequencyfloat, default=200 * units.MHz

Sampling frequency of LOFAR

Returns:

blocknumberint

Index of block corresponding to LORA timestamp

samplenumberint

Index of sample corresponding to LORA timestamp, within block blocknumber

NuRadioReco.modules.io.LOFAR.readLOFARData.LOFAR_event_id_to_unix(event_id)

NuRadioReco.modules.io.LOFAR.readLOFARData.tbb_filetag_from_unix(timestamp)

Returns TBB filename based on UNIX timestamp of an event.

Parameters:

timestamp: int

UNIX timestamp from GPS

Returns:

filename: str

The tag in the TBB filename identifying the files of the event.

Notes

Technically speaking there is no such thing as an “UTC timestamp”. The Coordinated Universal Time (UTC) is a time standard , which defines a reference for current time. It uses hours, minutes and seconds to divide a day. Crucially, it allows for the introduction of leap seconds.

The UNIX timestamp on the other hand is defined as the number of non-leap seconds passed since 00:00:00 UTC on 1 January 1970. However, when a leap second occurs the UNIX timestamp is actually reset (i.e. that same timestamp refers to two moments in time). As such, the UNIX timestamp remains synchronised with the UTC time (except for the one second that is a leap second).

Note that astropy has support for leap seconds. During the day a leap seconds occurs, UNIX timestamps are reported as floating point values (instead of the usual integers). Conversely, converting a UNIX timestamp to a datetime will give a millisecond contribution. Though the seconds are still accounted for as one would expect.

NuRadioReco.modules.io.LOFAR.readLOFARData.tbbID_to_nrrID(channel_id, mode)

Converts a TBB channel ID to the corresponding NRR channel ID given the antenna mode.

This simply adds a “9” as the fourth element of the channelID, if the antenna mode is “LBA_inner”. The function nrrID_to_tbbID() can be used to do the opposite.

As of February 2024, this function only supports “LBA_INNER” and “LBA_OUTER” as possible antenna modes. Note that the antenna mode is always converted to lowercase, so the comparison is case-insensitive (i.e. “LBA_inner” and “LBA_INNER” are both recognised as the same antenna set).

Parameters:

channel_id: str or int

TBB channel ID

mode: {“LBA_inner”, “LBA_outer”}

The antenna set for which to convert (case-insensitive).

Returns:

nrr_channel_id: str

The NuRadioReco channelID corresponding to the TBB channelID depending on whether the antenna mode is “lba_inner” or “lba_outer”

Notes

This function encodes the convention used in the LOFAR.json detector description. For the inner LBA antennas, a “9” was added as the fourth element of the channelID. However, the TBB files always use the same set of channel IDs (i.e. the same channel ID refers to different physical antennas depending on the antenna set). Given a channel ID from the TBB file and the antenna set, this function returns the channel ID of the corresponding channel in the NRR Detector description.

NuRadioReco.modules.io.LOFAR.readLOFARData.nrrID_to_tbbID(channel_id)

This function does the opposite of tbbID_to_nrrID().

It returns the TBB channel ID given a NRR channel ID. Following the convention used in the LOFAR detector description as of February 2024, this simply replaces the fourth element of the channelID with a “0”.

Parameters:

channel_id: str or int

Channel ID

Returns:

tbb_channel_id: str

The TBB channelID corresponding to the NuRadioReco channelID

See also

tbbID_to_nrrID

convert TBB channel ID to NRR channel ID

class NuRadioReco.modules.io.LOFAR.readLOFARData.getLOFARtraces(tbb_h5_filename, metadata_dir, time_unix, time_ns, trace_length_nbins)

Bases: object

A class to facilitate getting traces from LOFAR TBB HDF5 Files.

This class is used internally by readLOFARData to read in LOFAR traces. Most users will want to use the readLOFARData to be able to use the

Parameters:

tbb_h5_filenamestr

The TBB (.h5) file to read in.

metadata_dirstr

The path where the metadata for the desired event are stored.

time_unix: int

Event trigger timestamp in (UNIX) seconds

time_ns: int

Event trigger timestamp in ns past UTC second

trace_length_nbinsint

Desired length of trace to be loaded from TBB HDF5 files. This does not affect trace size read-in for RFI cleaning

Methods

check_trace_quality()	Check all traces recorded from the TBB against quality requirements.
close_file()	Closes the currently opened TBB file.
get_trace(dipole_id)	Return the trace for antenna dipole_id
setup_trace_loading()	Opens the file and sets some variables.

setup_trace_loading()

Opens the file and sets some variables.

This enables get_trace to be called repeatedly for different dipoles.

check_trace_quality()

Check all traces recorded from the TBB against quality requirements.

Returns two sets. The first is a list of dipole ids failing any of the following three requirements:

1. The starting sample number deviates by more than 25% (of the trace length) from the median starting sample number;

2. The starting sample number is later than the median starting sample number;

3. The length of the recorded trace deviates by more than 10% from the median trace length.

The second set corresponds to all dipole ids for which the matching dipole (with the other polarization) is missing.

Returns:

deviating_dipolesset of str

dipoles_missing_counterpartset of str

get_trace(dipole_id)

Return the trace for antenna dipole_id

Parameters:

dipole_id: str

The dipole id

Returns:

tracenp.ndarray

The trace for antenna dipole_id

close_file()

Closes the currently opened TBB file.

class NuRadioReco.modules.io.LOFAR.readLOFARData.readLOFARData(restricted_station_set=None, tbb_directory=None, json_directory=None, metadata_directory=None)

Bases: object

This class reads in the data from the TBB files and puts them into an Event structure.

This class uses the raw TBB file, (processed) LORA json and LOFAR metadata. If the directory paths are not provided, they default to the ones on COMA.

Parameters:

restricted_station_setlist, optional

Only read in data for stations in restricted_station_set. If not provided, read in all stations for which TBB files can be found.

tbb_directory: Path-like str, default=”/vol/astro5/lofar/astro3/vhecr/lora_triggered/data/”

The path to the directory containing the TBB files.

json_directory: Path-like str, default=”/vol/astro7/lofar/kratos_files/json”

The path to the directory containing the JSON files from LORA.

metadata_directory: Path-like str, default=”/vol/astro7/lofar/vhecr/kratos/data/”

The path to the directory containing the LOFAR metadata (antenna positions and timing calibrations).

Methods

begin(event_id[, logger_level])	Prepare the reader to ingest the event with ID event_id.
get_stations()	Return the internal dictionary which contains the paths to the TBB event files and the extracted metadata per stations.
run(detector[, trace_length])	Runs the reader with the provided detector.

end

get_stations()

Return the internal dictionary which contains the paths to the TBB event files and the extracted metadata per stations.

Returns:

stationsdict

Dictionary with station names as keys and dictionaries as values, who have a files key with as value a list of TBB filepaths and a metadata key which has a list with metadata as value.

Notes

The metadata key is only set in the readLOFARData.begin function, to avoid setting it multiple times if there is more than 1 TBB file for a given station.

Metadata is a list containing (in this order):

1. station name

2. antenna set

3. tbb timestamp (unix)

4. tbb timestamp (nanoseconds after last second)

5. station clock frequency

6. positions of antennas

7. dipole IDs

8. calibration delays per dipole

begin(event_id, logger_level=0)

Prepare the reader to ingest the event with ID event_id.

This resets the internal representation of the stations as well as the event ID. The timestamps are read from the LORA JSON file corresponding to the event. The function then globs through the TBB directory to find all files corresponding to the event and adds them to the corresponding station file list. It also loads the metadata for every station.

Parameters:

event_id: int

The ID of the event to load.

logger_levelint, default=logging.NOTSET

Use this parameter to override the logging level for this module.

run(detector, trace_length=65536)

Runs the reader with the provided detector.

For every station that has files associated with it, a Station object is created together with its channels (pulled from the detector description, depending on the antenna set (LBA_OUTER/INNER)). Every channel also gets a group ID which is retrieved from the Detector description. For LOFAR we use the integer value of the even dipole number, so channels ‘001000000’ and ‘001000001’, which are the two dipoles composing one physical antenna, both get group ID 1000000.

Parameters:

detector: Detector object

The detector description to be used for this event.

trace_length: int

Desired length of the trace to be loaded from TBB files.

Yields:

evt: Event object

The event containing all the loaded traces.

Notes

For each LOFAR station, one Station with the corresponding station_id will be created, which contains the voltage traces. Note that these voltage traces are already corrected for the calibration delays, using the corresponding delays stored in the input TBB file.

Additionally, the LORA reconstruction data is stored in the HybridShower, and an (empty) RadioShower is created to store the output of additional reconstruction modules.

end()