API Documentation

Logfile

PolySync uses a distributed Record and Replay system to handle system-level data logging/replay. All participating nodes use the Logfile API to facilitate the Record and Replay (RnR) functionality.

The API uses the term “session” to refer to the collection of files associated with a given logging session across the entire PolySync Runtime. Sessions are identified by their “session timestamp,” which typically corresponds to the UTC microsecond time the session was started at, see ps_rnr_session_id. A user can invoke arbitrary session time values. They are simply used to create directory names on each host that store the actual files. The default location is $PSYNC_HOME/rnr_logs.

Sessions are strongly bound to the SDF with which they were produced. The actual files are identified by the producer of the file. They are a binary file type and use the naming convention: <node_name>.<node_GUID>.plog.

Example Distributed Layout From Recorded Session ID of 123:

File Layout

  • Host A
    • rnr_base_dir/123/node-1.000000.plog
    • rnr_base_dir/123/node-2.000000.plog
  • Host B
    • rnr_base_dir/123/node-3.000000.plog
    • rnr_base_dir/123/node-4.000000.plog
  • Host C
    • rnr_base_dir/123/node-5.000000.plog

RnR commands

In the above example, if you wanted to instruct all five nodes to replay the session you would use the Record and Replay API to send commands to the nodes. These commands are provided to the Logfile API, and are handled by each node independently. Each node would receive the set mode command, with a session_id value of 123, causing them to load the actual plog file that matches their SDF key and session time.

Examples

There are six examples in the examples repo that implement the logfile API

  • logfile_iterator
    • An example that uses the Logfile API to iterate over a specified logfile and reads the contents. It then prints details from the header of each record. The iterator also allows the application to access, convert, or otherwise process the data.
  • logfile_iterator_for_velodyne
    • An example that uses the Logfile API like the logfile_iterator example except it is a concrete example for iterating over the Velodyne LiDAR logfile.
  • logfile_queue_reader
    • An example of a node that uses the Logfile API to replay a PolySync logfile using the meassage queue instead of subscribing to a high-level PolySync message type. This node accesses the raw non-abstracted plog data.
  • logfile_reader
    • An example that uses the Logfile API to open a read a PolySync logfile plog file for replay.
  • logfile_to_pcap_converter
    • An example that uses the logfile iterator method to traverse an existing plog file and convert a ps_byte_array_msg from a velodyne sensor and converts the buffer to a pcap formatted file.
  • logfile_writer
    • An example that usese the Logfile API to replay a PolySync logfile and shows how to write ps_byte_array_msg to a plog file.

Usage

#include <polysync_logfile.h>

Functions

psync_logfile_init ( )

This will initialize the logfile resources owned by a provided node. If this is the first node on the bus, for a given host, the system replay clock will be owned by the calling node (typically done by the PolySync Manager Node).

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node, which will have its logfile resources initialized.
Returns

psync_logfile_release ( )

This will release the logfile resources owned by a provided node. This should be called before psync_release.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will have its logfile resources released.
Returns

psync_logfile_get_mode ( )

This will get the current mode of the logfile.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out ps_logfile_mode_kind * const mode A pointer to ps_logfile_mode_kind that receives the mode value.
Returns

psync_logfile_set_mode ( )

This will set the desired logfile mode, possibly intermittently entering LOGFILE_MODE_OFF.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will have its mode changed.
in const ps_logfile_mode_kind mode The desired mode, either LOGFILE_MODE_OFF, LOGFILE_MODE_READ, or LOGFILE_MODE_WRITE.
in const ps_rnr_session_id session_id The session identifier associate with the mode, not used for LOGFILE_MODE_OFF.
Returns

psync_logfile_get_state ( )

This will get the current state of the logfile.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out ps_logfile_state_kind * const state A pointer to ps_logfile_state_kind that receives the state value.
Returns

psync_logfile_set_state ( )

This will set the desired logfile state given its mode, and an optional absolute start time for enabling. The start time allows a synchronized start point to be used across the PolySync domain.

The enabled state allows file writing when in MODE_WRITE, and file reading (replay) when in LOGFILE_MODE_READ. Toggling the state can be used to perform pause/resume logic.

Parameters
in/out type name description
in ps_node_ref node_ref A reference to the node that will have its state changed.
in const ps_logfile_state_kind state The desired state, either LOGFILE_STATE_DISABLED, or LOGFILE_STATE_ENABLED.
in const ps_timestamp start_time The absolute time to perform the enable state operation. [UTC timestamp]
Returns

psync_logfile_write_message ( )

This will write a PolySync message to a file using the logfile interface. The message is written to a file using the message header timestamp as its timestamp. The message is silently ignored if the logfile state is not enabled.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will write the message to its logfile.
in const ps_msg_ref message The message reference.
Returns

psync_logfile_enable_output_queue ( )

This will enable the use of the logfile replay message queue. The replay message queue provides an asynchronous message queue to logfile users’ application space. The queue gets all replay message data in replay-time from the replay thread, regardless of enabled publisher filters. The user is responsible for calling psync_message_free on the queue data.

Access to the queue reference is provided by psync_logfile_get_replay_msg_queue.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will have its logfile output queue enabled.
in const unsigned int enabled A flag indicating if enabled. A value of zero means disabled.
Returns

psync_logfile_get_replay_msg_queue ( )

This will get a reference to the logfile replay output message queue.

Use of the output queue is provided by psync_logfile_enable_output_queue.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out GAsyncQueue ** queue_ptr A pointer to the GAsyncQueue pointer that receives the queue reference.
Returns

psync_logfile_fill_rnr_status_message ( )

This will fill a ps_rnr_msg with the current logfile data.

The function sets the type field of ps_rnr_msg to RNR_MSG_STATUS, and fills the remaining message fields based on the current logfile mode and state.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node that will have its status copied to the ps_rnr_msg.
out ps_rnr_msg * const message A pointer to ps_rnr_msg that receives the logfile status data.
Returns

psync_logfile_get_replay_clock_owner ( )

This will get the replay clock owner process identifier and node GUID.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out ps_guid * const owner_guid A pointer to ps_guid that receives the owner process ID.
out ps_identifier * const owner_pid A pointer to ps_guid that receives the owner node GUID.
Returns

psync_logfile_get_replay_clock_scaler ( )

This will get the logfile replay clock scale on the current host.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out double * const scale A pointer to a double that receives the replay clock scale value.
Returns

psync_logfile_get_thread_status ( )

This will perform a status check on any/all logfile threads. Error messages will be generated if issues are detected.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node that will have its logfile threads’ statuses checked.
Returns

psync_logfile_handle_rnr_command()

This will process the RnR commands using the logfile API routines based on the type field of ps_rnr_msg.

This does not do any filtering based on the guid field of ps_rnr_msg.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node that will handle the RnR command message.
out const ps_rnr_msg * const message A pointer to ps_rnr_msg that specifies the possible command data.
Returns

psync_logfile_reader_get_eof_status ( )

This will get the logfile reader end-of-file status.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out unsigned int * eof_reached A pointer to unsigned int that receives the reader end-of-file status.
Returns

psync_logfile_reader_get_file_attributes()

This will get the logfile reader file attributes.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node.
out ps_rnr_logfile_attributes * const file_attributes A pointer to ps_rnr_logfile_attributes that receives the file attributes.
Returns

psync_logfile_set_message_type_filters ( )

This will set the writer/reader message type filters. If writer/reader lists are NULL, then the corresponding filter is disabled.

If the writer filter is enabled, the logfile writer will ignore writing messages with types that match the filter.

If the reader filter is enabled, the logfile reader will not publish messages with types that match the filter. The replay message queue will still get all the messages if it is enabled.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will have its logfile message filters set.
in ps_msg_type * const writer_disabled_type A pointer to ps_msg_type that specifies the array of message types to filter in the writer.
in const unsigned long writer_disabled_types_len The number of elements in the writer filter list.
in ps_msg_type * const reader_disabled_types A pointer to ps_msg_type that specifies the array of message types to filter in the reader.
in const unsigned long reader_disabled_types_len The number of elements in the reader filter list.
Returns

psync_logfile_reader_reset_to_beginning()

This will disable the logfile state, and seeks to the beginning of the logfile. It preserves the last known start deviation, and/or NCRT value.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node that will have its reader reset to the beginning.
Returns

psync_logfile_reader_seek_to()

This will disable the logfile state, and seeks to the closest point in the logfile that does not exceed the supplied timestamp.

Parameters
in/out type name description
in const ps_node_ref node_ref Reference to the node that will have its logfile reader seek to the given time.
in const ps_timestamp timestamp The timestamp to seek to in the logfile. [microseconds]
Returns

psync_logfile_set_file_path ( )

This will override the automatic file path generation logic with a supplied file path. If NULL, logfile will enable the use of the automatic generation logic.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will set its logfile path.
in const char * file_path A pointer to char buffer that specifies the file path string to use. If NULL, override to disabled.
Returns

psync_logfile_set_replay_clock_scaler ( )

This wills set the replay clock scale on the current host.

1.0 means no effect, Less than 1.0 means to slow down, and greater than 1.0 means to speed up.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will set the replay speed on the system.
in const double scale The replay clock scale value.
Returns

psync_logfile_foreach_iterator()

This will iterate over data in a PolySync logfile ( *.plog ).

The psync_logfile_iterator_callback will be called for each ps_rnr_log_record in the logfile.

Parameters
in/out type name description
in ps_node_ref node_ref Reference to the node that will provide the available data model support.
in const char * const file_path A pointer to the char array that specifies the path to the logfile.
in const psync_logfile_iterator_callback callback User callback function invoked for each ps_rnr_log_record in the logfile.
in void * const user_data A pointer to arbitrary user data to be provided to the callback function. NULL is allowed.
Returns

Typedefs

psync_logfile_iterator_callback

This is a convenient typedef of the function signature required for callback function provided to psync_logfile_foreach_iterator.

Parameters
in/out type description
in const ps_rnr_logfile_attributes * const A pointer to the attributes of the logfile being iterated.
in const ps_msg_type Message type identifier for the message in the data field of the ps_rnr_log_record.
in const ps_rnr_log_record * const The current record in the iteration.
in void * A pointer to the user data provided in psync_logfile_foreach_iterator.
Returns
  • void

Structures

ps_rnr_log_module

Log header module data.

Type Name Description
uint8_t version_major Version major.
uint8_t version_minor Version minor.
uint16_t version_subminor Version subminor.
uint32_t build_date Build date.
uint8_t build_hash PSYNC_MODULE_VERIFY_HASH_LEN Build md5 hash.
uint16_t name_len Size of ps_rnr_log_module.name_data
uint8_t *name_data Name of module, does not include a terminating character.

ps_rnr_type_support

Log file header type support data.

Type Name Description
uint32_t type Message type identifier.
uint16_t name_len Size of ps_rnr_type_support.name_data
uint8_t *name_data Name of type, does not include a terminating character.

ps_rnr_log_header

Log file header contents

Type Name Description
uint8_t version_major API build version major.
uint8_t version_minor API build version minor.
uint16_t version_subminor API build version subminor.
uint32_t build_date API build date. [UTC seconds]
uint64_t node_guid GUID of the producer.
uint32_t num_modules Number of elements in the ps_rnr_log_header.modules list.
ps_rnr_log_module *modules List of modules.
uint32_t num_type_supports Number of elements in the ps_rnr_log_header.type_supports list.
ps_rnr_type_support *type_supports List of type supports.

ps_rnr_log_record

Log file record entry.

Type Name Description
uint32_t index Index of this record in the log file.
uint32_t size Size of
uint32_t prev_size Size of previous ps_rnr_log_record.data field.
uint64_t timestamp Copy of the message timestamp that is contained in the data field.
void *data Pointer to record data, treat this like a ps_msg_ref type.

ps_logfile_attributes

Logfile attributes.

Type Name Description
ps_rnr_log_header log_header RnR header data.
ps_timestamp start_time First timestamp in the file.
ps_timestamp end_time Last timestamp in the file.
ps_timestamp duration File duration. [microseconds]
unsigned long file_size File size. [bytes]
unsigned long data_offset Offset of the first record in the file. [bytes]
unsigned long data_count Number of record entries in the file.
unsigned long avg_data_size Average data size. [bytes]