This document describes the log file abstraction used within the ORT server to allow the download of log files from external logging systems.

To diagnose problems with ORT runs or other subsystems of ORT Server, it is essential to have access to the logs generated by the affected components. For complex deployments of ORT Server, it is therefore expected that an external log management and aggregation tool is used to have logs over a longer timeframe available with sophisticated search and filter capabilities.

While this functionality and tooling is typically used by operations, it is not necessarily available to end users of ORT Server who trigger ORT runs on their repositories. Those users need access to the logs for their own runs on a reasonable level of detail. This information is available in the logging system, but users need an easy way to obtain it, which does not require to log into another system and to learn the syntax of search queries.

This use case is handled by the log file abstraction. It defines an interface that allows downloading log files for specific ORT runs and workers from an external log management system. ORT Server offers an endpoint that exposes this functionality, so users can retrieve their log files via the API. Behind the scenes, the concrete implementation in use fetches the data from an external system or whatever other source.

This section describes the interfaces that need to be implemented to integrate a source of log information with ORT Server.

The interaction with the log source is done via the LogFileProvider interface. To reduce the effort required for a concrete implementation, the interface defines a single method only to retrieve a log file for a specific ORT run and a single worker. The function expects the following parameters:

• The ID of the affected ORT run.
• A constant defining the worker for which the logs are to be retrieved.
• A set of log levels to include in the result.
• The time range for which logs are to be fetched. This is intended as a hint for an implementation. Some log systems may need a time range to perform efficient queries. Since the caller has access to the ORT Server database, it is straight-forward to obtain the start and end time for the affected ORT run.

Since log data can become large, especially when including the level DEBUG, it makes sense to use a single worker as granularity for retrieving data. The logic to bundle the logs of a whole run can then be implemented by the caller.

For the log sources (i.e., the workers) of a run and the log levels, enum classes are defined. Filtering based on a log level typically means that all logs of this level or higher levels should be included. This logic does not have to be implemented by the log file provider, as it is passed the complete set of levels to retrieve; so it can do a strict comparison of levels.

An implementation is free to throw arbitrary exceptions if something goes wrong. They are caught by the abstraction and wrapped into a standard exception.

The abstraction defines a typical factory interface for creating LogFileProvider instances based on the service loader mechanism: LogFileProviderFactory.

The factory function is passed a ConfigManager as parameter. So the provider instance can be configured for the external system it has to access. Credentials that may be required to interact with the system can be obtained from the ConfigManager as well.

With LogFileService, the Log File Abstraction provides a facade class that takes care of the creation of the underlying LogFileProvider and offers advanced functionality.

While LogFileProvider supports downloading only a single log file at once, LogFileService can be queried for a set of log sources for an ORT run. So, the logs of all workers could be retrieved in a single step. The downloaded log files are automatically added to a Zip archive, which can then be sent to the caller.

When creating a LogFileService instance via the static create() factory function, a ConfigManager object has to be provided. The configuration wrapped in this object must have a section named logFileService. This section defines the provider implementation to be used in a property named name. LogFileService reads this property and then searches on the classpath for a LogFileProviderFactory implementation with this name. The sub config manager for the logFileService section is then passed to the matched factory; thus it can contain additional properties to be evaluated by the concrete factory implementation. The listing below shows an example configuration:

logFileService {
  name = loki
  url = https://loki.example.org/
  ...
}