Monitoring a flow

Once a data flow is started and processing, built-in monitoring tools are available to watch the health, performance, and behavior of the flow. These monitoring tools provide real-time information to help users maintain smooth and efficient integrations. The primary tools likely to be utilized, include:

Bulletins and Bulletin Board: Bulletins are used to communicate real-time status messages or warnings, such as processing errors or validation issues, to help users monitor and troubleshoot their flows. Bulletins appear on individual components, the status bar, and the global Bulletin Board.

• Bulletin Board: Accessed from Global menu>Monitoring, the Bulletin Board serves as the centralized message center displaying all bulletins from individual components such as status, errors, and warnings. It provides all details from the reported bullet including timestamp, severity level, and the reported message within a single screen.

• Component bulletins: Components will report bulletins based on the bulletin level configured within the component’s settings tab. When a bulletin is reported, the bullet icon (red note) is displayed on that component within the canvas. Hovering over the bullet icon will display the details including time, severity level, and reported message.

• Status bar: Tenant bulletins are displayed from the note icon on the status bar, under the Global menu. Hovering over the bullet icon in the status bar will provide a quick look at the time, severity level, and message of the bulletin.

Bulletins automatically expire after five minutes.

Summary Page: Available from the Global menu, Summary provides a high-level view and visual information on how the tenant is functioning. It provides information about each of the components on the canvas, separated by type for easy viewing. Users can review component categories to quickly see the health and status of the tenant processing.

The Connections tab within the Summary screen provides a view of all connection queues. Sorting in descending size will allow you to see any queues which have reached their capacity. Using the right-arrow, you can navigate directly to the queue for further investigation.

Processor Status History: Many processors provide statistics on their recent performance such as throughput, latency, success and failure rates, and more. The information is akin to the Summary page details but only for the processor selected. Users can view status history from the context menu, by right-clicking on the processor.

Log Files: Log files generated from the LogMessage and LogAttribute processors provide robust debugging and monitoring capabilities:

• LogMessage: Writes custom messages, allowing users to trace data flow execution and include dynamic content from FlowFile attributes.

• LogAttribute: Logs all attributes of a FlowFile, creating a comprehensive snapshot of metadata at specific points in a data flow.

This custom logging improves the ability to monitor, troubleshoot, and gain insights into data flows without disrupting actual data processing. Logs are available in a dedicated Log Files screen, accessed from Global menu>Monitoring.

To access logs:

1. Select the Monitoring option from the Global menu.

2. Select the Log Files option.

3. The default view displays all logs for the current day, broken into hourly file blocks.

4. (Optional) Search past dates (or date range) to retrieve past log files.

5. Click the file hyperlink to download the log file.

The ‘Log message’ property is required to generate a message when using the LogMessage processor. Log files are kept for 30 days.