Guide for missing suggestions

Prerequisites

Check your VPN connection. Verify Agent Configuration and make sure if:

• textForwarding is enabled for all plugins

• set confidenceSuggestionThreshold to 0.1 (By default its 0.6)

• set fileTextEventDebounceMilis to 3000 (By default its 15000, lowering it will give faster suggestions)

Brief architecture overview

When troubleshooting missing suggestions, it's essential to understand the sequence of services that play a role in the suggestion generation process. This sequence involves a series of steps that a request passes through, including:

• agent: Desktop app. The initial component that handles user requests or queries. Agent communicates with agent-edge through gRPC. (responsible: delta team)

• agent-Edge: A back-end proxy service that serve as an intermediary layer between the agent and other back-end components. The agent-edge talks to the text-classification-pipeline using Kafka. (responsible: delta team or any Java BE engineer)

• text-classification-pipeline: A critical part of the process that performs various tasks, including data extraction, transformation and api calls. The text-classification-pipeline talks to the classifier using pure HTTP. (responsible: OT team or any Java BE engineer)

• classifier: Responsible for classifying or identifying relevant suggestions based on user input. (data-science team)

Requests are processed sequentially through these services. If missing suggestions are observed, the root cause of the issue can often be found somewhere along this sequence.

Localizing the Problem

Extracting Kafka Event Identifier

To troubleshoot and localize the problem regarding missing suggestions, we'll begin by inspecting the Kafka event associated with the classification process. Follow these steps to access the relevant Kafka event:

1. To initialize the classification process, start by opening Microsoft Word and writing a sample text.

2. Connect to Kafka UI Using kubectl: Ensure you have kubectl configured and access to your Kubernetes cluster. Use the following command to connect to the Kafka UI.

   kubectl port-forward svc/gv-kafka-ui 8080:80 --kubeconfig <path-to-kubeconfig-yaml>

   This command will create a local port forwarding to the Kafka UI.

3. Navigate through Kafka topics and find the Text topic: Open your web browser and go to http://localhost:8080. This will take you to the Kafka UI web interface. In the Kafka UI, navigate to the topic where text messages related to the classification process are stored. This topic may have a name related to the process or content classification. Once you've located the appropriate topic, click on it to view the list of messages stored within

4. Locate the message with the Text you wrote: Browse through the list of messages in the Kafka topic and find the message that corresponds to the text you wrote in Microsoft Word. This message will typically contain metadata and content related to the classification process. Within the selected Kafka message, look for an identifier (id field) in the event. This ID is crucial for tracking and troubleshooting the specific event related to the missing suggestions.

Finalization

Now that you have the ID from the Kafka event associated with the text classification process, you can proceed to find the corresponding event in the "text-classification-results" Kafka topic.

1. Find the event in "text-classification-results" Kafka topic: Access the "text-classification-results" Kafka topic where the processed classification events are stored. Use the ID you obtained from the Kafka event associated with the text you wrote in Microsoft Word to search for the corresponding event. This ID doesn’t serve as a unique identifier for the event, you need to find the closest event by time.

2. If the event is NOT found. The root cause of the missing suggestions likely lies within the text-classification-pipeline. Check flink-taskmanager logs and inform responsible team.

3. If the event is found:

   1. If the successCode is a non-zero value. In such cases, the problem lies within the classifier component. Inform responsible team.

   2. Otherwise, it suggests that the classification process successfully processed the text, and the issue might be further downstream in the process.

4. Access agent-edge logs Within the agent-edge logs, perform a search using the extracted ID as the search criteria. Once you've located a log entry that corresponds to the extracted ID, examine the content of the entry. Look for an entry that resembles the following format:

   [<ID>] Classification received. With key: {}. partition: {}

   1. If no matching entry is found: First you have to wait a minute, maybe classification is not ready yet. If you cannot find log entries in the agent-edge logs that match the specified format or if there are errors or anomalies in the entries, it suggests that the issue might be related to the agent-edge component. In such cases, you should investigate the agent-edge logs further to identify potential issues and errors that may be affecting the classification process.

   2. If you find log entries in the agent-edge logs with the specified format and there are no exceptions in logs, it indicates that the agent-edge successfully received the classification event. In this case, the agent-edge component appears to be functioning correctly.

5. The root cause lies within the agent or it’s more complex configuration issue.