Skip to content

refactor(ai-client-chat): optimize conversation memory handling in PromptChatMemoryAdvisor #3605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

refactor(ai-client-chat): optimize conversation memory handling in PromptChatMemoryAdvisor #3605

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions ...hat/src/main/java/org/springframework/ai/chat/client/advisor/PromptChatMemoryAdvisor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,29 +113,28 @@ public ChatClientRequest before(ChatClientRequest chatClientRequest, AdvisorChai
logger.debug("[PromptChatMemoryAdvisor.before] Memory before processing for conversationId={}: {}",
conversationId, memoryMessages);

// 2. Process memory messages as a string.
// 2. Add the new user message to the conversation memory.
UserMessage userMessage = chatClientRequest.prompt().getUserMessage();
this.chatMemory.add(conversationId, userMessage);
// 3. Check if memory is empty and return the request as is.
if (memoryMessages.isEmpty()) {
return chatClientRequest;
}
// 4. Process memory messages as a string.
String memory = memoryMessages.stream()
.filter(m -> m.getMessageType() == MessageType.USER || m.getMessageType() == MessageType.ASSISTANT)
.map(m -> m.getMessageType() + ":" + m.getText())
.collect(Collectors.joining(System.lineSeparator()));

// 3. Augment the system message.
// 5. Augment the system message.
SystemMessage systemMessage = chatClientRequest.prompt().getSystemMessage();
String augmentedSystemText = this.systemPromptTemplate
.render(Map.of("instructions", systemMessage.getText(), "memory", memory));

// 4. Create a new request with the augmented system message.
ChatClientRequest processedChatClientRequest = chatClientRequest.mutate()
// 6. Create a new request with the augmented system message.
return chatClientRequest.mutate()
.prompt(chatClientRequest.prompt().augmentSystemMessage(augmentedSystemText))
.build();

// 5. Add all user messages from the current prompt to memory (after system
// message is generated)
// 4. Add the new user message to the conversation memory.
UserMessage userMessage = processedChatClientRequest.prompt().getUserMessage();
this.chatMemory.add(conversationId, userMessage);

return processedChatClientRequest;
}

@Override
Expand Down
20 changes: 2 additions & 18 deletions ...-client-chat/src/test/java/org/springframework/ai/chat/client/ChatClientAdvisorTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,15 +97,7 @@ public void promptChatMemory() {

// Capture and verify the system message instructions
Message systemMessage = this.promptCaptor.getValue().getInstructions().get(0);
assertThat(systemMessage.getText()).isEqualToIgnoringWhitespace("""
Default system text.

Use the conversation memory from the MEMORY section to provide accurate answers.

---------------------
MEMORY:
---------------------
""");
assertThat(systemMessage.getText()).isEqualToIgnoringWhitespace("Default system text.");
assertThat(systemMessage.getMessageType()).isEqualTo(MessageType.SYSTEM);

// Capture and verify the user message instructions
Expand Down Expand Up @@ -175,15 +167,7 @@ public void streamingPromptChatMemory() {

// Capture and verify the system message instructions
Message systemMessage = this.promptCaptor.getValue().getInstructions().get(0);
assertThat(systemMessage.getText()).isEqualToIgnoringWhitespace("""
Default system text.

Use the conversation memory from the MEMORY section to provide accurate answers.

---------------------
MEMORY:
---------------------
""");
assertThat(systemMessage.getText()).isEqualToIgnoringWhitespace("Default system text.");
assertThat(systemMessage.getMessageType()).isEqualTo(MessageType.SYSTEM);

// Capture and verify the user message instructions
Expand Down