Add `model_context` to `SelectorGroupChat` for enhanced speaker selection #6330

Ethan0456 · 2025-04-17T17:47:55Z

Why are these changes needed?

This PR enhances the SelectorGroupChat class by introducing a new model_context parameter to support more context-aware speaker selection.

Changes

Added a model_context: ChatCompletionContext | None parameter to SelectorGroupChat.
Defaulted to UnboundedChatCompletionContext when None is provided like AssistantAgent.
Updated _select_speaker to prepend context messages from model_context to the main thread history.
Refactored history construction into a helper method construct_message_history.

Related issue number

Closes Issue #6301, enabling the group chat manager to utilize model_context for richer, more informed speaker selection decisions.

Checks

I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md to build and test documentation locally.
I've added tests (if relevant) corresponding to the changes introduced in this PR.
I've made sure all auto checks have passed.

…ker selection (microsoft#6301) Signed-off-by: Abhijeetsingh Meena <abhijeet040403@gmail.com>

Ethan0456 · 2025-04-21T16:41:46Z

Hi @ekzhu,

I’ve made some changes to use messages from model_context for speaker selection. For now, BufferedChatCompletionContext with a buffer size of 5 is set as the default for testing.

Would really appreciate any feedback on the approach — also curious which context class you'd prefer as the default.

ekzhu · 2025-04-22T01:02:29Z

...n/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_selector_group_chat.py

+            self._model_context = model_context
+        else:
+            # TODO: finalize the best default context class
+            self._model_context = BufferedChatCompletionContext(buffer_size=5)


Let's use UnboundedChatCompletionContext

Done! changed it to use UnboundedChatCompletionContext

ekzhu · 2025-04-22T01:05:13Z

...n/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_selector_group_chat.py

    async def select_speaker(self, thread: List[BaseAgentEvent | BaseChatMessage]) -> str:
        """Selects the next speaker in a group chat using a ChatCompletion client,
        with the selector function as override if it returns a speaker name.

        A key assumption is that the agent type is the same as the topic type, which we use as the agent name.
        """
+        # TODO: A hacky solution - Update model context from _message_thread at every speaker selection
+        # Add last BaseChatMessage to model context


Split the part of handle_agent_response in the base class that deal with updating message thread into a separate method: update_message_thread, which updates the message thread.

autogen/python/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_base_group_chat_manager.py

Lines 145 to 153 in d051da5

# Append the message to the message thread and construct the delta.

delta: List[BaseAgentEvent | BaseChatMessage] = []

if message.agent_response.inner_messages is not None:

for inner_message in message.agent_response.inner_messages:

self._message_thread.append(inner_message)

delta.append(inner_message)

self._message_thread.append(message.agent_response.chat_message)

delta.append(message.agent_response.chat_message)

Then, in SelectorGroupChatManager, override this method to also update the message context.

Same with handle_start, we also need to update model context there.

autogen/python/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_base_group_chat_manager.py

Lines 118 to 119 in d051da5

self._message_thread.extend(message.messages)

Done!

Created a update_message_thread method in BaseGroupChatManager to update the message thread and overrode it in SelectorGroupChatManager to update the model_context

ekzhu · 2025-04-22T04:50:18Z

@Ethan0456 I realized that #6350 may be doing a similar thing to this PR but from message thread point of view. Let's pause this PR for now and let's see if we can address context size problem using #6350 first.

SongChiYoung · 2025-04-22T10:19:32Z

@ekzhu @Ethan0456

As an AutoGen user who has been eagerly looking forward to this PR, I wanted to share my thoughts in detail. It's a bit long, but I hope it's clear. I would appreciate any feedback after reading.

Community Need

Based on ongoing community feedback, I believe there is a clear need for internal message summarization and management functionality within SelectorGroupChat. This has been raised repeatedly in Discord, Discussions (especially #6347), and even in Help channels with similar requests.

Personal Use Case

That said, I’m sharing my perspective here not as a contributor, but as a user who practically needs this functionality.

Limitations of #6350

While #6350 does address a similar issue, its TTL cutoff approach simply limits the number of messages. This doesn’t quite meet the need for summarizing or selectively preserving internal messages.

Specifically, in the case of SelectorGroupChat, TTL cutoff could potentially remove critical messages, including the initial user request, which raises a concern that the selector might lose context and misidentify the next agent. I am concerned that TTL alone may not address this effectively.

Why model_context Works Better for Me

The model_context-based approach proposed in this PR, particularly using HeadAndTailChatCompletionContext, allows for reliably preserving both the initial and most recent messages. This ensures that SelectorGroupChat can always reference the original user intent when choosing the next speaker, which is essential for the use cases I face. Achieving this kind of context preservation through a simple TTL mechanism seems difficult.

Concern About Expanding #6350 Scope

If #6350 were to expand beyond TTL cutoff into more complex message preservation or summarization, it might blur the responsibility between simple message cleanup and full history management. This could make the purpose of each mechanism less clear.

Conclusion

Therefore, I personally see #6350 as a clean and focused solution for trimming unnecessary messages, and I’m very supportive of that contribution moving forward. However, this PR enables more precise conversation flow control through internal message summarization and history context management, and it’s something I was also looking forward to seeing merged.

I believe the two are not in conflict—they solve different problems and can complement each other well.

Additional Note

AutoGen’s model_context structure is already designed to allow users to customize message management without requiring external extensions. That said, tools like the community extension autogen-contextplus (which I contributed to) or future model_context improvements could make history management within SelectorGroupChat even more flexible and powerful.

Ethan0456 · 2025-04-22T15:54:54Z

Hi @ekzhu, @SongChiYoung,

I also believe that model_context offers more flexibility in this scenario, particularly when it comes to controlling the tokens and the structure of message history used for speaker selection.

A Hypothetical Example

For example, a (hypothetical) workflow—similar to what @SongChiYoung described—could involve maintaining a list of "user query" -> ["task result" or "unsuccessful attempt + reflection"] entries inside the model_context. This kind of structured memory can help influence speaker selection in a more intentional and context-sensitive way, rather than just relying on the most recent n messages.

This approach may not be achievable with the current design proposed in PR #6350—not to say that the PR isn't useful, but rather that it targets a different problem space.

Another workflow where model_context could be especially beneficial is the following:

Hypothesis-Driven Agent Collaboration

Scenario: You're orchestrating a team of LLM agents, each responsible for a different stage of scientific reasoning—such as hypothesis generation, experiment design, result analysis, and reflection.

Why not traditional last_n_messages?
In such a setup, relying solely on the most recent messages can omit critical information, like earlier hypotheses or failed experiments, which might be essential for driving the next step of reasoning.

How model_context helps?
Instead of a linear transcript, model_context can maintain a structured list of "hypothesis" -> "attempt" -> "failure reason" triples. This richer form of context allows the SelectorGroupChat to select agents like ReflectionAgent to evaluate past attempts holistically and make informed decisions.

This enables goal-aware, context-rich memory selection, compared to a more straightforward time-based truncation approach, like the one proposed in PR #6350.

Would love to hear your thoughts on this!

ekzhu · 2025-04-22T16:27:11Z

@Ethan0456 @SongChiYoung Great points made. Let's resume work here.

There are many complaints about SelectorGroupChat, we can try to improve it here.

- Added `update_message_thread` method in `BaseGroupChatManager` to manage message thread updates. - Replaced direct `_message_thread` modifications with calls to this method. - Overrode `update_message_thread` in `SelectorGroupChat` to also update the `model_context`. Signed-off-by: Abhijeetsingh Meena <abhijeet040403@gmail.com>

ekzhu · 2025-04-22T20:35:53Z

...ckages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_base_group_chat_manager.py

                    delta.append(inner_message)
-            self._message_thread.append(message.agent_response.chat_message)
+            await self.update_message_thread([message.agent_response.chat_message])


To simplify, you can just call await self.update_message_thread(delta) after this?

Done! now message thread is updated only once with delta

ekzhu

Let's add some unit tests to show that model context is being managed, validate it using ReplayChatCompletionClient which records the calls.

Ethan0456 · 2025-04-23T16:30:07Z

Hi @ekzhu,

I've updated the code based on your suggestion and added a unit test to validate the selector group chat with model context.

Please let me know if you have any additional suggestions for improvement.

ekzhu

Can you resolve the merge conflict with the main branch?

ekzhu · 2025-04-23T18:45:27Z

...n/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_selector_group_chat.py

        else:
            agent_name = participants[0]
        self._previous_speaker = agent_name
        trace_logger.debug(f"Selected speaker: {agent_name}")
        return agent_name

-    async def _select_speaker(self, roles: str, participants: List[str], history: str, max_attempts: int) -> str:
+    def construct_message_history(
+        self, message_history: Sequence[Union[BaseChatMessage, BaseAgentEvent, UserMessage, AssistantMessage]]


At this point, there shouldn't be any BaseChatMessage or BaseAgentEvent in the input list right?

ekzhu · 2025-04-23T18:47:16Z

...n/packages/autogen-agentchat/src/autogen_agentchat/teams/_group_chat/_selector_group_chat.py

@@ -453,6 +502,7 @@ def __init__(
        candidate_func: Optional[CandidateFuncType] = None,
        custom_message_types: List[type[BaseAgentEvent | BaseChatMessage]] | None = None,
        emit_team_events: bool = False,
+        model_context: ChatCompletionContext | None = None,


We need to update the API doc (argument list) and include a code example of using a custom model context.

ekzhu · 2025-04-23T18:47:53Z

python/packages/autogen-agentchat/tests/test_group_chat.py

+    agent2 = AssistantAgent("agent2", model_client=agent_two_model_client, description="Assistant agent 2")
+
+    termination = TextMentionTermination("TERMINATE")
+    team = SelectorGroupChat(


I don't see the model context is being customized here?

Add model_context parameter to SelectorGroupChat for dynamic spea…

352cc80

…ker selection (microsoft#6301) Signed-off-by: Abhijeetsingh Meena <abhijeet040403@gmail.com>

Ethan0456 marked this pull request as ready for review April 17, 2025 17:48

Use model_context to select next speaker

d839ec3

ekzhu reviewed Apr 22, 2025

View reviewed changes

Ethan0456 added 2 commits April 22, 2025 22:44

Use UnboundedChatCompletionContext as default chat completion context

b37ec18

ekzhu reviewed Apr 22, 2025

View reviewed changes

Ethan0456 added 2 commits April 23, 2025 21:52

Update message thread in one call using delta

3bc42fe

Add unit test for in SelectorGroupChat

35e1da9

ekzhu requested changes Apr 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `model_context` to `SelectorGroupChat` for enhanced speaker selection #6330

Add `model_context` to `SelectorGroupChat` for enhanced speaker selection #6330

Ethan0456 commented Apr 17, 2025 •

edited

Loading

Ethan0456 commented Apr 21, 2025 •

edited

Loading

ekzhu Apr 22, 2025

Ethan0456 Apr 22, 2025

ekzhu Apr 22, 2025

ekzhu Apr 22, 2025

Ethan0456 Apr 22, 2025

ekzhu commented Apr 22, 2025

SongChiYoung commented Apr 22, 2025

Ethan0456 commented Apr 22, 2025 •

edited

Loading

ekzhu commented Apr 22, 2025

ekzhu Apr 22, 2025

Ethan0456 Apr 23, 2025

ekzhu left a comment

Ethan0456 commented Apr 23, 2025

ekzhu left a comment

ekzhu Apr 23, 2025

ekzhu Apr 23, 2025

ekzhu Apr 23, 2025

	# Append the message to the message thread and construct the delta.
	delta: List[BaseAgentEvent \| BaseChatMessage] = []
	if message.agent_response.inner_messages is not None:
	for inner_message in message.agent_response.inner_messages:
	self._message_thread.append(inner_message)
	delta.append(inner_message)
	self._message_thread.append(message.agent_response.chat_message)
	delta.append(message.agent_response.chat_message)

Add model_context to SelectorGroupChat for enhanced speaker selection #6330

Are you sure you want to change the base?

Add model_context to SelectorGroupChat for enhanced speaker selection #6330

Conversation

Ethan0456 commented Apr 17, 2025 • edited Loading

Why are these changes needed?

Changes

Related issue number

Checks

Ethan0456 commented Apr 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekzhu commented Apr 22, 2025

SongChiYoung commented Apr 22, 2025

Community Need

Personal Use Case

Limitations of #6350

Why model_context Works Better for Me

Concern About Expanding #6350 Scope

Conclusion

Additional Note

Ethan0456 commented Apr 22, 2025 • edited Loading

A Hypothetical Example

Hypothesis-Driven Agent Collaboration

ekzhu commented Apr 22, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ekzhu left a comment

Choose a reason for hiding this comment

Ethan0456 commented Apr 23, 2025

ekzhu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add `model_context` to `SelectorGroupChat` for enhanced speaker selection #6330

Add `model_context` to `SelectorGroupChat` for enhanced speaker selection #6330

Ethan0456 commented Apr 17, 2025 •

edited

Loading

Ethan0456 commented Apr 21, 2025 •

edited

Loading

Ethan0456 commented Apr 22, 2025 •

edited

Loading