Creating a Discussion

The simplest task you can accomplish with this library is to create a small discussion between LLMs.

This guide will teach you the basic setup of the library. You will understand how to setup models, user-agents and how to coordinate them in a discussion. By the end of htis guide, you will be able to run a small discussion with a moderator and save it to the disk for persistence.

Basics

The Model

SynDisco can theoretically support any LLM, as long as it is wrapped in a BaseModel wrapper. The BaseModel class is a very simple interface with one method. This method gives the underlying LLM input, and returns its output to the library.

There already exists a TransformersModel class which handles models from the transformers python library. In 90% of your applications, this will be enough. We can load a TransformersModel using the following code:

[1]:

from syndisco.backend.model import TransformersModel

llm = TransformersModel(
    model_path="unsloth/Meta-Llama-3.1-8B-Instruct",
    name="test_model",
    max_out_tokens=100,
)

/media/SSD_2TB/dtsirmpas_data/software/miniconda3/envs/syndisco/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Fetching 4 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [02:25<00:00, 36.49s/it]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:11<00:00,  2.75s/it]
Device set to use cuda:0

This will download a small LLM from huggingface. You can substitute the model_path for any similar model in HuggingFace supporting the Transformers library.

Creating personas

All actors can be defined by a persona, aka a set of attributes that define them. These attributes can be age, ethnicity, and even include special instructions on how they should behave.

Creating a persona programmatically is simple:

[2]:

from syndisco.backend.persona import LLMPersona

persona_data = [
    {
        "username": "Emma35",
        "age": 38,
        "sex": "female",
        "education_level": "Bachelor's",
        "sexual_orientation": "Heterosexual",
        "demographic_group": "Latino",
        "current_employment": "Registered Nurse",
        "special_instructions": "",
        "personality_characteristics": [
            "compassionate",
            "patient",
            "diligent",
            "overwhelmed",
        ],
    },
    {
        "username": "Giannis",
        "age": 21,
        "sex": "male",
        "education_level": "College",
        "sexual_orientation": "Pansexual",
        "demographic_group": "White",
        "current_employment": "Game Developer",
        "special_instructions": "",
        "personality_characteristics": [
            "strategic",
            "meticulous",
            "nerdy",
            "hyper-focused",
        ],
    },
]

personas = [LLMPersona(**data) for data in persona_data]

for persona in personas:
    print(persona)

LLMPersona(username='Emma35', age=38, sex='female', sexual_orientation='Heterosexual', demographic_group='Latino', current_employment='Registered Nurse', education_level="Bachelor's", special_instructions='', personality_characteristics=['compassionate', 'patient', 'diligent', 'overwhelmed'])
LLMPersona(username='Giannis', age=21, sex='male', sexual_orientation='Pansexual', demographic_group='White', current_employment='Game Developer', education_level='College', special_instructions='', personality_characteristics=['strategic', 'meticulous', 'nerdy', 'hyper-focused'])

Since creating a lot of distinct users is essential in running large-scale experiments, users are usually defined in JSON format. That way, you can change anything without touching your code!

Here is an applied example of how to mass-define user personas through JSON files. The LlmPersona class provides a method (LlmPersona.from_json_file()) which handles the IO and unpacking operations for you!

Creating the user-agents

Having a persona and a model we can finally create an actor. The actor will personify the selected persona using the model to talk.

Besides a persona and a model, the actors will also need instructions and a context. By convention, all actors share the same context, and all user-agents share the same instructions. Personalized instructions are defined in the actor’s persona.

[3]:

from syndisco.backend.actors import LLMActor, ActorType


CONTEXT = "You are taking part in an online conversation"
INSTRUCTIONS = "Act like a human would"

actors = [
    LLMActor(
        model=llm,
        name=p.username,
        attributes=p.to_attribute_list(),
        context=CONTEXT,
        instructions=INSTRUCTIONS,
        actor_type=ActorType.USER,
    )
    for p in personas
]

Managing turn-taking

In real-life discussions, who gets to speak at each point in time is determined by complex social dynamics, which are difficult to realistically model. However, there are ways with which we can simulate these dynamics.

SynDisco uses the TurnManager class to model turn taking. Two implementations are available by default: Round Robin, and Random Weighted

  • Round Robin is the simplest, most intuitive way to model turn-taking; everyone gets to talk once per round. Once everyone talks once, they get to talk again in the same sequence

[4]:

from syndisco.backend.turn_manager import RoundRobin


rrobin_turn_manager = RoundRobin(["John", "Mary", "Howard", "Todd"])

for i in range(10):
    print(next(rrobin_turn_manager))

John
Mary
Howard
Todd
John
Mary
Howard
Todd
John
Mary

  • RandomWeighted on the other hand throws a weighted coin on each round. If the coin flip succedes, the previous user gets to respond. If not, another user is selected at random

[5]:

from syndisco.backend.turn_manager import RandomWeighted


rweighted_turn_manager = RandomWeighted(
    names=["John", "Mary", "Howard", "Todd"], p_respond=0.5
)

for i in range(10):
    print(next(rweighted_turn_manager))

Howard
John
Mary
John
Howard
John
Howard
Todd
Howard
John

Generating a discussion

Let’s start with the most basic task; a single discussion between two user-agents.

Since we only have two users, a RoundRobin approach where each user takes a turn sequentially is sufficient.

Now we can run a simple discussion

[6]:

from syndisco.jobs import Discussion


turn_manager = RoundRobin([actor.name for actor in actors])
conv = Discussion(next_turn_manager=turn_manager, users=actors)
conv.begin()

User Emma35 posted:
Hey everyone, I'm Emma35, nice to meet you all. Just got back from a
long shift at the hospital and I'm exhausted. Anyone else have a tough
day at work?

User Giannis posted:
Hey Emma35, sorry to hear you had a tough day at the hospital. I can
relate, though - I've had my fair share of long days working on game
development projects. Sometimes it feels like I'm stuck in a never-
ending loop of debugging and coding, but it's worth it in the end when
I see the game come together. What kind of work do you do at the
hospital, if you don't mind me asking?

User Emma35 posted:
I don't mind at all, Giannis. I'm a registered nurse, so I work in the
medical field. It can be emotionally and physically draining at times,
but it's so rewarding to see my patients recover and get back on their
feet. I've been working in pediatrics for the past 5 years, and it's
an absolute joy to work with kids and their families. They always
bring a smile to my face, even on the toughest days. How about you,
what kind

User Giannis posted:
That's amazing, Emma35, working in pediatrics can be incredibly
rewarding. I'm sure it's not always easy, but it's great that you find
joy in it. As for me, I'm a game developer, which can be a bit of a
niche field, but I love it. I'm currently working on a project that
combines my passion for strategy and problem-solving with my love for
gaming. I'm designing a turn-based RPG with a unique combat system and
a rich storyline.

User Emma35 posted:
That sounds like a really cool project, Giannis. I've always been a
bit of a gamer myself, but I have to admit, I don't have as much time
for it as I'd like. Between work and taking care of my own family,
it's hard to find the time. But I'm always excited to hear about new
game developments and ideas. What inspired you to create a turn-based
RPG? Was there a particular game that sparked the idea for this
project?

Let’s add a moderator to oversee the discussion.

[7]:

MODERATOR_INSTRUCTIONS = "You are a moderator. Oversee the discussion"

moderator_persona = LLMPersona(
    **{
        "username": "Moderator",
        "age": 41,
        "sex": "male",
        "education_level": "PhD",
        "sexual_orientation": "Pansexual",
        "demographic_group": "White",
        "current_employment": "Moderator",
        "special_instructions": "",
        "personality_characteristics": [
            "strict",
            "neutral",
            "just",
        ],
    }
)

moderator = LLMActor(
    model=llm,
    name=moderator_persona.username,
    attributes=moderator_persona.to_attribute_list(),
    context=CONTEXT,
    instructions=MODERATOR_INSTRUCTIONS,
    actor_type=ActorType.USER,
)


# remember to update this!
turn_manager = RoundRobin([actor.name for actor in actors] + [moderator.name])
conv = Discussion(
    next_turn_manager=turn_manager,
    users=actors + [moderator],
    moderator=moderator,
)
conv.begin()

User Emma35 posted:
Not yet, I'm waiting for you to post something.

User Moderator posted:
Welcome to our community discussion forum, Emma35. I'm glad you're
here. Before we begin, I'd like to outline the ground rules for our
conversation. As the moderator, it's my duty to ensure that all
participants feel comfortable and respected. Let's keep the discussion
civil and focused on the topic at hand. I'll do my best to keep the
conversation flowing smoothly and address any questions or concerns
you may have.  To get us started, can you tell me a bit about what you

User Giannis posted:
User Giannis posted: Hey everyone, thanks for the warm welcome,
Moderator. I'm excited to be here and contribute to the discussion. As
for me, I'm Giannis, a 21-year-old game developer with a passion for
creating immersive gaming experiences. I'm currently working on my own
indie game project, and I'm always looking for feedback and new ideas
to improve it. I'm a bit of a nerd, so I'm excited to geek out with
all of you and share my

User Moderator posted:
Welcome, Giannis. It's great to have you on board. Your enthusiasm for
game development is infectious. Emma35, it seems like we have two new
participants to introduce themselves. Let's take turns sharing a bit
about ourselves. Emma35, would you like to go next?


You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset

User Moderator posted:
User Emma35 posted: I'm Emma35, nice to meet you both. I'm a 35-year-
old freelance writer,  specializing in science fiction and fantasy
genres. I've been writing for  about 10 years now, and I'm always
looking for new ideas and  collaboration opportunities. I'm excited to
be here and learn from  everyone's experiences.

User Moderator posted:
Welcome, Emma35. It's great to have you here. It sounds like you have
a rich background in writing, and I'm sure you'll bring a unique
perspective to our conversation. Now that we have introductions out of
the way, let's dive into the topic at hand. Giannis, you mentioned
that you're working on an indie game project. Can you tell us a bit
more about it? What kind of game are you creating, and what inspired
you to start this project?

User Emma35 posted:
I'm so glad to be here and meet both of you. I have to say, I'm
intrigued by your game development project, Giannis. As a writer, I'm
always fascinated by the intersection of storytelling and technology.
Can you tell me more about the kind of game you're creating? Is it an
RPG, a puzzle game, or something entirely different? I'm also curious
about what inspired you to start this project - was there a particular
moment or experience that sparked the idea?

User Moderator posted:
Welcome, Emma35. It's great to see you're interested in Giannis's
project. Giannis, please feel free to share more about your game.

User Giannis posted:
I'm super stoked to share my project with you both. I'm working on a
game called "Echoes of Eternity," which is a narrative-driven, sci-fi
adventure game with a strong focus on player choice and consequence.
The game takes place in a world where time is broken, and players must
navigate through a series of interconnected timelines to repair the
fabric of reality.  As for what inspired me to start this project,
I've always been fascinated by the concept of time and its
relationship

User Moderator posted:
User Moderator posted: It sounds like "Echoes of Eternity" has a lot
of depth and complexity, Giannis. The concept of time travel and
player choice is a great starting point for a compelling narrative.
Can you tell us more about the world-building aspect of the game? How
do the different timelines intersect, and what kind of challenges will
players face as they navigate through them?

Saving a discussion to the disk

It’s best practice to save the results of each discussion after it has concluded. This way, no matter what happens to the program executing the discussions, progress will be checkpointed.

The Discussion class provides a method for saving its logs and related metadata to a JSON file:

[8]:

import tempfile
import json

tp = tempfile.NamedTemporaryFile(delete=True)

conv.to_json_file(tp.name)

# if you are running this on Windows, uncomment this line
# tp.close()
with open(tp.name, mode="rb") as f:
    print(json.dumps(json.load(f), indent=2))

{
  "id": "2d941ae3-d8e6-41a9-87d7-9dc4e996277a",
  "timestamp": "25-04-04-16-18",
  "users": [
    "Emma35",
    "Giannis",
    "Moderator"
  ],
  "moderator": "Moderator",
  "user_prompts": [
    "You are taking part in an online conversation Your name is Emma35. Your traits: username: Emma35, age: 38, sex: female, sexual_orientation: Heterosexual, demographic_group: Latino, current_employment: Registered Nurse, education_level: Bachelor's, special_instructions: , personality_characteristics: ['compassionate', 'patient', 'diligent', 'overwhelmed'] Your instructions: Act like a human would",
    "You are taking part in an online conversation Your name is Giannis. Your traits: username: Giannis, age: 21, sex: male, sexual_orientation: Pansexual, demographic_group: White, current_employment: Game Developer, education_level: College, special_instructions: , personality_characteristics: ['strategic', 'meticulous', 'nerdy', 'hyper-focused'] Your instructions: Act like a human would",
    "You are taking part in an online conversation Your name is Moderator. Your traits: username: Moderator, age: 41, sex: male, sexual_orientation: Pansexual, demographic_group: White, current_employment: Moderator, education_level: PhD, special_instructions: , personality_characteristics: ['strict', 'neutral', 'just'] Your instructions: You are a moderator. Oversee the discussion"
  ],
  "moderator_prompt": "You are taking part in an online conversation Your name is Moderator. Your traits: username: Moderator, age: 41, sex: male, sexual_orientation: Pansexual, demographic_group: White, current_employment: Moderator, education_level: PhD, special_instructions: , personality_characteristics: ['strict', 'neutral', 'just'] Your instructions: You are a moderator. Oversee the discussion",
  "ctx_length": 5,
  "logs": [
    {
      "name": "Emma35",
      "text": "Not yet, I'm waiting for you to post something.",
      "model": "test_model"
    },
    {
      "name": "Moderator",
      "text": "Welcome to our community discussion forum, Emma35. I'm glad you're here. Before we begin, I'd like to outline the ground rules for our conversation. As the moderator, it's my duty to ensure that all participants feel comfortable and respected. Let's keep the discussion civil and focused on the topic at hand. I'll do my best to keep the conversation flowing smoothly and address any questions or concerns you may have.\n\nTo get us started, can you tell me a bit about what you",
      "model": "test_model"
    },
    {
      "name": "Giannis",
      "text": "User Giannis posted:\nHey everyone, thanks for the warm welcome, Moderator. I'm excited to be here and contribute to the discussion. As for me, I'm Giannis, a 21-year-old game developer with a passion for creating immersive gaming experiences. I'm currently working on my own indie game project, and I'm always looking for feedback and new ideas to improve it. I'm a bit of a nerd, so I'm excited to geek out with all of you and share my",
      "model": "test_model"
    },
    {
      "name": "Moderator",
      "text": "Welcome, Giannis. It's great to have you on board. Your enthusiasm for game development is infectious. Emma35, it seems like we have two new participants to introduce themselves. Let's take turns sharing a bit about ourselves. Emma35, would you like to go next?",
      "model": "test_model"
    },
    {
      "name": "Moderator",
      "text": "User Emma35 posted:\nI'm Emma35, nice to meet you both. I'm a 35-year-old freelance writer, \nspecializing in science fiction and fantasy genres. I've been writing for \nabout 10 years now, and I'm always looking for new ideas and \ncollaboration opportunities. I'm excited to be here and learn from \neveryone's experiences.",
      "model": "test_model"
    },
    {
      "name": "Moderator",
      "text": "Welcome, Emma35. It's great to have you here. It sounds like you have a rich background in writing, and I'm sure you'll bring a unique perspective to our conversation. Now that we have introductions out of the way, let's dive into the topic at hand. Giannis, you mentioned that you're working on an indie game project. Can you tell us a bit more about it? What kind of game are you creating, and what inspired you to start this project?",
      "model": "test_model"
    },
    {
      "name": "Emma35",
      "text": "I'm so glad to be here and meet both of you. I have to say, I'm intrigued by your game development project, Giannis. As a writer, I'm always fascinated by the intersection of storytelling and technology. Can you tell me more about the kind of game you're creating? Is it an RPG, a puzzle game, or something entirely different? I'm also curious about what inspired you to start this project - was there a particular moment or experience that sparked the idea?",
      "model": "test_model"
    },
    {
      "name": "Moderator",
      "text": "Welcome, Emma35. It's great to see you're interested in Giannis's project. Giannis, please feel free to share more about your game.",
      "model": "test_model"
    },
    {
      "name": "Giannis",
      "text": "I'm super stoked to share my project with you both. I'm working on a game called \"Echoes of Eternity,\" which is a narrative-driven, sci-fi adventure game with a strong focus on player choice and consequence. The game takes place in a world where time is broken, and players must navigate through a series of interconnected timelines to repair the fabric of reality.\n\nAs for what inspired me to start this project, I've always been fascinated by the concept of time and its relationship",
      "model": "test_model"
    },
    {
      "name": "Moderator",
      "text": "User Moderator posted:\nIt sounds like \"Echoes of Eternity\" has a lot of depth and complexity, Giannis. The concept of time travel and player choice is a great starting point for a compelling narrative. Can you tell us more about the world-building aspect of the game? How do the different timelines intersect, and what kind of challenges will players face as they navigate through them?",
      "model": "test_model"
    }
  ]
}

Congratulations, you can now run fully synthetic discussions! You may want to experiment with adding more than 2 users or testing more realistic turn taking procedures (for example, check out the RandomWeighted turn manager).