genaiscript/runtime expansion?

[CoffeeAtBedtime]

I find myself building classes that my genaiscripts use like the one below and importing them and newing them up to build more deterministic code alongside the genaiscript goodies with very reliable structured outputs. I've had success making the "agent" classes export types, use parsers, tools and send back structured outputs from runPrompt thanks to the runPrompt options.

It'd be so cool to build a Vue application and accept user inputs and continue to integrate genaiscript code in this way by importing something like the runtime. The api run offers running a script and getting the response but not as integrated. However if I go this route presently the genaiscript itself needs to be the entry point.

scribe.mts

export type ConversationSummary = {
  summary: string;
};

export class ScribeAgent {
  conversationHistory: string[] = [];
  constructor() {}

  record(message: string): void {
    const timestamp = new Date().toISOString();
    this.conversationHistory.push(`[${timestamp}] ${message}`);
  }

  getLastMessage(): string {
    return this.conversationHistory[this.conversationHistory.length - 1] || '';
  }

  async summarize(): Promise<ConversationSummary> {
    const res = await runPrompt(
      (ctx) => {
        ctx.$`
        Role: Scribe Agent.
        
        Your task is to summarize the following conversation into a concise summary that captures the key points and decisions:
        <CONVERSATION>

        Include direct quotes or important details that are relevant to the context.

        Ensure the response validates the schema.
        `;
        ctx.def('CONVERSATION', this.conversationHistory.join('\n'));
      },
      { responseSchema: this.conversationSummarySchema, responseType: 'json', system: ['system.assistant'] },
    );

    const conversationTokens = await tokenizers.count(this.conversationHistory.join('\n'));
    if (conversationTokens > 30000) {
      this.conversationHistory = [];
    }

    this.record(res.json.summary);
    return res.json as ConversationSummary;
  }

  conversationSummarySchema = {
    $schema: 'http://json-schema.org/draft-07/schema#',
    title: 'ConversationSummary',
    type: 'object',
    properties: {
      summary: {
        type: 'string',
      },
    },
    required: ['summary'],
  };
}

[pelikhan]

The api run offers running a script and getting the response but not as integrated.

Yes support for running outside the "cli" is possible but you can rely on the global types anymore. Well, should be doable.

[pelikhan]

other idea is to rip out the message rendering engine into an isomorphic API since it's fairly agnostic (until you hit the vendor SDKs which also carry tons of other dependencies).

[CoffeeAtBedtime]

Ahhh I understand the node aspect better now. I think maybe it is a desire to launch in "server" mode and communicate with a front end. I will post some sample code later today of how I do it now.

[pelikhan]

There is a built in server in the cli that is currently used to power the VScode integration and the web app maybe we can build on that.

There is a long standing work item to move to grpc for the protocol to clean it up a bit.

[CoffeeAtBedtime]

Here's my silly demo https://github.com/CoffeeAtBedtime/CatChat/ to see what I'm trying to accomplish. There aren't any structured outputs. But this pattern I think could be used to run a serious multi step research with judges and human in the loop inside a GenAIScript like I keep seeing python versions of on HN. They all have to write the helper functions you already have and it could be pure Typescript 😄 This uses a run loop and waits for input. I know the host.input can halt and the conversation could be done through terminal. In a web UI though I can assemble and manage data structures or create wacky multi-lane conversations.

[pelikhan]

Love this!