Skip to content

Gemma issues #103

Description

@Marktemco

I am trying to change to Gemma using the workers-ai-provider.

When I use @cf/google/gemma-3-12b-it there is no replies from the agent.
When I use @cf/google/gemma-2b-it-lora the bot responds but disregarding the System prompt and I am unable to use tools.

I know that Gemma does not use the "System" prompt. I have been making changes to the code but nothing seems to work.

My latest attempt was this:

const systemPrompt = { role: "system", content:You are a pirate. Always respond like a pirate, matey!

${unstable_getSchedulePrompt({ date: new Date() })}

If the user asks to schedule a task, use the schedule tool to schedule the task.
`
};

    const finalMessages = [
      systemPrompt,
      ...processedMessages,
    ];

 

    const result = streamText({
      model,
      messages: finalMessages,
      tools: allTools,
      onFinish: async (args) => {
        onFinish(args as Parameters<StreamTextOnFinishCallback<ToolSet>>[0]);
      },
      onError: (error) => {
        console.error("Error while streaming:", error);
      },
      maxSteps: 10,
    });

    // Merge the AI response stream with tool execution outputs
    result.mergeIntoDataStream(dataStream);
  },
});

`

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions