Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/python_gaas_tools/gaas_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,6 @@ The `py/gaas_gpt_model.py` module provides classes for interacting with various
* Constructs a Pixel script: `LLM(engine="<engine_id>", command="<question>", useHistory=<use_history>, context=["<context>"], paramValues=[<param_dict>]);`.
* Executes this Pixel script via `super().callReactor()`.
* Returns the response from the LLM, typically including the generated text and token counts.
* `ner(text: str, entities: List[str], mask_entities: List[str] = [], param_dict: Optional[Dict] = None, insight_id: Optional[str] = None)`:
* Executes an `NER(...)` Pixel command for Named Entity Recognition.
* `get_conversation_history(insight_id: Optional[str] = None) -> List[Dict]`:
* Executes a `GetRoomMessages(roomId="<insight_id>");` Pixel script to retrieve chat history from the `ModelInferenceLogsDatabase`.
* `embeddings(strings_to_embed: List[str], param_dict: Optional[Dict] = None, insight_id: Optional[str] = None) -> List[Dict]`:
Expand Down Expand Up @@ -74,7 +72,7 @@ The `py/gaas_gpt_model.py` module provides classes for interacting with various
* **Initialization `__init__(self, model_engine_class: Optional[str] = "TOMCAT", **kwargs)`**:
* `model_engine_class` (str, default: "TOMCAT"): Determines the type of engine to create. Can be "TOMCAT" or "LOCAL". "HF_PIPELINE" is mentioned but not fully implemented in the constructor logic shown.
* `**kwargs`: Passed to the constructor of the chosen model engine class (e.g., `engine_id` for `TomcatModelEngine`).
* **Key Methods**: All methods (`get_model_type`, `ask`, `instruct`, `embeddings`, `keyword_extraction`, `ner`, `do_call`, `get_model_engine_id`, `get_conversation_history`) are wrappers that delegate the call to the underlying `self.model_engine` instance.
* **Key Methods**: All methods (`get_model_type`, `ask`, `instruct`, `embeddings`, `keyword_extraction`, `do_call`, `get_model_engine_id`, `get_conversation_history`) are wrappers that delegate the call to the underlying `self.model_engine` instance.
* **Langchain Integration**:
* `to_langchain_embedder()`: Wraps the `ModelEngine` to make its `embeddings` method compatible with Langchain's `Embeddings` interface.
* `to_langchain_chat_model()`: Wraps the `ModelEngine` to make its `ask` and `get_conversation_history` methods compatible with Langchain's `BaseChatModel` interface. This includes converting message formats.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: model
description: Use when writing code in an app that calls an LLM, embedding model, or other model engine, OR when listing/selecting models the user has access to. Covers the LLM() and MyEngines() pixel commands via @semoss/sdk's runPixel, including prompt/completion calls, conversational history, structured outputs, attaching media of any type (image, PDF, Word, spreadsheet, audio, video) with the media argument, and parsing model responses. Also covers which top-level arguments full_prompt drops, media included, plus the other model pixels: Embeddings(), MultiModalEmbeddings(), Vision(), NER(), GetContextWindow(), GetUserModelUsage(), and the provider batch API (BatchLLM, GetModelBatchStatus, GetModelBatchResults). Do not use for vector database queries (see vector).
description: Use when writing code in an app that calls an LLM, embedding model, or other model engine, OR when listing/selecting models the user has access to. Covers the LLM() and MyEngines() pixel commands via @semoss/sdk's runPixel, including prompt/completion calls, conversational history, structured outputs, attaching media of any type (image, PDF, Word, spreadsheet, audio, video) with the media argument, and parsing model responses. Also covers which top-level arguments full_prompt drops, media included, plus the other model pixels: Embeddings(), MultiModalEmbeddings(), Vision(), GetContextWindow(), GetUserModelUsage(), and the provider batch API (BatchLLM, GetModelBatchStatus, GetModelBatchResults). Do not use for vector database queries (see vector).
---

# Model Engine
Expand Down Expand Up @@ -212,14 +212,13 @@ Embeddings(engine="${EMBEDDING_MODEL_ID}", values=["first text", "second text"])

Returns a map with the vectors in `pixelReturn[0].output`. If the engine is not an embedding model, the output is the literal string "This model does not support embeddings." - check for it. `MultiModalEmbeddings(engine=, text=, image=, video=)` embeds any combination of modalities (image/video accept base64, data URLs, or remote URLs), with results broken out by modality.

### Vision and NER
### Vision

```
Vision(engine="${MODEL_ID}", command=["Describe this diagram"], image=["${urlOrBase64}"]);
NER(engine="${NER_MODEL_ID}", prompt=["<encode>${text}</encode>"], entities=["PERSON","EMAIL"], maskEntities=[true]);
```

`Vision` is the image-text-to-text task for models exposing it. `NER` extracts (and with `maskEntities` redacts) named entities - the engine must actually be an NER model or the call fails.
`Vision` is the image-text-to-text task for models exposing it.

### Token budget and usage

Expand Down
38 changes: 0 additions & 38 deletions py/gaas_gpt_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,35 +206,6 @@ def ask(

return pixelReturn

def ner(
self,
text: str,
entities: List[str],
mask_entities: List[str] = [],
param_dict: Optional[Dict] = None,
insight_id: Optional[str] = None,
):
if insight_id is None:
insight_id = self.insight_id

epoc = super().get_next_epoc()

pixel = f'NER(engine="{self.engine_id}", prompt="{text}", entities={entities}, maskEntities={mask_entities});'

pixelReturn = super().callReactor(
epoc=epoc,
pixel=pixel,
insight_id=insight_id,
)

if pixelReturn is not None and len(pixelReturn) > 0:
output = pixelReturn[0]["pixelReturn"][0]
if "ERROR" in output["operationType"]:
raise Exception(output["output"])
return output["output"]

return pixelReturn

def get_conversation_history(self, insight_id: Optional[str] = None) -> List[Dict]:
"""This method is responsible to get message history back from the model logs database.

Expand Down Expand Up @@ -608,15 +579,6 @@ def keyword_extraction(
):
return self.model_engine.keyword_extraction(**kwargs)

def ner(
self,
insight_id: Optional[
str
] = None, # TODO remove once users stop using it. No longer needs to be set.
**kwargs,
):
return self.model_engine.ner(**kwargs)

def do_call(self, **kwargs: Any) -> Any:
return self.model_engine.embeddings(**kwargs)

Expand Down
53 changes: 0 additions & 53 deletions src/prerna/cluster/util/IModelZKListener.java

This file was deleted.

110 changes: 0 additions & 110 deletions src/prerna/cluster/util/IRemoteClientServer.java

This file was deleted.

98 changes: 0 additions & 98 deletions src/prerna/cluster/util/ModelAvailableListener.java

This file was deleted.

Loading
Loading