ChatAI21

Overview

This notebook covers how to get started with AI21 chat models. Note that different chat models support different parameters. See the AI21 documentation to learn more about the parameters in your chosen model. See all AI21's LangChain components.

Integration details

Class	Package	Local	Serializable	JS support	Package downloads	Package latest
ChatAI21	langchain-ai21	❌	beta	✅

Model features

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Native async	Token usage	Logprobs
✅	✅	❌	❌	❌	❌	✅	✅	✅	❌

Setup

Credentials

We'll need to get an AI21 API key and set the AI21_API_KEY environment variable:

import os
from getpass import getpass

os.environ["AI21_API_KEY"] = getpass()

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# os.environ["LANGCHAIN_TRACING_V2"] = "true"
# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")

Installation

!pip install -qU langchain-ai21

Instantiation

Now we can instantiate our model object and generate chat completions:

from langchain_ai21 import ChatAI21

llm = ChatAI21(model="jamba-instruct", temperature=0)

Invocation

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg

AIMessage(content="J'adore programmer.", id='run-2e8d16d6-a06e-45cb-8d0c-1c8208645033-0')

Chaining

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)

API Reference:ChatPromptTemplate

AIMessage(content='Ich liebe das Programmieren.', id='run-e1bd82dc-1a7e-4b2e-bde9-ac995929ac0f-0')

API reference

For detailed documentation of all ChatAI21 features and configurations head to the API reference: https://python.langchain.com/v0.2/api_reference/ai21/chat_models/langchain_ai21.chat_models.ChatAI21.html

Related

• Chat model conceptual guide
• Chat model how-to guides