AI21 Labs’ Jamba 1.5 family of models is now available on Amazon Bedrock

Today we are announcing the availability of AI21 Labs’ powerful new Jamba 1.5 family of large language models (LLMs) in The bedrock of the AmazonThese models represent a significant advancement in the capabilities of the long context language, offering speed, efficiency, and performance across a wide range of applications. The Jamba 1.5 model family includes Jamba 1.5 Mini and Jamba 1.5 Large. Both models support a 256K token context window, structured JSON output, function calling, and are capable of processing document objects.

AI21 Labs is a leader in building foundational artificial intelligence (AI) models and systems for the enterprise. Together, AI21 Labs and AWS are empowering customers across industries to build, deploy, and scale AI-powered systems. Generative AI Applications that solve real-world challenges and spur innovation through strategic collaboration. With AI21 Labs’ advanced, production-ready models coupled with dedicated services and Amazon’s powerful infrastructure, customers can leverage LLMs in a secure environment to shape the future of how we process information, communicate, and learn.

What is Jamba 1.5?
The Jamba 1.5 models leverage a unique hybrid architecture that combines the architecture of the transformer model with Structured State Space Model (SSM) Technology. This innovative approach enables Jamba 1.5 models to manage long context windows of up to 256,000 tokens while maintaining the high-performance characteristics of traditional transformer models. You can learn more about this hybrid SSM/transformer architecture at Jamba: A hybrid language model between Transformer and Mamba White paper.

You can now use two new Jamba 1.5 models from AI21 on Amazon Bedrock:

• Jamb 1.5 large It excels at complex reasoning tasks at all cue lengths, making it ideal for applications requiring high-quality results from long and short inputs.
• Jamb 1.5 Mini It is optimized for low-latency processing of long prompts, enabling rapid analysis of large documents and data.

Strengths of the Jamba 1.5 models include:

• Long context handling – With a token context length of 256K, Jamba 1.5 models can improve the quality of enterprise applications such as large document analysis and summarization, as well as agent and RAG workflows.
• Multilingual – Support for English, Spanish, French, Portuguese, Italian, Dutch, German, Arabic and Hebrew.
• Developer friendly – Native support for structured JSON output, function calling, and the ability to digest document objects.
• Speed ​​and efficiency – AI21 measured the performance of Jamba 1.5 models and shared that the models demonstrate up to 2.5x faster inference on long contexts than other comparable sized models. For detailed performance results, please visit Jamba model family announced on AI21 website.

Get started with Jamba 1.5 models on Amazon Bedrock
To get started with the new Jamba 1.5 models, go to the Bedrock Console from Amazonchoose Access to the model in the bottom left panel and request access to Jamba 1.5 Mini or Jamba 1.5 Large.

To test the Jamba 1.5 models on the Amazon Bedrock console, choose the Text either Chat Playground in the left menu panel. Then select Select model and select AI21 as the category and Jamb 1.5 Mini either Jamb 1.5 large as a model.

When choosing View API requestYou can get a code example of how to invoke the model using the AWS Command Line Interface (AWS CLI) with the current example message.

You can follow the Code examples in the Amazon Bedrock documentation to access the available models using AWS SDK and build their applications using multiple programming languages.

The following Python code example shows how to send a text message to Jamba 1.5 models using the Amazon Bedrock Converse API for text generation.

import boto3
from botocore.exceptions import ClientError

# Create a Bedrock Runtime client.
bedrock_runtime = boto3.client("bedrock-runtime", region_name="us-east-1")

# Set the model ID.
# modelId = "ai21.jamba-1-5-mini-v1:0"
model_id = "ai21.jamba-1-5-large-v1:0"

# Start a conversation with the user message.
user_message = "What are 3 fun facts about mambas?"
conversation = (
    {
        "role": "user",
        "content": ({"text": user_message}),
    }
)

try:
    # Send the message to the model, using a basic inference configuration.
    response = bedrock_runtime.converse(
        modelId=model_id,
        messages=conversation,
        inferenceConfig={"maxTokens": 256, "temperature": 0.7, "topP": 0.8},
    )

    # Extract and print the response text.
    response_text = response("output")("message")("content")(0)("text")
    print(response_text)

except (ClientError, Exception) as e:
    print(f"ERROR: Can't invoke '{model_id}'. Reason: {e}")
    exit(1)

Jamba 1.5 models are perfect for use cases like paired document analysis, compliance analysis, and answering questions for long documents. They can easily compare information from multiple sources, check whether passages comply with specific guidelines, and handle very long or complex documents. You can find example code at AI21 GitHub Repository on AWSFor more information on how to effectively incite Jamba models, see AI21 Documentation.

Now available
AI21 Labs Jamba 1.5 model family now available on Amazon Bedrock in the Eastern US (N. Virginia) AWS Region. Check the Full list of regions for future updates. For more information, see the AI21 Labs on Amazon Bedrock Product page and pricing page.

Try the Jamba 1.5 models in the Bedrock Console from Amazon Today and send comments to AWS re:Post for Amazon Bedrock or through your usual AWS support contacts.

Visit our community.aws site to find in-depth technical content and discover how our Builder communities use Amazon Bedrock in their solutions.

— Ant