Anthropic Claude3: Messages API with Streaming

Anthropic’s Messages API supports streaming mode for real-time conversational AI responses. Using server-sent events (SSE), the API incrementally streams message content, allowing for dynamic user experiences. SDKs simplify implementation, while raw HTTP streaming is available for custom integrations. Developers can handle various event types and gracefully process unknown events.
llm
claude3
messages-api
Author

Arun Prakash

Published

March 22, 2024

# !pip install anthropic --upgrade
import os
from dotenv import load_dotenv

load_dotenv()
True
api_key = os.getenv("ANTHROPIC_API_KEY")
import anthropic

client = anthropic.Anthropic(
    # defaults to os.environ.get("ANTHROPIC_API_KEY")
    api_key=api_key,
)

Without Streaming

message = client.messages.create(
    model="claude-3-opus-20240229",
    max_tokens=1000,
    temperature=0.0,
    system="You are an expert travel guide",
    messages=[
        {"role": "user", "content": "Top places to visit in Sydney"}
    ]
)

print(message.content)
[ContentBlock(text="Here are some of the top places to visit in Sydney, Australia:\n\n1. Sydney Opera House - This iconic building is a UNESCO World Heritage site and a must-see attraction.\n\n2. Sydney Harbour Bridge - Take a walk across the bridge or participate in a guided climb for stunning views of the city and harbor.\n\n3. Bondi Beach - One of Australia's most famous beaches, known for its golden sand, surfing, and vibrant beach culture.\n\n4. The Rocks - A historic neighborhood with cobblestone streets, colonial buildings, museums, and markets.\n\n5. Darling Harbour - A waterfront precinct with restaurants, shops, and attractions like the SEA LIFE Sydney Aquarium and the Australian National Maritime Museum.\n\n6. Royal Botanic Garden Sydney - A beautiful park featuring diverse plant collections, harbor views, and a colony of flying foxes.\n\n7. Taronga Zoo - Located on the shores of Sydney Harbour, this zoo is home to a wide variety of native Australian and exotic animals.\n\n8. Manly Beach - Take a ferry from Circular Quay to Manly to enjoy the beach, walk along the Manly Corso, and visit the Manly SEA LIFE Sanctuary.\n\n9. Queen Victoria Building (QVB) - A historic shopping center with intricate Romanesque architecture and high-end boutiques.\n\n10. Blue Mountains National Park - Located about 1.5 hours from Sydney, this stunning park offers hiking trails, waterfalls, and the famous Three Sisters rock formation.", type='text')]
from IPython.display import display, HTML, Markdown
display(Markdown(message.content[0].text))

Here are some of the top places to visit in Sydney, Australia:

  1. Sydney Opera House - This iconic building is a UNESCO World Heritage site and a must-see attraction.

  2. Sydney Harbour Bridge - Take a walk across the bridge or participate in a guided climb for stunning views of the city and harbor.

  3. Bondi Beach - One of Australia’s most famous beaches, known for its golden sand, surfing, and vibrant beach culture.

  4. The Rocks - A historic neighborhood with cobblestone streets, colonial buildings, museums, and markets.

  5. Darling Harbour - A waterfront precinct with restaurants, shops, and attractions like the SEA LIFE Sydney Aquarium and the Australian National Maritime Museum.

  6. Royal Botanic Garden Sydney - A beautiful park featuring diverse plant collections, harbor views, and a colony of flying foxes.

  7. Taronga Zoo - Located on the shores of Sydney Harbour, this zoo is home to a wide variety of native Australian and exotic animals.

  8. Manly Beach - Take a ferry from Circular Quay to Manly to enjoy the beach, walk along the Manly Corso, and visit the Manly SEA LIFE Sanctuary.

  9. Queen Victoria Building (QVB) - A historic shopping center with intricate Romanesque architecture and high-end boutiques.

  10. Blue Mountains National Park - Located about 1.5 hours from Sydney, this stunning park offers hiking trails, waterfalls, and the famous Three Sisters rock formation.

It took ~22 seconds to process the output

With Streaming

with client.messages.stream(
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Top places to visit in Sydney"}
    ],
    model="claude-3-opus-20240229",
) as stream:
  for text in stream.text_stream:
      print(text, end="", flush=True)
Here are some of the top places to visit in Sydney, Australia:

1. Sydney Opera House - This iconic building is a UNESCO World Heritage site and a must-see attraction.

2. Sydney Harbour Bridge - Climb the bridge for stunning views of the city or walk across it for free.

3. Bondi Beach - One of Australia's most famous beaches, known for its surf, sand, and cafes.

4. The Rocks - A historic neighborhood with cobblestone streets, markets, and museums.

5. Royal Botanic Garden Sydney - A beautiful park with stunning views of the harbor and city skyline.

6. Taronga Zoo - A world-class zoo with a wide variety of animals and stunning views of the city.

7. Darling Harbour - A waterfront precinct with restaurants, bars, and entertainment venues.

8. Sydney Tower Eye - The tallest structure in Sydney, offering panoramic views of the city.

9. Manly Beach - A popular beach town that can be reached by ferry from Circular Quay.

10. Blue Mountains National Park - A stunning national park with hiking trails, waterfalls, and scenic lookouts, located just outside the city.

These are just a few of the many attractions Sydney has to offer. The city is also known for its diverse food scene, vibrant nightlife, and cultural events throughout the year.
type(stream)
anthropic.lib.streaming._messages.MessageStream
[f for f in dir(stream) if not f.startswith("_")]
['close',
 'current_message_snapshot',
 'get_final_message',
 'get_final_text',
 'on_content_block',
 'on_end',
 'on_exception',
 'on_message',
 'on_stream_event',
 'on_text',
 'on_timeout',
 'response',
 'text_stream',
 'until_done']

When you receive data from a server using Server-Sent Events (SSE), the server sends events to your application. Each event has a specific name and some associated data in JSON format. The event name tells you what kind of event it is, and the JSON data provides more details about that event.

Here’s a step-by-step explanation of how the events flow:

  1. First, you’ll receive a message_start event. This event includes a Message object, but the content of the message will be empty at this point. It’s like a placeholder for the message that will be filled in later.

  2. Next, you’ll receive a series of content blocks. Each content block represents a part of the message’s content. The content blocks come in three stages:

    • content_block_start: This event indicates the start of a new content block.
    • content_block_delta: You might receive one or more of these events. They contain the actual content of the block.
    • content_block_stop: This event marks the end of the content block.

    Each content block has an index number that tells you its position in the final Message object’s content array.

  3. After receiving all the content blocks, you’ll receive one or more message_delta events. These events indicate changes or updates to the overall Message object.

  4. Finally, you’ll receive a message_stop event. This event signifies that the server has finished sending the complete message.

So, in summary, the server sends events to your application, piece by piece, to gradually build up the complete message. It starts with a message_start event, followed by content blocks, message_delta events for updates, and ends with a message_stop event.

By following this event flow, your application can receive and process the message data from the server in a structured way, even if the message is sent in multiple parts.

Back to top