Lesson 16 Google Gemini
Theme:

Theory: Google Gemini

Exploring Google’s multimodal AI models, long-context capabilities, and the Gemini ecosystem.

Gemini Architecture

Gemini is Google DeepMind’s family of multimodal AI models, built from the ground up to handle text, images, audio, video, and code in a single architecture. Unlike some competitors that bolt vision capabilities onto a text model, Gemini was trained natively on multimodal data, which tends to produce more coherent cross-modal reasoning.

The architecture uses a Transformer-based design with modifications for efficient long-context processing. Google has not published full architectural details, but the models demonstrate strong performance on benchmarks across all modalities, suggesting deep integration of multimodal understanding rather than separate processing pipelines.

Model Family

Google offers Gemini in several tiers designed for different trade-offs between capability, speed, and cost:

ModelStrengthsContextUse Case
Gemini 2.5 ProStrongest reasoning, best quality1M tokensComplex analysis, research, coding
Gemini 2.5 FlashFast, cost-effective, thinking mode1M tokensProduction workloads, high volume
Gemini 2.0 FlashSpeed, free tier, real-time1M tokensPrototyping, real-time applications

Gemini 2.5 Flash is particularly noteworthy because it includes a “thinking” mode (similar to OpenAI’s o-series reasoning) while remaining significantly cheaper. This makes it a strong candidate for tasks that benefit from reasoning but where Gemini Pro’s cost is prohibitive.

All current Gemini models support the full 1M-token context window, which is the longest commercially available context among major providers. Claude offers 200K tokens and GPT-4o offers 128K tokens for comparison.

Long Context (1M Tokens)

The 1M-token context window is one of Gemini’s most distinctive features. To put this in perspective: 1 million tokens is roughly 750,000 words, or about 10 average-length novels. This enables entirely new use cases that were previously impossible or required complex RAG pipelines.

Practical applications of long context include:

However, long context is not always the right approach. The “needle in a haystack” problem means models can struggle to find specific information buried in very long contexts. RAG with focused retrieval often produces better results for targeted question-answering over large corpora. Additionally, the cost scales linearly with context length: a 1M-token input at Gemini Pro rates costs $10 per request.

Practical rule: Use long context when you need the model to reason across the entire document (summarisation, comparison, pattern finding). Use RAG when you need precise answers to specific questions from a large corpus.

Multimodal Inputs

Gemini accepts multiple input types natively. The generate_content() method takes a list of parts, where each part can be text, an image, a video, an audio file, or a document.

import google.generativeai as genai

model = genai.GenerativeModel("gemini-2.5-flash")

# Image analysis
response = model.generate_content([
    "What's happening in this image? Describe in detail.",
    genai.upload_file("photo.jpg"),
])

# Video analysis (unique to Gemini)
video_file = genai.upload_file("presentation.mp4")
response = model.generate_content([
    "Summarise the key points from this video.",
    video_file,
])

# Audio transcription and analysis
audio_file = genai.upload_file("podcast.mp3")
response = model.generate_content([
    "Transcribe this audio and identify the main topics discussed.",
    audio_file,
])

Video and audio analysis are currently unique to Gemini among the major API providers. Claude supports images and PDFs natively, while GPT-4o supports images and has separate audio models. If your application involves video or audio processing, Gemini is often the natural choice.

Google AI Studio vs Vertex AI

Google provides two platforms for accessing Gemini models, each targeting a different audience:

Google AI Studio is the developer-friendly platform. It uses simple API key authentication, has a generous free tier, and is accessed via the google-generativeai Python SDK. It is ideal for learning, prototyping, and small-scale applications. The main limitation is that it lacks enterprise features like VPC-SC, customer-managed encryption keys, and SLA guarantees.

Vertex AI is Google Cloud’s enterprise AI platform. It uses GCP IAM authentication, provides access to Gemini plus third-party models (including Claude and Llama), and includes enterprise features like model monitoring, A/B testing, and custom model training. For production workloads, Vertex AI is the recommended platform.

The API surface is nearly identical between the two platforms. Code written for Google AI Studio can be migrated to Vertex AI by changing the authentication method and import paths, with minimal logic changes.

Grounding with Google Search

Gemini offers a unique “grounding” feature that allows the model to search Google in real-time before generating a response. This reduces hallucination for factual queries and ensures the model has access to current information beyond its training data.

model = genai.GenerativeModel("gemini-2.5-flash")

response = model.generate_content(
    "What were the major tech announcements this week?",
    tools=[genai.Tool(google_search=genai.GoogleSearch())],
)
print(response.text)
# Response includes citations from Google Search results

Grounding is particularly valuable for applications that need current information (news, stock prices, weather) or factual accuracy on topics where the model might otherwise hallucinate. It is conceptually similar to building a web search tool for Claude, but integrated natively into the API.

Pro tip: Grounding adds latency and cost because it performs a Google Search for each request. Use it selectively for queries that genuinely need current information, not for every request.

Further Reading