How to Test Email Signup Flows in CI/CD Pipelines Using a Temporary Email API
A practical guide to end-to-end email verification testing with pytest and GitHub Actions — using real temporary inboxes, WebSocket event streams, and zero mocking.
How to Test Email Signup Flows in CI/CD Pipelines Using a Temporary Email API
Email confirmation is one of the most common flows in modern web applications, and one of the most reliably broken in automated test suites. Registration flows, password resets, magic links, and OTP codes all depend on email delivery — and that dependency is usually where CI pipelines stall, flake, or get skipped entirely.
This article walks through the full implementation: creating a real temporary inbox via API, triggering a signup, waiting for the confirmation email, extracting the link or OTP, and asserting success — all inside pytest, all runnable in GitHub Actions.
1. Why Email Flows Break CI/CD Pipelines
The problem is not that testing email is hard. It is that most teams use approaches that are either too slow, too fragile, or require manual intervention.
The common failure modes:
- The test sends email to a hardcoded address and checks a shared inbox manually
- The test mocks the email entirely and never sends anything
- The test uses a staging SMTP server with no API for message retrieval
- The test polls a generic inbox service that has rate limits and no auth
- The test simply skips email verification in CI with a feature flag
Each of these introduces a gap between what the test covers and what actually runs in production. Email verification is not a peripheral concern — it is often the first authenticated action a user takes. If you never test it end-to-end, you will eventually ship a broken registration flow to production.
2. The Problem with Mocking Email
Mocking is the default recommendation and the right tool for unit-testing individual functions. It is not sufficient for testing email flows end-to-end.
When you mock email in a test, you are asserting that your code calls a send function with the right arguments. You are not asserting:
- That the email is actually deliverable from your infrastructure
- That your email template renders correctly
- That the confirmation link is well-formed and points to the right environment
- That the OTP in the email matches what the backend expects
- That your email provider's SDK version is compatible with your configuration
A mocked test passes when your code calls sendgrid.send(to=address, subject=...). A real end-to-end test passes when a rendered email containing a working link arrives in an inbox and the link successfully completes the registration flow. These are different assertions.
The Mailtrap approach is a step forward: it routes email to a real SMTP server with a web interface and basic API. But Mailtrap introduces its own problems in CI:
- Shared inboxes across test runs cause cross-contamination
- API rate limits become a bottleneck in parallel pipelines
- You still need a paid account to get per-test inbox isolation
- The API response format requires SDK integration, not raw HTTP
What automated pipelines actually need is a temporary inbox API: one request to create an isolated inbox, a push notification when mail arrives, and a simple HTTP call to read the body.
3. Why You Need a Real Temporary Inbox
A real temporary inbox API provides properties that mocks and shared inboxes cannot:
| Property | Mock | Shared SMTP | Temp Inbox API |
|---|---|---|---|
| Inbox isolation per test | — | Manual | Automatic |
| Push notification on arrival | — | — | WebSocket |
| API for message retrieval | — | Sometimes | Yes |
| Deterministic expiration | — | Manual cleanup | Automatic (TTL) |
| Works in headless CI | Yes | Depends | Yes |
| Tests real delivery path | No | Partial | Yes |
The infrastructure described in this article accepts real SMTP delivery via AWS SES in production, meaning the email you receive in your test inbox passed through the same ingestion path as any production email — spam filtering, DKIM verification, and all.
4. Step-by-Step Implementation
Base URL and Authentication
The API base URL is https://uncorreotemporal.com. All endpoints are under /api/v1.
Anonymous use requires no signup: the first call returns a session_token which authenticates all subsequent calls to that mailbox. For CI pipelines where you need multiple mailboxes or persistence across runs, register for an API key and use Authorization: Bearer <key>.
# conftest.py or helpers.py
import urllib.parse
BASE_URL = "https://uncorreotemporal.com"
def encode_address(address: str) -> str:
"""URL-encode the mailbox address for use in path segments."""
return urllib.parse.quote(address, safe="")
Step 1: Create a Temporary Inbox
import httpx
def create_inbox(ttl_minutes: int = 10) -> dict:
"""
Create an anonymous temporary inbox.
Returns:
{"address": "...", "expires_at": "...", "session_token": "..."}
"""
resp = httpx.post(
f"{BASE_URL}/api/v1/mailboxes",
params={"ttl_minutes": ttl_minutes},
)
resp.raise_for_status()
return resp.json()
A successful response:
{
"address": "swift-eagle-17@uncorreotemporal.com",
"expires_at": "2024-11-15T14:32:00.000000+00:00",
"session_token": "dGhpcyBpcyBhIHNhbXBsZSB0b2tlbg"
}
The session_token is your authentication credential for this inbox. Store it; you need it for every subsequent call.
Step 2: Trigger Signup on the Application Under Test
This step is specific to your application. The inbox address is passed wherever a real user would enter their email:
def trigger_signup(email: str, password: str = "Test1234!") -> None:
"""Register on the application under test using the temp inbox address."""
resp = httpx.post(
"https://your-app-staging.example.com/api/auth/register",
json={"email": email, "password": password},
)
resp.raise_for_status()
The key point: the email value passed here is the address returned by the inbox creation call. Your application sends the confirmation email to a real inbox that your test controls.
Step 3: Wait for the Email
The API supports two retrieval strategies: polling the message list, or subscribing to a WebSocket event stream.
Polling approach (simpler, works everywhere):
import time
def wait_for_message(
address: str,
session_token: str,
timeout: int = 30,
poll_interval: float = 2.0,
) -> dict:
"""
Poll the inbox until a message arrives or timeout is reached.
Returns the first message summary (id, from_address, subject, etc.).
Raises TimeoutError if no message arrives within `timeout` seconds.
"""
encoded = encode_address(address)
headers = {"X-Session-Token": session_token}
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
resp = httpx.get(
f"{BASE_URL}/api/v1/mailboxes/{encoded}/messages",
headers=headers,
)
resp.raise_for_status()
messages = resp.json()
if messages:
return messages[0] # most recent first
time.sleep(poll_interval)
raise TimeoutError(f"No message arrived in {address} within {timeout}s")
WebSocket approach (lower latency, no wasted requests):
import asyncio
import json
import websockets
async def wait_for_message_ws(
address: str,
session_token: str,
timeout: int = 30,
) -> str:
"""
Subscribe to the inbox WebSocket and return the first message_id received.
"""
encoded = encode_address(address)
uri = f"wss://uncorreotemporal.com/ws/inbox/{encoded}?token={session_token}"
async with websockets.connect(uri) as ws:
async def _read():
async for raw in ws:
event = json.loads(raw)
if event.get("event") == "new_message":
return event["message_id"]
return await asyncio.wait_for(_read(), timeout=timeout)
The WebSocket endpoint sends {"event": "new_message", "message_id": "<uuid>"} as soon as mail is ingested. Combined with a 30-second timeout, this approach eliminates polling latency and avoids hammering the API in parallel test runs.
Step 4: Fetch the Full Message Body
The message list endpoint returns summaries without body content. Fetch the full content using the message ID:
def get_message_body(address: str, session_token: str, message_id: str) -> dict:
"""
Retrieve the full message content (body_text, body_html, attachments).
Automatically marks the message as read.
Returns:
{
"id": "...",
"from_address": "...",
"subject": "...",
"body_text": "...",
"body_html": "...",
"attachments": [],
"received_at": "...",
"is_read": true
}
"""
encoded = encode_address(address)
resp = httpx.get(
f"{BASE_URL}/api/v1/mailboxes/{encoded}/messages/{message_id}",
headers={"X-Session-Token": session_token},
)
resp.raise_for_status()
return resp.json()
Step 5: Extract the Confirmation Link and Assert
import re
def extract_confirmation_link(body_html: str | None, body_text: str | None) -> str:
"""
Extract a confirmation URL from the email body.
Tries HTML first, falls back to plain text.
"""
# Adjust the pattern to match your app's confirmation URL format
pattern = r'https://your-app-staging\.example\.com/confirm\?token=[A-Za-z0-9._-]+'
for content in [body_html, body_text]:
if content:
match = re.search(pattern, content)
if match:
return match.group(0)
raise ValueError("Confirmation link not found in email body")
def complete_signup(confirmation_url: str) -> None:
resp = httpx.get(confirmation_url, follow_redirects=True)
assert resp.status_code == 200, f"Confirmation failed: {resp.status_code}"
5. Integrating into Pytest
A clean pytest fixture encapsulates the inbox lifecycle. The fixture creates the inbox before the test and deletes it (or lets it expire) after:
# conftest.py
import pytest
import httpx
import urllib.parse
BASE_URL = "https://uncorreotemporal.com"
def _encode(address: str) -> str:
return urllib.parse.quote(address, safe="")
@pytest.fixture
def temp_inbox():
"""
Pytest fixture that creates a temporary inbox for one test.
Yields a dict with 'address' and 'session_token'.
Deletes the inbox after the test completes.
"""
resp = httpx.post(f"{BASE_URL}/api/v1/mailboxes", params={"ttl_minutes": 10})
resp.raise_for_status()
inbox = resp.json()
address = inbox["address"]
token = inbox["session_token"]
yield inbox
# Cleanup: soft-delete the mailbox regardless of test outcome
try:
httpx.delete(
f"{BASE_URL}/api/v1/mailboxes/{_encode(address)}",
headers={"X-Session-Token": token},
)
except Exception:
pass # Inbox will expire automatically anyway
The full test using this fixture:
# test_registration.py
import re
import time
import httpx
import pytest
BASE_URL = "https://uncorreotemporal.com"
APP_URL = "https://your-app-staging.example.com"
def _encode(address: str) -> str:
import urllib.parse
return urllib.parse.quote(address, safe="")
def wait_for_message(address: str, token: str, timeout: int = 30) -> dict:
import time
encoded = _encode(address)
headers = {"X-Session-Token": token}
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
resp = httpx.get(f"{BASE_URL}/api/v1/mailboxes/{encoded}/messages", headers=headers)
resp.raise_for_status()
msgs = resp.json()
if msgs:
return msgs[0]
time.sleep(2)
raise TimeoutError(f"No message in {timeout}s")
def get_full_message(address: str, token: str, message_id: str) -> dict:
encoded = _encode(address)
resp = httpx.get(
f"{BASE_URL}/api/v1/mailboxes/{encoded}/messages/{message_id}",
headers={"X-Session-Token": token},
)
resp.raise_for_status()
return resp.json()
def test_email_confirmation_flow(temp_inbox):
address = temp_inbox["address"]
token = temp_inbox["session_token"]
# 1. Register on the application
reg = httpx.post(f"{APP_URL}/api/auth/register", json={
"email": address,
"password": "TestPassword1!",
})
assert reg.status_code == 201
# 2. Wait for the confirmation email
summary = wait_for_message(address, token, timeout=30)
assert "confirm" in summary["subject"].lower()
# 3. Fetch the full body
message = get_full_message(address, token, summary["id"])
assert message["body_text"] or message["body_html"]
# 4. Extract the confirmation link
pattern = rf"{re.escape(APP_URL)}/confirm\?token=[A-Za-z0-9._-]+"
content = message["body_html"] or message["body_text"]
match = re.search(pattern, content)
assert match, "Confirmation link not found in email"
confirm_url = match.group(0)
# 5. Click the confirmation link
confirm_resp = httpx.get(confirm_url, follow_redirects=True)
assert confirm_resp.status_code == 200
# 6. Assert the account is now active
login = httpx.post(f"{APP_URL}/api/auth/login", json={
"email": address,
"password": "TestPassword1!",
})
assert login.status_code == 200
assert "access_token" in login.json()
This test is fully isolated. Each run creates its own inbox, its own account, and its own confirmation link. Running 20 instances in parallel produces 20 isolated inboxes with no coordination needed.
6. Integrating into GitHub Actions
# .github/workflows/e2e.yml
name: E2E Tests
on:
push:
branches: [main, staging]
pull_request:
jobs:
e2e:
runs-on: ubuntu-latest
env:
# No signup needed for anonymous use; add API key for authenticated access
UCT_API_KEY: ${{ secrets.UCT_API_KEY }}
APP_URL: ${{ vars.STAGING_URL }}
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install pytest httpx websockets
- name: Run email flow tests
run: pytest tests/e2e/test_registration.py -v --timeout=60
The UCT_API_KEY secret is only needed if you use authenticated access (Bearer token). For fully anonymous use, no credentials are required and the job needs no secrets beyond your application's staging URL.
Parallel matrix runs for different user tiers or regions work without any changes — each matrix leg creates its own inbox:
strategy:
matrix:
plan: [free, pro]
region: [us, eu]
fail-fast: false
7. Handling OTP Extraction
OTP flows follow the same pattern but with a regex targeting a numeric code instead of a URL:
import re
def extract_otp(body_text: str | None, body_html: str | None) -> str:
"""
Extract a 6-digit OTP from email body.
Adjust the pattern to match your application's OTP format.
"""
# Try plain text first — usually cleaner to parse
for content in [body_text, body_html]:
if content:
match = re.search(r'\b(\d{6})\b', content)
if match:
return match.group(1)
raise ValueError("OTP not found in email body")
def test_otp_login_flow(temp_inbox):
address = temp_inbox["address"]
token = temp_inbox["session_token"]
# Request a magic link / OTP
httpx.post(f"{APP_URL}/api/auth/otp", json={"email": address}).raise_for_status()
# Wait and retrieve
summary = wait_for_message(address, token, timeout=30)
message = get_full_message(address, token, summary["id"])
otp = extract_otp(message["body_text"], message["body_html"])
assert len(otp) == 6
# Submit the OTP
resp = httpx.post(f"{APP_URL}/api/auth/otp/verify", json={
"email": address,
"code": otp,
})
assert resp.status_code == 200
For OTPs embedded in HTML (inside a styled <div> or <td>), parsing body_text is more reliable than regex on HTML. If body_text is empty and you must parse HTML, use html.parser from the standard library or BeautifulSoup rather than a fragile HTML regex.
8. Cleaning Up Expired Inboxes
Inboxes expire automatically based on the TTL set at creation. A mailbox created with ttl_minutes=10 is marked inactive 10 minutes later by the background expiry worker. No manual cleanup is required for the common case.
For explicit cleanup in fixture teardown:
httpx.delete(
f"{BASE_URL}/api/v1/mailboxes/{_encode(address)}",
headers={"X-Session-Token": token},
)
This performs a soft-delete (is_active=False). The mailbox and its messages remain in the database but the inbox stops accepting new mail and the address becomes unreachable.
For authenticated users with API keys, you can retrieve all active mailboxes created by your key and clean them up as a batch at the end of a test session:
# Teardown: delete all mailboxes created by this API key
resp = httpx.get(
f"{BASE_URL}/api/v1/mailboxes",
headers={"Authorization": f"Bearer {api_key}"},
)
for mailbox in resp.json():
httpx.delete(
f"{BASE_URL}/api/v1/mailboxes/{_encode(mailbox['address'])}",
headers={"Authorization": f"Bearer {api_key}"},
)
9. Best Practices
Use short TTLs. For test pipelines, ttl_minutes=5 or ttl_minutes=10 is enough and prevents orphaned inboxes from accumulating if the test fails before cleanup.
Handle timeouts defensively. Email delivery is subject to external latency — your application's SMTP provider, SES queue depth, DNS propagation. A timeout=30 on wait_for_message is usually sufficient for staging environments. Set it to 60 in CI where network latency is unpredictable.
Do not share inboxes between tests. Create one inbox per test function, not per module or session. Two tests using the same inbox can receive each other's emails if the application under test sends multiple messages (welcome email + confirmation, for example).
Use the WebSocket approach for high-concurrency pipelines. If you are running 50 parallel test workers, polling every 2 seconds generates 1,500 requests per minute against the message list endpoint. The WebSocket stream eliminates that load — each worker holds one persistent connection and receives events instantly.
Filter by subject when multiple emails arrive. Some applications send both a welcome email and a separate confirmation email. The message list is ordered by received_at descending. If you expect multiple messages, filter by subject rather than taking messages[0] blindly.
messages = [m for m in all_messages if "confirm" in m["subject"].lower()]
Store the session token securely within the test process. The session_token is the only credential protecting the inbox. Do not log it. Do not pass it as a URL query parameter in requests your application logs. In test output, redact it if you are capturing HTTP traffic.
Respect rate limits. The free plan imposes a quota on active mailboxes. For large parallel pipelines, consider an authenticated API key on a higher-tier plan so quota enforcement is per-account rather than global. If your tests hit 429, add jitter to inbox creation calls or pre-create a pool of inboxes at session start.
10. Conclusion
Email confirmation flows are a well-understood engineering problem with a poorly-understood testing problem. The gap between "we mock the email sender" and "we verify the entire flow from registration through confirmation" is where production bugs live.
A temporary email API turns email into a first-class test primitive: create an inbox in one HTTP call, get push-notified when mail arrives, read the body, extract the token, assert the outcome, and expire the inbox when you are done. The entire flow fits in a pytest fixture. It runs in a GitHub Actions job with no external services beyond a staging environment and a temporary inbox endpoint.
The API described in this article — real SMTP ingestion via AWS SES, REST inbox management, and WebSocket event streaming — is available at https://uncorreotemporal.com. Anonymous use requires no signup: POST /api/v1/mailboxes returns a working inbox immediately. If you are building a pipeline that needs multiple inboxes, controlled TTLs, or authenticated access, an API key is available via account registration.
Your email confirmation tests should be as reliable as your unit tests. They can be.
Written by
Software Engineer · Sr. Python Developer · AWS Certified Solutions Architect
Software engineer with 20 years of experience building Python backends, cloud infrastructure, and AI agent tooling. Builder of UnCorreoTemporal.
LinkedInReady to give your AI agents a real inbox?
Create your first temporary mailbox in 30 seconds. Free plan available.
Create your free mailbox