Advanced

Enhancements

You now have a fully functional AI email assistant. This lesson covers advanced enhancements: multi-account support, Outlook integration, email scheduling, privacy and security best practices, production deployment, and a comprehensive FAQ.

Multi-Account Support

To manage multiple Gmail accounts, extend the configuration and authentication to support multiple credential sets:

# app/config.py - Multi-account extension

@dataclass
class AccountConfig:
    """Configuration for a single email account."""
    name: str
    provider: str  # "gmail" or "outlook"
    credentials_path: str
    token_path: str
    email_address: str


@dataclass
class MultiAccountConfig:
    """Multi-account configuration."""
    accounts: list = field(default_factory=list)

    @classmethod
    def from_env(cls):
        """Load accounts from environment variables.

        Format:
        ACCOUNT_1_NAME=Personal Gmail
        ACCOUNT_1_PROVIDER=gmail
        ACCOUNT_1_CREDENTIALS=credentials/personal_client_secret.json
        ACCOUNT_1_TOKEN=credentials/personal_token.json
        ACCOUNT_1_EMAIL=user@gmail.com
        """
        accounts = []
        i = 1
        while True:
            name = os.getenv(f"ACCOUNT_{i}_NAME")
            if not name:
                break
            accounts.append(AccountConfig(
                name=name,
                provider=os.getenv(f"ACCOUNT_{i}_PROVIDER", "gmail"),
                credentials_path=os.getenv(f"ACCOUNT_{i}_CREDENTIALS", ""),
                token_path=os.getenv(f"ACCOUNT_{i}_TOKEN", ""),
                email_address=os.getenv(f"ACCOUNT_{i}_EMAIL", ""),
            ))
            i += 1
        return cls(accounts=accounts)

# app/gmail/auth.py - Multi-account authentication

class MultiAccountGmailAuth:
    """Manage OAuth credentials for multiple Gmail accounts."""

    def __init__(self):
        self.services = {}

    def get_service(self, account: AccountConfig):
        """Get or create a Gmail service for a specific account."""
        if account.email_address in self.services:
            return self.services[account.email_address]

        creds = None
        if os.path.exists(account.token_path):
            creds = Credentials.from_authorized_user_file(
                account.token_path, config.gmail_scopes
            )

        if not creds or not creds.valid:
            if creds and creds.expired and creds.refresh_token:
                creds.refresh(Request())
            else:
                flow = InstalledAppFlow.from_client_secrets_file(
                    account.credentials_path, config.gmail_scopes
                )
                creds = flow.run_local_server(port=0)

            with open(account.token_path, "w") as f:
                f.write(creds.to_json())

        service = build("gmail", "v1", credentials=creds)
        self.services[account.email_address] = service
        return service

Outlook / Microsoft Graph Support

To add Outlook support, use the Microsoft Graph API. The authentication flow uses MSAL (Microsoft Authentication Library):

# Install the Microsoft Graph SDK
pip install msal msgraph-sdk

# app/outlook/client.py
"""Microsoft Outlook email client using Graph API."""
import msal
import requests
from app.config import config


class OutlookClient:
    """Client for interacting with Microsoft Graph API for Outlook."""

    GRAPH_URL = "https://graph.microsoft.com/v1.0"
    SCOPES = [
        "https://graph.microsoft.com/Mail.Read",
        "https://graph.microsoft.com/Mail.Send",
        "https://graph.microsoft.com/Mail.ReadWrite",
    ]

    def __init__(self, client_id: str, tenant_id: str, client_secret: str):
        self.app = msal.ConfidentialClientApplication(
            client_id,
            authority=f"https://login.microsoftonline.com/{tenant_id}",
            client_credential=client_secret,
        )
        self._token = None

    def _get_token(self) -> str:
        """Get or refresh access token."""
        result = self.app.acquire_token_for_client(scopes=self.SCOPES)
        if "access_token" in result:
            self._token = result["access_token"]
            return self._token
        raise Exception(f"Auth failed: {result.get('error_description')}")

    def _headers(self) -> dict:
        """Get authorization headers."""
        return {
            "Authorization": f"Bearer {self._get_token()}",
            "Content-Type": "application/json",
        }

    def fetch_messages(self, folder: str = "inbox", top: int = 50) -> list:
        """Fetch messages from a folder."""
        url = (
            f"{self.GRAPH_URL}/me/mailFolders/{folder}/messages"
            f"?$top={top}&$orderby=receivedDateTime desc"
        )
        resp = requests.get(url, headers=self._headers())
        resp.raise_for_status()
        return resp.json().get("value", [])

    def send_message(self, to: str, subject: str, body: str) -> dict:
        """Send an email message."""
        url = f"{self.GRAPH_URL}/me/sendMail"
        payload = {
            "message": {
                "subject": subject,
                "body": {"contentType": "Text", "content": body},
                "toRecipients": [
                    {"emailAddress": {"address": to}}
                ],
            }
        }
        resp = requests.post(url, headers=self._headers(), json=payload)
        resp.raise_for_status()
        return {"status": "sent"}

    def reply_to_message(
        self, message_id: str, body: str
    ) -> dict:
        """Reply to a specific message."""
        url = f"{self.GRAPH_URL}/me/messages/{message_id}/reply"
        payload = {
            "message": {
                "body": {"contentType": "Text", "content": body}
            }
        }
        resp = requests.post(url, headers=self._headers(), json=payload)
        resp.raise_for_status()
        return {"status": "sent"}

Email Scheduling

Add the ability to schedule draft sends for a specific date and time:

# app/scheduler.py - Scheduled sends

from datetime import datetime
from app.database import SessionLocal, Draft, Email


class ScheduledSender:
    """Send drafts at scheduled times."""

    def check_and_send_scheduled(self):
        """
        Check for drafts scheduled to send and send them.
        Called by the background scheduler every minute.
        """
        session = SessionLocal()
        try:
            now = datetime.utcnow()

            # Find drafts with status='scheduled' and send_at <= now
            scheduled = (
                session.query(Draft)
                .filter(
                    Draft.status == "scheduled",
                    Draft.sent_at <= now,  # Reusing sent_at as schedule time
                )
                .all()
            )

            for draft in scheduled:
                email = session.query(Email).get(draft.email_id)
                if not email:
                    continue

                try:
                    from app.gmail.client import GmailClient
                    client = GmailClient()
                    client.send_message(
                        to=email.sender,
                        subject=f"Re: {email.subject}",
                        body=draft.content,
                        thread_id=email.thread_id,
                    )
                    draft.status = "sent"
                    draft.sent_at = now
                    print(f"Scheduled send complete: Re: {email.subject}")
                except Exception as e:
                    print(f"Scheduled send failed: {e}")
                    draft.status = "send_failed"

            session.commit()

        except Exception as e:
            session.rollback()
            print(f"Scheduled send check error: {e}")
        finally:
            session.close()


# Add to create_scheduler():
# scheduler.add_job(
#     func=ScheduledSender().check_and_send_scheduled,
#     trigger=IntervalTrigger(minutes=1),
#     id="scheduled_sends",
#     name="Process scheduled sends",
# )

Privacy and Security Best Practices

When building an AI email assistant that processes personal communications, privacy is critical:

Data Handling

• Local-first storage: All email data stays in your local SQLite database. No data is sent to third parties except OpenAI for classification and drafting.
• Minimize data sent to LLM: Only send the minimum necessary context (subject, truncated body). Never send attachments or full HTML to the LLM.
• Token retention: Store OAuth tokens securely. Use file permissions (chmod 600) on credential files. Never commit them to version control.
• Database encryption: For production, use SQLCipher for encrypted SQLite or migrate to PostgreSQL with encrypted storage.

OpenAI API Privacy

# Privacy-focused OpenAI configuration
# As of 2024, OpenAI does NOT use API data for training.
# But you can add extra safeguards:

PRIVACY_SYSTEM_PROMPT = """
IMPORTANT: Do not memorize, store, or reference any personal
information from these emails beyond this conversation.
Treat all email content as confidential.
"""

# Use this in every LLM call:
messages = [
    {"role": "system", "content": PRIVACY_SYSTEM_PROMPT + original_prompt},
    {"role": "user", "content": email_content},
]

Access Controls

# app/web/auth.py - Simple authentication for the dashboard
"""Basic authentication middleware for the web dashboard."""
from functools import wraps
from flask import request, Response


def require_auth(f):
    """Decorator to require basic auth on routes."""
    @wraps(f)
    def decorated(*args, **kwargs):
        auth = request.authorization
        if not auth or not _check_credentials(auth.username, auth.password):
            return Response(
                "Authentication required",
                401,
                {"WWW-Authenticate": 'Basic realm="AI Email Assistant"'},
            )
        return f(*args, **kwargs)
    return decorated


def _check_credentials(username: str, password: str) -> bool:
    """Verify credentials against environment variables."""
    import os
    expected_user = os.getenv("DASHBOARD_USERNAME", "admin")
    expected_pass = os.getenv("DASHBOARD_PASSWORD", "changeme")
    return username == expected_user and password == expected_pass

Production Deployment

For production use, add these improvements:

# Dockerfile
FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Production WSGI server
RUN pip install gunicorn

COPY . .

# Create credentials directory
RUN mkdir -p credentials

EXPOSE 5000

CMD ["gunicorn", "-w", "2", "-b", "0.0.0.0:5000", "run:create_app()"]

# docker-compose.yml
version: "3.8"

services:
  email-assistant:
    build: .
    ports:
      - "5000:5000"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - OPENAI_MODEL=gpt-4o-mini
      - DATABASE_URL=sqlite:///data/email_assistant.db
      - FLASK_SECRET_KEY=${FLASK_SECRET_KEY}
      - DASHBOARD_USERNAME=${DASHBOARD_USERNAME}
      - DASHBOARD_PASSWORD=${DASHBOARD_PASSWORD}
    volumes:
      - ./data:/app/data
      - ./credentials:/app/credentials
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:5000/api/health"]
      interval: 30s
      timeout: 10s
      retries: 3

Cost Optimization

Keep your OpenAI costs minimal with these strategies:

• Use gpt-4o-mini: At $0.15/1M input tokens and $0.60/1M output tokens, it is 30x cheaper than gpt-4o with comparable quality for classification tasks.
• Batch processing: Classify and generate drafts in batches rather than one-at-a-time to reduce API overhead.
• Skip newsletters: Do not classify or draft replies for emails labeled as "newsletter" or "notification" by Gmail.
• Cache classifications: Once an email is classified, never reclassify it. The classification is stored in the database permanently.
• Template over LLM: Use the template system for common response types instead of generating with the LLM every time.
• Estimated monthly cost: For 200 emails/day with classification + selective drafting: approximately $1.50-3.00/month.

Frequently Asked Questions

Can I use a different LLM instead of OpenAI?

Yes. The code uses the OpenAI Python client, which is compatible with any provider that supports the OpenAI API format. To use a local model (Ollama, llama.cpp), change the base URL:

client = OpenAI(
    api_key="not-needed",
    base_url="http://localhost:11434/v1"  # Ollama
)

Will this work with Google Workspace (business Gmail)?

Yes. The Gmail API works identically for personal Gmail and Google Workspace accounts. For Workspace, the admin may need to allow the OAuth app in the Admin Console.

How do I handle rate limits?

The Gmail API allows 250 quota units per second (a message.get costs 5 units). Our polling approach stays well within limits. For OpenAI, gpt-4o-mini allows 30,000 requests per minute on Tier 1, which is more than enough.

Can I process attachments?

The Gmail API supports downloading attachments. You could add a step to the ingestion pipeline that extracts text from PDF/DOCX attachments and includes it in the classification context. Be mindful of token costs when sending attachment text to the LLM.

How do I add custom classification categories?

Edit the CLASSIFICATION_PROMPT in app/ai/classifier.py. Add your new categories to the prompt description and to the validation list in _validate_enum(). No code changes needed beyond the prompt and validation.

Is my email data sent to OpenAI?

Only email subject lines and truncated body text (up to 3,000 characters) are sent to the OpenAI API for classification and draft generation. Attachments, images, and full HTML are never sent. As of 2024, OpenAI does not use API data for model training. You can further limit data exposure by reducing the body truncation limit.

Can I run this without the web interface?

Yes. All features work as standalone Python modules. You can run email fetching, classification, and draft generation from the command line or scripts without starting the Flask server. The scheduler can run independently too.

What You Have Built

Congratulations! You now have a complete, production-ready AI email assistant with:

• Gmail OAuth 2.0 integration with automatic token refresh
• LLM-powered email classification (priority, category, sentiment)
• Context-aware draft generation with tone matching and templates
• Email summarization and action item extraction
• Follow-up reminders and daily digest generation
• A web dashboard with priority inbox, draft review, and send workflow
• Background scheduling for automatic email processing
• Extensible architecture supporting multi-account and multi-provider

The total codebase is under 1,500 lines of Python. Every component is modular and can be extended or replaced independently. The monthly running cost is under $3 with gpt-4o-mini.