Zelflayer: The Fully-Local macOS Assistant That Understands Your Documents Through Voice

How one powerful AppleScript tool + vector + graph search created the ultimate private AI assistant

I was drowning in documents. Research papers scattered across my Downloads folder, meeting notes buried in subdirectories, contracts I signed months ago that I suddenly needed to reference. Like most knowledge workers, I had built a personal digital library over years, but it was opaque—searchable only by filename and hope.

Meanwhile, the AI assistants I used were cloud-based black boxes that couldn't access my local files, required constant internet connectivity, and raised serious privacy concerns when dealing with sensitive documents.

I needed something different: a fully local, voice-activated assistant that could actually understand my document collection and help me navigate my digital life.

The breakthrough came from an unexpected place: AppleScript.

Most AI agent frameworks use rigid, pre-defined tools. You want to check your calendar? Here's a get_calendar_events() function. Need to send an email? Use send_email(). But what happens when Apple changes their API? Or when you need to interact with a niche application? The whole system breaks.

I realized the solution wasn't more tools—it was one infinitely flexible tool that could do everything: dynamic AppleScript generation.

Here's the entire "tool" that powers Zelflayer's macOS automation:

async def execute_applescript(script: str) -> str:
    """Execute any AppleScript code and return the result"""
    stdout, stderr = run_applescript_capture(script)
    return stdout if not stderr else f"Error: {stderr}"

That's it. One function. But it's Turing-complete access to every macOS application, service, and system function. No API limits, no version dependencies, no maintenance overhead.

Let me show you what this looks like in practice:

User Request: "Find the latest email from Sarah and create a calendar event for tomorrow at 2pm titled 'Follow up with Sarah' with a reminder"

Agent's Thought Process:

1. Query Mail app for Sarah's latest email
2. Extract the subject line for context
3. Create calendar event with proper date formatting
4. Create reminder linked to the event
5. Confirm everything was created successfully

The Generated AppleScript:

tell application "Mail"
    set theMessages to messages of inbox whose sender contains "sarah"
    if (count of theMessages) > 0 then
        set theLastMessage to item -1 of theMessages
        return subject of theLastMessage & " sent on " & (date sent of theLastMessage as string)
    end if
end tell

The agent iteratively writes, executes, and refines AppleScript until the task is complete. If one approach fails, it tries another. No human intervention required.

But macOS automation was only half the battle. I needed the assistant to actually understand my documents, not just execute commands.

Traditional RAG (Retrieval-Augmented Generation) systems use vector search—finding semantically similar text chunks. This works well for general similarity but misses the rich relationships between entities in your documents.

Zelflayer's breakthrough combines both approaches:

similar_chunks = await vector_search_chunks(query_embedding, top_k=3)

graph_data = await get_1hop_bidirectional_graph("Apple Inc")

combined_context = f"""
Vector Knowledge:
{vector_context}

Graph Knowledge:
{graph_context}
"""

This hybrid approach means when you ask "What were Apple's key partnerships last quarter?", the system can:

• Find relevant financial documents via vector similarity
• Extract entity relationships showing Apple's partnerships
• Combine both for a comprehensive answer

Here's how this plays out in practice:

• Agent checks Calendar app via AppleScript

• Searches Mail for meeting-related keywords

• Cross-references attendees with Contacts

• Provides consolidated briefing

• Converts PDF to structured document using Docling

• Extracts knowledge graph of entities and relationships

• Creates searchable chunks with embeddings

• Answers questions about the content with full context

• Searches through processed documents

• Identifies action items and deadlines

• Creates Reminders with appropriate due dates

• Links back to source documents

Everything runs locally:

• Whisper tiny.en: Voice transcription on-device
• Ollama: Local LLM inference (Granite models)
• SurrealDB: Embedded graph+vector database
• MLX: Apple's machine learning framework for Mac optimization

No data leaves your machine. Your documents, voice commands, and personal information stay private.

• Voice Interface: Faster-Whisper + PyAudio for recording
• Document Processing: Docling for PDF→structured text + REBEL for entity extraction
• Storage: SurrealDB for hybrid graph/vector storage
• Agent: Code-generating LLM that writes AppleScript
• Integration: One AppleScript execution tool that does everything

This architecture scales beyond documents:

• Voice-controlled workflows: "Process all PDFs in this folder and summarize them"
• Cross-application automation: "Find receipts in Mail and add them to a Numbers spreadsheet"
• Intelligent scheduling: "Based on my calendar availability and these meeting notes, schedule follow-ups"
• Research assistance: "Find all references to 'climate change' in my documents and create a summary note"

Zelflayer represents a fundamental shift: from cloud-dependent AI assistants that know nothing about your life, to personal AI that lives in your digital world. It sees your documents, understands your schedule, knows your contacts, and can act across all your applications.

The combination of dynamic AppleScript generation with vector+graph RAG creates something unique: an assistant that doesn't just respond to commands, but understands your context and takes action across your entire digital ecosystem.

Fully local. Voice-activated. Infinitely capable.

That's the power of one simple AppleScript tool combined with intelligent document understanding.

Try it yourself: The Zelflayer project is open source. One AppleScript function. Endless possibilities.