Get the FREE Ultimate OpenClaw Setup Guide →

automating-mac-apps

npx machina-cli add skill SpillwaveSolutions/automating-mac-apps-plugin/automating-mac-apps --openclaw
Files (1)
SKILL.md
10.3 KB

Automating macOS Apps (Apple Events, AppleScript, JXA)

Technology Status

JXA and AppleScript are legacy (last major updates: 2015-2016). Modern alternatives:

  • PyXA: Active Python automation (see installation below)
  • Shortcuts App: Visual workflow builder
  • Swift/Objective-C: Production-ready automation

macOS Sequoia 15 Notes

Test scripts on target macOS due to:

  • Stricter TCC permissions
  • Enhanced Apple Events security
  • Sandbox improvements

Core framing (use this mental model)

  • Apple Events: The underlying inter-process communication (IPC) transport for macOS automation.
  • AppleScript: The original DSL (Domain-Specific Language) for Apple Events: best for discovery, dictionaries, and quick prototypes.
  • JXA (JavaScript for Automation): A JavaScript binding to the Apple Events layer: best for data handling, JSON, and maintainable logic.
  • ObjC Bridge: JavaScript access to Objective-C frameworks (Foundation, AppKit) for advanced macOS capabilities beyond app dictionaries.

When to use which

  • Use AppleScript for discovery, dictionary exploration, UI scripting, and quick one-offs.
  • Use JXA for robust logic, JSON pipelines, integration with Python/Node, and long-lived automation.
  • Hybrid pattern (recommended): discover in AppleScript, implement in JXA for production use.

When to use this skill

  • Foundation for app-specific skills (Calendar, Notes, Mail, Keynote, Excel, Reminders).
  • Run the automation warm-up scripts before first automation to surface macOS permission prompts.
  • Use as the base reference for permissions, shell integration, and UI scripting fallbacks.

Workflow (default)

  1. Identify target app + dictionary using Script Editor.
  2. Prototype a minimal command that works.
    • Example: tell application "Finder" to get name of first item in desktop
  3. Decide language using the rules above.
  4. Harden: add error handling, timeouts, and permission checks.
    • JXA: try { ... } catch (e) { console.log('Error:', e.message); }
    • AppleScript: try ... on error errMsg ... end try
  5. Validate: run a read-only command and check outputs.
    • Example: osascript -e 'tell application "Finder" to get name of home'
    • Confirm: Output matches expected (e.g., user home folder name)
  6. Integrate via osascript for CLI or pipeline use.
  7. UI scripting fallback only when the dictionary is missing/incomplete.
    • Example UI Script: tell application "System Events" to click button 1 of window 1 of process "App"
    • Example JXA: Application('System Events').processes.byName('App').windows[0].buttons[0].click()

Validation Checklist

  • Automation/Accessibility permissions granted (System Settings > Privacy & Security)
  • App is running: Application("App").running() returns true
  • State checked before acting (e.g., folder exists)
  • Dictionary method used (not UI scripting)
  • Delays/retries added for UI operations
  • Read-only test command succeeds
  • Output matches expected values

Automation permission warm-up

  • Use before first automation run or after macOS updates to surface prompts:
    • All apps at once: skills/automating-mac-apps/scripts/request_automation_permissions.sh (or .py).
    • Per-app: skills/automating-<app>/scripts/set_up_<app>_automation.{sh,py} (calendar, notes, mail, keynote, excel, reminders).
    • Voice Memos (no dictionary): skills/automating-voice-memos/scripts/set_up_voice_memos_automation.sh to activate app + check data paths; enable Accessibility + consider Full Disk Access.
  • Run from the same host you intend to automate with (Terminal vs Python) so the correct app gets Automation approval.
  • Each script runs read-only AppleScript calls (list accounts/calendars/folders, etc.) to request Terminal/Python control; click “Allow” when prompted.

Modern Python Alternatives to JXA

PyXA (Python for macOS Automation) - Preferred for new projects:

PyXA Features

  • Active development (v0.2.3+), modern Python syntax
  • App automation: Safari, Calendar, Reminders, Mail, Music
  • UI scripting, clipboard, notifications, AppleScript integration
  • Method chaining: app.lists().reminders().title()

PyXA Installation {#pyxa-installation}

# Install PyXA
pip install mac-pyxa

# Or with pip3 explicitly
pip3 install mac-pyxa

# Requirements:
# - Python 3.10+ (check with: python3 --version)
# - macOS 12+ (Monterey or later recommended)
# - PyObjC is installed automatically as a dependency

# Verify installation
python3 -c "import PyXA; print(f'PyXA {PyXA.__version__} installed successfully')"

Note: All app-specific skills in this plugin that show PyXA examples assume PyXA is installed. See this section for installation.

PyXA Example (Safari Automation)

import PyXA

# Launch Safari and navigate
safari = PyXA.Safari()
safari.activate()
safari.open_location("https://example.com")

# Get current tab URL
current_url = safari.current_tab.url
print(f"Current URL: {current_url}")

PyXA Example (Reminders)

import PyXA

reminders = PyXA.Reminders()
work_list = reminders.lists().by_name("Work")

# Add new reminder
new_reminder = work_list.reminders().push({
    "name": "Review PyXA documentation",
    "body": "Explore modern macOS automation options"
})

PyXA Official Resources:


PyObjC (Python-Objective-C Bridge) - For Low-Level macOS Integration:

PyObjC Capabilities

  • Direct Framework Access: AppKit, Foundation, and all macOS frameworks
  • Apple Events: Send Apple Events via Scripting Bridge
  • Script Execution: Run AppleScript or JXA from Python
  • System APIs: Direct access to CalendarStore, AddressBook, SystemEvents

Installation

pip install pyobjc
# Installs bridges for major frameworks

PyObjC Example (AppleScript Execution)

from Foundation import NSAppleScript

# Execute AppleScript from Python
script_source = '''
tell application "Safari"
    return URL of current tab
end tell
'''

script = NSAppleScript.alloc().initWithSource_(script_source)
result, error = script.executeAndReturnError_(None)

if error:
    print(f"Error: {error}")
else:
    print(f"Current Safari URL: {result.stringValue()}")

PyObjC Example (App Control via Scripting Bridge)

from ScriptingBridge import SBApplication

# Control Mail app
mail = SBApplication.applicationWithBundleIdentifier_("com.apple.Mail")
inbox = mail.inboxes()[0]  # Access first inbox

# Get unread message count
unread_count = inbox.unreadCount()
print(f"Unread messages: {unread_count}")

PyObjC Official Resources:


JXA Status

JXA has no updates since 2016. Use PyXA for new projects when possible.

When Not to Use

  • Cross-platform automation (use Selenium/Playwright for web)
  • Full UI testing (use XCUITest or Appium)
  • Environments blocking Automation/Accessibility permissions
  • Non-macOS platforms
  • Simple shell scripting tasks (use Bash directly)

Related Skills

  • App-specific automation (create automating-[app] skills as needed)
  • ci-cd-tcc for advanced permission management in automated environments
  • mastering-applescript for AppleScript-focused workflows

Security Best Practices

Permission Management:

  • Request minimal required permissions to reduce security risks
  • Use code signing for production scripts (Developer ID certificate)
  • Store credentials securely (Keychain, not hardcoded)
  • Validate all inputs to prevent injection attacks

Official Apple Security Guidance:

Output expectations

  • Keep examples minimal and runnable.
  • JSON Output: For CLI pipelines, use JSON.stringify(result) in JXA.
    • Example: console.log(JSON.stringify({files: files, count: files.length}))
  • Exit Codes: Ensure osascript exits with 0 for success, non-zero for failure.

What to load

Tier 1: Essentials (Start Here)

  • JXA Syntax & Patterns: automating-mac-apps/references/basics.md
  • AppleScript Basics: automating-mac-apps/references/applescript-basics.md
  • Cookbook (Common Recipes): automating-mac-apps/references/recipes.md

Tier 2: Advanced & Production

  • JXA Cookbook (Condensed): automating-mac-apps/references/cookbook.md
  • Performance Patterns: automating-mac-apps/references/applescript-performance.md
  • CI/CD & Permissions: automating-mac-apps/references/ci-cd-tcc.md
  • Shell Environment: automating-mac-apps/references/shell-environment.md
  • UI Scripting Inspector: automating-mac-apps/references/ui-scripting-inspector.md

Tier 3: Specialized & Reference

  • PyXA Core API Reference (complete class/method docs): automating-mac-apps/references/pyxa-core-api-reference.md
  • PyXA Basics: automating-mac-apps/references/pyxa-basics.md (Modern Python automation fundamentals)
  • AppleScript → PyXA Conversion: automating-mac-apps/references/applescript-to-pyxa-conversion.md (Migration guide with examples)
  • Translation Checklist (AppleScript → JXA): automating-mac-apps/references/translation-checklist.md (Comprehensive guide with examples and pitfalls)
  • JXA Helpers Library: automating-mac-apps/references/helpers.js
  • whose Batching Patterns: automating-mac-apps/references/whos-batching.md
  • Dictionary Strategies: automating-mac-apps/references/dictionary-strategies.md
  • ASObjC Helpers: automating-mac-apps/references/applescript-asobjc.md

Related Skills:

  • web-browser-automation: Complete browser automation guide (Chrome, Edge, Brave, Arc)

Source

git clone https://github.com/SpillwaveSolutions/automating-mac-apps-plugin/blob/main/plugins/automating-mac-apps-plugin/skills/automating-mac-apps/SKILL.mdView on GitHub

Overview

Automating macOS Apps uses Apple Events to control apps with AppleScript for discovery, JXA for robust logic, and PyXA for modern Python automation. It covers when to use each path, how to prototype, harden, and validate automation, and positions PyXA and other modern options as current alternatives.

How This Skill Works

Automation uses Apple Events as the IPC layer. Start by identifying the target app and its dictionary in Script Editor, prototype a minimal command, then decide among AppleScript, JXA, or PyXA based on the task. Harden with error handling, timeouts, and permission checks, and validate outputs with read only tests before integrating via osascript.

When to Use It

  • Foundation for app specific automation like Calendar, Notes, Mail, Keynote, Excel, and Reminders
  • Need discovery and dictionary exploration or quick UI scripting with AppleScript
  • Require robust logic and JSON pipelines or long running automation with JXA
  • Prefer modern Python automation using PyXA
  • Use the hybrid pattern: discover in AppleScript, implement in JXA for production

Quick Start

  1. Step 1: Identify target app + dictionary using Script Editor
  2. Step 2: Prototype a minimal command and decide between AppleScript, JXA, or PyXA
  3. Step 3: Harden, validate with a read only test, and integrate via osascript

Best Practices

  • Use AppleScript for discovery and dictionary exploration; keep scripts small
  • Leverage JXA for data handling, JSON pipelines, and longer workflows
  • Adopt the hybrid approach: discover with AppleScript, implement with JXA
  • Harden scripts with error handling, timeouts, and permission checks
  • Validate with read only tests and integrate via osascript

Example Use Cases

  • tell application Finder to get name of first item in desktop
  • osascript -e 'tell application Finder to get name of home'
  • try { ... } catch (e) { console.log('Error:', e.message); }
  • tell application 'System Events' to click button 1 of window 1 of process 'App'
  • Application('System Events').processes.byName('App').windows[0].buttons[0].click()

Frequently Asked Questions

Add this skill to your agents
Sponsor this space

Reach thousands of developers