web-browser-automation
Comprehensive macOS browser automation using PyXA, Playwright, Selenium, and Puppeteer for desktop web testing, scraping, and workflow automation. Use when asked to "automate web browsers", "Selenium Chrome automation", "Playwright testing", "Puppeteer scraping", or "cross-browser automation". Supports Chrome, Edge, Brave, Arc browsers.
Packaged view
This page reorganizes the original catalog entry around fit, installability, and workflow context first. The original raw source lives below.
Install command
npx @skill-hub/cli install spillwavesolutions-automating-mac-apps-plugin-web-browser-automation
Repository
Skill path: plugins/automating-mac-apps-plugin/skills/web-browser-automation
Comprehensive macOS browser automation using PyXA, Playwright, Selenium, and Puppeteer for desktop web testing, scraping, and workflow automation. Use when asked to "automate web browsers", "Selenium Chrome automation", "Playwright testing", "Puppeteer scraping", or "cross-browser automation". Supports Chrome, Edge, Brave, Arc browsers.
Open repositoryBest for
Primary workflow: Ship Full Stack.
Technical facets: Full Stack, Testing.
Target audience: everyone.
License: Unknown.
Original source
Catalog source: SkillHub Club.
Repository owner: SpillwaveSolutions.
This is still a mirrored public skill entry. Review the repository before installing into production workflows.
What it helps with
- Install web-browser-automation into Claude Code, Codex CLI, Gemini CLI, or OpenCode workflows
- Review https://github.com/SpillwaveSolutions/automating-mac-apps-plugin before adding web-browser-automation to shared team environments
- Use web-browser-automation for development workflows
Works across
Favorites: 0.
Sub-skills: 0.
Aggregator: No.
Original source / Raw SKILL.md
---
name: web-browser-automation
description: Comprehensive macOS browser automation using PyXA, Playwright, Selenium, and Puppeteer for desktop web testing, scraping, and workflow automation. Use when asked to "automate web browsers", "Selenium Chrome automation", "Playwright testing", "Puppeteer scraping", or "cross-browser automation". Supports Chrome, Edge, Brave, Arc browsers.
allowed-tools:
- Bash
- Read
- Write
---
# macOS Web Browser Automation Guide
## Table of Contents
1. [Overview](#overview)
2. [Browser Compatibility Matrix](#browser-compatibility-matrix)
3. [PyXA Integration](#pyxa-integration)
4. [Playwright Automation](#playwright-automation)
5. [Selenium WebDriver](#selenium-webdriver)
6. [Puppeteer Node.js](#puppeteer-nodejs)
7. [Comprehensive Automation Workflows](#comprehensive-automation-workflows)
- [Workflow 1: Multi-Browser Tab Management](#workflow-1-multi-browser-tab-management)
- [Workflow 2: Automated Research and Data Collection](#workflow-2-automated-research-and-data-collection)
- [Workflow 3: Cross-Browser Testing Suite](#workflow-3-cross-browser-testing-suite)
- [Workflow 4: Web Scraping and Data Extraction](#workflow-4-web-scraping-and-data-extraction)
8. [Brief Automation Patterns](#brief-automation-patterns)
9. [Advanced Techniques](#advanced-techniques)
10. [Troubleshooting and Validation](#troubleshooting-and-validation)
11. [Security Considerations](#security-considerations)
12. [Performance Optimization](#performance-optimization)
13. [Integration Examples](#integration-examples)
## Overview
This guide covers comprehensive web browser automation on macOS desktop, focusing on automation (not testing). We cover four major automation frameworks with practical examples for real-world scenarios.
**PyXA Installation:** To use PyXA examples in this skill, see the installation instructions in `automating-mac-apps` skill (PyXA Installation section).
### Primary Automation Tools
- **PyXA**: macOS-native Python wrapper with direct browser integration
- **Playwright**: Cross-platform framework with Python bindings for modern web automation
- **Selenium**: Industry-standard automation with ChromeDriver integration
- **Puppeteer**: Node.js framework for Chrome/Chromium automation
### Tool Selection Guide
| Tool | Primary Use | Key Advantages |
|------|-------------|----------------|
| **PyXA** | macOS-native control | Direct OS integration, Arc spaces |
| **Playwright** | Cross-browser testing | Auto-waiting, mobile emulation |
| **Selenium** | Legacy enterprise | Mature ecosystem, wide language support |
| **Puppeteer** | Headless Chrome | Fast execution, PDF generation |
See `references/browser-compatibility-matrix.md` for detailed browser support.
## Getting Started
1. **Choose your framework** based on your needs (see Tool Selection Guide above)
2. **Install dependencies** for your chosen framework
3. **Follow framework-specific guides** linked below
4. **Review workflows** for common automation patterns
## Framework Guides
### PyXA Browser Integration
- **Best for**: macOS-native browser control with Arc spaces support
- **Installation**: `pip install PyXA`
- **Guide**: `references/pyxa-integration.md`
### Playwright Automation
- **Best for**: Cross-browser testing with auto-waiting
- **Installation**: `pip install playwright && playwright install`
- **Guide**: `references/playwright-automation.md`
### Selenium WebDriver
- **Best for**: Legacy enterprise automation
- **Installation**: `pip install selenium`
- **Guide**: `references/selenium-webdriver.md`
### Puppeteer Node.js
- **Best for**: Headless Chrome with PDF generation
- **Installation**: `npm install puppeteer`
- **Guide**: `references/puppeteer-automation.md`
## Automation Workflows
Complete workflow examples for common automation scenarios:
### Multi-Browser Tab Management
**Guide**: `references/workflows.md#workflow-1-multi-browser-tab-management`
### Automated Research and Data Collection
**Guide**: `references/workflows.md#workflow-2-automated-research-and-data-collection`
### Cross-Browser Testing Suite
**Guide**: `references/workflows.md#workflow-3-cross-browser-testing-suite`
### Web Scraping and Data Extraction
**Guide**: `references/workflows.md#workflow-4-web-scraping-and-data-extraction`
## Brief Automation Patterns
### Browser Launch and Profile Management
```python
# PyXA approach for Chrome
chrome = PyXA.Application("Google Chrome")
chrome.new_window("https://example.com")
```
### Tab Organization and Grouping
```python
# PyXA tab filtering
tabs = chrome.windows()[0].tabs()
work_tabs = [tab for tab in tabs if "meeting" in tab.title().lower()]
for tab in work_tabs:
tab.close()
```
### JavaScript Injection for Content Extraction
```python
# PyXA JavaScript execution
content = tab.execute_javascript("document.body.innerText")
links = tab.execute_javascript("Array.from(document.querySelectorAll('a')).map(a => a.href)")
```
### Cross-Browser Synchronization
```python
# PyXA multi-browser control
browsers = [PyXA.Application("Google Chrome"), PyXA.Application("Microsoft Edge")]
for browser in browsers:
browser.new_tab("https://shared-resource.com")
```
### Form Filling and Interaction
```python
# Playwright auto-waiting
page.fill("#username", "[email protected]")
page.click("text=Submit") # Auto-waits for element
```
### Screenshot and Content Capture
```javascript
// Puppeteer screenshot
await page.screenshot({ path: 'capture.png', fullPage: true });
await page.pdf({ path: 'page.pdf', format: 'A4' });
```
### Playwright Quickstart
```python
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("https://example.com")
page.click("text=Get Started") # Auto-waits for element
page.fill("#search-input", "automation")
browser.close()
```
For advanced Playwright features (contexts, viewports, dynamic content), see `references/playwright-automation.md`.
## Additional Resources
### Advanced Techniques
- **Network Interception**: `references/playwright-automation.md#network-interception`
- **Parallel Browser Automation**: `references/selenium-webdriver.md#parallel-testing`
- **Performance Monitoring**: `references/puppeteer-automation.md#performance-monitoring`
- **Browser Context Management**: `references/selenium-webdriver.md#browser-contexts-and-pages`
### Validation Checklist
After implementing browser automation:
- [ ] Verify browser launches without errors
- [ ] Confirm page navigation completes successfully
- [ ] Test element selectors locate expected elements
- [ ] Validate extracted data matches page content
- [ ] Check screenshots/PDFs are generated correctly
- [ ] For PyXA: verify macOS permissions are granted
### Troubleshooting
- **Element Not Found Errors**: Common solutions across frameworks
- **Stale Element References**: Handling dynamic content
- **Browser Detection**: Avoiding automation detection
- **Network Timeout Issues**: Timeout configuration
### Security Considerations
- **Credential Management**: Secure storage of login credentials
- **Certificate Handling**: SSL/TLS certificate validation
- **Sandbox and Isolation**: Running automation in isolated environments
- **Data Sanitization**: Cleaning extracted data
### Performance Optimization
- **Browser Configuration**: Disabling unnecessary features
- **Network Optimization**: Blocking unwanted resources
- **Parallel Execution**: Running tests concurrently
- **Resource Pooling**: Managing browser instances efficiently
## When Not to Use
- For mobile browser automation (use Appium or native testing frameworks)
- For Windows/Linux-only environments (some PyXA features are macOS-only)
- When CAPTCHA solving is required (use specialized services)
- For production scraping without respecting robots.txt
## What to Load
- **PyXA**: `references/pyxa-integration.md` - macOS-native browser control
- **Playwright**: `references/playwright-automation.md` - Cross-browser testing
- **Selenium**: `references/selenium-webdriver.md` - Enterprise automation
- **Puppeteer**: `references/puppeteer-automation.md` - Node.js Chrome automation
- **Workflows**: `references/workflows.md` - Complete automation scenarios
---
## Referenced Files
> The following files are referenced in this skill and included for context.
### references/pyxa-integration.md
```markdown
# PyXA Browser Integration
## Installation
```bash
pip install PyXA
```
## Browser Classes
```python
# Chrome automation
chrome = PyXA.Application("Google Chrome")
# Arc with spaces
arc = PyXA.Application("Arc")
work_space = arc.spaces().by_title("Work")
work_space.focus()
```
## Tab Management
```python
# Get all tabs
tabs = chrome.windows()[0].tabs()
# Create new tab
new_tab = chrome.new_tab("https://example.com")
# Close specific tabs
for tab in tabs:
if "unwanted" in str(tab.url()):
tab.close()
```
## JavaScript Execution
```python
# Execute JavaScript in current tab
content = chrome.current_tab().execute_javascript("document.body.innerText")
# Extract page metadata
meta_description = chrome.current_tab().execute_javascript("""
const meta = document.querySelector('meta[name="description"]');
return meta ? meta.getAttribute('content') : null;
""")
```
## Bookmark Operations
```python
# Add current page to bookmarks
current_tab = chrome.windows()[0].active_tab
chrome.bookmarks_bar.make("bookmark", {
"title": current_tab.title(),
"url": str(current_tab.url())
})
# Access bookmarks
bookmarks = chrome.bookmarks_bar.bookmark_items()
for bookmark in bookmarks:
if "research" in bookmark.title().lower():
print(f"Research: {bookmark.title()} - {bookmark.url()}")
```
## Window Management
```python
# Create new window
new_window = chrome.new_window("https://example.com")
# Minimize inactive windows
windows = chrome.windows()
for window in windows[1:]: # Skip front window
if not window.minimized():
window.minimized = True
```
## Arc Browser Spaces
```python
# Arc-specific features
arc = PyXA.Application("Arc")
# List all spaces
spaces = arc.spaces()
for space in spaces:
print(f"Space: {space.name()}")
# Switch to specific space
work_space = spaces.by_title("Work")
work_space.focus()
```
## Error Handling
```python
try:
chrome = PyXA.Application("Google Chrome")
tab = chrome.current_tab()
if tab:
title = tab.title()
print(f"Current tab: {title}")
else:
print("No active tab found")
except Exception as e:
print(f"Browser automation error: {e}")
```
```
### references/playwright-automation.md
```markdown
# Playwright Browser Automation
## Installation
```bash
pip install playwright
playwright install # Downloads browser binaries
```
## Basic Usage
```python
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("https://example.com")
title = page.title()
print(f"Page title: {title}")
browser.close()
```
## Browser Types
```python
# Chromium (Chrome/Edge/Chromium)
browser = p.chromium.launch()
# Firefox
browser = p.firefox.launch()
# WebKit (Safari)
browser = p.webkit.launch()
```
## Advanced Launch Options
```python
browser = p.chromium.launch(
headless=False, # Show browser window
slow_mo=1000, # Slow down operations by 1 second
args=[
'--disable-blink-features=AutomationControlled',
'--no-sandbox',
'--disable-dev-shm-usage'
]
)
```
## Page Navigation and Interaction
```python
page = browser.new_page()
# Navigation
page.goto("https://example.com")
page.go_back()
page.go_forward()
page.reload()
# Auto-waiting interactions
page.click("text=Get Started") # Waits for element automatically
page.fill("#username", "[email protected]")
page.press("#username", "Enter")
# Wait for specific conditions
page.wait_for_selector(".results")
page.wait_for_load_state("networkidle")
```
## Element Selection and Manipulation
```python
# CSS selectors
button = page.locator("#submit-button")
button.click()
# Text-based selection
page.click("text=Sign In")
# XPath
page.click("//button[contains(text(), 'Submit')]")
# Multiple elements
links = page.locator("a").all()
for link in links:
print(link.get_attribute("href"))
```
## Screenshots and Recording
```python
# Screenshot full page
page.screenshot(path="fullpage.png")
# Screenshot specific element
element = page.locator(".content")
element.screenshot(path="element.png")
# Video recording
context = browser.new_context(record_video_dir="videos/")
page = context.new_page()
# ... automation ...
context.close() # Saves video
```
## Network Interception
```python
# Intercept and modify requests
def handle_request(request):
if "analytics" in request.url:
request.abort() # Block analytics
elif "api.example.com" in request.url:
# Modify API calls
request.continue_(headers={**request.headers, "X-Custom": "value"})
else:
request.continue_()
page.route("**/*", handle_request)
```
## File Upload and Download
```python
# File upload
page.set_input_files('input[type="file"]', 'path/to/file.pdf')
# Handle downloads
with page.expect_download() as download_info:
page.click('a[download]')
download = download_info.value
download.save_as("downloads/file.pdf")
```
## Mobile Emulation
```python
# iPhone simulation
context = browser.new_context(
viewport={"width": 375, "height": 667},
user_agent="Mozilla/5.0 (iPhone; CPU iPhone OS 14_0 like Mac OS X) AppleWebKit/605.1.15"
)
page = context.new_page()
```
## Cross-Browser Testing
```python
browsers = ["chromium", "firefox", "webkit"]
results = {}
for browser_name in browsers:
browser_type = getattr(p, browser_name)
browser = browser_type.launch()
try:
page = browser.new_page()
page.goto("https://httpbin.org/user-agent")
ua = page.text_content("body")
results[browser_name] = f"SUCCESS: {ua[:50]}..."
except Exception as e:
results[browser_name] = f"FAILED: {e}"
finally:
browser.close()
print("Cross-browser results:", results)
```
## Error Handling and Timeouts
```python
try:
page.goto("https://slowsite.com", timeout=10000)
page.click(".might-not-exist", timeout=5000)
except Exception as e:
print(f"Operation failed: {e}")
# Take screenshot for debugging
page.screenshot(path="error_screenshot.png")
```
## Configuration Files
```python
# playwright.config.py
import os
from playwright.sync_api import expect
os.environ["PWDEBUG"] = "1" # Enable debug mode
expect.set_options(timeout=10000) # Global timeout
# Configuration for different environments
config = {
"chromium": {
"headless": True,
"args": ["--no-sandbox"]
},
"firefox": {
"headless": True
},
"webkit": {
"headless": True
}
}
```
```
### references/selenium-webdriver.md
```markdown
# Selenium WebDriver Automation
## Installation
```bash
pip install selenium
# ChromeDriver automatically managed in v4.11+
```
## Basic Usage
```python
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://example.com")
title = driver.title
print(f"Page title: {title}")
driver.quit()
```
## Browser Configuration
```python
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless=new")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--window-size=1920,1080")
driver = webdriver.Chrome(options=options)
```
## Element Location and Interaction
```python
# Find elements
element = driver.find_element(By.ID, "username")
elements = driver.find_elements(By.CLASS_NAME, "item")
# Interactions
element.click()
element.send_keys("text input")
element.clear()
# Advanced selectors
from selenium.webdriver.common.by import By
email_field = driver.find_element(By.CSS_SELECTOR, "input[type='email']")
submit_btn = driver.find_element(By.XPATH, "//button[@type='submit']")
```
## Waiting Strategies
```python
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Explicit wait
wait = WebDriverWait(driver, 10)
element = wait.until(EC.presence_of_element_located((By.ID, "results")))
# Fluent wait with conditions
element = WebDriverWait(driver, 10, poll_frequency=1).until(
lambda driver: driver.find_element(By.ID, "dynamic-element")
)
```
## Actions and Advanced Interactions
```python
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(driver)
actions.move_to_element(menu).click(submenu).perform()
# Drag and drop
actions.drag_and_drop(source, target).perform()
# Keyboard shortcuts
from selenium.webdriver.common.keys import Keys
element.send_keys(Keys.CONTROL, 'a') # Select all
element.send_keys(Keys.DELETE) # Delete
```
## Window and Tab Management
```python
# Handle multiple windows
main_window = driver.current_window_handle
driver.find_element(By.LINK_TEXT, "Open New Window").click()
for handle in driver.window_handles:
if handle != main_window:
driver.switch_to.window(handle)
print(f"New window title: {driver.title}")
driver.close()
driver.switch_to.window(main_window)
```
## File Upload and Download
```python
# File upload
file_input = driver.find_element(By.CSS_SELECTOR, "input[type='file']")
file_input.send_keys("/path/to/file.pdf")
# Download handling
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_experimental_option("prefs", {
"download.default_directory": "/tmp/downloads",
"download.prompt_for_download": False,
"download.directory_upgrade": True
})
driver = webdriver.Chrome(options=options)
```
## Cookie Management
```python
# Get all cookies
cookies = driver.get_cookies()
# Add cookie
driver.add_cookie({"name": "session", "value": "abc123"})
# Delete cookie
driver.delete_cookie("session")
driver.delete_all_cookies()
```
## Screenshot and Visual Testing
```python
# Full page screenshot
driver.save_screenshot("page.png")
# Element screenshot
element = driver.find_element(By.ID, "content")
element.screenshot("element.png")
# PDF generation (Chrome only)
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_experimental_option("prefs", {
"printing.print_preview_sticky_settings.appState": json.dumps({
"recentDestinations": [{"id": "Save as PDF", "origin": "local"}],
"selectedDestinationId": "Save as PDF",
"version": 2
})
})
```
## Parallel Testing
```python
import threading
from selenium import webdriver
def test_browser(url):
driver = webdriver.Chrome()
driver.get(url)
title = driver.title
driver.quit()
return title
# Run tests in parallel
urls = ["https://site1.com", "https://site2.com", "https://site3.com"]
threads = []
for url in urls:
thread = threading.Thread(target=test_browser, args=(url,))
threads.append(thread)
thread.start()
for thread in threads:
thread.join()
```
## Grid and Remote Execution
```python
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
# Connect to Selenium Grid
driver = RemoteWebDriver(
command_executor="http://localhost:4444/wd/hub",
desired_capabilities=DesiredCapabilities.CHROME
)
driver.get("https://example.com")
```
## Page Object Model
```python
class LoginPage:
def __init__(self, driver):
self.driver = driver
self.username_field = (By.ID, "username")
self.password_field = (By.ID, "password")
self.login_button = (By.ID, "login")
def login(self, username, password):
self.driver.find_element(*self.username_field).send_keys(username)
self.driver.find_element(*self.password_field).send_keys(password)
self.driver.find_element(*self.login_button).click()
# Usage
driver = webdriver.Chrome()
login_page = LoginPage(driver)
login_page.login("[email protected]", "password")
```
```
### references/puppeteer-automation.md
```markdown
# Puppeteer Node.js Automation
## Installation
```bash
npm install puppeteer
```
## Basic Usage
```javascript
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await puppeteer.new_page();
await page.goto('https://example.com');
const title = await page.title();
console.log(`Page title: ${title}`);
await browser.close();
})();
```
## Launch Options
```javascript
const browser = await puppeteer.launch({
headless: 'new', // Modern headless mode
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-dev-shm-usage'
],
defaultViewport: {
width: 1280,
height: 720
}
});
```
## Page Navigation
```javascript
await page.goto('https://example.com', {
waitUntil: 'networkidle0',
timeout: 30000
});
await page.goBack();
await page.goForward();
await page.reload();
```
## Element Interaction
```javascript
// Click elements
await page.click('button.primary');
await page.click('#submit-btn');
// Fill forms
await page.type('#username', '[email protected]');
await page.type('#password', 'securepassword');
// Select from dropdowns
await page.select('#country', 'US');
// Checkboxes and radio buttons
await page.check('#agree-terms');
await page.uncheck('#newsletter');
```
## Content Extraction
```javascript
// Get text content
const text = await page.$eval('h1', el => el.textContent);
// Extract multiple elements
const links = await page.$$eval('a', anchors =>
anchors.map(a => ({ text: a.textContent, href: a.href }))
);
// Get page HTML
const html = await page.content();
```
## Screenshots and PDFs
```javascript
// Full page screenshot
await page.screenshot({
path: 'screenshot.png',
fullPage: true
});
// Element screenshot
const element = await page.$('.content');
await element.screenshot({ path: 'element.png' });
// Generate PDF
await page.pdf({
path: 'page.pdf',
format: 'A4',
printBackground: true,
margin: {
top: '1cm',
right: '1cm',
bottom: '1cm',
left: '1cm'
}
});
```
## JavaScript Execution
```javascript
// Execute JavaScript in page context
const dimensions = await page.evaluate(() => {
return {
width: document.documentElement.clientWidth,
height: document.documentElement.clientHeight,
deviceScaleFactor: window.devicePixelRatio
};
});
console.log('Page dimensions:', dimensions);
// Pass data to page context
await page.evaluate((data) => {
console.log('Data from Node.js:', data);
}, { message: 'Hello from Puppeteer' });
```
## Network Interception
```javascript
// Block unwanted resources
await page.setRequestInterception(true);
page.on('request', (request) => {
const resourceType = request.resourceType();
if (resourceType === 'image' || resourceType === 'media') {
request.abort();
} else {
request.continue();
}
});
// Monitor network activity
page.on('response', response => {
console.log(`${response.status()} ${response.url()}`);
});
```
## File Upload and Download
```javascript
// File upload
const input = await page.$('input[type="file"]');
await input.uploadFile('/path/to/file.pdf');
// Handle downloads
const [download] = await Promise.all([
page.waitForEvent('download'),
page.click('a[download]')
]);
await download.saveAs('/tmp/downloaded-file.pdf');
```
## Authentication and Cookies
```javascript
// Set authentication
await page.authenticate({
username: 'user',
password: 'pass'
});
// Cookie management
await page.setCookie({
name: 'session',
value: 'abc123',
domain: 'example.com'
});
const cookies = await page.cookies();
console.log('Cookies:', cookies);
```
## Mobile Emulation
```javascript
// iPhone simulation
await page.setViewport({
width: 375,
height: 667,
deviceScaleFactor: 2,
isMobile: true,
hasTouch: true
});
await page.setUserAgent(
'Mozilla/5.0 (iPhone; CPU iPhone OS 14_0 like Mac OS X) AppleWebKit/605.1.15'
);
```
## Performance Monitoring
```javascript
// Measure page load time
const startTime = Date.now();
await page.goto('https://example.com');
const loadTime = Date.now() - startTime;
console.log(`Page loaded in ${loadTime}ms`);
// Monitor performance metrics
const metrics = await page.metrics();
console.log('Performance metrics:', metrics);
// Capture traces
await page.tracing.start({ path: 'trace.json' });
// ... perform actions ...
await page.tracing.stop();
```
## Error Handling
```javascript
try {
await page.goto('https://example.com', { timeout: 10000 });
await page.click('.might-not-exist', { timeout: 5000 });
} catch (error) {
console.error('Operation failed:', error.message);
// Take screenshot for debugging
await page.screenshot({ path: 'error-screenshot.png' });
}
```
## Browser Contexts and Pages
```javascript
// Multiple pages
const page1 = await browser.newPage();
const page2 = await browser.newPage();
// Incognito context
const context = await browser.createIncognitoBrowserContext();
const privatePage = await context.newPage();
// Session isolation
const session1 = await browser.newContext();
const session2 = await browser.newContext();
```
```
### references/workflows.md
```markdown
# Browser Automation Workflows
## Workflow 1: Multi-Browser Tab Management (PyXA)
**Objective**: Synchronize tabs across Chrome, Edge, and Brave browsers for research workflows.
```python
import PyXA
from time import sleep
def sync_research_tabs():
# Initialize browser applications
chrome = PyXA.Application("Google Chrome")
edge = PyXA.Application("Microsoft Edge")
brave = PyXA.Application("Brave Browser")
browsers = [chrome, edge, brave]
# Open research URLs in each browser
research_urls = [
"https://scholar.google.com",
"https://arxiv.org",
"https://github.com"
]
for browser in browsers:
if not browser.frontmost:
browser.frontmost = True
sleep(0.5)
# Create new window for research session
research_window = browser.new_window()
# Open research tabs
for url in research_urls:
research_window.new_tab(url)
sleep(0.3)
# Organize tabs by pinning important ones
tabs = research_window.tabs()
if len(tabs) > 0:
# Pin first tab (Google Scholar)
tabs[0].execute_javascript("javascript:void(0)") # Focus tab
# Create summary report
summary = {
"chrome_tabs": len(chrome.windows()[0].tabs()) if chrome.windows() else 0,
"edge_tabs": len(edge.windows()[0].tabs()) if edge.windows() else 0,
"brave_tabs": len(brave.windows()[0].tabs()) if brave.windows() else 0
}
return summary
# Execute workflow
result = sync_research_tabs()
print(f"Research session initialized: {result}")
```
## Workflow 2: Automated Research and Data Collection (PyXA)
**Objective**: Extract and organize research data from multiple browser tabs using JavaScript execution and clipboard integration.
```python
import PyXA
import json
from datetime import datetime
def collect_research_data():
chrome = PyXA.Application("Google Chrome")
if not chrome.windows():
print("No Chrome windows open")
return {}
front_window = chrome.windows()[0]
research_data = {
"timestamp": datetime.now().isoformat(),
"tabs": [],
"bookmarks": []
}
# Collect tab information
tabs = front_window.tabs()
for tab in tabs:
if not tab.loading: # Only process loaded tabs
tab_info = {
"title": tab.title,
"url": str(tab.url),
"id": tab.id
}
# Extract page metadata using JavaScript
try:
meta_description = tab.execute_javascript("""
const meta = document.querySelector('meta[name="description"]');
return meta ? meta.getAttribute('content') : null;
""")
tab_info["description"] = meta_description
except:
tab_info["description"] = None
research_data["tabs"].append(tab_info)
# Collect relevant bookmarks
try:
bookmarks_bar = chrome.bookmarks_bar
research_bookmarks = bookmarks_bar.bookmark_items()
for bookmark in research_bookmarks:
if any(keyword in bookmark.title.lower() for keyword in
["research", "paper", "study", "science"]):
research_data["bookmarks"].append({
"title": bookmark.title,
"url": str(bookmark.url)
})
except Exception as e:
print(f"Error collecting bookmarks: {e}")
# Save to organized format
filename = f"research_session_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
with open(filename, 'w') as f:
json.dump(research_data, f, indent=2)
return research_data
# Execute data collection
data = collect_research_data()
print(f"Collected data from {len(data.get('tabs', []))} tabs and {len(data.get('bookmarks', []))} bookmarks")
```
## Workflow 3: Cross-Browser Testing Suite (Playwright)
**Objective**: Run automated tests across multiple browsers with comprehensive reporting.
```python
from playwright.sync_api import sync_playwright
import json
from datetime import datetime
def run_cross_browser_tests():
"""Run automated tests across Chrome, Firefox, and Safari"""
test_results = {
"timestamp": datetime.now().isoformat(),
"tests": []
}
browsers_to_test = ["chromium", "firefox", "webkit"]
with sync_playwright() as p:
for browser_name in browsers_to_test:
browser_type = getattr(p, browser_name)
try:
browser = browser_type.launch()
context = browser.new_context()
page = context.new_page()
# Test 1: Basic navigation
start_time = datetime.now()
page.goto("https://httpbin.org/html")
load_time = (datetime.now() - start_time).total_seconds()
# Test 2: JavaScript execution
title = page.evaluate("document.title")
# Test 3: Element interaction
page.click("text=Sample")
page.wait_for_selector("h1")
# Test 4: Screenshot
screenshot_path = f"screenshot_{browser_name}.png"
page.screenshot(path=screenshot_path)
test_results["tests"].append({
"browser": browser_name,
"status": "passed",
"load_time": load_time,
"title": title,
"screenshot": screenshot_path
})
browser.close()
except Exception as e:
test_results["tests"].append({
"browser": browser_name,
"status": "failed",
"error": str(e)
})
# Save results
with open("cross_browser_test_results.json", "w") as f:
json.dump(test_results, f, indent=2)
return test_results
# Run the test suite
results = run_cross_browser_tests()
passed_tests = sum(1 for test in results["tests"] if test["status"] == "passed")
print(f"Test Results: {passed_tests}/{len(results['tests'])} tests passed")
```
## Workflow 4: Web Scraping and Data Extraction (Puppeteer)
**Objective**: Extract structured data from web pages with error handling and rate limiting.
```javascript
const puppeteer = require('puppeteer');
const fs = require('fs');
async function scrapeProductData() {
const browser = await puppeteer.launch({
headless: 'new',
args: ['--no-sandbox', '--disable-setuid-sandbox']
});
const page = await browser.new_page();
const scrapedData = [];
try {
// Navigate to target page
await page.goto('https://example.com/products', {
waitUntil: 'networkidle0',
timeout: 30000
});
// Wait for products to load
await page.waitForSelector('.product-item', { timeout: 10000 });
// Extract product data
const products = await page.$$eval('.product-item', items => {
return items.map(item => {
const title = item.querySelector('.product-title')?.textContent?.trim();
const price = item.querySelector('.price')?.textContent?.trim();
const link = item.querySelector('a')?.href;
return {
title,
price,
link,
timestamp: new Date().toISOString()
};
}).filter(product => product.title && product.price);
});
scrapedData.push(...products);
// Handle pagination if present
const nextButton = await page.$('a[aria-label="Next page"]');
if (nextButton) {
await nextButton.click();
await page.waitForTimeout(2000); // Rate limiting
// Recursively scrape next page
const nextPageData = await page.$$eval('.product-item', items => {
return items.map(item => ({
title: item.querySelector('.product-title')?.textContent?.trim(),
price: item.querySelector('.price')?.textContent?.trim(),
link: item.querySelector('a')?.href,
timestamp: new Date().toISOString()
})).filter(product => product.title && product.price);
});
scrapedData.push(...nextPageData);
}
} catch (error) {
console.error('Scraping error:', error.message);
} finally {
await browser.close();
}
// Save data
fs.writeFileSync('product_data.json', JSON.stringify({
scrape_timestamp: new Date().toISOString(),
total_products: scrapedData.length,
products: scrapedData
}, null, 2));
return scrapedData;
}
// Run the scraper
scrapeProductData().then(data => {
console.log(`Scraped ${data.length} products`);
}).catch(console.error);
```
```