scrapingbee-automation
Scannednpx machina-cli add skill ComposioHQ/awesome-claude-skills/scrapingbee-automation --openclawScrapingbee Automation via Rube MCP
Automate Scrapingbee operations through Composio's Scrapingbee toolkit via Rube MCP.
Toolkit docs: composio.dev/toolkits/scrapingbee
Prerequisites
- Rube MCP must be connected (RUBE_SEARCH_TOOLS available)
- Active Scrapingbee connection via
RUBE_MANAGE_CONNECTIONSwith toolkitscrapingbee - Always call
RUBE_SEARCH_TOOLSfirst to get current tool schemas
Setup
Get Rube MCP: Add https://rube.app/mcp as an MCP server in your client configuration. No API keys needed — just add the endpoint and it works.
- Verify Rube MCP is available by confirming
RUBE_SEARCH_TOOLSresponds - Call
RUBE_MANAGE_CONNECTIONSwith toolkitscrapingbee - If connection is not ACTIVE, follow the returned auth link to complete setup
- Confirm connection status shows ACTIVE before running any workflows
Tool Discovery
Always discover available tools before executing workflows:
RUBE_SEARCH_TOOLS
queries: [{use_case: "Scrapingbee operations", known_fields: ""}]
session: {generate_id: true}
This returns available tool slugs, input schemas, recommended execution plans, and known pitfalls.
Core Workflow Pattern
Step 1: Discover Available Tools
RUBE_SEARCH_TOOLS
queries: [{use_case: "your specific Scrapingbee task"}]
session: {id: "existing_session_id"}
Step 2: Check Connection
RUBE_MANAGE_CONNECTIONS
toolkits: ["scrapingbee"]
session_id: "your_session_id"
Step 3: Execute Tools
RUBE_MULTI_EXECUTE_TOOL
tools: [{
tool_slug: "TOOL_SLUG_FROM_SEARCH",
arguments: {/* schema-compliant args from search results */}
}]
memory: {}
session_id: "your_session_id"
Known Pitfalls
- Always search first: Tool schemas change. Never hardcode tool slugs or arguments without calling
RUBE_SEARCH_TOOLS - Check connection: Verify
RUBE_MANAGE_CONNECTIONSshows ACTIVE status before executing tools - Schema compliance: Use exact field names and types from the search results
- Memory parameter: Always include
memoryinRUBE_MULTI_EXECUTE_TOOLcalls, even if empty ({}) - Session reuse: Reuse session IDs within a workflow. Generate new ones for new workflows
- Pagination: Check responses for pagination tokens and continue fetching until complete
Quick Reference
| Operation | Approach |
|---|---|
| Find tools | RUBE_SEARCH_TOOLS with Scrapingbee-specific use case |
| Connect | RUBE_MANAGE_CONNECTIONS with toolkit scrapingbee |
| Execute | RUBE_MULTI_EXECUTE_TOOL with discovered tool slugs |
| Bulk ops | RUBE_REMOTE_WORKBENCH with run_composio_tool() |
| Full schema | RUBE_GET_TOOL_SCHEMAS for tools with schemaRef |
Powered by Composio
Source
git clone https://github.com/ComposioHQ/awesome-claude-skills/blob/master/composio-skills/scrapingbee-automation/SKILL.mdView on GitHub Overview
Automate Scrapingbee operations through Composio's Scrapingbee toolkit using Rube MCP. The process emphasizes discovering current tool schemas, establishing an active connection, and executing with memory-aware, session-managed workflows. Keeping tool schemas up-to-date ensures reliable automation.
How This Skill Works
The workflow starts by calling RUBE_SEARCH_TOOLS to fetch current Scrapingbee tool slugs and schemas. Next, you verify or establish a Scrapingbee connection with RUBE_MANAGE_CONNECTIONS and ensure the status is ACTIVE. Finally, execute the discovered tool using RUBE_MULTI_EXECUTE_TOOL, passing the required arguments, a memory object, and a session_id to maintain state across steps.
When to Use It
- You’re starting a new Scrapingbee workflow and need the latest tool schemas before selecting a task slug.
- You must connect or re-verify a Scrapingbee account in Rube MCP and ensure ACTIVE before running workflows.
- You have a specific Scrapingbee task and want to execute the corresponding tool slug with schema-compliant arguments.
- You’re running a multi-tool workflow and need to preserve state with memory and a consistent session_id.
- You’re debugging tool schemas or handling pagination; you should re-search tools to fetch current tokens and fields.
Quick Start
- Step 1: Add the Rube MCP server at https://rube.app/mcp and run RUBE_SEARCH_TOOLS to fetch Scrapingbee tool schemas.
- Step 2: Call RUBE_MANAGE_CONNECTIONS with toolkits: ["scrapingbee"] and ensure the connection status is ACTIVE.
- Step 3: Use RUBE_MULTI_EXECUTE_TOOL with the discovered TOOL_SLUG and a schema-compliant arguments object, including memory and session_id.
Best Practices
- Always call RUBE_SEARCH_TOOLS first to obtain current tool schemas and avoid hardcoding slugs.
- Check that RUBE_MANAGE_CONNECTIONS reports an ACTIVE status before any execution.
- Use exact field names and types from the search results to ensure schema compliance.
- Include memory in every RUBE_MULTI_EXECUTE_TOOL call, even if empty ({}).
- Reuse session IDs within a workflow and generate new ones for new workflows; handle pagination tokens as they appear.
Example Use Cases
- Discover a Scrapingbee task slug via RUBE_SEARCH_TOOLS and run a single scrape with memory across steps.
- Connect a Scrapingbee account using RUBE_MANAGE_CONNECTIONS, verify ACTIVE, then execute a tool slug for product scraping.
- Execute a sequence of Scrapingbee tools in one workflow with a shared session_id and memory payload.
- Handle a paginated scrape by continuing to fetch with updated tool schemas until all pages are retrieved.
- Initiate a fresh workflow from scratch by re-searching tools, establishing a new session_id, and running the first tool.
