What formats does data-processing support?

JSON, YAML, and TOML via jq and yq.

Do I need to install tools?

Yes — install and verify jq and yq (brew install jq yq on macOS).

Can I modify files in place?

Yes — yq supports in-place edits (for example, yq -i '.version = "2.0.0"' config.yaml).

data-processing

Scanned

npx machina-cli add skill aiskillstore/marketplace/data-processing --openclaw

Files (1)

SKILL.md

2.8 KB

Data Processing

Query, filter, and transform structured data (JSON, YAML, TOML) efficiently from the command line.

Tools

Tool	Command	Use For
jq	`jq '.key' file.json`	JSON processing
yq	`yq '.key' file.yaml`	YAML/TOML processing

jq Essentials

# Extract single field
jq '.name' package.json

# Extract nested field
jq '.scripts.build' package.json

# Extract from array
jq '.dependencies[0]' package.json

# Extract multiple fields
jq '{name, version}' package.json

# Navigate deeply nested
jq '.data.users[0].profile.email' response.json

# Filter by condition
jq '.users[] | select(.active == true)' data.json

# Transform each element
jq '.users | map({id, name})' data.json

# Count elements
jq '.users | length' data.json

# Raw string output
jq -r '.name' package.json

yq Essentials

# Extract field
yq '.name' config.yaml

# Extract nested
yq '.services.web.image' docker-compose.yml

# List all keys
yq 'keys' config.yaml

# List all service names (Docker Compose)
yq '.services | keys' docker-compose.yml

# Get container images (K8s)
yq '.spec.template.spec.containers[].image' deployment.yaml

# Update value (in-place)
yq -i '.version = "2.0.0"' config.yaml

# TOML to JSON
yq -p toml -o json '.' config.toml

Quick Reference

Task	jq	yq
Get field	`jq '.key'`	`yq '.key'`
Array element	`jq '.[0]'`	`yq '.[0]'`
Filter array	`jq '.[] \| select(.x)'`	`yq '.[] \| select(.x)'`
Transform	`jq 'map(.x)'`	`yq 'map(.x)'`
Count	`jq 'length'`	`yq 'length'`
Keys	`jq 'keys'`	`yq 'keys'`
Pretty print	`jq '.'`	`yq '.'`
Compact	`jq -c`	`yq -o json -I0`
Raw output	`jq -r`	`yq -r`
In-place edit	-	`yq -i`

When to Use

Reading package.json dependencies
Parsing Docker Compose configurations
Analyzing Kubernetes manifests
Processing GitHub Actions workflows
Extracting data from API responses
Filtering large JSON datasets
Config file manipulation
Data format conversion

Additional Resources

For complete pattern libraries, load:

./references/jq-patterns.md - Arrays, filtering, transformation, aggregation, output formatting
./references/yq-patterns.md - Docker Compose, K8s, GitHub Actions, TOML, YAML modification
./references/config-files.md - package.json, tsconfig, eslint/prettier patterns

Source

git clone https://github.com/aiskillstore/marketplace/blob/main/skills/0xdarkmatter/data-processing/SKILL.mdView on GitHub

Overview

This skill enables you to query, filter, and transform structured data from the command line using jq for JSON and yq for YAML/TOML. It’s ideal for parsing configs, manifests, and workflows—Docker Compose, Kubernetes, and GitHub Actions—and extracting actionable insights.

How This Skill Works

You use jq and yq CLI tools to read, navigate, and transform data. Build pipelines to select fields, filter arrays, and map results across JSON, YAML, or TOML, and you can edit files in place with yq when needed.

When to Use It

Reading package.json dependencies
Parsing Docker Compose configurations
Analyzing Kubernetes manifests
Processing GitHub Actions workflows
Filtering large JSON datasets

Quick Start

Step 1: Install jq and yq (e.g., brew install jq yq) and verify versions
Step 2: Extract a field from JSON (e.g., jq '.name' package.json)
Step 3: Convert TOML to JSON (e.g., yq -p toml -o json '.' config.toml)

Best Practices

Install and verify jq and yq before use
Use jq -r for raw output when integrating with other tools
Leverage yq -i for in-place edits and preserve original files
Combine jq and yq in pipelines to handle mixed data formats
Test paths with small samples to avoid errors and confirm field names

Example Use Cases

Extract a package.json field: jq '.name' package.json
Read a nested field: jq '.scripts.build' package.json
List Kubernetes container images: yq '.spec.template.spec.containers[].image' deployment.yaml
Convert TOML to JSON: yq -p toml -o json '.' config.toml
Update a YAML value in place: yq -i '.version = "2.0.0"' config.yaml

Frequently Asked Questions

Add this skill to your agents