Mastering jq for JSON on the Command Line

jq is a lightweight, zero-dependency command-line JSON processor. It can filter, transform, reshape, and produce JSON with a compact expression language that composes cleanly with shell pipelines. Once you move past the basics, it replaces ad-hoc grep/sed hacks on API responses and log files entirely. This guide covers the filters, operators, and functions you will reach for daily.

Installation

Install the latest stable jq (1.7+) from your distro's repositories.

# Debian / Ubuntu
sudo apt install jq

# Fedora / RHEL / Rocky (EPEL not needed on Fedora; on RHEL enable EPEL first)
sudo dnf install jq

# Arch
sudo pacman -S jq

Confirm the version:

jq --version
# jq-1.7.1  (output will vary)

Core Concepts

Every jq program is a filter. A filter receives an input value and produces one or more output values. The identity filter . passes input through unchanged and is the foundation for every more complex expression.

Pretty-printing and basic field access

curl -s https://api.github.com/repos/stedolan/jq | jq .

# Field access — dot-notation
curl -s https://api.github.com/repos/stedolan/jq | jq '.stargazers_count'

# Nested field
echo '{"user":{"name":"ada","age":36}}' | jq '.user.name'
# "ada"

Array indexing and iteration

# Index a specific element
echo '[10,20,30]' | jq '.[1]'
# 20

# Slice
echo '[10,20,30,40]' | jq '.[1:3]'
# [20,30]

# Iterate — explode every element into separate outputs
echo '["a","b","c"]' | jq '.[]'

The iterator .[] is important: it turns an array (or object values) into a stream of individual values that downstream filters operate on one at a time.

Pipes and Multi-Step Filters

The pipe | feeds the output of one filter into the next, exactly like a shell pipe but operating on JSON values rather than text.

# Pull the login field from every item in a GitHub org members list
curl -s https://api.github.com/orgs/github/members | jq '.[] | .login'

# Chain multiple field accesses
echo '{"repo":{"owner":{"login":"octocat"}}}' | jq '.repo | .owner | .login'
# "octocat"

Building new objects and arrays

Use object construction {key: expr} and array construction [expr] to reshape output.

# Reshape: keep only two fields from each element
curl -s https://api.github.com/orgs/github/members \
  | jq '[.[] | {login: .login, id: .id}]'

# Shorthand: {login, id} expands to {login: .login, id: .id}
curl -s https://api.github.com/orgs/github/members \
  | jq '[.[] | {login, id}]'

map and select

map(f) applies filter f to every element of an array and collects results back into an array. It is exactly equivalent to [.[] | f] but far more readable.

# Double every number
echo '[1,2,3,4]' | jq 'map(. * 2)'
# [2,4,6,8]

# Extract a field from every object in an array
echo '[{"name":"Alice","score":90},{"name":"Bob","score":74}]' \
  | jq 'map(.name)'
# ["Alice","Bob"]

select(condition) passes a value through only when the boolean expression is true; otherwise it produces no output. Combined with map or a bare iterator it acts as a filter in the Unix sense.

# Keep only high scorers
echo '[{"name":"Alice","score":90},{"name":"Bob","score":74}]' \
  | jq '[.[] | select(.score >= 80)]'
# [{"name":"Alice","score":90}]

# Equivalent using map_values on an object (keeps keys, filters values)
# For arrays, map + select is idiomatic:
echo '[{"name":"Alice","active":true},{"name":"Bob","active":false}]' \
  | jq 'map(select(.active))'

Combining map and select

# From a package list, return names of packages with size > 5000
cat packages.json | jq '[.packages[] | select(.size > 5000) | .name]'

Useful Built-in Functions

String operations

# Test with regex
echo '["foo.log","bar.json","baz.log"]' \
  | jq '[.[] | select(test("\\.log$"))]'
# ["foo.log","baz.log"]

# Split and join
echo '"hello world"' | jq 'split(" ") | join("-")'

# String interpolation
echo '{"user":"ada","lang":"Python"}' \
  | jq '"\(.user) codes in \(.lang)"'

Numeric and aggregate operations

# length works on strings, arrays, objects, and null
echo '[1,2,3]' | jq 'length'   # 3
echo '"hello"' | jq 'length'   # 5

# add: sum an array of numbers, or concatenate strings/arrays
echo '[10,20,30]' | jq 'add'
# 60

# min, max, sort_by, group_by, unique_by
echo '[{"k":"b"},{"k":"a"},{"k":"c"}]' | jq 'sort_by(.k)'

Type checking and conversion

echo '42' | jq 'type'          # "number"
echo 'null' | jq 'type'        # "null"
echo '"hello"' | jq 'tonumber' # will error — use only on numeric strings
echo '"42"' | jq 'tonumber'   # 42
echo '99' | jq 'tostring'      # "99"

Transforming and Producing JSON

Converting CSV or flat data to JSON

jq can also act as a JSON producer when you feed it raw text with the -R (raw input) and -s (slurp) flags.

# Turn lines of text into a JSON array of strings
ls /etc | jq -R -s 'split("\n") | map(select(length > 0))'

# Parse a simple CSV (no quoted commas) into objects
printf 'alice,90\nbob,74\n' \
  | jq -R 'split(",") | {name: .[0], score: (.[1] | tonumber)}'

Merging and updating objects

# Recursive merge with *
echo 'null' | jq '{a:1,b:2} * {b:99,c:3}'
# {"a":1,"b":99,"c":3}

# Update a field non-destructively with |=
echo '{"count":5}' | jq '.count |= . + 1'
# {"count":6}

# Add a computed field to every element in an array
echo '[{"price":10},{"price":25}]' \
  | jq '[.[] | . + {tax: (.price * 0.2)}]'

Producing compact output and raw strings

# -c: compact (single line) — useful for piping to other tools
echo '{"a":1}' | jq -c .

# -r: raw output (strips JSON string quotes) — useful for shell variables
VERSION=$(curl -s https://api.github.com/repos/stedolan/jq/releases/latest \
  | jq -r '.tag_name')
echo "$VERSION"

Passing Shell Variables into jq

Never interpolate shell variables directly into a jq expression — that is a code injection risk. Use --arg (string) or --argjson (JSON value) instead.

THRESHOLD=80
echo '[{"name":"Alice","score":90},{"name":"Bob","score":74}]' \
  | jq --argjson t "$THRESHOLD" '[.[] | select(.score >= $t)]'

USER=alice
echo '{"alice":"admin","bob":"viewer"}' \
  | jq --arg u "$USER" '.[$u]'
# "admin"

Verification

A quick end-to-end check using only local data — no network needed:

echo '[{"name":"nginx","active":true,"port":80},
       {"name":"sshd","active":true,"port":22},
       {"name":"ftp","active":false,"port":21}]' \
  | jq '[.[] | select(.active) | {name, port}]'
# Expected:
# [{"name":"nginx","port":80},{"name":"sshd","port":22}]

Troubleshooting

• null output instead of a value — the field name is wrong or the input structure differs from what you expect. Add | keys after the object to inspect available fields.
• Cannot index array with string / object with number — you are mixing up array vs. object access. Check type at that point in the pipeline: jq '. | type'.
• jq: error (at <stdin>:1): Invalid numeric literal — the input is not valid JSON. Validate it first with jq empty file.json; echo $? (exit 0 means valid).
• Unexpected results with special characters in keys — keys containing hyphens or spaces must be quoted: ."content-type".
• Performance on large files — use --stream for multi-GB JSON to avoid loading the whole document into memory, or consider jaq (a Rust reimplementation) for heavy workloads.