Implementing Semgrep: custom rules and a Supply Chain scan

Part 3 ended with a division of labor: Aikido owns the malware layer, and Semgrep was going to own the code-and-config lane. This is the part where I build that lane: reachability-aware dependency scanning, SAST, and a set of custom rules for the two specific things the Shai-Hulud worm did to this class of repo, planting a lifecycle script and editing config that runs on open.

Two things share the malware-detection job now, so it's worth being precise about where they overlap and where they don't. Aikido Safe Chain intercepts an install and checks each package against a malware feed before the tarball lands. Semgrep Supply Chain reads the lockfile and checks the same kind of feed without installing anything at all. Same threat, two different moments to catch it. I wanted to see both actually fire on the same poisoned package before I trusted the overlap.

Where Semgrep fits

Semgrep is three products stitched into one CLI: SAST (pattern-based static analysis across languages), Supply Chain (reachability-aware scanning of your dependency lockfile), and Secrets. semgrep scan --config auto runs the first for free, no account. Supply Chain and the full Secrets product need a login and a token.

The Supply Chain side has a distinction worth understanding before the results make sense. It separates Malicious findings (a specific package name and version is known-bad, full stop) from Reachability findings (a dependency has a known vulnerability, but only flagged if your code actually calls the vulnerable code path). A raw CVE-in-lockfile scanner produces a lot of noise, most of it dependencies you import for one function that never touches the vulnerable one. Reachability analysis is the fix for that noise. Malicious analysis doesn't need it, because there's no "safe way" to depend on a package that's already been proven to run a credential stealer, reachable or not.

Custom rules for this threat

Part 2's plan named three specific patterns to catch: a lifecycle script added to package.json, an auto-run task added to .vscode/tasks.json, and a download-then-execute dropper. I wrote one rule per pattern, plus a fourth for the JavaScript version of the same dropper idea.

The lifecycle-script rule is the most direct translation of the threat. preinstall, install, and postinstall scripts run automatically the moment someone runs pnpm install, with no prompt, the exact mechanism Part 1 walked through.

- id: npm-lifecycle-script
  languages: [json]
  severity: ERROR
  message: >-
    A `preinstall`/`install`/`postinstall` script in package.json runs
    automatically on `npm install` / `pnpm install`, with no prompt and no
    review step of its own. This is the persistence mechanism used by
    npm supply-chain worms (e.g. Shai-Hulud) to execute arbitrary code the
    moment a victim installs the package.
  patterns:
    - pattern-inside: '"scripts": {...}'
    - pattern-either:
        - pattern: '"preinstall": "..."'
        - pattern: '"install": "..."'
        - pattern: '"postinstall": "..."'
  paths:
    include:
      - "package.json"

The second rule targets .vscode/tasks.json's runOn: "folderOpen", the other persistence trick Part 1 catalogued: code that runs the instant a folder is opened in the editor, no click required. Unlike lifecycle scripts, this one is rare enough in legitimate use that flagging its mere presence, not just suspicious content inside it, is the right bar.

The third and fourth rules both target the same underlying idea (fetch something from the network, then execute it without ever writing it to disk for review) in the two languages this repo actually has: shell and JavaScript.

- id: network-fetch-pipe-exec
  languages: [generic]
  severity: ERROR
  pattern-regex: '(curl|wget)\b(?:[^\n|]|\\\n)*\|\s*(sudo\s+)?(sh|bash|zsh)\b'

That regex went through one revision before I trusted it. The first version stopped at any newline, which meant a backslash-continued command (curl -fsSL \` on one line, the rest on the next) evaded it entirely. That's not a hypothetical: it's the exact style this repo's own `supply-chain.yml uses for its safe, non-piped install step. A PR reviewer caught the gap. The fixed regex spans a continuation but still stops at a bare newline or an unrelated pipe, so it doesn't bleed into whatever command comes next.

The JavaScript rule had a bigger problem. My first pass was three syntactic patterns trying to match eval(await res.text()) and its variants directly. Three independent reviewers (Greptile, CodeRabbit, and my own repo's Claude auto-review) converged on the same finding: the pattern meant to catch eval() on an intermediate variable wasn't actually checking that the variable came from the fetch. It matched any eval() call that happened to occur somewhere after any unrelated fetch() in the same function. A completely unrelated eval("1 + 1") three lines below a config fetch would have tripped it.

The fix was to stop writing syntax and start writing dataflow. Semgrep's taint mode does exactly that:

- id: js-fetch-then-eval
  mode: taint
  pattern-sources:
    - pattern: fetch(...)
  pattern-sinks:
    - pattern: eval(...)
    - pattern: new Function(...)

Instead of matching a shape, this tracks whether a value returned by fetch() actually reaches eval() or new Function(), through however many variables and method calls sit in between. const text = await res.text(); eval(text) now matches. An unrelated eval() after an unrelated fetch doesn't. Three lines replaced twelve, and they're more correct: the kind of trade a fixed-pattern tool almost never offers you, because most of them can't see dataflow at all.

Proving the rules actually fire

A rule you haven't watched fail is a rule you're guessing about. Each of the four has a fixture pair committed alongside it (.semgrep/fixtures/bad/, which the rule must flag, and .semgrep/fixtures/clean/, the legitimate-looking equivalent it must leave alone), checked by a small regression script that fails if any rule stops matching its own fixture, or starts matching something it shouldn't.

Not .semgrep/tests/, notably. Semgrep ships a default ignore list that silently skips any directory literally named test or tests. A directory of intentionally-malicious-looking fixtures would have been invisible to every scan, itself, without ever throwing an error. Small thing, would have been a quiet total loss of coverage.

Fixtures prove the rule works on paper. I wanted one to fail on a live pull request, the same way Part 3 planted safe-chain-test in CI. So I added a real postinstall to package.json, uncommitted:

"postinstall": "node ./scripts/backdoor.js"

semgrep.rules.npm-lifecycle-script
A `preinstall`/`install`/`postinstall` script in package.json runs automatically...
50┆ "postinstall": "node ./scripts/backdoor.js"

Caught, on the diff, before I ever ran pnpm install. Reverted immediately.

Supply Chain scan, on the actual poisoned package

The custom rules cover code and config. The lockfile itself needed the other half of Semgrep, Supply Chain, and I wanted to prove it against something more concrete than a synthetic test string. Part 1 named five poisoned packages by exact version. Part 3 planted one of them, keyv@6.0.0, and watched Safe Chain refuse the install. I wanted to know whether Semgrep would catch the same version at the lockfile level, before an install is even attempted.

On a throwaway branch, never merged, I hand-edited pnpm-lock.yaml to resolve keyv at 6.0.0 instead of the real 4.5.4, and opened a PR against it. No pnpm install: the file edit alone is enough for a lockfile scanner. semgrep ci came back with two blocking, critical findings:

keyv - MAL-0000-0062 (CRITICAL)
Self-propagating npm worm tracked as ChainDrop republished 444 packages across
unrelated maintainer scopes, running an install-time credential harvester that
steals the publish tokens it uses to spread further.

keyv - MAL-2026-11524 (CRITICAL)
keyv contains malicious code. Remove it immediately.

Semgrep's intel tracks the campaign under its own name, "ChainDrop." Vendors don't always agree on what to call the same worm, which is itself a small, useful data point about how fragmented threat intel still is. But the description lines up exactly with Part 1's account: install-time credential theft, self-propagation through stolen publish tokens. Same package, same version, two tools, two different moments in the pipeline. Safe Chain stops it as pnpm install tries to fetch the tarball; Semgrep Supply Chain flags it the moment the version lands in a lockfile, before anyone runs install at all. Closed the PR without merging and deleted the branch. The file never touched a real install.

Gating CI on all of it

The registry auto ruleset isn't clean on this repo. It has 44 to 45 pre-existing findings on main right now: 37 GitHub Actions steps using mutable version tags instead of pinned SHAs, three createDecipheriv calls missing an explicit GCM auth-tag length in the backup scripts, a couple of missing Dependabot cooldown windows, one string-concatenation-in-a-log-line, one test file that triggers an XSS heuristic on a literal "<script>" string used as test data. None of that is what this PR is about, and none of it should turn every unrelated future PR red.

semgrep scan supports --baseline-commit, which reports only findings introduced since a given commit. No login required for that flag, it's a plain diff against git history. CI passes the PR's base SHA:

semgrep scan \
  --config auto \
  --config .semgrep/rules/supply-chain.yml \
  --exclude .semgrep/fixtures \
  --baseline-commit "${{ github.event.pull_request.base.sha }}" \
  --error \
  .

Two bugs surfaced while wiring this up, both the kind that look fine until you actually watch them fail to fail. First: semgrep scan exits 0 regardless of findings unless you pass --error. I ran the whole pipeline once, watched a blocking finding print to the console, checked the exit code out of habit, and found it was zero. Without that flag the entire gate is theater: findings show up in logs, nothing ever turns the check red. Second: the authenticated semgrep ci job originally gated on if: ${{ secrets.SEMGREP_APP_TOKEN != '' }} at the job level. actionlint, and independently CodeRabbit, flagged it: the secrets context isn't available in a job-level if: at all, only github, inputs, needs, and vars are. The fix moved the presence check into the step's own shell instead, which sidesteps the restriction entirely rather than working around it with an extra job.

The registry ruleset caught one more thing in this very PR. The new semgrep.yml itself used actions/checkout@v7, a mutable tag, the same class of supply-chain risk this whole series is about, this time in my own new file. Pinned to the commit SHA, matching the convention shell.yml and supply-chain.yml already use.

Honest notes: overlap, gaps, and what "done" actually looks like here

Overlap with Aikido. Both now do dependency-malware detection, and I'm keeping both anyway, because they catch it at different moments (Safe Chain at the install itself, Semgrep at the lockfile), and a maintainer-account compromise that slips past one signature feed might not slip past the other's. Redundancy in a detection layer is a feature here, not waste.

A gap against Part 2's own checklist. It said a PR that "edits .claude/settings.json" should trip a rule. What I actually shipped flags suspicious content inside that file, a curl | bash pattern specifically, not the mere fact of an edit. I considered a blanket presence rule, the same shape as the VS Code task rule, and backed off: unlike runOn: "folderOpen", which is genuinely rare, this repo edits .claude/settings.json hooks regularly as part of normal maintenance (three of them are already committed, doing ordinary session setup). A rule that fires on every edit to a file this repo touches routinely would train me to click through it, which is worse than no rule. The content-based version is narrower than the plan promised, and I'd rather say that plainly than round up.

Secrets, status unclear. semgrep ci's own log reported "Enabled products: Code, Supply Chain." Secrets wasn't listed as a third enabled product for this scan. Some hardcoded-credential patterns exist inside the free auto/Code ruleset regardless, and none fired on this repo, but I can't currently claim the dedicated Secrets product ran here. Filing that as an open question rather than a checked box.

Noise and effort. 44-to-45 pre-existing findings, diffed away rather than fixed, is real technical debt sitting on main. The honest move was making the gate diff-aware instead of pretending the debt doesn't exist. The Pro engine the authenticated job downloads is over 300MB and adds real minutes to CI. And the rule-writing itself took three iterations on the JavaScript rule alone before three independent reviewers stopped finding holes in it. Custom Semgrep rules are not a write-once artifact. They're code, with the same need for adversarial review as anything else in this repo.

Where this leaves the plan

Two more rows of Part 2's table are real now: reachability-aware dependency scanning and SAST through Semgrep in CI, and custom rules catching the worm's specific persistence trick, both gating pull requests. Between Part 3 and this one, a routine pnpm install on this project now has to clear a malware check, and a routine pull request has to clear a static-analysis and dependency check, before either can do anything at all.

What's left is the part with no vendor to install: scoping every credential in the threat model to least privilege, and writing the rotation runbook down instead of keeping it in my head. Part 5 closes the series by asking the plan's last honest question, what would a fresh Shai-Hulud-class package hit first here, now, and answering it against everything the last three parts actually built, not what they were supposed to.