I built an automated job search that queries 14 APIs every morning

I’m a Principal Engineer in India looking for US remote work with visa sponsorship. That’s a narrow search. Manually checking job boards every day is a poor use of my time, so I built a system that does it for me.

The system runs on a $10/month Linode VPS. Every morning at 6am UTC it queries 14 job source APIs, scores each listing against my profile, and queues high-scoring matches for AI analysis. When I open the dashboard, there’s a ranked list of opportunities with tailored cover letters ready to send.

This post is about how it works, what broke in interesting ways, and what I spent too much money on before fixing it.

The architecture

14 Job Source APIs
    │ (Lever, Greenhouse, Ashby, RemoteOK, HN, Himalayas,
    │  Jobicy, Remotive, WeWorkRemotely, Composio,
    │  + JSearch, Adzuna, Findwork, USAJobs with API keys)
    ▼
Go Service (Gin + SQLite)
    │
    ├─ Scheduler: daily cron at 6am UTC
    ├─ Scorer: keyword-based 0-100 scoring (6 dimensions)
    ├─ Content Index: 36 product files matched to JD requirements
    ├─ Doc Generator: Claude CLI for cover letters + emails
    ├─ Umami: analytics dashboard
    └─ Dashboard: Gin-rendered HTML with theme sync

The service is ~4,000 lines of Go across 25 files. It’s a single binary with an embedded SQLite database. No external dependencies except the Claude Code CLI for AI generation.

Three tiers of job sources

Not all APIs are equal. I split sources into three tiers based on what they return.

Tier 1: Structured APIs with full descriptions

These return title, company, salary, description, location — everything needed for scoring without any page scraping.

Himalayas (himalayas.app/jobs/api): 107K+ jobs, has seniority field, structured salary. No auth.
RemoteOK (remoteok.com/api): Full feed as JSON array. I cache it for 30 minutes and filter locally per query.
Remotive (remotive.com/api/remote-jobs): Remote-only. Max 4 requests/day — I cache for 6 hours.
Jobicy (jobicy.com/api/v2/remote-jobs): Tag-based, not full-text search. 1-hour cache.

Tier 2: ATS direct polling (the highest signal source)

These are the job boards of specific companies I want to work at. Zero auth, zero aggregator delay.

Lever (api.lever.co/v0/postings/{company}): 20 target companies — Stripe, Netflix, Figma, Anthropic, Databricks, Supabase, etc.
Greenhouse (boards-api.greenhouse.io/v1/boards/{company}/jobs): 20 companies — Cloudflare, Datadog, Airbnb, Duolingo, etc.
Ashby (GraphQL API): 15 companies — Ramp, Notion, Linear, Vercel. Includes compensation data.

I fetch each company’s full listing once per hour and cache it. Then filter locally per search query. This cuts API calls from ~300/run to ~55.

The ATS tier is the best source by far. You’re polling the company’s own job board directly — no aggregator lag, no missing listings, no stale data. If Cloudflare posts a Staff Platform Engineer at 9am, my dashboard shows it by 9:01.

Tier 3: Search wrappers + special sources

Composio: Google search wrapper. Returns (title, url, snippet) — needs page scraping for descriptions.
HN Who’s Hiring: Monthly thread parsed via the Algolia API. Pipe-delimited first line (Company | Role | Location | Remote | Salary), rest is description. Best organic source for startup roles.
WeWorkRemotely: RSS feed. Title format is Company: Job Title.

Tier 3 sources are the lowest quality per-listing but catch things the structured APIs miss.

The scoring engine

Every job gets scored 0-100 across six dimensions:

Dimension	Max	What it measures
Tech	35	How many of my languages/frameworks appear in the JD
Seniority	20	Title matching (Principal=20, Staff=18, Senior=12)
Problem fit	15	Rare problem shapes (offline-first, DRM, IoT)
Architecture	15	Distributed systems, platform engineering, event-driven
Visa	15	Explicit sponsorship mention vs “must be authorized”
Domain	10	EdTech (+8), GovTech (+7), Healthcare (+6)

This is pure keyword matching. No ML, no embeddings, no LLM calls. It runs in <1ms per job and is deterministic. A Staff Platform Engineer role at an EdTech company mentioning distributed systems, Go, and visa sponsorship scores 80+.

The scoring is intentionally crude. False positives (high score, bad fit) are caught by the AI analysis step. False negatives (low score, good fit) are the real risk — I periodically scan low-scoring jobs manually to calibrate.

Two-pass document generation

This is where it gets interesting. The old system used hardcoded templates — 6 canned paragraphs shuffled by keyword match. Every cover letter read identical. Classic AI slop, except it wasn’t even AI — it was string concatenation.

The new system uses a two-pass approach:

Pass 1: Requirement extraction. The JD is sent to Claude (Haiku) with a prompt that returns structured JSON:

{
  "must_have_tech": ["Go", "Kubernetes", "PostgreSQL"],
  "nice_to_have_tech": ["Terraform", "gRPC"],
  "domain": "fintech",
  "key_problems": ["scale API gateway", "reduce deploy time"],
  "seniority_signals": ["staff", "tech lead"],
  "visa_signal": "sponsorship_explicit"
}

Pass 2: Context matching + generation. The extracted requirements are matched against a content index of 36 product files embedded in the Go binary via go:embed. Each product has structured metadata:

name: Digital Examination Platform
tech: [Go, Java, Kotlin, RabbitMQ, MQTT, SQLite]
metrics:
  commits: 602
  ownership: 56.8%
  team_size: 5
role: Technical Lead, System Architect

The matcher scores each product by tech overlap + domain signals + role match. Top 4 products get their metadata and markdown body injected into the generation prompt alongside the JD and extracted requirements.

Every prompt includes explicit anti-slop rules:

STRICT RULES:
1. Use ONLY the projects/metrics listed below. Do NOT invent experience.
2. Reference specific project names, commit counts, ownership percentages.
3. NO generic phrases: avoid "passionate about", "excited to apply",
   "proven track record", "results-driven".
4. Write like a principal engineer to a fellow engineer.
5. Every sentence must convey concrete information. No filler.

The result: cover letters that reference specific projects with real metrics, mapped to specific JD requirements. Not perfect, but dramatically better than templates.

The cost disaster (and the fix)

For the first two weeks, the AI analysis cron was burning $2.74 per run. Twenty runs cost $34.84. The cron script had MODEL="sonnet" hardcoded (line 7, cron-analyze.sh) — it should have been haiku.

Sonnet at $3/MTok input was 3.75x more expensive than Haiku at $0.80/MTok. Combined with a 56,000-token extended thinking budget (wildly excessive for job matching), each analysis batch consumed ~850K input tokens.

The fix was four changes:

Model: Sonnet → Haiku (MODEL="${CLAUDE_MODEL:-haiku}")
Thinking budget: 56K → 16K tokens
Analysis threshold: score >= 55 → score >= 70 (40% fewer analyses)
Prompt size: 3K char description cap → 1.5K, condensed candidate profile

Before: ~$1.86/run, ~$56/month After: ~$0.15/run, ~$5/month

The extended thinking budget was the sneaky one. Claude uses thinking tokens proportional to the budget ceiling, not proportional to the problem complexity. A 56K budget for “should this engineer apply to this job?” is like renting a warehouse to store a bicycle.

API compliance: what I learned

Running 14 APIs from one cron job taught me which providers care about rate limits and which don’t:

Greenhouse: Explicitly “not rate limited” for their Board API. I still throttle at 500ms because I’m polite.
RemoteOK: Will suspend your access without warning if you don’t include User-Agent and a visible attribution link.
Remotive: “Max 4 times a day” — my 6-hour cache aligns perfectly.
Adzuna: 250 hits/day on free tier. I use 6.
HN Algolia: 10,000/hour. I use 2/day.

The general pattern: APIs that want attribution (RemoteOK, Himalayas, Jobicy, Adzuna) require you to show where the listing came from. I store source as a field on every job and render it as a badge on the dashboard.

For the ATS sources (Lever, Greenhouse, Ashby), the critical optimization was caching. Without it, 55 companies x 6 search queries = 330 API calls per run. With company-level caching (fetch once, filter locally), it’s 55 calls. Same data, 83% fewer requests.

What’s actually on the dashboard

The dashboard is server-rendered HTML with Gin templating. It syncs with my portfolio’s theme system (Minimal/Galaxy/Nord) via localStorage. Features:

Job list with score rings, source badges, salary, visa status
Score breakdown per job (which dimensions contributed what)
One-click generation of cover letters, emails, resume tailoring
AI analysis with match assessment, strengths, gaps, approach strategy
Umami analytics showing portfolio visitor traffic (spoiler: it’s low)
14-source status panel showing enabled/disabled, last run, error counts

The llms.txt angle

I also added llms.txt and llms-full.txt to my portfolio site — a machine-readable profile following the llmstxt.org spec. The idea: as AI-powered recruiting tools emerge, having a well-known URL with structured profile data gives me early-mover advantage for discovery.

Whether this drives real traffic in 2026 is an open question. The cost was zero (two static files generated from existing resume.json), so the downside is also zero.

What I’d build differently

Use NATS instead of RabbitMQ for the distributed exam sync layer. (Wrong project, but I keep thinking about it.) For this project specifically:

Skip Composio entirely. It’s a Google search wrapper that returns snippets, not job data. Every other source returns structured data. Composio exists because it was the first source I integrated; inertia kept it.
Build the ATS polling first. Lever + Greenhouse + Ashby cover the companies I actually want to work at. The aggregator APIs (Himalayas, RemoteOK) are supplementary. I built them first because they were easier.
Don’t hardcode the candidate profile. My analysis prompt has Vivek’s background baked in as a string. It should read from the same content index the cover letter generator uses.
Track application outcomes. The dashboard tracks job status (new/saved/applied/interview/rejected) but doesn’t measure conversion rates. I have no data on which cover letter style or approach strategy works best.

Numbers

After two weeks of running:

662 jobs scored across 14 sources
10 sources active (4 waiting for free API keys)
55 target companies polled directly via ATS APIs
~$5/month projected AI analysis cost (down from $56)
36 products indexed for skill-matching context
39 portfolio visitors/month (the distribution problem is harder than the engineering problem)

The last number is the honest one. I built a technically capable system. The bottleneck isn’t the system — it’s getting recruiters to see my profile. That’s a distribution problem, and no amount of Go code solves it.

I’m Vivek Yarra, a Principal Engineer with 15 years of this kind of systems work. The portfolio is at viveky.com, the career metrics at viveky.com/insights, and I’m open to US remote Principal/Staff roles with visa sponsorship. Let’s talk.