For developers

The stack

Two Convex deployments: dev determined-cheetah-971 and prod tremendous-seahorse-330. The prod site is https://tremendous-seahorse-330.convex.site.

Set up

pnpm install
pnpm exec convex dev          # once, to connect to the dev deployment (needs a seat on the Convex team)
pnpm dev:web                  # the web app; add pnpm dev:docs for this site

apps/web/.env.local needs VITE_CLERK_PUBLISHABLE_KEY (public by design) and VITE_CONVEX_URL, with VITE_API_MODE=live. The full list is in .env.example.

Deployment environment variables (Convex, never VITE_*)

Set them with pnpm exec convex env set NAME value (add --prod for production). Never put a value in a commit, a doc or chat.

How data flows

1. Reddit. A cron (convex/crons.ts) runs watch:tick every 10 minutes. For each Reddit phrase it reads the community: Reddit's official API if configured, else the www feed, else a mirror. keepMetrics stops a feed without scores from zeroing them.
2. X and Facebook. The helper calls GET /session?platform=x (your sealed login, decrypted only for that call) and GET /phrases?platform=x, reads the site, then calls POST /ingest.
3. Ingest. convex/ingest.ts validates each post, stores it, matches it against your phrases (convex/lib/match.ts), and writes a hits row.
4. Scoring. Each new match schedules scoring:scoreHits, which asks OpenAI for a 0 to 100 score, an intent and a one-line reason. A 10-minute cron scores anything missed.
5. Website reading. brand:extractFromWebsite (public action, needs sign-in) calls Firecrawl's POST /v2/scrape with a JSON schema, cleans the reply in parseBrandFacts, stores it in brands, and scoring:forScoring adds a short business summary to each scoring prompt.
6. Email alerts. A 5-minute cron runs alerts:sweep: for each person with alerts on it picks their strongest fresh unsent matches, sends one plain-text email through AgentMail (POST /v0/inboxes/{inbox}/messages/send), and only then marks them sent.
7. Web app. Reads keywords, hits, sessions and the alert settings through live Convex queries.

HTTP API for helpers

All three take Authorization: Bearer lk_ingest_.... The owner comes from the key, never from the request.

POST /ingest
{ "platform": "facebook",
  "posts": [ { "externalId": "pfbid0...", "authorName": "Pat", "body": ["need a bookkeeper"],
               "url": "https://www.facebook.com/...", "likes": 1, "comments": 0 } ] }
→ 200 { "accountId": "...", "ingested": 1, "skipped": 0 }

GET /session?platform=x     → the cookie jar, decrypted, only for a helper holding a valid key
GET /phrases?platform=x     → { "platform": "x", "phrases": ["need a helper"] }
GET /proxy                  → { "required": true, "proxy": { "server": "http://host:8080", "username": "u", "password": "p" } }
                              (503 when the deployment has no proxy: the helper must stop)

Up to 100 posts per batch. Invalid rows are skipped and counted, not fatal. platform is facebook, x or reddit.

The public API

At /api/v1 on the deployment's site address, authenticated with a key made in the dashboard (lk_api_..., stored as a SHA-256 hash, shown once; convex/apiKeys.ts). Every route goes through apiRoute in convex/http.ts: key check, scope check, a per-minute counter (60 a minute per key), one JSON error shape, no CORS (keys must not live in browsers). Reference: The API.

• Scopes (convex/lib/scopes.ts): read always, write:phrases, webhooks. A key made before scopes existed has none stored and means read. A route declares the scope it needs.
• Writing phrases goes through convex/lib/keywordOps.ts, the same code the dashboard uses, so validation and the plan limit are identical. POST /keywords takes an Idempotency-Key (table apiIdempotency, one day).
• Webhooks (convex/webhooks.ts): when scoring stores a score, webhooks:fanOut queues a delivery for each active webhook that wants it; webhooks:deliver signs (convex/lib/webhookSign.ts, HMAC-SHA256 over timestamp.body, secret sealed with the same key as connected logins), POSTs with a 5 second timeout and redirect: 'manual', never reads the response body, and retries at 1, 5 and 30 minutes. Five failed deliveries in a row switch the webhook off. Addresses are checked when saved and again at send time (convex/lib/webhookUrl.ts). They are a Pro feature: PLAN_WEBHOOKS_PER_PERSON is 0 unless the operator raises it.
• Known limit: the address check cannot resolve DNS, so a public name that points at a private address is not caught by the check itself. Redirects are never followed and response bodies are never read, which limits what such a name could do.

Plans

Everyone is on Free, enforced on the server in keywords:create and the account creators (convex/lib/plan.ts): one phrase and one account per platform. Only new things are refused; existing ones stay. plan:mine reports plan and usage for Settings, Billing. Pro is display-only and nothing charges. Tests that create several phrases per platform raise the limits with vi.stubEnv('PLAN_PHRASES_PER_PLATFORM', '100').

The mandatory proxy

X and Facebook reading always goes through the operator's proxy, and a person never sees or sets one. fetch_proxy in clients/listeningkit_ingest.py asks GET /proxy before any browser starts; launch_chrome(show, proxy) passes it to Chrome. There is deliberately no --proxy or --no-proxy flag. The proxy is never logged, never in an error, and its username and password are added to the helper's hidden-secrets list. clients/tests/test_proxy.py proves with real Chrome and a local proxy that demands a password that traffic goes through it. Limit to know: the helper runs on the person's own computer, so only server-side reading would fully hide the proxy from a technical person. Reddit is read by the Convex server, which cannot use a proxy.

Security rules (do not weaken)

• Every query and mutation calls requireOwner(ctx) and filters by owner. No table is readable across accounts.
• Logins are sealed with AES-256-GCM. No query returns a jar to a browser.
• Ingest keys are stored as SHA-256 hashes and shown once.
• Error messages must never echo cookie values. Tests use obvious fake values.
• Post text from the internet is untrusted: it is fenced inside the scoring prompt, and the reply is parsed and clamped.

Commands

Convex tests use convex-test, which finds functions only through the modules map in the test file. List every module you use. Web tests blank VITE_CONVEX_URL, so run them from apps/web (or with --filter web).

If you change the token format in apps/extension/lib.js or convex/lib/token.ts, keep apps/web/src/lib/__tests__/extension-contract.test.ts green and rebuild the zip.

Testing a platform reader honestly

A stand-in HTML page proves your code; it does not prove the real site. Both X and Facebook passed their stand-in tests and then needed a fix on the first real run (X rejected Camoufox, Facebook used a different page layout). So:

1. Test the reader in real Chrome against a stand-in page (see clients/tests/test_x_browser.py, test_facebook.py).
2. Check it once against the real site with a fake login. It must report "login refused", not crash.
3. Run it with a real throwaway account before you call it done.

When X or Facebook changes its page

The readers depend on page structure, so they break when the site changes. To fix one:

1. Set the debug folder and re-run with --show:

   export LISTENINGKIT_FB_DEBUG_DIR=/tmp/fb      # or LISTENINGKIT_X_DEBUG_DIR for X
   python clients/facebook_push.py --phrases "need a helper" --count 5 --dry-run --verbose --show

2. When it finds nothing it writes fb_page.png (or x_page.png), the page text *.txt, and for Facebook the page HTML fb_page.html without scripts.
3. Read the screenshot first: is the site showing results, a login page, "Something went wrong", or a checkpoint?
4. If results are showing but not read, find the real markup in the saved HTML and change the selectors in clients/facebook_browser.py (EXTRACT_JS) or clients/x_browser.py.
5. Add a stand-in test for the new layout, then re-run for real.

Things learned the hard way: X rejects Camoufox's browser fingerprint even with a good login, so the readers use real Chrome via Playwright. Facebook only fills in a post's address while the mouse is over its timestamp, so the reader hovers each one and falls back to a stable id plus a search link. The reader also keeps only posts whose text contains the phrase, because sites pad quiet searches with unrelated posts.

Adding another platform

1. Add it to convex/lib/token.ts (cookie names and domains), the platform union in convex/schema.ts, and apps/extension/lib.js.
2. Write a reader class with search_posts(phrase, count) like clients/facebook_browser.py, raising Unauthorized, TooManyRequests and AccountLocked (matched by class name).
3. Write a *_push.py that calls x_push.run_once(args, client, phrases, platform=..., search=..., to_post=...).
4. Add it to LIVE_PLATFORMS in apps/web/src/lib/platform-support.ts and to the helper note in DashboardKeywordsLive.tsx.
5. Test as above, and run the extension contract test.

Ownership and what is left

Who owns the GitHub repo, the Convex team and the Clerk app, and the ordered to-do list (OpenAI key, Reddit app, submission), are in HANDOFF.md. The public build log is hackathon.md.