Monitors
A suite of purpose-built web scrapers powering the Proxcop and Pokecop alert networks. Each monitor runs an independent fetch → diff → notify → persist loop tailored to its target site. Go monitors handle Shopify products.json polling (detecting new products, new variants, and restocks) and sitemap monitoring (Chrome Hearts). Python handles JS-rendered targets requiring browser emulation and fingerprint bypass. Node.js integrates with platform APIs. All monitors use exponential backoff, disk-persisted state to prevent duplicate notifications, heartbeat pings for liveness verification, and PM2 for production process management.
Overview
The Monitors are a suite of specialized web scraping processes that form the intelligence engine behind the Proxcop and Pokecop communities. Each monitor is purpose-built for a specific target website, handling the unique page structure, anti-bot measures, and data extraction patterns of that site. When stock appears or changes, monitors fire Discord webhook notifications to community members within seconds.
The system spans multiple languages — Go for high-performance polling monitors, Python for sites requiring browser emulation or advanced fingerprinting bypass, and Node.js for API-integrated monitors. All monitors share the same core loop: fetch → diff against known state → notify on change → persist new state.
Reliability engineering is a first-class concern: exponential backoff on network failures, heartbeat pings to verify liveness, state persistence to disk to survive restarts without duplicate notifications, and PM2 process management for production uptime.
Screenshots
Monitor Types
Shopify Monitor
Polls any Shopify store's public products.json endpoint and diffs it against persisted state to detect three event types: new products, new size variants, and restocks (variant going from unavailable → available). On first boot, state is silently initialized so existing products don't trigger false positives.
Chrome Hearts Monitor
Monitors the Chrome Hearts website sitemap for new product page URLs and redirect changes. A 302 redirect on a known URL signals a stock drop or availability change. The monitor differentiates between 301 permanent redirects (ignored) and 302 temporary redirects (notified). Optional keyword filtering limits alerts to relevant products.
Affluence Backend Monitor
A generic monitor that polls the MonitorManager backend API to fetch configured products for one or more websites, then scrapes each product according to its configuration. Supports multiple sites in a single process run. Results are dispatched to per-product webhook URLs.
Extended Monitor Fleet
Beyond the Go monitors, the system includes a fleet of Python and Node.js monitors targeting sites with more complex anti-bot defenses — browser fingerprint challenges, CAPTCHA, JS-rendered content, and session-based rate limiting. Python handles scraping logic requiring Selenium or Playwright for headless browser emulation. Node.js monitors integrate directly with platform APIs where available (Shopify Storefront API, brand webhook feeds).
Core Loop
Fetch
HTTP request to the target endpoint (products.json, sitemap, API). Headers mimic legitimate browser traffic. Retry with exponential backoff on network failures.
Diff
Compare fetched data against state persisted to disk from the last cycle. New product IDs, new variant IDs, or availability changes trigger notifications.
Notify
Build a rich Discord embed with product name, image, URL, variant details, and price. POST to all configured webhook URLs for that product.
Persist
Write new state to the JSON state file. On the next cycle, this becomes the baseline for diffing. Restart-safe — no duplicate notifications after a process restart.
Sleep
Wait for the configured interval (typically 10–60 seconds) before the next cycle. Interval tuned per target to balance freshness against rate limiting risk.