21/32proof of conceptMarket Intelligence
Market Pulse
Monitors a curated source list and builds reviewed items into a weekly market report.
What it solves
Market research done by hand is a repeated sweep of the same sources. This collects, classifies and summarises each item, so an editor reviews instead of searching.
How it runs
Seven steps, input to output
Goes inA curated list of sources, each with a feed an administrator has validated
- 01
Sources
An administrator adds a source, then validates and enables its feed.
- 02
Collect
Feeds are fetched on a schedule or on demand, under strict limits, with every URL checked first.
- 03
Clean
Feed text is treated as untrusted and reduced to plain text.
- 04
Deduplicate
Layered checks and database constraints stop the same story arriving twice.
- 05
Enrich
Each item gets classification, sentiment and a summary as validated structured output.
- 06
Review
An editor works the review queue, and every status change is audited.
- 07
Report
Reviewed items build into the weekly report.
Comes outA weekly market report, exported as PDF or Word
What it does
Market-intelligence application that monitors a curated source inventory, collects feed metadata under strict fetch limits, and runs every item through classification, sentiment and summary before it reaches an editor.
Layered deduplication and database constraints stop the same story arriving twice, every status change is audited, and reviewed items build into weekly reports exported as PDF or Word: so the research and reporting cycle stops being a manual sweep.
Three decisions
What shaped the build
A source is not permission to collect
A source record is an inventory entry. Collection stays disabled until a feed is separately validated and enabled.
The database is the last duplicate guard
Deduplication is layered, and a uniqueness constraint in Postgres is the final guard when two jobs run at once.
Model output cannot create taxonomy
AI output is checked against a strict schema and mapped to the existing taxonomy. Unknown values are left out, never added.
Built with
8 parts
- OpenAI
- classify · sentiment · summary
- rss-parser
- feed collection
- Apify
- extended collection
- Readability · linkedom
- article extraction
- sanitize-html
- untrusted feed text
- Drizzle · Neon
- Postgres store + dedupe
- Zod
- env and payload validation
- pdf-lib · docx
- report export