Skip to content

olostep-api/OloTender

Repository files navigation

OloTender

OloTender is an AI-powered tender matching app for government contracts. It starts from a user profile, scrapes active opportunities from public procurement portals using Olostep, extracts and scores the results, stores them in Qdrant for semantic search, and lets you keep only the tenders that match your trade, location, project size, and description.

Untitled.design.6.mp4

What this app does

  • Builds a profile around your trade, region, project size, email, and optional description.
  • Scrapes fresh tenders from SAM.gov and NYSCR.
  • Uses OpenAI for extraction, embedding, and relevance scoring.
  • Stores indexed tenders in Qdrant, so search is fast and historical.
  • Marks tenders as saved or applied.
  • Download saved tenders as CSV.
  • Flags old results as stale(automatically) and refreshes them in the background.
  • Sends email digests when the pipeline is configured to do so.
  • Supports Olostep scheduling as well for automated refreshes and sending mail when a new opportunity is found.

How it works

flowchart TD
	A[Create profile] --> B[Save profile to data/profile.json]
	B --> C[Fetch tenders from Dashboard]
	C --> D[Scrape SAM.gov and NYSCR]
	D --> E[Extract title, deadline, value, scope, contact]
	E --> F[Map project size and score relevance]
	F --> G[Embed and upsert into Qdrant]
	G --> H[Dashboard renders fresh tenders]
	G --> I[Search reads historical Qdrant results]
	H --> J[Save or mark applied]
	B --> K[Create Olostep schedule]
	K --> L[Scheduled webhook calls /api/scrape]
Loading

The core pipeline is in lib/pipeline.ts and does this in order:

  1. Scrapes the configured portal search pages.
  2. Opens tender detail pages and extracts structured fields.
  3. Maps contract value to project size locally.
  4. Generates embeddings for the tender text.
  5. Scores the tender against trade, location, size, and description.
  6. Upserts the final payload into Qdrant.
  7. Optionally sends an email digest when an email is passed into the pipeline.

Main tabs

Dashboard

This is the live matching screen. It shows the latest tenders that match your saved profile. Use the Fetch Tenders / Refresh Tenders button to run a fresh scrape based on your profile description, project size, location, and trade.

If the newest indexed tender is older than 24 hours, the dashboard marks the list as stale and kicks off a background refresh.

Tender History

This is the search tab. It does not scrape the website. It performs vector search over the tenders already stored in Qdrant, so it returns past indexed results instead of live web results.

You can search, filter by source and status, sort by relevance or deadline, and open tender details from the saved history.

Saved

This tab shows only the tenders you bookmarked. It is useful for tracking opportunities you want to revisit, mark applied, or compare later.

Profile setup

The home screen is the profile form. It captures the matching criteria that power the dashboard and scheduling flow:

  • Trade or keywords
  • Location
  • Project size
  • Description
  • Email for digests and schedule setup

Tech stack

  • Next.js 16
  • React 19
  • TypeScript
  • Olostep for scraping and scheduling
  • OpenAI for extraction, embeddings, and scoring
  • Qdrant for vector storage and search
  • Resend for email digests
  • Zustand for client state
  • Tailwind CSS and custom UI components

Local setup

  1. Clone the repo and move into the project folder.

    git clone <repo-url>
    cd olotender
  2. Install dependencies.

    npm install
  3. Create a .env.local file in the project root.

    OLOSTEP_API_KEY=your_olostep_key
    OPENAI_API_KEY=your_openai_key
    QDRANT_URL=your_qdrant_cluster_url
    QDRANT_API_KEY=your_qdrant_api_key
    RESEND_API_KEY=your_resend_api_key
    NEXT_PUBLIC_APP_URL=http://localhost:3000
  4. Start the app.

    npm run dev
  5. Open the app, fill out your profile, and save it.

  6. Go to the Dashboard and click Fetch Tenders / Refresh Tenders to index the first set of matches.

Local scheduling

OloTender uses Olostep schedules instead of node-cron or Vercel Cron jobs. The app registers a schedule with Olostep, and Olostep calls your public /api/scrape endpoint on the interval you choose.

Create a schedule locally

Because Olostep must reach your app over the internet, expose your local server with ngrok first:

ngrok http 3000

Then send a POST request in Postman to the local schedule endpoint:

http://localhost:3000/api/schedule

Use this JSON body:

{
  "action": "create"
}

The schedule endpoint reads your saved profile, creates the Olostep schedule, and stores the returned schedule ID in data/profile.json.

Delete a schedule locally

Copy the schedule ID from the profile data or the create response, then send this body in Postman:

{
  "action": "delete",
  "scheduleId": "your_schedule_id_here"
}

This removes the Olostep schedule and clears the stored schedule ID.

Email note

The profile email is used for the matching workflow and optional digest delivery. If your pipeline and schedule are configured to send digests, you will receive email updates when the scheduled run fires.

API routes

  • POST /api/profile saves the profile.
  • GET /api/profile loads the saved profile.
  • POST /api/scrape runs the scraping and indexing pipeline.
  • GET /api/tenders loads indexed tenders from Qdrant.
  • POST /api/search performs vector search over historical tenders in Qdrant.
  • PATCH /api/tender/[id] updates saved, applied, or viewed state.
  • POST /api/schedule creates or deletes an Olostep schedule.
  • GET /api/export downloads the stored tenders as CSV.

Data and behavior notes

  • The app stores the profile locally in data/profile.json.
  • Qdrant automatically stores and returns historical tender records.
  • Search uses embeddings and does not fetch the live web.
  • Dashboard results can be stale; the app marks them stale after 24 hours and refreshes in the background.
  • Project size is mapped locally from contract value because the source portals do not provide strict size labels.
  • You can save tenders from the dashboard or search results and mark them applied later.

Adding more portals

To add another portal, update config/portals.ts with the new search and detail URLs, then extend lib/pipeline.ts so it can scrape, parse, score, and upsert that source too.

Troubleshooting

  • If search returns no results, run a dashboard scrape first so Qdrant has data.
  • If scheduling does not work locally, make sure ngrok is running and the public URL is reachable.
  • If scraping fails, verify OLOSTEP_API_KEY, OPENAI_API_KEY, QDRANT_URL, and QDRANT_API_KEY.
  • If email digests do not arrive, confirm RESEND_API_KEY is set and your profile email is valid.

Refrences

Docs & references:

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages