Find Founders

This script finds founder/co-founder names for a list of companies using open-source tools and a local LLM. This tool requires a working transformers installation and a backend (PyTorch or TensorFlow). See setup below.

Files

• main.py - script
• companies.txt - input, list of 10 companies
• companies_50.txt - input, list of 50 companies
• requirements.txt - Python package dependencies

Output

• founders.json - output (company -> JSON list of founder names)
• founders.provenance.json - evidence (everything used to determine the founder names)

Approach

• Prefer Wikipedia infobox as a source of truth: parse infobox + sentences on the company page that contain founder-related keywords.
• If Wikipedia doesn't provide usable snippets, run DuckDuckGo searches and collect sentence snippets (specifically, sentence-level snippets that mention the startup/company and include a founder-related keyword and a person-like name).
• Feed the collected snippets and the company name to an open-source LLM (via transformers) which is prompted to output the most likely founder names.
• The script performs simple validation (check that returned names or surnames appear in the snippets) and writes results to JSON.

Assumptions Made

• We can't rely on easily finding founders on a company's own webpage, rather, relying on Wikipedia + google search fallback is a better approach.
• The Wikipedia infobox for a company, if present, accurately lists the company’s founders under a label such as “Founders”, “Founder”, or “Founded by”.
• For non-Wikipedia sources, DuckDuckGo returns relevant English language pages with readable HTML content accessible via simple GET requests (no JavaScript rendering required).
• Text snippets from a web search mentioning both a relevant keyword (“founded by”, “co-founder”) and a person-like name are sufficient evidence to infer founders.
• The LLM will always return free-form text that can be extracted into usable names and formatted into a JSON list with some simple post processing.
• Founder names are names of individuals and not group names or organization names

Future Improvements

• Implement a weighted confidence system where different evidence sources (Wikipedia, news articles, company website, etc.) are scored by reliability.
• The company’s official website should be treated as the ground truth source, but this will also require building a recursive site crawler to traverse and extract relevant text from the entire website (i.e. scanning /about, /team, /history pages for founder mentions).
• Before feeding in snippets to the LLM do some better pre-processing: maybe score with BM25 or simple keyword+name heuristics and pass only the top ranked, unique snippets to the LLM to reduce noise.
• Sometimes it's very hard to get high quality sources from web search and the LLM used currently (flan-t5-base) has a limited input window, so switching to a model with a larger context size (or fine-tuning one) would definitely result in better snippet coverage and higher quality output.

Setup

1) create & activate venv

python -m venv venv

mac/linux

source venv/bin/activate

windows (PowerShell)

venv\Scripts\Activate

2) install dependencies (+ torch)

install dependencies

pip install -r requirements.txt

install torch for CPU

python -m pip install torch --index-url https://download.pytorch.org/whl/cpu

install torch for NVIDIA GPU (example: CUDA 11.8 / cu118)

Go to https://pytorch.org/get-started/locally/ and find the pip install command example:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126

To run

python main.py --input companies.txt --output founders.json

or run the list of 50 companies

python main.py --input companies_50.txt --output founders.json

add the --verbose flag to print progress logs (useful for tracking how each company is being processed)

python main.py --input companies.txt --output founders.json --verbose

The first time the script runs, transformers will automatically download the model (about 850 MB for flan-t5-base).