ScrapeGraphAI: build free lead lists by scraping names, emails, phones

Here is the quick start. You will install ScrapeGraphAI and run a tiny check to confirm the import works. Later you can wire it to a local model with Ollama (a free local LLM runtime) so you pay zero per request.

Install into a fresh virtual environment so your global Python stays clean.

bash
python3 -m venv .venv && source .venv/bin/activate
pip install -U pip scrapegraphai

Verify the package loads. If this prints ok, you are set.

bash
python -c "from scrapegraphai.graphs import SmartScraperGraph; print('ok')"

From here you feed ScrapeGraphAI a URL and a short prompt like “return company, contact name, email, phone as JSON.” It parses the page and gives you a clean structure you can paste into Google Sheets or import into your CRM. Stop here if you are browsing on your phone. Comment AI on the reel and I will DM the full setup with local models, dynamic pages, and batch runs.

Here is the quick start. You will install ScrapeGraphAI and run a tiny check to confirm the import works. Later you can wire it to a local model with Ollama (a free local LLM runtime) so you pay zero per request.

Install into a fresh virtual environment so your global Python stays clean.

bash
python3 -m venv .venv && source .venv/bin/activate
pip install -U pip scrapegraphai

Verify the package loads. If this prints ok, you are set.

bash
python -c "from scrapegraphai.graphs import SmartScraperGraph; print('ok')"

From here you feed ScrapeGraphAI a URL and a short prompt like “return company, contact name, email, phone as JSON.” It parses the page and gives you a clean structure you can paste into Google Sheets or import into your CRM.

Setup

Pick your platform, then add a local model so runs stay free.

macOS

bash
# Python env
python3 -m venv .venv && source .venv/bin/activate
pip install -U pip scrapegraphai playwright
python -m playwright install chromium

# Local model runtime
brew install ollama
ollama pull llama3

Linux (Debian/Ubuntu)

bash
# Python 3.11 and venv
sudo apt update && sudo apt install -y python3.11 python3.11-venv
python3.11 -m venv .venv && source .venv/bin/activate
pip install -U pip scrapegraphai playwright
python -m playwright install chromium

# Local model runtime
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3

Windows 11

bash
# Python and venv
winget install Python.Python.3.11 -s winget
py -3.11 -m venv .venv
.\.venv\Scripts\activate
pip install -U pip scrapegraphai playwright
python -m playwright install chromium

# Local model runtime
winget install Ollama.Ollama -s winget
ollama pull llama3

Verify

Check the Python import works.

bash
python -c "from scrapegraphai.graphs import SmartScraperGraph; print('ok')"

Expected output:

bash
ok

Check Ollama returns a response. This confirms your local model is ready.

bash
ollama run llama3 "Say hi in one short sentence."

Expected output starts with something like:

Hello, good to meet you.

Minimal script

Save this as lead_scrape.py in your project folder. It tells ScrapeGraphAI to grab companies and contact fields as JSON from a single URL using your local model.

py
from scrapegraphai.graphs import SmartScraperGraph

# Target a single directory page for the first run
url = "https://example.com/partners"

prompt = (
    "Extract a list of entries on this page. For each entry return a JSON object "
    "with keys: company, contact_name, email, phone. Use null if a field is missing."
)

config = {
    # Use a local model through an OpenAI-compatible interface
    # ScrapeGraphAI reads the model string and routes through the provider
    "llm": {"model": "ollama/llama3", "temperature": 0},
}

graph = SmartScraperGraph(prompt=prompt, source=url, config=config)
result = graph.run()
print(result)

Run it:

bash
python lead_scrape.py

You should see JSON like this:

json
[
  {"company": "Acme Co", "contact_name": "Jane Roe", "email": "[email protected]", "phone": "+1 555-0100"},
  {"company": "Beta Labs", "contact_name": null, "email": "[email protected]", "phone": null}
]

Configuration tips

• Be explicit in the prompt. List the exact field names you want, and say “use null if missing.” This keeps shape consistent for CSV export.
• Keep temperature at 0 for contact extraction. It stops the model from rewriting names or guessing phones.
• Save HTML for tricky pages. Use Playwright (a headless browser) to render, save to a file, then pass the file path to source. Example:

py
# quick one-off render and save with Playwright
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://example.com/directory", wait_until="networkidle")
    html = page.content()
    open("page.html", "w", encoding="utf-8").write(html)
    browser.close()

Then update your script:

py
graph = SmartScraperGraph(prompt=prompt, source="page.html", config=config)

• Batch over many pages. Drop your URLs into a txt file and loop, writing one JSON per page so you can resume mid-run.

py
import json, time
from scrapegraphai.graphs import SmartScraperGraph

config = {"llm": {"model": "ollama/llama3", "temperature": 0}}

with open("urls.txt") as f:
    urls = [u.strip() for u in f if u.strip()]

for i, u in enumerate(urls, 1):
    g = SmartScraperGraph(prompt=prompt, source=u, config=config)
    out = g.run()
    open(f"out_{i}.json", "w", encoding="utf-8").write(json.dumps(out, ensure_ascii=False, indent=2))
    time.sleep(1)  # be polite

• Normalize emails and phones before CSV. A post step keeps your CRM happy.

py
import re

def clean_phone(s: str | None):
    if not s:
        return None
    digits = re.sub(r"\D", "", s)
    return "+" + digits if digits else None

Troubleshooting

• Error: ModuleNotFoundError: No module named scrapegraphai. Fix: activate your venv before running. On macOS and Linux run source .venv/bin/activate. On Windows run .venv\Scripts\activate.
• Error: playwright.errors.BrowserType.launch: Executable doesn’t exist. Fix: install the browser bundle once. Run python -m playwright install chromium.
• Output is empty on a JavaScript page. Fix: render with Playwright and pass the saved HTML file path to source as shown above. Static fetching misses content that loads after page render.

When it beats manual research

• Conference sponsor pages where each card shows a company logo, a blurb, and a contact link. You get a structured list in one pass instead of copying one by one.
• Local association directories with consistent entry blocks. The model locks onto the pattern and returns uniform rows.
• Job boards for tracking fresh hires. Prompt for company, role title, posting date, and application email, then run it nightly.
• Partner lists on SaaS sites. Pull company name and generic contact email, then feed that into your first-touch sequence.

Sources