<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[ScrapeMole]]></title><description><![CDATA[ScrapeMole]]></description><link>https://scrapemole.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Fri, 30 Oct 2026 16:54:01 GMT</lastBuildDate><atom:link href="https://scrapemole.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[77 pay-per-call endpoints for AI agents over x402: what the 402 looks like, what a paid call costs, and what we've actually earned ($0.50)]]></title><description><![CDATA[Maker disclosure: I run ScrapeMole, which sells these endpoints. All numbers below come from our own live endpoints and our on-chain ledger, measured on 28…]]></description><link>https://scrapemole.hashnode.dev/77-pay-per-call-endpoints-for-ai-agents-over-x402-what-the-402-looks-like-what-a-paid-call-costs-and-what-we-ve-actually-earned-0-50</link><guid isPermaLink="true">https://scrapemole.hashnode.dev/77-pay-per-call-endpoints-for-ai-agents-over-x402-what-the-402-looks-like-what-a-paid-call-costs-and-what-we-ve-actually-earned-0-50</guid><category><![CDATA[x402]]></category><category><![CDATA[ai agents]]></category><category><![CDATA[crypto]]></category><category><![CDATA[langchain]]></category><dc:creator><![CDATA[ScrapeMole]]></dc:creator><pubDate>Wed, 28 Oct 2026 11:22:57 GMT</pubDate><content:encoded><![CDATA[<blockquote>
<p>Maker disclosure: I run ScrapeMole, which sells these endpoints. All numbers below come from our own live endpoints and our on-chain ledger, measured on 28 Oct 2026. They're small, and I'm not rounding them up.</p>
</blockquote>
<p>x402 turns the old HTTP status <code>402 Payment Required</code> into a working payment flow. The client asks for a resource and gets a 402 with a price. It signs a USDC transfer, retries, and gets the data. There's no API key, no account and no monthly plan, which is why it fits AI agents that need one piece of data, once, from a service they've never seen.</p>
<p>We run 12 x402 origins on the Cloudflare Workers free plan. Here's what they look like from the outside, plus what happened when we used x402 as a <em>buyer</em> inside LangChain and Cloudflare Agents.</p>
<h2>The catalog, measured</h2>
<p>Reading the 12 public <code>openapi.json</code> files on 28 Oct 2026 gives:</p>
<ul>
<li><strong>77 paid endpoints</strong>, every one with a price in <code>x-payment-info</code></li>
<li><strong>Min $0.001, median $0.002, max $0.01</strong> per call</li>
<li>Categories: web and news search, page reader, YouTube, Google Trends, domains/DNS/WHOIS/SSL, on-chain prices and balances, SEC company filings, Wikipedia, FX/time/weather, npm/PyPI packages, arXiv and more</li>
</ul>
<h2>What an agent sees on the first request</h2>
<pre><code class="language-bash">curl -i "https://pulse.scrapemole.workers.dev/v1/time/now?tz=Europe/Madrid"
</code></pre>
<p>Decoded <code>payment-required</code> header (trimmed):</p>
<pre><code class="language-json">{"accepts": [
  {"network": "eip155:8453", "amount": "1000", "asset": "0x8335…2913"},
  {"network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp", "amount": "1000", "asset": "EPjF…Dt1v"}
]}
</code></pre>
<p>That's $0.001 in USDC, payable on Base or Solana. Measured round-trip for the unpaid 402, five calls in a row from Europe: <strong>766, 826, 39, 37, 27 ms</strong>. The first two were cold isolates; warm ones answer in about 30–40 ms.</p>
<p>Two design choices that matter for agents:</p>
<ol>
<li><strong>Bad input never costs money.</strong> Inputs are checked before the 402 is issued. In our last validation matrix, 573 of 574 malformed inputs got a free <code>400</code> instead of a price quote.</li>
<li><strong>Failures are never settled.</strong> Payment only settles on a response below 400, so a timeout or upstream error costs $0.</li>
</ol>
<p>There's also one MCP endpoint per origin (<code>/mcp</code>), so MCP clients that handle x402 can call the same tools.</p>
<h2>Using x402 as a buyer: two framework integrations</h2>
<p>To test the buyer side, we wrote two small integrations against a third-party x402 API (Blockchain IQ's market brief):</p>
<table>
<thead>
<tr>
<th>Integration</th>
<th>Size</th>
<th>Paid test call</th>
<th>Proof</th>
</tr>
</thead>
<tbody><tr>
<td>LangChain tool (Python)</td>
<td>298 lines (client 165, tools 110, init 23)</td>
<td>$0.01 USDC on Base</td>
<td><a href="https://basescan.org/tx/0xa6269f589e477345244689de339d496115b003a1a58b0bda0f0a436cb31746d4">https://basescan.org/tx/0xa6269f589e477345244689de339d496115b003a1a58b0bda0f0a436cb31746d4</a></td>
</tr>
<tr>
<td>Cloudflare Agents example (TS)</td>
<td>164 lines</td>
<td>$0.01 USDC on Base</td>
<td><a href="https://basescan.org/tx/0xdf23be05e48991f340998b84206e3c0a944b22ba8a54945831faa330fabfb6cf">https://basescan.org/tx/0xdf23be05e48991f340998b84206e3c0a944b22ba8a54945831faa330fabfb6cf</a></td>
</tr>
</tbody></table>
<p>The pattern that makes x402 safe inside an agent is a <strong>per-call spend cap checked before signing</strong>:</p>
<pre><code class="language-python">client = BlockchainIQClient(max_usdc_per_call="0.01")
price = client.quote("brief")   # free: reads the unpaid 402
if price &gt; 10_000:              # atomic USDC, 6 decimals
    raise SystemExit("quote above $0.01, refusing to pay")
</code></pre>
<p>The model chooses <em>whether</em> to call the tool, but it never chooses <em>how much</em> to pay. Both repos can be cloned from <a href="https://scrapemole.pages.dev/code/">https://scrapemole.pages.dev/code/</a>.</p>
<h2>What we've actually earned</h2>
<p>From our on-chain ledger (Base, last checked 27 Oct 2026, 21:50 CET):</p>
<ul>
<li><strong>$0.501 USDC lifetime, 122 settled payments, 4 paying wallets</strong></li>
<li>Two of those wallets are very likely the same operator (one funded the other)</li>
<li>Almost every paid call we could attribute was YouTube search or transcripts</li>
<li>Solana went live on 26 Oct and has received $0 so far</li>
</ul>
<p>So x402 works mechanically: unknown agents did pay, and one came back many times. We're listed on x402scan, x402dash and 402 Index, but we can't tell which listing sent them. But volume per seller is tiny. Public x402 stats we read on 19 Oct showed a median seller earning about $0.01 over 30 days. If you're building here, design for many small integrations rather than one big customer.</p>
<h2>If you want to try it</h2>
<ul>
<li>No wallet: most of the same data is on the Apify Store, billed per result: <a href="https://apify.com/scrapemole">https://apify.com/scrapemole</a></li>
<li>With a wallet: <a href="https://scrapemole.pages.dev">https://scrapemole.pages.dev</a> (each tool page shows the curl, the price and the MCP config)</li>
</ul>
<p>Questions about the cap pattern or the 402 validation order are welcome. I'll answer in the comments.</p>
]]></content:encoded></item><item><title><![CDATA[Getting 200 top-level YouTube comments took 60 s and 1,346 rows with a DIY library, versus 4.5 s with an Actor (plus a transcript test)]]></title><description><![CDATA[Maker disclosure: I build ScrapeMole, which publishes the YouTube Comments and Transcript Actors used below. The open-source libraries I compare against…]]></description><link>https://scrapemole.hashnode.dev/getting-200-top-level-youtube-comments-took-60-s-and-1-346-rows-with-a-diy-library-versus-4-5-s-with-an-actor-plus-a-transcript-test</link><guid isPermaLink="true">https://scrapemole.hashnode.dev/getting-200-top-level-youtube-comments-took-60-s-and-1-346-rows-with-a-diy-library-versus-4-5-s-with-an-actor-plus-a-transcript-test</guid><category><![CDATA[Python]]></category><category><![CDATA[youtube]]></category><category><![CDATA[nlp]]></category><category><![CDATA[webscraping ]]></category><dc:creator><![CDATA[ScrapeMole]]></dc:creator><pubDate>Wed, 28 Oct 2026 11:21:34 GMT</pubDate><content:encoded><![CDATA[<blockquote>
<p>Maker disclosure: I build ScrapeMole, which publishes the YouTube Comments and Transcript Actors used below. The open-source libraries I compare against are good. This is where they differ, measured on 28 Oct 2026.</p>
</blockquote>
<p>If you're feeding YouTube into a sentiment model or a RAG index, you need two things: comments and transcripts. I ran the free Python libraries and our Actors on the same videos from the same Linux box, with no proxy.</p>
<h2>Comments: 200 top comments from one video</h2>
<p>The video is 3Blue1Brown's "But what is a neural network?" (<code>aircAruvnKk</code>, about 9,442 comments).</p>
<p><strong>DIY with <code>youtube-comment-downloader</code> 0.1.84:</strong></p>
<pre><code class="language-python">from youtube_comment_downloader import YoutubeCommentDownloader, SORT_BY_POPULAR
d = YoutubeCommentDownloader()
for c in d.get_comments_from_url(URL, sort_by=SORT_BY_POPULAR, language="en"):
    ...
</code></pre>
<p>What I measured:</p>
<ul>
<li>The first 200 rows held only <strong>20 top-level comments</strong>. The other 180 were replies, interleaved with their parents.</li>
<li>Collecting <strong>200 top-level comments meant pulling 1,346 rows and took 60.2 s.</strong></li>
<li>Without <code>language="en"</code>, the box's IP got localized strings back: likes as <code>"14 tūkst."</code> and dates as <code>"prieš 7 metus"</code> (Lithuanian). With <code>language="en"</code> you get <code>"14K"</code> and <code>"7 years ago"</code>, which you still have to parse.</li>
</ul>
<p><strong>Our YouTube Comments Scraper</strong> (same code as on Apify, run locally):</p>
<pre><code class="language-json">{ "videos": ["https://www.youtube.com/watch?v=aircAruvnKk"], "maxComments": 200, "sortBy": "top" }
</code></pre>
<ul>
<li><strong>200 top-level comments in 4.5 s</strong>, 0 errors</li>
<li>Replies are off by default and can be turned on with <code>includeReplies</code></li>
<li><code>likeCount</code> comes back as an integer (14000), <code>publishedAtApprox</code> as a date, <code>totalCommentsOnVideo</code> as 9442, plus <code>isPinned</code>, <code>isHeartedByCreator</code>, <code>isByVideoCreator</code>, <code>replyCount</code>, <code>videoTitle</code> and 23 fields in total</li>
</ul>
<p>Cost: the Actor is $0.001 per comment on Apify's free tier, so these 200 comments are $0.20. Other public prices per comment on 28 Oct 2026 (Apify Store API): streamers/youtube-comments-scraper $0.002, apidojo $0.0005 plus $0.001 per query, solidcode $0.00035, supreme_coder $0.00025 plus start. We're not the cheapest. The point is clean top-level rows with numeric fields. The DIY library is free if you're happy to filter and parse.</p>
<h2>Transcripts: 10 well-known videos</h2>
<p><strong>DIY with <code>youtube-transcript-api</code> 1.2.4, <code>languages=["en"]</code>:</strong> <strong>9 of 10 in 7.1 s total</strong> (0.6–0.9 s each). The miss was PSY's "Gangnam Style" (<code>9bZkp7q19f0</code>), which raised <code>NoTranscriptFound</code> because it has Korean auto-captions and no English track. You can handle that in a few more lines with <code>list()</code>. Out of the box, though, you get text and timestamps only, with no video metadata.</p>
<p><strong>Our YouTube Transcript Scraper on Apify</strong> (same 10 IDs, <code>fallbackToAnyLanguage: true, includeSrt: true</code>):</p>
<ul>
<li><strong>10 of 10 ok, 7.2 s run time.</strong> Gangnam Style came back in Korean (<code>ko</code>, auto-generated, 67 segments).</li>
<li>Word counts matched the DIY run exactly on all 9 shared videos (for example 3,357 words / 286 segments on <code>aircAruvnKk</code>, and 21,030 / 2,955 on Karpathy's "Let's build GPT")</li>
<li>29 fields per video, including SRT, title, channel, view count, duration, publish date and available languages</li>
<li>Apify platform usage for the run: <strong>$0.0024</strong>. The transcript event costs $0.004 per video on the free tier ($4 per 1,000).</li>
</ul>
<p>For comparison, the official YouTube Data API's <code>captions.download</code> costs 200 quota units and "requires the user to have permission to edit the video", so it only works on your own videos. Its <code>commentThreads.list</code> costs 1 unit for up to 100 comments, and the default quota is 10,000 units/day for non-search endpoints. That's fine for comments if you have a Google Cloud project and a key.</p>
<h2>So which should you use?</h2>
<ul>
<li><strong>One-off notebook:</strong> the free libraries. Remember <code>language="en"</code> and filter out replies.</li>
<li><strong>Scheduled pipeline, many videos, or a no-code tool (n8n, Make, Zapier):</strong> the Actors. You get one JSON schema, numeric fields, no IP-locale surprises, and you don't pay for videos without captions.</li>
<li><strong>AI agents with a wallet:</strong> the same transcript and comments endpoints are on x402 at $0.002 per call, with no API key.</li>
</ul>
<p>Links:</p>
<ul>
<li>Comments Actor: <a href="https://apify.com/scrapemole/youtube-comments-scraper">https://apify.com/scrapemole/youtube-comments-scraper</a></li>
<li>Transcript Actor: <a href="https://apify.com/scrapemole/youtube-transcript-scraper">https://apify.com/scrapemole/youtube-transcript-scraper</a></li>
<li>Sentiment walkthrough (VADER, weighted by likes): <a href="https://scrapemole.pages.dev/guides/youtube-comments-sentiment-analysis-python/">https://scrapemole.pages.dev/guides/youtube-comments-sentiment-analysis-python/</a></li>
<li>ScrapeMole: <a href="https://scrapemole.pages.dev">https://scrapemole.pages.dev</a></li>
</ul>
<p><em>Method: single runs on 28 Oct 2026, around 12:00 CET, from one datacenter IP. Timings vary run to run, and your IP may be treated differently by YouTube.</em></p>
]]></content:encoded></item><item><title><![CDATA[Hiring intent from public job boards: 1,307 jobs from Greenhouse, Lever and Ashby in 8 seconds (DIY vs Actor, measured)]]></title><description><![CDATA[Maker disclosure: I build ScrapeMole, including the ATS Jobs Scraper used below. The DIY script is free and works; this post shows exactly where each…]]></description><link>https://scrapemole.hashnode.dev/hiring-intent-from-public-job-boards-1-307-jobs-from-greenhouse-lever-and-ashby-in-8-seconds-diy-vs-actor-measured</link><guid isPermaLink="true">https://scrapemole.hashnode.dev/hiring-intent-from-public-job-boards-1-307-jobs-from-greenhouse-lever-and-ashby-in-8-seconds-diy-vs-actor-measured</guid><category><![CDATA[sales]]></category><category><![CDATA[Python]]></category><category><![CDATA[webscraping ]]></category><category><![CDATA[data]]></category><dc:creator><![CDATA[ScrapeMole]]></dc:creator><pubDate>Wed, 28 Oct 2026 11:18:36 GMT</pubDate><content:encoded><![CDATA[<blockquote>
<p>Maker disclosure: I build ScrapeMole, including the ATS Jobs Scraper used below. The DIY script is free and works; this post shows exactly where each approach wins.</p>
</blockquote>
<p>Job postings are one of the few public signals that tell you what a company is about to spend money on. A burst of "Go To Market" roles means a sales push; ten new "Scaling" roles at an AI lab means data-center spend. Most companies publish these through an applicant tracking system (ATS), and Greenhouse, Lever and Ashby all expose a public, no-login JSON feed of open roles.</p>
<p>So the data is easy to get. The real work is the cleanup afterwards. I measured both routes on 28 Oct 2026 against three real boards: Airbnb (Greenhouse), Palantir (Lever) and OpenAI (Ashby).</p>
<h2>Route 1: DIY with requests (17 lines)</h2>
<pre><code class="language-python">import requests
def fetch(ats, slug):
    if ats == "greenhouse":
        js = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs", params={"content": "true"}).json()["jobs"]
        return [{"title": j["title"], "location": (j.get("location") or {}).get("name"), "url": j["absolute_url"]} for j in js]
    if ats == "lever":
        js = requests.get(f"https://api.lever.co/v0/postings/{slug}", params={"mode": "json"}).json()
        return [{"title": j["text"], "location": j["categories"].get("location"), "url": j["hostedUrl"], "salary": j.get("salaryRange")} for j in js]
    if ats == "ashby":
        js = requests.get(f"https://api.ashbyhq.com/posting-api/job-board/{slug}", params={"includeCompensation": "true"}).json()["jobs"]
        return [{"title": j["title"], "location": j.get("location"), "url": j["jobUrl"],
                 "salary": (j.get("compensation") or {}).get("compensationTierSummary")} for j in js]
</code></pre>
<p>Measured result (one run from a datacenter box, no proxy):</p>
<table>
<thead>
<tr>
<th>Board</th>
<th>Jobs</th>
<th>Time</th>
<th>Structured salary from the API</th>
</tr>
</thead>
<tbody><tr>
<td>greenhouse:airbnb</td>
<td>196</td>
<td>2.37 s</td>
<td>0</td>
</tr>
<tr>
<td>lever:palantir</td>
<td>306</td>
<td>3.22 s</td>
<td>0</td>
</tr>
<tr>
<td>ashby:openai</td>
<td>805</td>
<td>0.42 s</td>
<td>641 (as a display string, e.g. "$212K – $295K • Offers Equity")</td>
</tr>
<tr>
<td><strong>Total</strong></td>
<td><strong>1,307</strong></td>
<td><strong>6.0 s</strong></td>
<td><strong>641</strong></td>
</tr>
</tbody></table>
<p>That's fast and free. But three schemas means three field maps, three date formats and three location shapes. Greenhouse and Lever (for this board) give you no structured salary at all. The pay range is buried in the HTML description ("India Annual Pay Range ₹1,610,000 — ₹2,110,000 INR").</p>
<h2>Route 2: the ATS Jobs Scraper on Apify</h2>
<p>Input:</p>
<pre><code class="language-json">{ "companies": ["greenhouse:airbnb", "lever:palantir", "ashby:openai"], "maxJobsPerCompany": 1000 }
</code></pre>
<p>Measured result (same day, the same Actor code run locally with descriptions on):</p>
<ul>
<li><strong>1,307 jobs, 0 errors, 5.7 s</strong> (7.9 s on an earlier run the same morning).</li>
<li>One flat schema with 27 fields for all three ATSs: <code>title, companyName, location(s), country, remote, workplaceType, department, team, employmentType, postedAt, salaryMin, salaryMax, salaryCurrency, salaryInterval, salaryText, salarySource, description, applyUrl…</code></li>
<li><code>postedAt</code> was present on 1,307 of 1,307 rows. For Greenhouse it uses <code>first_published</code>, so edits don't make old jobs look new.</li>
<li>Salary found on <strong>1,051 of 1,307 jobs (80%)</strong>: OpenAI 657/805, Airbnb 160/196, Palantir 234/306. For Greenhouse and Lever, the salary is parsed out of the description text.</li>
</ul>
<p>Both ends of the range come through: <code>salaryMax</code> is filled on 1,001 of the 1,051 jobs with a salary (Airbnb 159/160, Palantir 201/234, OpenAI 641/657). The rest have a single figure in the posting (e.g. a $6,700 monthly internship rate). Greenhouse's "₹1,610,000 — ₹2,110,000 INR" becomes <code>salaryMin: 1610000, salaryMax: 2110000, salaryCurrency: "INR", salaryInterval: "year"</code>.</p>
<h2>The hiring-intent part: what changed this week</h2>
<p>Add <code>"postedWithinDays": 7</code> and the same three boards return <strong>99 jobs posted in the last 7 days</strong> (5.7 s): OpenAI 75, Airbnb 19, Palantir 5. Grouped by department, the top three were OpenAI Go To Market (12), OpenAI Scaling (10) and OpenAI Marketing (7).</p>
<p>That's the useful output for a sales or research workflow. You get a weekly diff of who is hiring for what, not a 1,307-row dump. For a scheduled run, <code>onlyNew: true</code> keeps state between runs and only returns (and only charges for) jobs it hasn't seen before.</p>
<h2>Cost per 1,000 jobs (public Apify Store prices, read 28 Oct 2026)</h2>
<table>
<thead>
<tr>
<th>Actor</th>
<th>Free-tier price per job</th>
<th>Per 1,000 jobs</th>
</tr>
</thead>
<tbody><tr>
<td>scrapemole/ats-jobs-scraper</td>
<td>$0.0015</td>
<td>$1.50</td>
</tr>
<tr>
<td>bovi/greenhouse-lever-ashby-job-scraper</td>
<td>$0.0015 (+$0.00005 start)</td>
<td>$1.50</td>
</tr>
<tr>
<td>yugenox/ats-jobs-search</td>
<td>$0.002</td>
<td>$2.00</td>
</tr>
<tr>
<td>factpipe/ats-jobs-feed</td>
<td>$0.003</td>
<td>$3.00</td>
</tr>
<tr>
<td>DIY script above</td>
<td>$0</td>
<td>$0 (plus your time and parser upkeep)</td>
</tr>
</tbody></table>
<p>The honest verdict: if you track one ATS and don't need salary, use the 17-line script. Use the Actor when you track many companies across several ATSs, need salary normalized into numbers, or want "only new since last run" on a schedule without running your own state store.</p>
<ul>
<li>Actor: <a href="https://apify.com/scrapemole/ats-jobs-scraper">https://apify.com/scrapemole/ats-jobs-scraper</a></li>
<li>Full Python walkthrough: <a href="https://scrapemole.pages.dev/guides/greenhouse-lever-ashby-jobs-python/">https://scrapemole.pages.dev/guides/greenhouse-lever-ashby-jobs-python/</a></li>
<li>ScrapeMole: <a href="https://scrapemole.pages.dev">https://scrapemole.pages.dev</a></li>
</ul>
<p><em>Method: both scripts were run on 28 Oct 2026, around 12:00 CET, from one Linux box with no proxy. The timings come from a single run each, so expect variance.</em></p>
]]></content:encoded></item></channel></rss>