All JSON endpoints return 500 (error_type: unknown_http) since ~2026-05-26; web UI renders fine

[knaughtykat01-prog]

Summary

Every scraping endpoint on the public instance (https://faexport.spangle.org.uk, reporting v2025.12.1) has been returning HTTP 500 since at least 2026-05-26 (~2 weeks at time of filing). The web frontend itself renders normally.

Evidence (probed 2026-06-10)

user/fender.json                 -> 500
user/fender/gallery.json?page=1  -> 500
user/fender/scraps.json?page=1   -> 500
user/fender/journals.json?page=1 -> 500
user/fender/watchers.json?page=1 -> 500
submission/4483957.json          -> 500
home.json                        -> 500

Response body is the same for all of them:

{
  "error_type": "unknown_http",
  "error": "FAExport received an unexpected status code when fetching data from FurAffinity"
}

Notes

• Fails identically for multiple unrelated usernames (tested with fender as a known-good control), so it is not account-specific.
• Reproduced from both a residential IP and a GCP datacenter IP, so it is not client-IP-specific.
• FurAffinity itself responds normally to direct requests from the same IPs, so FA is up — the failure appears to be in FAExport's own session/scraper against FA (cookie expired, or FA serving the instance a block/challenge page).
• First observed 2026-05-26; still failing 2026-06-10 with no interruption that I noticed across ~4-hourly polling.

Thanks for running the service! Happy to provide more probes if useful.

[knaughtykat01-prog]

Did some digging in the source to narrow this down.

unknown_http is the Sinatra handler for an uncaught OpenURI::HTTPError (lib/faexport.rb ~1077), so FA must be returning a non-200 that the rescue block in Scraper#fetch (lib/faexport/scraper.rb ~1316) does not recognise. The recognised cases are:

• 403 with an #challenge-error-title element → fa_cloudflare
• 503 + "requesting web pages too fast" page → fa_slowdown
• 400 + "user cannot be found" → fa_no_user
• 502/520 → retried 5x, then re-raised

Everything else falls through to unknown_http.

A few things this rules out:

• Expired login cookie: FA serves login walls / guest pages with HTTP 200, which check_errors() reports as fa_login / fa_guest_access / fa_style — never unknown_http. Also home.json (guest-accessible) fails identically.
• UA block: FA serves a bare FAExport user-agent fine from a residential IP (200), so it is not the UA on its own.

So the picture that fits: FA's Cloudflare front is answering this instance specifically with a non-200 in a format the detection does not match. The #challenge-error-title check is older challenge-page markup — modern Cloudflare IP-block pages (1006/1007 "Access denied") and managed challenges use different markup, and bot management can also answer 429, none of which match a branch. The actual status should be visible in your logs / the $http_errors counter.

If that is right, the fix is probably on the egress-IP side (plus maybe refreshing the Cloudflare block-page detection so these surface as fa_cloudflare instead of unknown_http).

[bshahin101]

the "falls through as unknown_http" is the tell — Cloudflare's 1xxx error codes come back with non-standard statuses your fa_cloudflare matcher isn't keyed on. quick map: 1006/1007/1008 = IP banned, 1015 = rate-limited (back off + rotate, not something you solve), 1010 = browser-integrity/automation flagged, 1020 = firewall rule. these are body-embedded ("Error 1015") with a 403/503 and usually NO cf-mitigated header — different from a managed challenge, which sets cf-mitigated: challenge plus a turnstile/JS interstitial. so two detector buckets: BLOCKED_IP/RATE (1006/1015 → rotate IP / slow down, no token) vs BLOCKED_CHALLENGE (managed/turnstile → real browser or token). disambiguate by matching the Error \d{4} string in the body alongside the cf-ray header.

[Deer-Spangle]

Can't believe I missed this ticket so long, sorry about that! I had received a couple emails about the same, and I had been looking into it since my alerting told me about it on the day. It's been a bit of a pain!

And yes, the scraper is hitting a cloudflare challenge page. Initially it was hitting a text-only error response, but now it's the standard cloudflare challenge page. I've tried changing the IP address of my VPS, and I'm still hitting the same errors. I moved my local services to use a locally hosted version of FAExport, and that's working fine for me, but the website has often been a best-effort kinda thing.

In the distant past, FA used to go under anti-bot protection for a few hours at most, or a few days, but this time it has been quite a long time without ceasing. I used to do some cloudflare bypass magic too, but I've given up on that too. It's tricky!

I may file a trouble ticket with FA and see if they're kind enough to let the public service continue. It does offer caching and things which mean it should hit FA less hard than most things, but it's still very much a scraping system.
I hoped that the ban might mean they were about to publish a public API, as they hint towards such, but no luck yet.
I'll file a trouble ticket, see what they say