[DRAFT] fix(#21): deprecate paper7 repo (Papers With Code API discontinued)#28
Draft
Vigtu wants to merge 1 commit into
Draft
[DRAFT] fix(#21): deprecate paper7 repo (Papers With Code API discontinued)#28Vigtu wants to merge 1 commit into
Vigtu wants to merge 1 commit into
Conversation
…inued Papers With Code's API is gone. Every paperswithcode.com path I probed returns HTTP 302 to huggingface.co/papers/trending — the /api/v1 endpoints, /api/v0, /api/v2, /graphql, the per-paper HTML pages, even /search. The whole site is a CloudFront redirect now. paper7 repo has returned 'Papers With Code decode failure' for every paper since. I researched a replacement (the brainstorming session picked the HF Papers API as the natural successor) and confirmed there is none: - HF Papers API (/api/papers/<id>) returns metadata only: id, authors, publishedAt, summary, title. No githubRepos / repos / code field exists on any paper I checked, including BERT and ReAct. - /api/papers/<id>/repos, /models, /spaces, /datasets all 404. - /api/models?paper=<id> ignores the paper filter (returns trending global models). - The huggingface.co/papers/<id> HTML page rarely renders github.com links — scraping ReAct returned 0 GitHub URLs in the DOM. - Semantic Scholar externalIds (already in paper7) does not include GitHub for any paper I queried. - OpenAlex has no code_links field. The paper-to-code mapping PWC published was a curated dataset, not just an API. Hugging Face acquired the brand but did not republish the curated data. There is no drop-in replacement. This PR: - Deletes src/repo.ts (the PWC client and decoder). - Rewrites src/commands/repo.ts to print a deprecation notice explaining the situation and pointing the user at paper7 get <id> --abstract-only. - Drops the PaperIdentifier argument from the repo CliCommand variant (parser.ts) and makes the CLI id argument optional so existing invocations like paper7 repo 2210.03629 still work — the id is accepted and ignored. - Removes RepositoryDiscoveryClient / RepositoryDiscoveryLive / RepositoryDiscoveryError plumbing from cli.ts, including the formatRepositoryDiscoveryError handler. - Removes the PWC service provision from browse/cache/vault tests that only ever provided an 'unused' instance. - Rewrites tests/refs-repo.test.ts: drops the old PWC integration tests and the PWC error-class assertions; adds two deprecation tests covering paper7 repo (no id) and paper7 repo <id>. Closes p7dotorg#21.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
paper7 repohas been broken since Papers With Code's API went away (the brainstorm session originally planned to replace it with the Hugging Face Papers API). After implementing and probing, I confirmed there is no drop-in replacement for the curated paper-to-code mapping PWC published. This PR deprecates the command with a clear notice instead of substituting a worse data source.The discussion question
Three options were on the table:
github.com/*in HF Paperssummarytext (~10–30% coverage)I went with (1) because (2) is a worse product (silent low coverage looks like bugs) and (3) is a heavier change with mediocre signal. If you'd rather keep the command working with reduced coverage, say so and I'll re-scope to option (2) — happy to add a
paper7 repo --best-effortmode.Evidence the API is gone (all probed live during implementation)
paperswithcode.com/api/v1/*huggingface.co/papers/trendingpaperswithcode.com/api/v0/,v2/,/graphql,/search,/api/v1/datasets/paperswithcode.com/paper/<slug>(HTML page)GET /api/papers/<id>(BERT 1810.04805)id, authors, publishedAt, summary, title. No GitHub fieldGET /api/papers/<id>(ReAct 2210.03629)ai_summary, ai_keywords, upvotes. No GitHub fieldGET /api/papers/<id>/repos,/models,/spaces,/datasets/api/models?paper=<id>huggingface.co/papers/2210.03629forgithub.com/*URLsexternalIds(Transformer 1706.03762)MAG, DBLP, ArXiv, CorpusId— no GitHubWhat this PR changes
src/repo.ts(the PWC client + decoder, 318 lines).src/commands/repo.tsbecomes a one-shot deprecation message — no clients, no Effect environment requirements.paper7 reporuns with or without an arxiv/pubmed/doi id (the id arg is now optional and ignored). Existing script invocations likepaper7 repo 2210.03629still work, they just print the deprecation notice.RepositoryDiscoveryClient/RepositoryDiscoveryLive/RepositoryDiscoveryErrorfromcli.ts, including theformatRepositoryDiscoveryErrorhandler and theEffect.provide(RepositoryDiscoveryLive)frommain.browse.test.ts,cache.test.ts,vault.test.ts(they only ever provided an "unused" stub).tests/refs-repo.test.ts: drops the PWC integration tests and the PWC error-class assertions; adds two deprecation tests.User-facing output
Test plan
npx vitest run→ 165 pass / 0 fail (-3 from baseline: removed 5 PWC integration tests, added 2 deprecation tests).npx tsc --noEmitclean.paper7 repowith and without an id, both producing the deprecation notice + exit 0.CHANGELOG note for next release
paper7 repois now deprecated. The Papers With Code API was discontinued. No drop-in replacement is currently available. Trypaper7 get <id> --abstract-onlyfor paper metadata, or check the abstract for a code link.Closes #21.