New limits in nuget.org's Search API

[jcjiang]

Discussion thread

Context

We will introduce new limits on nuget.org's Search API on July 13th, 2021. We will conduct a 24 hour dry run on June 29, 2021 to help you identify systems that may be affected.

If you need help with these new Search API limits, do reach out to us at support@nuget.org or by commenting on the discussion issue: NuGet/Home#10945

Affected endpoints

The following nuget.org endpoints will be impacted:

Endpoint	Affected parameter	Old limit	New limit	Example URL
Search packages	page	5001	1,501	https://www.nuget.org/packages?page=2
Search V3 API	skip	100,000	3,000	https://azuresearch-usnc.nuget.org/query?skip=10
Search V2 API	$skip	100,000	30,000	https://www.nuget.org/api/v2/Search()?$skip=10
Search V1 API	$skip	100,000	30,000	https://www.nuget.org/v1/FeedService.svc/Search()?$skip=10
Find packages by ID V2 API	$skip	100,000	30,000	https://www.nuget.org/api/v2/FindPackagesById()?id=%27newtonsoft.json%27&$skip=10
Find packages by ID V1 API	$skip	100,000	30,000	https://www.nuget.org/api/v1/FindPackagesById()?id=%27newtonsoft.json%27&$skip=10

Workarounds

Here are some workarounds for common scenarios. If these workarounds don't cover your needs, please let us know and we'd be happy to help you on the discussion issue:

Query for all packages

If you need to enumerate all packages published to nuget.org, please follow this guide: https://docs.microsoft.com/en-us/nuget/guides/api/query-for-all-published-packages

Find a specific package's downloads

A common use case for the Search API is to lookup a package's downloads. You can do so using nuget.org's search syntax and the NuGet client SDK:

ILogger logger = NullLogger.Instance; 
CancellationToken cancellationToken = CancellationToken.None; 

SourceRepository repository = Repository.Factory.GetCoreV3("https://api.nuget.org/v3/index.json"); 
PackageSearchResource resource = await repository.GetResourceAsync<PackageSearchResource>(); 
SearchFilter searchFilter = new SearchFilter(includePrerelease: true); 

IEnumerable<IPackageSearchMetadata> results = await resource.SearchAsync( 
    "packageid:Newtonsoft.Json",
    searchFilter,
    skip: 0,
    take: 20,
    logger,
    cancellationToken); 

foreach (IPackageSearchMetadata result in results) 
{
    Console.WriteLine($"Package {result.Identity.Id} has {result.DownloadCount} downloads");
}