Repository navigation
How is the Resource Unit normalization for CSOM requests calculated to determine when app should be throttled? #10213
Description
What type of issue is this?
Question
What SharePoint development model, framework, SDK or API is this about?
SharePoint CSOM
Target SharePoint environment
SharePoint Online
What browser(s) / client(s) have you tested
- 💥 Internet Explorer
- 💥 Microsoft Edge
- 💥 Google Chrome
- 💥 FireFox
- 💥 Safari
- mobile (iOS/iPadOS)
- mobile (Android)
- not applicable
- other (enter in the "Additional environment details" area below)
Additional environment details
SharePoint online CSOM / REST API
Issue description
In chapter Application Throttling there is the following text:
SharePoint provides various APIs. Different APIs have different costs depending on the complexity of the API. The cost of APIs is normalized by SharePoint and expressed by resource units. Application’s limits are also defined using resource units.
We want to rate limit on the client side to avoid throttling by the server. Therefore we need to know how much each request will cost or has cost.
In particular we want to know Resource Unit of the following CSOM queries/actions:
- Context.Load(siteUsers, us => us.Include(u => u.Title, u => u.Id, u => u.LoginName, u => u.PrincipalType, u => u.Email, u => u.UserPrincipalName, u => u.AadObjectId));
- var refUserAsPrincipal = rootWeb.EnsureUser(logOnName);
- SecurableObjectSharePoint.Context.Load(refUserAsPrincipal, _user => _user.Email, _user => _user.LoginName);
- SecurableObjectSharePoint.ResetRoleInheritance();
- SecurableObjectSharePoint.BreakRoleInheritance(true, true);
- SecurableObjectSharePoint.Context.Load(SecurableObjectSharePoint, s => s.HasUniqueRoleAssignments);
- _spfile = ClientContext.Web.GetFileByUrl(AbsoluteUrl);
- ClientContext.Load(SPFile, f => f.TimeLastModified, f => f.Exists);
- ClientContext.Load(SPFile, f => f.Exists);
- ClientContext.Load(SPFile, f => f.Exists, f => f.TimeLastModified, f => f.MajorVersion, f => f.MinorVersion, f => f.Length, f => f.ListItemAllFields.Id, f => f.ListItemAllFields.ParentList.ForceCheckout, f => f.CheckOutType, f => f.CheckedOutByUser.UserPrincipalName, f => f.LockedByUser.UserPrincipalName, f => f.ListItemAllFields.HasUniqueRoleAssignments);
- ClientContext.Load(SPFile, f => f.ListItemAllFields.RoleAssignments.Include(rs => rs.Member, rs => rs.RoleDefinitionBindings.Include(rdb => rdb.RoleTypeKind, rdb => rdb.Name)));
- ClientContext.Load(member, u => u.AadObjectId);
- ClientContext.Load(SPFile, f => f.Exists, f => f.ListItemAllFields.ParentList.Fields.Include(fld => fld.InternalName, fld => fld.FieldTypeKind), f => f.ListItemAllFields);
- ClientContext.Load(web, s => s.ServerRelativeUrl);
- ClientContext.Load(checkedoutedFile, f => f.Where(fi => fi.ServerRelativePath.DecodedUrl == fileRelativeUrl).Include(fi => fi.CheckedOutBy.Email));
- SPFile.CheckIn(comment, (CheckinType)checkinType);
- SPFile.CheckOut();
- SPFile.UndoCheckOut();
- SPFile.DeleteObject();
- var mstream = SPFile.OpenBinaryStream();
- ClientContext.Load(SPFile, r => r.ListItemAllFields.HasUniqueRoleAssignments);
- ClientContext.Load(SPFile, r => r.ListItemAllFields.ParentList.ParentWeb.RoleDefinitions.Where(rd => !(rd.RoleTypeKind == RoleType.Guest || rd.RoleTypeKind == RoleType.RestrictedGuest)).Include(rd => rd.Name, rd => rd.BasePermissions, rd => rd.RoleTypeKind));
- ClientContext.Load(SPFile, r => r.ListItemAllFields.RoleAssignments.Include(ra => ra.Member.LoginName, ra => ra.Member.PrincipalType, ra => ra.Member.Id, ra => ra.RoleDefinitionBindings.Include(rdb => rdb.Id, rdb => rdb.RoleTypeKind, rdb => rdb.Name)));
- ClientContext.Load(SPFile, _file => _file.Exists, _file => _file.ListItemAllFields.ParentList.Fields.Include(_field => _field.InternalName, _field => _field.ReadOnlyField, _field => _field.FieldTypeKind), _file => _file.ListItemAllFields.ContentType.Name);
- var uploadFile = folder.Files.Add(newfile); ClientContext.Load(uploadFile);
- bytesUploaded = uploadFile.StartUpload(uploadId, startStream);
- uploadFile = uploadFile.FinishUpload(uploadId, fileoffset, finishStream);
- bytesUploaded = uploadFile.ContinueUpload(uploadId, fileoffset, continueStream);
- uploadFile.CancelUpload(uploadId);
- site.Context.Load(oWeb, w => w.Title, w => w.Language, w => w.Url, w => w.Navigation.UseShared, w => w.Description, w => w.AllProperties, w => w.HasUniqueRoleAssignments, w => w.ServerRelativeUrl, w => w.Id);
- site.Context.Load(oWeb, w => w.RoleAssignments.Include(ra => ra.Member.LoginName, ra => ra.Member.Id, ra => ra.RoleDefinitionBindings.Include(rdb => rdb.Id, rdb => rdb.RoleTypeKind, rdb => rdb.Name)));
- site.Context.Load(oWeb, r => r.RoleDefinitions.Where(rd => !(rd.RoleTypeKind == RoleType.Guest || rd.RoleTypeKind == RoleType.RestrictedGuest)).Include(rd => rd.Id, rd => rd.Name, rd => rd.BasePermissions, rd => rd.RoleTypeKind));
We want also know the points for more complex calls/actions, such as creating or updating lists, contenttypes, lists, site columns.
We now have made estimations what each call will probably costs based on the Graph API call costs table with values 1,2 or 5
If there is a way to return the actual resource unit costs of each action/query in the response headers, that would be awesome. We then can implement rate limiting on the client side with actual values.
The Metered API charges on price per 1000 requests (thus not on points). We therefore cannot use the metered (pay-per-use) API to determine the points per request.
Can you assist us to clarify the Resource Unit costs of the CSOM operations/queries?
Activity
Ashlesha-MSFT commentedon Apr 29, 2025
Hello Robin Bouwmeester (@digiofficerobin),
Thank you for bringing this issue to our attention. We will look into it and get back to you shortly.
Tanddant commentedon Apr 29, 2025
Robin Bouwmeester (@digiofficerobin) - You might find some of your answers in this repo
https://github.com/OneDrive/samples/tree/master/scenarios/throttling-ratelimit-handling
While it doesn't log the "cost" of a request, you could look at your rate-limit remaining, and do the math yourself to some extend.
But it also shows you how to implement Rate-limit handling and have your code "back off" before it becomes a real problem
This is also built into the latest version of PnP Framework: pnp/pnpframework@eec5fb2
digiofficerobin commentedon Apr 30, 2025
Dan Toft (@Tanddant)
While it doesn't log the "cost" of a request, you could look at your rate-limit remaining, and do the math yourself to some extend.In theory that is a good idea. Unfortunately this is not possible in practice, because rate limit headers are most of the time not returned. See issue #10069 . And video's https://www.youtube.com/playlist?list=PLnX8DsYQ38gOUhzNBG0UnPHL7NqKBobX1 .
And if they return, then I need to use multiple threads to create enough requests which makes it difficult to calculate the consumed quota. It would have been better if the consumed points are returned with every response.
By the way load testing is not allowed
You're running a load-testing script on SharePoint Online and you get throttled. Load testing isn't allowed on SharePoint Online.according to this article: https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#common-throttling-scenarios-in-sharepoint-onlineBut it also shows you how to implement Rate-limit handling and have your code "back off" before it becomes a real problemWe have implemented this, but it won't work if no Rate-Limit headers are returned. And even if we get the response headers and wait the RateLimit-Reset time in seconds, then the quota isn't reset. See issue #10070
Tanddant commentedon May 1, 2025
Robin Bouwmeester (@digiofficerobin)
I agree, the whole system feels "half baked" - but I'll say with some minor exception handling, and the latest version of PnP Framework, I managed to do 400k "heavy" CSOM requests (Versions and Permissions primarily) against a "pretty bad tenant", that's constantly being throttled due to another vendor not knowing what they're doing - without my code ever breaking.
I did see some of the other random errors that sometimes pops up, that isn't strictly speaking throttling, such as PnP Framework timeouts - but with the following setup I managed to get through
SocketExceptionErrorCodewas503504429MessagewasAn existing connection was forcibly closed by the remote host.TaskCanceledExceptionwhereMessageisA task was canceled.and theStackTracecontainedPnP.Framework(when PnP Framework times out)ServerExceptionServerErrorTypeNamewasMicrosoft.Office.Server.Search.SubstrateSearch.QueryTimeoutExceptionMicrosoft.Office.Server.Search.Query.InternalQueryErrorExceptionMicrosoft.SharePoint.SPQueryThrottledExceptionMessagewasTimeoutWebExceptionStatusCodewas503504429I simply wrote a generic wrapper for my CSOM request, and then catch those types of exceptions, and then retry - and while it's not perfect, it actually worked wonders, and my code just kept crunching for almost two days until it completed successfully
digiofficerobin commentedon May 1, 2025
Dan Toft (@Tanddant) Thanks for your solution. We have similar exception handling and retry logic for these types of errors.
Our app includes both background services and end-user (GUI) services, which share the same app registration and service account (ROPC flow). When the background application is throttled for a few minutes, all end-users experience delays, timeouts, hangs, exceptions, etc. This is a significant issue.
To address this, we opted for a softer solution that allows us to smoothly delay requests from the client side during high demand periods, preventing us from exhausting our quota too quickly. We have implemented this solution, but we need to understand the costs of each request.
As a surrogate for real costs, we have estimated the costs as follows:
We would like to know if these estimates reflect the real costs and how we can better smooth out the delays between requests. It really matters if a request costs 1, 2, or even 5 points.
Additionally, we need to account for the number of requests a service account can make. For years, we have used the ROPC flow (delegated permissions) because it is easy to set up with CSOM and provides fine-grained control over service account access. The ClientCredentials flow, which uses application permissions, offers an all-or-nothing approach on a site with Sites.Selected permission. For CSOM, requests require client certificates (client secrets are not allowed).
Since September 2024, we have experienced increased throttling at our customers using the ROPC flow. The article Avoid getting throttled or blocked in SharePoint Online was updated in March with more details about actual limits, which is helpful but not sufficient.
We suspect that since September 2024, the ROPC flow is now treated as a user and thus subject to 'User Throttling,' whereas previously it was only subject to 'Application Throttling.' If someone can confirm or deny this, it would greatly aid our issue research. Despite searching for release notes and the SharePoint Online roadmap, we could not find an official statement about this change.
The difference between User Throttling and Application Throttling is significant. A user can make 3000 requests (not points) per 5 minutes, which translates to only 600 requests per minute. An application has no request limit but operates with Resource Units (points).
If this change occurred around September 2024 in SharePoint Online, it would explain the issues we have been experiencing with our customers since that time.
Ashlesha-MSFT commentedon May 2, 2025
Robin Bouwmeester (@digiofficerobin),
Your detailed breakdown of CSOM request costs and the challenges around opaque throttling logic highlights a real gap in SharePoint Online’s developer experience.
Since SharePoint currently doesn't expose resource unit consumption per request—making proactive rate limiting difficult—it would be valuable to raise a feature request to Microsoft.
You can submit this through below link
https://feedbackportal.microsoft.com/feedback/forum/06735c62-321c-ec11-b6e7-0022481f8472
digiofficerobin commentedon May 2, 2025
Ashlesha-MSFT Thanks for your response.
I've added the feature request to the provided feedback poral: https://feedbackportal.microsoft.com/feedback/idea/84324290-6d27-f011-9d48-000d3a0d77f1
digiofficerobin commentedon May 15, 2025
Video explaining the importance of this issue: https://youtu.be/alSYB1Su41g
digiofficerobin commentedon Jun 19, 2025
Any updates on this question?
digiofficerobin commentedon Aug 18, 2025
Any updates on this question?
VesaJuvonen commentedon Aug 19, 2025
No updates - the currently owning PM on this feature is on a vacation with delays. We should be able to get a response on this within upcoming weeks. Thanks for pushing for resolution and input on this. Appreciate the push to keep us tracking there 👍
SimonBourdages commentedon Sep 2, 2025
Greetings,
While I appreciate and can understand the want for transparency on those Resources Cost. this is not something that we can expose at this time nor is it the current goal. I simply wanted to acknowledge that we are working on provide transparency where we can and you can see it by having more granular limit exposed in our documentations. Our next step on this journey is working on a plan for you to be able to understand how much Resource utilization you have done to better balance out future requests. There are no time or commitment for now, but this is high on our planning list.
While i understand this might not be the answer you wanted I hope this helps.
VesaJuvonen commentedon Nov 3, 2026
This issue is more than 12 months old and has not had recent activity. We are cleaning up the backlog so we can manage incoming reports more efficiently. If the issue is still relevant, please submit a new issue with updated details and current reproduction steps. We apologize for any inconvenience caused by this updated process, and thank you for being part of the SharePoint developer ecosystem.
github-actions commentedon Nov 3, 2026
Thank you for taking the time to file an issue. We periodically archive older or inactive issues as part of our issue management process, which automatically closes them once they are archived.
If you’d like to understand more about why and how we handle archived (closed) issues, please see Our approach to closed issues.
We appreciate your contribution and if this is still an active issue with the latest SPFx versions, please do resubmit the details. We needed to perform a cleanup, so that we can start with a clean table with a new process. We apologize for the inconvenience this might cause.