Rate Limits are being hit, Bot is being blocked, IP Banned about dsharpplus

[OoLunar]

That the library would "queue" messages and not send them if they were to hit the rate limit.

This is what the library should be doing at the moment. In your testing, does it not?

[ecrocombe]

I was under the impression that "Error" type messages:

[2024-02-13 17:28:16 +11:00] [114 /RatelimitHit] [Error] Ratelimit hit, requeueing request to https://discord.com/api/v10/channels/**************53536/messages

Is actually hitting the limits, as apposed to the "Warning" messages that simply say they will be queuing future message?
Please excuse my naiveness if this is not the case

[OoLunar]

The error message means your last request hit the ratelimit and the current request will be delayed until the ratelimit resets

[ecrocombe]

So once "Message A" hits the limit and gets queued, what happens to the next "Message B" that gets sent on a different thread?
Does:

1. "Message B" wait for "Message A" before trying? honouring "Message A"'s "X-Retry-After"
2. "Message B" try to send itself, then honour its own "X-Retry-After", completely ignoring the fact "Message A" is queued?

If its number2, then I think that is where I'm going wrong.
I'm setting up some logging as we speak to hopefully get some more insights into this.

[akiraveliara]

so... ratelimits are vastly different depending on which version you use, v4.x uses a different implementation to v5.x nightly, which uses a different implementation to v6.x not-even-alphas.

first off, since i can see a sharded client there, please make sure you're on the latest version of either stable or nightly builds - we caught a ratelimiting-related bug in the sharded client not too long ago, and i recently fixed another scenario where we would queue up too many requests in nightly.

the second important thing is that there are 429s we cannot prevent, since we can only act upon receiving ratelimit information from discord, and if a 429 happens before that, that is a problem we can't solve.

the third premise here is that ratelimiting absolutely does not support multiprocess sharding, if that's something you want to do - there is currently no way to transmit ratelimits between processes.

if you want to understand how it works, i would need to know what version you're using - if you want me to look into it deeper, i would need a trace log (redacting tokens of course, but all other information intact) and lastly i would recommend you batch multiple embeds into one message, which reduces ratelimit strain and would also deal with the possibility of discord considering this API abuse, at least to some extent.

[ecrocombe]

I would love any and all help on this, the bot has "premium" features which customers are paying for.

<PackageReference Include="DSharpPlus" Version="4.4.6" />
<PackageReference Include="DSharpPlus.Interactivity" Version="4.4.6" />
<PackageReference Include="DSharpPlus.SlashCommands" Version="4.4.6" />

The bot is using in-process sharding, only 1 instance of the application is running at any given time.
I am however using the 1x DiscordSharedClient (Singleton) across multiple threads in the same application. I hope this answers your multi-process question?

Trace?? I'll see what i can do, Much appreciated your input!

[ecrocombe]

How should i get this trace log to you @akiraveliara ? its not something I would want to post here, i hope you understand. my discord handle is "private_wire#9001" if you would prefer?

[akiraveliara]

my DMs on discord are open, handle is @akiraveliara

[ecrocombe]

Hi team, thanks for your quick response, time and effort in producing a fast update.

I have migrated from 4.4.6 to5.0.0-nightly-02069.
I have also disable one of the more spammy feeds. (hopefully temporarily)

The ban has been lifted in the last 30min, and with this most recent patch, slash commands are no longer working, although they did for a short while... maybe 5min¿

I have also noticed a pattern with the previous bans, they appear to be occurring after connected for pretty much spot on 1 hour, and the ban lasts for pretty much spot on 1 hour.

[OoLunar]

The ban has been lifted in the last 30min, and with this most recent patch, slash commands are no longer working, although they did for a short while... maybe 5min¿

Which command framework are you using? DSharpPlus.SlashCommands or DSharpPlus.Commands?

[ecrocombe]

Which command framework are you using? DSharpPlus.SlashCommands or DSharpPlus.Commands?

<PackageReference Include="DSharpPlus" Version="5.0.0-nightly-02069" />
<PackageReference Include="DSharpPlus.Interactivity" Version="5.0.0-nightly-02069" />
<PackageReference Include="DSharpPlus.SlashCommands" Version="5.0.0-nightly-02069" />

This appears as a recurring log entry:
[WRN] Slash commands failed to register properly on startup. (1209a574)
System.InvalidOperationException: Slash commands failed to register properly on startup.
at DSharpPlus.SlashCommands.SlashCommandsExtension.<>c__DisplayClass47_0.<b__0>d.MoveNext()

Startup/Program

HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);
builder.Services.AddHostedService<BotWorker>();
// Discord
var discordConfig = new DiscordConfiguration()
{
 Intents = DiscordIntents.AllUnprivileged | DiscordIntents.GuildMembers,
 Token = builder.Configuration.GetValue<string>("DISCORD_BOT_TOKEN") ?? throw new Exception("DISCORD_BOT_TOKEN"),
 TokenType = TokenType.Bot,
 AlwaysCacheMembers = true,
 AutoReconnect = true,
 GatewayCompressionLevel = GatewayCompressionLevel.Stream,
 LargeThreshold = 250,
 MinimumLogLevel = LogLevel.Trace,
 LogUnknownEvents = false       
};
builder.Services.AddSingleton(discordConfig);
var discordShardedClient = new DiscordShardedClient(discordConfig);
builder.Services.AddSingleton<DiscordShardedClient>(discordShardedClient);
using (IHost host = builder.Build())
{
 // Run Host Applications.
 await host.RunAsync();
}

BotWorker Hosted Service

    public class BotWorker : IHostedService
 {
     private readonly IServiceProvider _provider;
     private readonly DiscordShardedClient _discordClient;

     public BotWorker(IServiceProvider provider,
                    DiscordShardedClient discordClient
                    )
     {
         _provider = provider ?? throw new ArgumentNullException(nameof(provider));
         _discordClient = discordClient ?? throw new ArgumentNullException(nameof(discordClient));
         _discordClient.SessionCreated += _discordClient_SessionCreated;  //_discordClient_Ready;
     }

     private async Task _discordClient_SessionCreated(DiscordClient sender, SessionReadyEventArgs args)
     {
         if (sender.ShardId == 0)
             await sender.UpdateStatusAsync(new DSharpPlus.Entities.DiscordActivity(_stringLocalizer[Resources.BOT_ACTIVITY], DSharpPlus.Entities.ActivityType.Playing), DSharpPlus.Entities.UserStatus.Online);
     }
     public async Task StartAsync(CancellationToken cancellationToken)
     {
         var slash = await _discordClient.UseSlashCommandsAsync(new SlashCommandsConfiguration() { Services = _provider });
         var inter = await _discordClient.UseInteractivityAsync(new DSharpPlus.Interactivity.InteractivityConfiguration()
         {
             PollBehaviour = PollBehaviour.DeleteEmojis,
             Timeout = TimeSpan.FromSeconds(30)
         });
         slash.RegisterCommands<CategoriesModule>();
         slash.RegisterCommands<ChatModule>();
         slash.RegisterCommands<CommandsModule>();
         slash.RegisterCommands<EconomyModule>();
         slash.RegisterCommands<FeedsModule>();
         slash.RegisterCommands<GameserverModule>();
         slash.RegisterCommands<PlayersModule>();
         slash.RegisterCommands<PremiumModule>();
         slash.RegisterCommands<ProductsModule>();
         slash.RegisterCommands<TimedCommandsModule>();
         slash.RegisterCommands<ReportModule>();

         slash.RegisterCommands<Bal>();
         slash.RegisterCommands<Buy>();
         slash.RegisterCommands<Dep>();
         slash.RegisterCommands<InstallKit>();
         slash.RegisterCommands<Link>();
         slash.RegisterCommands<Mute>();
         slash.RegisterCommands<Pay>();
         slash.RegisterCommands<Run>();
         slash.RegisterCommands<Spin>();
         slash.RegisterCommands<Stats>();
         slash.RegisterCommands<With>();


         foreach (var shard in slash)
         {
             shard.Value.SlashCommandErrored += Slash_SlashCommandErrored;
             shard.Value.AutocompleteErrored += Value_AutocompleteErrored;
         }

         await _discordClient.StartAsync();

     }