Many LLM providers have a rate limit, specifically RPM. Add an interval between requests can meet the rate limit.
I've added it successfully with the help of AI. I hope this can be added into new versions.
1. src/config.py
A. Add to _RELOADABLE_ENV_SETTINGS (near the other performance settings, e.g. after PARALLEL_TRANSLATIONS):
('PARALLEL_TRANSLATIONS', 'PARALLEL_TRANSLATIONS', '1'),
('REQUEST_INTERVAL', 'REQUEST_INTERVAL', '0'), # <-- add this
('MAX_TOKENS_PER_CHUNK', 'MAX_TOKENS_PER_CHUNK', '450'),B. Treat it as a float in _apply_reloadable_env_settings.
Find:
_INT_ATTRS = {'PARALLEL_TRANSLATIONS', 'MAX_TOKENS_PER_CHUNK'}Change to something like:
_INT_ATTRS = {'PARALLEL_TRANSLATIONS', 'MAX_TOKENS_PER_CHUNK'}
_FLOAT_ATTRS = {'REQUEST_INTERVAL'}Then inside the loop that converts values:
if attr in _NOTIFY_BOOL_ATTRS:
g[attr] = str(raw).strip().lower() == 'true'
elif attr in _NOTIFY_INT_ATTRS or attr in _INT_ATTRS:
try:
g[attr] = int(raw)
except (TypeError, ValueError):
g[attr] = int(default)
elif attr in _FLOAT_ATTRS:
try:
g[attr] = max(0.0, float(raw))
except (TypeError, ValueError):
g[attr] = float(default)
else:
g[attr] = raw
2. .env.example
Add near the performance section:
# Minimum delay (seconds) between successful LLM requests.
# 0 = no delay. Useful to stay under provider rate limits.
REQUEST_INTERVAL=0
3. src/api/blueprints/config_routes.py
A. Return the value in GET /api/config (around the parallel_translations lines):
"parallel_translations": int(_config.PARALLEL_TRANSLATIONS),
"max_parallel_translations": int(_config.MAX_PARALLEL_TRANSLATIONS),
"request_interval": float(getattr(_config, "REQUEST_INTERVAL", 0) or 0), # <-- addB. Allow saving it – add to allowed_keys in save_settings():
'PARALLEL_TRANSLATIONS',
'REQUEST_INTERVAL', # <-- add
'DISABLE_AUTO_PAUSE',C. Validate / clamp (in the same block that clamps PARALLEL_TRANSLATIONS):
elif key == 'REQUEST_INTERVAL':
try:
n = float(safe_value)
except (TypeError, ValueError):
continue
safe_value = str(max(0.0, n))
4. Web UI – HTML
In src/web/templates/translation_interface.html, right after the Parallel requests block (#parallelWorkersGroup), add:
<!-- Request interval (global minimum delay between LLM calls) -->
<div class="form-group" style="margin-bottom: 15px;">
<label data-i18n="settings:request_interval_label" for="requestInterval">Request interval (seconds)</label>
<div class="neu-inset-light">
<input type="number" class="form-control" id="requestInterval"
min="0" step="0.5" value="0"
data-i18n-attr="title:settings:request_interval_input_title"
title="Minimum delay between successful LLM requests. 0 = no delay.">
</div>
<small data-i18n="settings:request_interval_help"
style="color: var(--text-muted-light); font-size: 0.6875rem; margin-top: 0.5rem; display: block;">
Hard minimum delay after every successful LLM response before the next request is sent.
Useful to stay under provider rate limits. 0 disables the delay.
</small>
</div>
5. src/web/static/js/core/settings-manager.js
A. Mark the field as dirty (in envDirtyElements):
{ id: 'parallelWorkers', event: 'input' },
{ id: 'maxTokensPerChunk', event: 'input' },
{ id: 'requestInterval', event: 'input' }, // <-- addB. Save it (inside saveAllSettings, next to the parallel workers block):
// Save request interval (minimum delay between successful LLM calls)
const requestIntervalInput = DomHelpers.getElement('requestInterval');
if (requestIntervalInput) {
const ri = parseFloat(requestIntervalInput.value);
envSettings['REQUEST_INTERVAL'] = String(
Number.isFinite(ri) && ri >= 0 ? ri : 0
);
}
6. src/web/static/js/ui/form-manager.js
In loadDefaultConfig(), after the parallel workers seeding:
// Request interval (global minimum delay between LLM requests)
if (config.request_interval !== undefined && config.request_interval !== null) {
const requestIntervalInput = DomHelpers.getElement('requestInterval');
if (requestIntervalInput) {
requestIntervalInput.value = String(config.request_interval);
}
}
7. Enforce the delay src/core/llm_client.py.
A. Add the import at the top of the file
import asyncio
import src.config as cfg
(Keep any existing imports.)
B. Replace the two methods
async def generate(self, prompt: str, system_prompt: Optional[str] = None,
timeout: int = None) -> Optional[LLMResponse]:
"""
Generate a response from the LLM (alias for make_request for backward compatibility)
Args:
prompt: The user prompt to send
system_prompt: Optional system prompt (role/instructions)
timeout: Request timeout in seconds
Returns:
LLMResponse with content and token usage info, or None if failed
"""
provider = self._get_provider()
if timeout:
response = await provider.generate(prompt, timeout, system_prompt=system_prompt)
else:
response = await provider.generate(prompt, system_prompt=system_prompt)
# Hard minimum interval between successful LLM requests
if response is not None:
interval = float(getattr(cfg, "REQUEST_INTERVAL", 0) or 0)
if interval > 0:
await asyncio.sleep(interval)
return response
async def make_request(self, prompt: str, model: Optional[str] = None,
timeout: int = None, system_prompt: Optional[str] = None) -> Optional[LLMResponse]:
"""
Make a request to the LLM API with error handling and retries
Args:
prompt: The user prompt to send (content to process)
model: Model to use (defaults to instance model)
timeout: Request timeout in seconds
system_prompt: Optional system prompt (role/instructions)
Returns:
LLMResponse with content and token usage info, or None if failed
"""
provider = self._get_provider()
# Update model if specified
if model:
provider.model = model
if timeout:
response = await provider.generate(prompt, timeout, system_prompt=system_prompt)
else:
response = await provider.generate(prompt, system_prompt=system_prompt)
# Hard minimum interval between successful LLM requests
if response is not None:
interval = float(getattr(cfg, "REQUEST_INTERVAL", 0) or 0)
if interval > 0:
await asyncio.sleep(interval)
return response
Many LLM providers have a rate limit, specifically RPM. Add an interval between requests can meet the rate limit.
I've added it successfully with the help of AI. I hope this can be added into new versions.
1.
src/config.pyA. Add to
_RELOADABLE_ENV_SETTINGS(near the other performance settings, e.g. afterPARALLEL_TRANSLATIONS):B. Treat it as a float in
_apply_reloadable_env_settings.Find:
Change to something like:
Then inside the loop that converts values:
2.
.env.exampleAdd near the performance section:
3.
src/api/blueprints/config_routes.pyA. Return the value in
GET /api/config(around the parallel_translations lines):B. Allow saving it – add to
allowed_keysinsave_settings():C. Validate / clamp (in the same block that clamps
PARALLEL_TRANSLATIONS):4. Web UI – HTML
In
src/web/templates/translation_interface.html, right after the Parallel requests block (#parallelWorkersGroup), add:5.
src/web/static/js/core/settings-manager.jsA. Mark the field as dirty (in
envDirtyElements):B. Save it (inside
saveAllSettings, next to the parallel workers block):6.
src/web/static/js/ui/form-manager.jsIn
loadDefaultConfig(), after the parallel workers seeding:7. Enforce the delay
src/core/llm_client.py.A. Add the import at the top of the file
(Keep any existing imports.)
B. Replace the two methods