GST API Credential, Session & Rate-Limit Management

A design reference for the transport, credential and session layer shared by every GST government-API integration

1. Requirements

1.1 Functional requirements

• A legal entity can hold several registrations (GSTINs), and each registration needs independent login material for two distinct government-facing service families: an e-Way Bill / e-Invoice family and a Returns filing family (GSTR-1, GSTR-3B, GSTR-2A/2B, IMS). A registration’s credential for one family must never be confused with its credential for the other.
• Every outbound call to either family — and a third, credential-less public-lookup family used for GSTIN/PAN verification — must go through one shared transport layer, so header injection, encryption, response classification and logging are implemented once, not once per family.
• A successful authentication handshake produces a short-lived session (an auth token plus a symmetric session key) that must be cached and reused across calls rather than re-negotiated on every request.
• Every call attempt, successful or not, must be recorded with its request and response, so failures are diagnosable without needing the government portal’s own logs.
• Recorded payloads must have credentials, tokens and session keys masked before persistence, regardless of which family produced them.
• Operators must be able to see call volume broken down by endpoint, by date, or by source document, without reading raw logs.
• A registration’s public status (active, cancelled, suspended, blocked) must be cached and periodically refreshed, for any registration ever looked up — a counterparty’s as well as the company’s own — not only registrations the company files under.
• A sandbox/test mode must redirect calls to a test path and, for one family, substitute fixed test login material so no real registration is needed to exercise the integration.

1.2 Non-functional requirements

• Secrecy in stored logs: passwords, session keys, auth tokens and the shared transport-level key must never appear in plaintext inside a persisted call record.
• Bounded blast radius for a stale session: an expired or rejected session must be detected immediately and trigger a defined recovery path, not an indefinite retry loop.
• Distinguishable failure classes: an outage, an exhausted call allowance, an invalid transport key and an expired session must surface as different conditions, not one generic error.
• Safety around non-idempotent operations: calls that generate a filing identifier cannot be safely re-issued blindly. Any retry built on this layer must be paired with a way to recognize “this was already done.”

1.3 Constraints

• One configuration singleton and one credential table hold the login material for every registration and every service family — no separate administration surface per family.
• Header and secret injection happens in exactly one shared base class; a family-specific class cannot bypass masking, because it never constructs the outbound request directly.
• The government portal is treated as an opaque external system with a request/response contract. Its internal behavior and infrastructure are not observable and are not modeled here.

2. High-Level Design

2.1 Component diagram

2.2 Data flow — issuing a call through the shared transport

Two details this diagram simplifies, expanded in §3.4: the “family-specific handshake” branch behaves very differently per family — one recovers unattended, the other cannot — and a third outcome (a “queued, retry later” response) sits between success and failure for some Returns-family calls.

3. Deep Dive

3.1 Data model

Legal Entity (reused) — the company that owns a registration and its credentials. Every credential row is scoped to exactly one legal entity.

GST API Credential — the stored, registration-scoped login record. One row exists per (legal entity, registration number, service family):

API Session — not a separate stored record, but the live state carried by the last four fields above once a handshake succeeds. A credential row with no current session simply has those fields empty or stale.

GST API Client — the shared transport base every family builds on: one code path for resolving a credential, attaching the shared key and family headers, executing the call, classifying the result, masking, and logging. Each family subclasses it only to add its own handshake and payload encryption.

External Call Log (reused from the shared integration-logging pattern) — one entry per call attempt, with request, response or error, and a link back to the triggering document. Written asynchronously, so a slow log write never lengthens the call’s own response time. It is the same generic mechanism used for other external integrations, not a GST-specific table.

API Usage Ledger — not a counter, but a retrospective aggregation over the External Call Log, grouped by endpoint, date, or source document, surfaced to operators as the API usage report. It answers “how much have we called,” never “are we about to be limited” — nothing consults it before making a call.

Registration Status Record — a cached snapshot of a registration’s public status, its registration/cancellation dates, and, for the Returns family, how recently a filing was made against it. Refreshed from the public-lookup family on a configurable interval rather than on every transaction, and populated for any registration number the system encounters, not only the filer’s own.

Module Configuration Singleton — one record per installation: whether the integration is enabled, the sandbox toggle, the shared transport-level key (or a site-level fallback), the credential rows, and cached copies of the government portal’s own public key/certificate used in the e-Way Bill / e-Invoice handshake.

3.2 API contract (illustrative)

client.request_for(service_family, registration) -> family-specific interface

interface.get(...) / .post(...) / .put(...)
  -> resolves/refreshes the API Session as needed
  -> encrypts payload if the family requires it
  -> returns decrypted result, or an ignorable structured outcome
     ("no documents found", "already generated", "queued, retry after N minutes"),
     or raises a classified error: outage, allowance exhausted,
     invalid transport key, or invalid/expired session

The three families differ only in their handshake and header shape (§3.4), not in this contract shape — which is what lets one transport base serve all three.

3.3 Session lifecycle and error handling

• Mid-operation expiry, e-Way Bill / e-Invoice family: the client recognizes a specific “invalid token” response code, clears the cached token, redoes the username/password handshake, and retries the same request once — fully automatic.
• Mid-operation expiry, Returns family: validity is checked before the call. A missing/expired session can only be repaired by requesting a fresh OTP. Called interactively, that surfaces the OTP prompt to the operator. Called from a background job, the attempt fails outright and the stale token is explicitly cleared once the job ends, so the next attempt starts clean rather than repeating against a token already known dead. A background job that outlives its Returns session cannot self-heal.
• Egress-IP binding: the Returns family binds a session to the outbound IP captured when the OTP was verified, and sends that IP on every later call. A changed outbound IP between authentication and use — a different worker, host, or NAT path — can be rejected by the portal on IP mismatch. That specific rejection is a recognized error code in the codebase but is not wired into the “ignorable and recoverable” set, so today it surfaces as an ordinary hard failure rather than an automatic re-authentication.
• Rate-limit / allowance discovery: there is no client-side counter or throttle anywhere in this layer. Limits are discovered only when the portal, or the shared transport-level provider, rejects a call — a specific HTTP status for allowance exhaustion, plus in-band error-code substrings for a broader server/account-limit condition. Discovery is entirely server-rejection-driven, after the fact. A separate, distinct outcome — “queued,” with a server-suggested wait in minutes — applies to some return-data fetches and is handled by scheduling a later retry at that delay, not by treating it as success or failure.
• Retry safety for non-idempotent calls: the transport itself has no blanket retry-on-failure — a failed call logs and raises. The one automatic retry (invalid-token-then-retry-once, above) re-sends a generation request that may have already succeeded on a prior attempt whose response was lost. This is made safe not by an idempotency key but by business-layer duplicate detection: a “duplicate” response for the same generation request is treated as a recoverable outcome, with its already-issued result extracted, rather than as failure. A separate scheduled sweep re-attempts generation calls that failed for transient connectivity reasons specifically — not arbitrary failures — gated by a per-document “retry pending” flag, and relies on that same duplicate-detection safety net rather than any request-level idempotency token.

4. Scale and Reliability

• Load pattern: bursty around invoicing activity and filing windows rather than steady-state; the call-log write is enqueued rather than synchronous, so logging pressure never adds latency to the call itself.
• No proactive rate governance: because allowance exhaustion is discovered only through a rejected call, this layer cannot slow down in advance of a filing-window crunch — it reacts after the government side has already said no. The usage ledger is descriptive, not a gate.
• Session state under concurrency: session fields live on the same row as the long-lived secret they were derived from, keyed by (registration, service family) — not per worker or process. Two processes racing to refresh the same registration’s session both write to that one row; whichever handshake completes last wins, silently overwriting the other’s freshly negotiated (and possibly still valid) session key. The egress-IP binding sharpens this into an operational constraint: horizontally-scaled or multi-egress deployments risk one worker’s session being unusable from another worker’s IP.
• Coarse failover: sandbox/fallback mode is a whole-family switch that reroutes every call in the e-Way Bill / e-Invoice family through an intermediary that manages its own handshake with the government portal — an administrative toggle, not an automatic per-call fallback.
• Monitoring: the External Call Log’s failure entries are the primary signal — a rising failure rate for one registration/service pair distinguishes a credential problem from a portal outage. A rising count of documents parked in the transient-retry state signals the retry sweep is not keeping up, since that state is set only for connectivity-class failures.

5. Trade-off Analysis

6. What to Revisit as the System Grows

• Add a lightweight proactive throttle ahead of the reactive rejection path, informed by the usage ledger’s own historical counts, so bursts around filing deadlines are smoothed before the portal has to reject anything.
• Separate volatile session state from the long-lived credential row, or add row-level locking around session refresh, once concurrent workers routinely authenticate the same registration.
• Give the Returns family an unattended recovery path, or at least a clear operational alert, for a session expiring inside a background job — today that path fails and clears the token with no automatic next step.
• Wire up the egress-IP-mismatch failure as a recognized, recoverable condition, once deployments spread calls across multiple egress points rather than a single fixed outbound address.
• Extend transient-error-triggered retry beyond generation calls, and consider a genuine request-level idempotency key rather than relying solely on each family’s own duplicate-response codes, as new endpoints are added whose “already done” signal isn’t yet mapped.