Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions .secrets.baseline
Original file line number Diff line number Diff line change
Expand Up @@ -157,21 +157,14 @@
"filename": "packages/cachekit/src/backends/cachekitio.test.ts",
"hashed_secret": "9242986caf890b01f4265ee595e88ee087af4b90",
"is_verified": false,
"line_number": 19
"line_number": 23
},
{
"type": "Secret Keyword",
"filename": "packages/cachekit/src/backends/cachekitio.test.ts",
"hashed_secret": "a62f2225bf70bfaccbc7f1ef2a397836717377de",
"is_verified": false,
"line_number": 45
},
{
"type": "Secret Keyword",
"filename": "packages/cachekit/src/backends/cachekitio.test.ts",
"hashed_secret": "310ca4c3a208b51df7a1d454bab0730b474e0d99",
"is_verified": false,
"line_number": 358
"line_number": 49
}
],
"packages/cachekit/src/intents.test.ts": [
Expand Down
143 changes: 77 additions & 66 deletions packages/cachekit-core-ts/index.js

Large diffs are not rendered by default.

73 changes: 70 additions & 3 deletions packages/cachekit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Production-ready Redis caching for TypeScript/Node.js. Hybrid TypeScript-Rust de

- **Dual-layer caching**: L1 in-memory (~50ns) + pluggable L2 (Redis, CacheKit SaaS, Memcached, local File)
- **Stale-while-revalidate**: Serve stale data while refreshing in background
- **Stampede protection**: Cold-miss single-flight per process (always on) + opt-in cross-process distributed locks
- **Zero-knowledge encryption**: Optional AES-256-GCM client-side encryption
- **Circuit breaker**: Automatic failure isolation with exponential backoff
- **TypeScript-first**: Full type safety with strict mode
Expand Down Expand Up @@ -133,6 +134,50 @@ const cache = createCache({
});
```

## Stampede Protection

A cold cache key hit by N concurrent callers would normally execute the wrapped
function N times and issue N backend reads — on the metered-misses SaaS backend
that is N billed misses for one key.

**In-process single-flight is always on**: concurrent `wrap()` calls for the
same cache key share one in-flight promise, so the herd costs one L2 read, one
compute, and one write. If the flight fails, every waiting caller receives the
same rejection and the next call retries fresh.

**Cross-process locking is opt-in** via `stampede.distributedLock`, for fleets
where many processes can go cold on the same key simultaneously. It mirrors
cachekit-py's flow: acquire the backend lock, double-check L2, compute, write,
release. Contested processes retry the lock on an interval (never polling
`get()` — on metered-misses backends a poll against a still-cold key is itself
a billed miss) and fall through to computing after `lockWaitMs`; the lock is
best-effort mitigation, never a correctness gate, so lock-endpoint failures
degrade to an unlocked compute.

```typescript
const cache = createCache({
backend: { url: 'redis://localhost:6379' }, // Redis and CachekitIO backends support locks
stampede: {
distributedLock: true, // default false
lockTimeoutMs: 30000, // lock lease; size at/above expected recompute time
lockWaitMs: 5000, // max wait for the lock holder before computing anyway
lockPollMs: 100, // lock retry interval while contested
},
});
```

Requires a lock-capable backend: Redis, a `CachekitIOBackendConfig` (the SaaS
lock endpoint is selected automatically), `cachekitioWithLocking()`, or
`cachekitioFull()`. `createCache` throws `ConfigurationError` if
`distributedLock` is requested on a backend without `acquireLock`/`releaseLock`.

There is deliberately no general admission-control cap beyond L1's
`maxConcurrentRefreshes`: on Node's single-threaded event loop concurrent
misses don't compete for threads, single-flight collapses the per-key herd,
and distinct-key miss floods are already bounded by backend timeouts plus the
circuit breaker. A global semaphore would add queueing latency without a
failure mode it prevents.

## Backends

Four backends implement the same `Backend` interface (raw bytes in/out) and plug into `createCache({ backend })` interchangeably:
Expand Down Expand Up @@ -262,13 +307,35 @@ npm install prom-client
```typescript
const cache = createCache({
backend: { url: 'redis://localhost:6379' },
metrics: true, // Enable Prometheus metrics
metrics: true, // or { prefix: 'myapp', defaultLabels: { env: 'prod' }, registry }
});

// Metrics: cachekit_operations_total, cachekit_hits_total, cachekit_misses_total,
// cachekit_errors_total, cachekit_operation_duration_seconds
// Counters: cachekit_operations_total{operation,status}, cachekit_hits_total{layer},
// cachekit_misses_total, cachekit_errors_total{error_type}
// Histogram: cachekit_operation_duration_seconds{operation}
// Gauges: cachekit_l1_entries, cachekit_l1_memory_bytes, cachekit_circuit_breaker_state
```

If `metrics` is enabled but `prom-client` is not installed, the SDK reports the
failure once through the library logger and metrics degrade to no-ops — never
silently. (On Cloudflare Workers, where prom-client cannot run, the `metrics`
option degrades to a no-op the same way.)

Internal error reporting (background refresh, invalidation channel, Redis
connection events) defaults to `console.error`; route it into your own logging
pipeline with `setLogger`:

```typescript
import { setLogger } from '@cachekit-io/cachekit';

setLogger((message, error) => myLogger.warn({ error }, message));
setLogger(null); // restore the default
```

With the CachekitIO backend, the `X-CacheKit-L1-*` telemetry headers are wired
automatically from the cache's live L1/L2 hit and miss counters; pass your own
`metricsProvider` in the backend config to override.

## Cloudflare Workers

The SDK ships a Workers-native entrypoint: `@cachekit-io/cachekit/workers`
Expand Down
79 changes: 78 additions & 1 deletion packages/cachekit/src/backends/cachekitio.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ import { cachekitio } from './cachekitio-factory.js';
import type { Backend, CachekitIOBackendConfig } from './types.js';
import { BackendError, ConfigurationError, TimeoutError } from '../errors.js';

// Deliberately fake credential for fixtures — obviously-fake value so secret
// scanners and reviewers never mistake it for a live key.
const FAKE_API_KEY = 'ck_test_fake-not-a-secret'; // pragma: allowlist secret

// Helper to create a mock Response
function mockResponse(
status: number,
Expand Down Expand Up @@ -355,7 +359,7 @@ describe('CachekitIO Backend', () => {
// This should not throw — it should detect apiKey and use cachekitio()
const cache = createCache({
backend: {
apiKey: 'ck_test_xyz',
apiKey: FAKE_API_KEY,
apiUrl: 'https://api.test.cachekit.io',
allowCustomHost: true,
},
Expand All @@ -369,5 +373,78 @@ describe('CachekitIO Backend', () => {

await cache.close();
});

// LAB-517: the SaaS telemetry headers are auto-wired from the cache's
// live hit/miss counters — no user-supplied metricsProvider needed.
it('auto-wires X-CacheKit-L1-* telemetry headers from live cache counters', async () => {
const { createCache } = await import('../cache.js');

const cache = createCache({
backend: {
apiKey: FAKE_API_KEY,
apiUrl: 'https://api.test.cachekit.io',
allowCustomHost: true,
},
l1: { enabled: true },
});

fetchSpy.mockResolvedValue(mockResponse(404));
await cache.get('ns:first'); // miss
await cache.get('ns:second'); // second request sees the first miss

const [, secondOpts] = fetchSpy.mock.calls[1] as [string, RequestInit];
const headers = secondOpts.headers as Record<string, string>;
expect(headers['X-CacheKit-L1-Status']).toBe('miss');
expect(headers['X-CacheKit-Misses']).toBe('1');
expect(headers['X-CacheKit-L1-Hits']).toBe('0');

await cache.close();
});

it('reports L1 telemetry as disabled when the cache has no L1', async () => {
const { createCache } = await import('../cache.js');

const cache = createCache({
backend: {
apiKey: FAKE_API_KEY,
apiUrl: 'https://api.test.cachekit.io',
allowCustomHost: true,
},
l1: { enabled: false },
});

fetchSpy.mockResolvedValueOnce(mockResponse(404));
await cache.get('ns:key');

const [, opts] = fetchSpy.mock.calls[0] as [string, RequestInit];
const headers = opts.headers as Record<string, string>;
expect(headers['X-CacheKit-L1-Status']).toBe('disabled');

await cache.close();
});

it('a user-supplied metricsProvider overrides the auto-wired one', async () => {
const { createCache } = await import('../cache.js');

const cache = createCache({
backend: {
apiKey: FAKE_API_KEY,
apiUrl: 'https://api.test.cachekit.io',
allowCustomHost: true,
metricsProvider: () => ({ l1Hits: 42, l2Hits: 7, misses: 3, l1Enabled: true }),
},
l1: { enabled: true },
});

fetchSpy.mockResolvedValueOnce(mockResponse(404));
await cache.get('ns:key');

const [, opts] = fetchSpy.mock.calls[0] as [string, RequestInit];
const headers = opts.headers as Record<string, string>;
expect(headers['X-CacheKit-L1-Hits']).toBe('42');
expect(headers['X-CacheKit-Misses']).toBe('3');

await cache.close();
});
});
});
4 changes: 2 additions & 2 deletions packages/cachekit/src/backends/redis.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { randomUUID } from 'node:crypto';
import { Redis as IoRedis, type RedisOptions } from 'ioredis';
import { LockableBackend, RedisBackendConfig, TTLBackend } from './types.js';
import { BackendError, TimeoutError } from '../errors.js';
import { logError } from '../logger.js';
import {
DEFAULT_TTL_SECONDS,
DEFAULT_REDIS_CONNECT_TIMEOUT,
Expand Down Expand Up @@ -99,8 +100,7 @@ export class RedisBackend implements LockableBackend, TTLBackend {
};

const safeMessage = sanitize(err.message) || 'Unknown Redis error';
// eslint-disable-next-line no-console -- Library intentionally uses console for error visibility
console.error(`[cachekit] Redis error: ${safeMessage}`);
logError(`[cachekit] Redis error: ${safeMessage}`);
});
}

Expand Down
Loading
Loading