Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .spelling
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ Fundle
Fundle's
HRESULT
Hardcoded
Heapwatch
How-tos
Hyper
Comment thread
martintmk marked this conversation as resolved.
IOCP
Expand Down Expand Up @@ -345,6 +346,7 @@ growable
hashbrown
hasher
hashers
heapwatch
hedges
honour
hostname
Expand Down
4 changes: 4 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ futures-core = { version = "0.3.31", default-features = false }
futures-util = { version = "0.3.31", default-features = false }
gungraun = { version = "0.19.2", default-features = false }
hashbrown = { version = "0.17.0", default-features = false }
heapwatch = { path = "crates/heapwatch", default-features = false, version = "0.1.0" }
heck = { version = "0.5.0", default-features = false }
http = { version = "1.4.1", default-features = false, features = ["std"] }
http-body = { version = "1.0.1", default-features = false }
Expand Down
1 change: 1 addition & 0 deletions crates/heapwatch/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Changelog
35 changes: 35 additions & 0 deletions crates/heapwatch/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.

[package]
name = "heapwatch"
version = "0.1.0"
description = "A low-overhead global allocator wrapper that continuously accounts heap usage"
readme = "README.md"
keywords = ["allocator", "heap", "memory", "metrics", "diagnostics"]
categories = ["memory-management", "development-tools::profiling"]

edition.workspace = true
rust-version.workspace = true
authors.workspace = true
license.workspace = true
homepage.workspace = true
include.workspace = true
repository = "https://github.com/microsoft/oxidizer/tree/main/crates/heapwatch"

# The crate is scaffolding: it carries the design documents but no code, so
# there is nothing to instrument and no test to run. Without this opt-out the
# impact-scoped coverage run fails with nextest's "no tests to run" (exit 4).
# REMOVE THIS SECTION when the implementation lands, so the workspace-default
# 100% line-coverage gate applies again.
[package.metadata.coverage-gate]
min-lines-percent = 0

[package.metadata.docs.rs]
rustdoc-args = ["--cfg", "docsrs"]
all-features = true

# >>> anvil-managed: anvil-lints
[lints]
workspace = true
# <<< anvil-managed: anvil-lints
67 changes: 67 additions & 0 deletions crates/heapwatch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<div align="center">
<img src="https://raw.githubusercontent.com/microsoft/oxidizer/refs/heads/main/logo.svg" alt="Heapwatch Logo" width="96">

# Heapwatch

[![crate.io](https://img.shields.io/crates/v/heapwatch.svg)](https://crates.io/crates/heapwatch)
[![docs.rs](https://docs.rs/heapwatch/badge.svg)](https://docs.rs/heapwatch)
[![MSRV](https://img.shields.io/crates/msrv/heapwatch)](https://crates.io/crates/heapwatch)
[![CI](https://github.com/microsoft/oxidizer/actions/workflows/main.yml/badge.svg?event=push)](https://github.com/microsoft/oxidizer/actions/workflows/main.yml)
[![Coverage](https://codecov.io/gh/microsoft/oxidizer/graph/badge.svg?token=FCUG0EL5TI)](https://codecov.io/gh/microsoft/oxidizer)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/microsoft/oxidizer/blob/main/LICENSE)
<a href="https://github.com/microsoft/oxidizer"><img src="https://raw.githubusercontent.com/microsoft/oxidizer/refs/heads/main/logo.svg" alt="This crate was developed as part of the Oxidizer project" width="20"></a>

</div>

A global allocator wrapper that continuously accounts heap usage.

Heapwatch wraps another allocator and, installed as a binary’s
`#[global_allocator]`, reports how many bytes that binary’s Rust heap holds
and how much has moved through it. It answers *how much*, never *who* — no
backtraces, no per-allocation metadata, no pointer-to-size shadow map —
which is what keeps the per-allocation cost to a few non-atomic adds, low
enough to leave enabled in production.

## Status

**The API is not implemented yet — this crate is currently scaffolding.**
The architecture is settled and written up in [`DESIGN.md`][__link0], and the ideas
deliberately left out of it are recorded in [`TODO.md`][__link1]. Both land ahead of
the code so the design can be reviewed on its own terms.

## The mechanism

Each thread accumulates its own totals with plain, non-atomic arithmetic and
publishes them into the allocator instance’s atomic totals in batches — once
*bytes allocated plus bytes freed* since its last publication crosses a
compile-time threshold, and again when the thread exits. That removes the
atomic read-modify-write per allocation that an exact accounting wrapper
pays, which is the cost that scales badly with core count.

The trade is a small, bounded, stated inaccuracy: a reading omits whatever
each live thread has not yet published, at most the threshold times the
number of live threads. That bound depends on neither the allocation rate
nor how long the process has been running. Reading is O(1) in the thread
count — a handful of relaxed loads through a handle obtained from the
allocator, with no registry to walk.

## Measurement boundary

Heapwatch counts successful calls through the registered `GlobalAlloc`, and
counts them as *requested* rather than as reserved, since the trait has no
hook to ask the inner allocator what it actually rounded up to. Allocations
that never reach the global allocator — native and FFI heaps, direct OS
mappings, anything routed to `std::alloc::System` — are outside the
boundary, as are thread stacks, static data, and the inner allocator’s own
metadata and fragmentation. It therefore complements, rather than replaces,
allocator-native statistics and process-level metrics such as resident set
size.


<hr/>
<sub>
This crate was developed as part of <a href="https://github.com/microsoft/oxidizer">The Oxidizer Project</a>. Browse this crate's <a href="https://github.com/microsoft/oxidizer/tree/main/crates/heapwatch">source code</a>.
</sub>

[__link0]: https://github.com/microsoft/oxidizer/blob/main/crates/heapwatch/docs/DESIGN.md
[__link1]: https://github.com/microsoft/oxidizer/blob/main/crates/heapwatch/docs/TODO.md
Loading
Loading