Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ export default defineConfig(withMermaid({
useTitleFromFrontmatter: true,
useFolderLinkFromIndexFile: true,
useFolderTitleFromIndexFile: true,
sortMenusByFrontmatterOrder: true,
}),

socialLinks: [
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/arbiter.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Arbiter
order: 60
---

# Arbiter
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/ceph.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Ceph
order: 30
---

# Ceph
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/chorus.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Chorus
order: 50
---

# Chorus
Expand Down
3 changes: 2 additions & 1 deletion docs/architecture/cloud-storage/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Cloud Storage
order: 50
---

# Cloud Storage
Expand Down Expand Up @@ -41,7 +42,7 @@ Ceph exposes three storage interfaces that CobaltCore services consume:
Applications / VMs
┌───────┴────────────────────┐
│ RBD │ CephFS │ RGW │ ← Ceph interfaces
│ RBD │ CephFS │ RGW │ ← Ceph interfaces
└───────┴────────────────────┘
RADOS (Reliable Autonomic Distributed Object Store)
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/liquid-ceph.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Liquid-Ceph
order: 70
---

# Liquid-Ceph
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/observability/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Observability & Audit
order: 80
---

# Observability & Audit Overview
Expand Down
75 changes: 75 additions & 0 deletions docs/architecture/cloud-storage/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: Overview - Cloud Storage
order: 15
---

# Overview - Cloud Storage

CobaltCore's cloud storage layer is built on [Ceph](./ceph.md), a distributed storage system that delivers object, block, and file storage in a single unified platform. The surrounding components handle lifecycle automation, data replication, high-availability quorum, observability, and liquid storage allocation — each with a focused responsibility.

## Architecture

The storage stack is organized into three layers:

**Foundation** — Ceph provides the core distributed storage engine. All other components either operate it, extend it, or observe it.

**Operations** — [Rook](./rook.md) runs as a Kubernetes operator and manages the full lifecycle of Ceph daemons (monitors, managers, OSDs, MDS, RGW) as containerized workloads. [Arbiter](./arbiter.md) extends quorum into stretched cluster topologies by deploying external Ceph monitors that Rook does not manage directly.

**Data Services** — [Chorus](./chorus.md) provides zero-downtime data replication and migration between object storage systems (S3 and Swift). [Liquid-Ceph](./liquid-ceph.md) enables dynamic, on-demand storage allocation across the cluster.

## Components

| Component | Layer | Role |
|-----------|-------|------|
| [Ceph](./ceph.md) | Foundation | Distributed storage engine — block (RBD), file (CephFS), object (RGW) |
| [Rook](./rook.md) | Operations | Kubernetes operator for Ceph lifecycle management |
| [Arbiter](./arbiter.md) | Operations | External Ceph monitors for quorum in stretched clusters |
| [Chorus](./chorus.md) | Data Services | Zero-downtime object storage replication and migration |
| [Liquid-Ceph](./liquid-ceph.md) | Data Services | Dynamic storage allocation across the Ceph cluster |
| [Observability & Audit](./observability/) | Observability | Metrics, dashboards, alerting, and audit — Prometheus, Perses, Prysm |

## Storage Interfaces

Ceph exposes three storage interfaces that CobaltCore services consume:

- **RBD (RADOS Block Device)** — thin-provisioned, resizable block volumes used by virtual machines and databases. Striped across OSDs for parallel I/O and backed by RADOS snapshots and replication.
- **CephFS** — POSIX-compliant distributed filesystem. Metadata is managed by a dedicated MDS cluster; data is striped across OSDs. Supports snapshots, quotas, and multiple active MDS daemons for horizontal metadata scaling.
- **RGW (RADOS Gateway)** — S3 and Swift-compatible object storage gateway. Supports multi-tenancy, versioning, lifecycle policies, server-side encryption, and multi-site active-active replication.

## Data Flow

```text
Applications / VMs
┌───────┴────────────────────┐
│ RBD │ CephFS │ RGW │ ← Ceph interfaces
└───────┴────────────────────┘
RADOS (Reliable Autonomic Distributed Object Store)
OSDs across cluster nodes
┌────┴─────┐
│ Rook │ ← manages daemon lifecycle via Kubernetes CRDs
└──────────┘
┌────┴──────┐ ┌─────────┐ ┌────────────┐
│ Arbiter │ │ Chorus │ │ Liquid-Ceph│
└───────────┘ └─────────┘ └────────────┘
(quorum) (replication) (allocation)
┌────┴──────────────────────────┐
│ Observability & Audit │
│ Prometheus · Perses · Prysm │
└───────────────────────────────┘
```

## High Availability

Ceph achieves HA through monitor quorum (typically 3 or 5 monitors), OSD replication or erasure coding, and MDS standby daemons. In stretched deployments that span two sites, [Arbiter](./arbiter.md) deploys a third monitor at a tiebreaker site so that quorum is maintained even if one full site goes offline.

## See Also

- [Observability & Audit](./observability/) — Prometheus metrics, Perses dashboards, and Prysm CLI for the storage stack
- [Ceph upstream architecture docs](https://docs.ceph.com/en/latest/architecture/)
- [Rook documentation](https://rook.io/docs/rook/latest-release/Getting-Started/intro/)
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/rook-ceph.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Installing Rook-Ceph on Kubernetes
order: 25
---

# Installing Rook-Ceph on Kubernetes
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cloud-storage/rook.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Rook
order: 20
---

# Rook
Expand Down
1 change: 1 addition & 0 deletions docs/architecture/cluster.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Kubernetes Cluster
order: 10
---

# Kubernetes Cluster
Expand Down
3 changes: 2 additions & 1 deletion docs/architecture/cortex.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Cortex
order: 40
---

# Cortex
Expand Down Expand Up @@ -66,4 +67,4 @@ graph LR
External <-->|receive placement request\nreturn placement suggestion| IniPlacement
External <-->|trigger migration| Scheduler
API <--> User["User / Dashboard"]
```
```
3 changes: 2 additions & 1 deletion docs/architecture/hypervisor.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Hypervisor
order: 30
---

# Hypervisor
Expand Down Expand Up @@ -168,4 +169,4 @@ NAME ORIGIN DESTINATION TYPE
12e479eb-6bef-4fdb-bfdc-0388df68bed9 node002-bb086 node008-bb086 completed migration_in 74d 2.755s 4.0 GiB 10.4 MiB 0 B 16.5 MiB/s 0/s 3
13679335-291c-4405-9e08-5911032599dd node007-bb086 node009-bb086 completed migration_out 78d 2.766s 4.0 GiB 398.0 MiB 718.8 MiB 588.9 MiB/s 0/s 1
1552e60a-bdba-4850-84da-07dd635bce2c node006-bb086 node003-bb087 completed migration_out 22d 35.078s 64.0 GiB 58.4 GiB 0 B 1.8 GiB/s 1250/s 4
```
```
3 changes: 2 additions & 1 deletion docs/architecture/openstack.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: OpenStack
order: 20
---

# OpenStack
Expand Down Expand Up @@ -141,4 +142,4 @@ C4Deployment
Rel(computeagent, cinderapi, "Calls")
Rel(mount, netapp, "Accesses", "NFS")
UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="1")
```
```
Loading
Loading