Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions dpsynth/relational/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# `dpsynth.relational` (Multi-Table Relational DP Synthesis)

> [!WARNING]
> **UNDER ACTIVE DEVELOPMENT / EXPERIMENTAL**: This module is in an early
> experimental stage and is under active development. The APIs, internal
> interfaces, and algorithms are subject to breaking changes. It is not yet
> recommended for production use.

---

## Overview

The `dpsynth.relational` package extends DP Synth to support **hierarchical
multi-table relational databases** under differential privacy.

In relational databases (e.g. `Household -> Person -> Activity`), privacy is
protected at the **root parent record** (e.g. the household), while valid
foreign-key linkages and cross-table statistical correlations are preserved
across child tables without generating orphaned records or requiring
intractable flat Cartesian joins.

### Core Approach

- **Cascading Relational Synthesis**: Synthesizes databases table-by-table
down foreign-key hierarchy trees, ensuring root-entity differential privacy
without materializing expensive Cartesian joins.
- **Permutation Modeling**: Captures rich parent-child and sibling-to-sibling
correlations through slot permutation and exchangeability, inspired by
PrivPetal ([Cai et al., 2025](https://arxiv.org/abs/2503.22970)).
- **Built on `dpsynth` & `mbi` Core**: Reuses existing discrete mechanisms (e.g.
AIM, MST) and the `mbi` (Private-PGM) graphical model engine under the hood.
- **Unified Mechanism API**: Integrates directly with the `MechanismConfig` /
`CalibratedMechanism` paradigm used throughout `dpsynth`.

---

## Planned API

```python
import dpsynth
from dpsynth.relational import ForeignKeyRelation, MultiTableConfig

# Define schemas and foreign key relations
foreign_keys = [
ForeignKeyRelation(
parent_table="households",
parent_primary_key="household_id",
child_table="persons",
child_foreign_key="household_id",
max_children_per_parent=5,
),
]

# Configure relational synthesizer
config = MultiTableConfig(
domains=table_domains,
foreign_keys=foreign_keys,
discrete_mechanism=dpsynth.discrete_mechanisms.AIMConfig(),
)

# Calibrate and run
calibrated = config.calibrate(epsilon=1.0, delta=1e-5)
result = calibrated(rng, {"households": df_h, "persons": df_p})
synth_tables = result.synthetic_tables
```
35 changes: 35 additions & 0 deletions dpsynth/relational/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Public API for multi-table relational differential privacy synthesis."""

# pylint: disable=g-importing-member

from dpsynth.relational.domain import ForeignKeyRelation
from dpsynth.relational.domain import from_dict
from dpsynth.relational.domain import from_yaml_file
from dpsynth.relational.domain import topological_sort_hierarchy
from dpsynth.relational.synthesizer import MultiDataGenerationResult
from dpsynth.relational.synthesizer import MultiTableConfig
from dpsynth.relational.synthesizer import MultiTableMechanism

__all__ = [
'ForeignKeyRelation',
'MultiDataGenerationResult',
'MultiTableConfig',
'MultiTableMechanism',
'from_dict',
'from_yaml_file',
'topological_sort_hierarchy',
]
119 changes: 119 additions & 0 deletions dpsynth/relational/domain.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Domain representations, schema definitions, and DAG validators for relational data."""

from __future__ import annotations

from collections.abc import Mapping, Sequence
import dataclasses
from typing import Any

from dpsynth import domain
from etils import epath
import yaml

PathType = epath.PathLike

# pylint: disable=unused-import
_YAML_UNUSED = yaml
# pylint: enable=unused-import


@dataclasses.dataclass(frozen=True)
class ForeignKeyRelation:
"""Defines a directed foreign key relationship between parent and child tables.

Attributes:
parent_table: Name of the parent table (e.g. 'households').
parent_primary_key: Name of the parent primary key column (e.g.
'household_id').
child_table: Name of the child table (e.g. 'persons').
child_foreign_key: Name of the child foreign key column referencing parent.
max_children_per_parent: Maximum number of children associated with a single
parent record (group size capacity bound s). Must be >= 1. Determines the
wide MRF generation slot count (s) and directly scales cascading DP
sensitivity (Delta_k = prod s_ancestors) for downstream child tables.
"""

parent_table: str
parent_primary_key: str
child_table: str
child_foreign_key: str
max_children_per_parent: int

def __post_init__(self):
if self.max_children_per_parent < 1:
raise ValueError(
'max_children_per_parent must be >= 1, got'
f' {self.max_children_per_parent}.'
)


def topological_sort_hierarchy(
tables: Sequence[str],
foreign_keys: Sequence[ForeignKeyRelation],
) -> list[tuple[int, str, ForeignKeyRelation | None]]:
"""Validates DAG tree structure and computes topological synthesis levels.

Args:
tables: Sequence of all table names in the database.
foreign_keys: Sequence of foreign key relationships between tables.

Returns:
An ordered list of (depth, table_name, foreign_key_relation) tuples, where
depth is 0 for root tables (foreign_key_relation is None) and depth >= 1 for
child tables (foreign_key_relation links the table to its immediate parent).

Raises:
ValueError: If foreign keys contain cycles, missing tables, or if a child
table references more than one parent table (in-degree > 1).
"""
del tables, foreign_keys
raise NotImplementedError(
'topological_sort_hierarchy is not yet implemented.'
)


def from_dict(
config: Mapping[str, Any],
) -> tuple[dict[str, domain.Schema], list[ForeignKeyRelation]]:
"""Parses multi-table schema and foreign keys from a dictionary.

Args:
config: Dictionary with 'tables' and optional 'foreign_keys' blocks.

Returns:
A tuple of (table_domains, foreign_keys).

Raises:
ValueError: If configuration format or attribute specifications are invalid.
"""
del config
raise NotImplementedError('from_dict is not yet implemented.')


def from_yaml_file(
filepath: str | PathType,
) -> tuple[dict[str, domain.Schema], list[ForeignKeyRelation]]:
"""Reads multi-table schema and foreign keys from a YAML file.

Args:
filepath: Path to the YAML schema file.

Returns:
A tuple of (table_domains, foreign_keys).
"""
del filepath
raise NotImplementedError('from_yaml_file is not yet implemented.')
134 changes: 134 additions & 0 deletions dpsynth/relational/synthesizer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Multi-table relational differential privacy synthesizer."""

from __future__ import annotations

from collections.abc import Mapping, Sequence
import dataclasses
from typing import Any, Literal

from absl import logging
import dp_accounting
from dpsynth import api
from dpsynth import discrete_mechanisms
from dpsynth import domain
from dpsynth.relational import domain as rel_domain
import numpy as np
import pandas as pd

# pylint: disable=unused-import
_LOGGING_UNUSED = logging
# pylint: enable=unused-import


@dataclasses.dataclass(frozen=True)
class MultiDataGenerationResult:
"""Results of multi-table relational DP synthetic data generation.

Attributes:
synthetic_tables: Mapping from table names to synthetic DataFrames.
discrete_mechanism_results: Mapping from link/table names to mechanism
diagnostics.
"""

synthetic_tables: Mapping[str, pd.DataFrame]
discrete_mechanism_results: Mapping[str, Any] = dataclasses.field(
default_factory=dict
)


@dataclasses.dataclass
class MultiTableMechanism(api.CalibratedMechanism):
"""Calibrated, runnable multi-table relational differential privacy mechanism.

Attributes:
domains: Mapping from table name to per-column attribute specifications.
foreign_keys: Sequence of foreign key relationships defining the hierarchy.
calibrated_discrete_mechanisms: Mapping from link names to calibrated
discrete mechanisms.
calibrated_initializers: Mapping from table and column to calibrated
initializers.
total_count_sigma: Sigma for the root table total-count mechanism.
num_permutation_slots: Permutation exploration slot count (o), default 2.
exploration_strategy: Exploration strategy ('empty_token' or 'size_sliced').
max_records_per_user: Assumed upper bound on records a single root user
contributes.

Note: For simplicity, user-defined contraints are not supported yet.
"""

domains: Mapping[str, domain.Schema]
foreign_keys: Sequence[rel_domain.ForeignKeyRelation]
calibrated_discrete_mechanisms: Mapping[str, api.CalibratedMechanism]
calibrated_initializers: Mapping[str, Mapping[str, api.CalibratedMechanism]]
total_count_sigma: float = dataclasses.field(repr=False)
num_permutation_slots: int = 2
exploration_strategy: Literal['empty_token', 'size_sliced'] = 'empty_token'
max_records_per_user: int = 1

@property
def dp_event(self) -> dp_accounting.DpEvent:
"""Returns the composed DpEvent for all sub-mechanisms."""
raise NotImplementedError('dp_event is not yet implemented.')

def __call__(
self,
rng: np.random.Generator,
data: Mapping[str, pd.DataFrame],
) -> MultiDataGenerationResult:
"""Generates synthetic multi-table relational data."""
del rng, data
raise NotImplementedError(
'MultiTableMechanism.__call__ is not yet implemented.'
)


@dataclasses.dataclass
class MultiTableConfig(api.MechanismConfig):
"""Configuration recipe for multi-table relational differential privacy synthesis.

Attributes:
domains: Mapping from table name to per-column attribute domain
specifications.
foreign_keys: Sequence of foreign key relationships defining the hierarchy.
discrete_mechanism: Discrete mechanism config (e.g. AIM, MST) for relational
links.
numerical_bins: Number of bins for numerical attribute discretization.
init_budget_fraction: Fraction of total zCDP budget allocated to Phase 1.
num_permutation_slots: Permutation exploration slot count (o), default 2.
exploration_strategy: Exploration strategy ('empty_token' or 'size_sliced').
"""

domains: Mapping[str, domain.Schema]
foreign_keys: Sequence[rel_domain.ForeignKeyRelation] = ()
discrete_mechanism: discrete_mechanisms.DiscreteMechanismConfig = (
dataclasses.field(default_factory=discrete_mechanisms.AIMConfig)
)
numerical_bins: int = 32
init_budget_fraction: float = 0.1
num_permutation_slots: int = 2
exploration_strategy: Literal['empty_token', 'size_sliced'] = 'empty_token'

def configure(
self,
*,
zcdp_rho: float,
delta: float = 0.0,
max_records_per_user: int = 1,
) -> MultiTableMechanism:
"""Configures privacy budgets across Phase 1 and Phase 2 relational links."""
del zcdp_rho, delta, max_records_per_user
raise NotImplementedError('configure is not yet implemented.')
Loading
Loading