Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
3e41df2
Remove torch dependency and nn subpackage
Jul 17, 2026
b7d78bf
Replace private/deprecated sklearn and scipy imports with public APIs
Jul 17, 2026
664d2b7
Migrate estimators to the scikit-learn 1.6+ developer API
Jul 17, 2026
7df94b0
Refactor metrics to use the public scikit-learn API
Jul 17, 2026
acaeb9c
Update repository URLs to the PlasmaControl organization
Jul 17, 2026
dce389e
Modernize packaging metadata for the scikit-learn 1.6+ upgrade
Jul 17, 2026
e0a6f07
Modernize CI and test configuration
Jul 17, 2026
df6f75d
Modernize type hints and unify imports across the package
Jul 17, 2026
9a08d1f
Clean up outdated coding practices
Jul 17, 2026
fde3e4a
Fix non-reproducible sparse weight initialization
Jul 17, 2026
3851d13
Follow scikit-learn conventions for constructor parameters
Jul 17, 2026
d444a8f
Make the package pass strict mypy and enforce it in CI
Jul 18, 2026
3b89a07
Modernize the test suite to match the package
Jul 18, 2026
5381e88
Add backend redesign plan
Jul 18, 2026
1b0c8dd
Phase A0: declare torch dependency and add backend package scaffolding
Jul 18, 2026
f5e49ba
Plan: note to drop the private-API disclaimer when pyrcn.nn goes public
Jul 18, 2026
74ef242
Phase A1: add check_sequences normalizer (TDD)
Jul 18, 2026
b14584c
Phase A2: torch reservoir module with NodeToNode parity (TDD)
Jul 18, 2026
f9f8b71
Plan: document torch-module reuse strategy (D14)
Jul 18, 2026
04f8aa5
Phase A2: rebuild the reservoir on nn.RNNCell (D14 reuse)
Jul 18, 2026
1a2c84d
Phase A2: bidirectional reservoir with NodeToNode parity
Jul 18, 2026
0f0cf27
Phase A2: torch-native reservoir weight init (TDD)
Jul 18, 2026
90bd60a
Phase A2: Euler + Hebbian reservoir variants (TDD)
Jul 18, 2026
3149fb6
Phase A2: torch InputFeatureMap (nn.Linear) + input init (TDD)
Jul 18, 2026
d9a2a6e
Fix InputToNode k_in: connect k_in inputs per hidden node
Jul 18, 2026
7d5c428
Phase A3: torch-native closed-form ridge readout (TDD)
Jul 18, 2026
93133e8
Plan: mark Phase A0-A3 workstreams done
Jul 18, 2026
66219ed
Phase A4: config-block -> torch-backend bridge (TDD)
Jul 18, 2026
d9d7a31
Phase A4: wire ELM estimators onto the torch backend (fast path + fal…
Jul 18, 2026
6cdd988
Phase A4: wire ESN estimators onto the torch backend (fast path + fal…
Jul 19, 2026
f81bcb6
Plan: mark A4 core (estimator torch integration) done
Jul 19, 2026
14ebf85
Phase A4b: ESN washout + predict initial_state / return_state
Jul 19, 2026
aa83a70
Plan: mark A4b (washout + state carry) done
Jul 19, 2026
bb34b17
Phase A5: drop the joblib/loky sequence-parallel path
Jul 19, 2026
fb16a33
Phase A5: parity harness + model-selection tests on the torch backend
Jul 19, 2026
2c9c1ba
Plan: mark A5 (parity + model-selection tests, drop joblib) done
Jul 19, 2026
09ed0ce
Phase A6: finalize - remove dead merge operators, device test, deps
Jul 19, 2026
99bfc2e
Plan: mark A6 (finalize) done; Phase A complete
Jul 19, 2026
97f02ed
Improve activation validation error message (#54)
Jul 19, 2026
e19d0cc
P1: promote the torch backend to a public pyrcn.nn package
Jul 19, 2026
c989799
P1: document the public pyrcn.nn API (Sphinx page)
Jul 19, 2026
7e28d4d
Plan: mark P1 (public pyrcn.nn) done
Jul 19, 2026
76c6ee6
Phase B1 core: trainable LinearReadout + train_readout helper
Jul 19, 2026
7c28a0d
Phase B1: gradient solver for the ELM estimators
Jul 19, 2026
15bfdfd
Phase B1: gradient solver for the ESN estimators
Jul 19, 2026
3392563
Plan: mark Phase B1 (gradient readout solver) done
Jul 19, 2026
1521a56
Phase B1: make gradient training reproducible via random_state
Jul 20, 2026
48a91de
Plan: gradient training reproducible via random_state
Jul 20, 2026
12c9f34
Phase B1: tight gradient<->closed-form consistency demo (real reservoir)
Jul 20, 2026
8717159
Phase B2: trainable reservoir (end-to-end gradient training / BPTT)
Jul 20, 2026
f590bf1
Plan: mark Phase B2 (trainable reservoir) done
Jul 20, 2026
58c2c4a
Update examples for the modernized API (no torch usage)
Jul 20, 2026
09bb4eb
Examples: use matplotlib_inline for set_matplotlib_formats
Jul 20, 2026
95a364d
Phase B2+: trainable_input flag + BPTT mini-batching (ESN)
Jul 20, 2026
da406cf
Plan: mark B2+ (trainable_input + BPTT mini-batching) done
Jul 20, 2026
4d3266a
Docs: complete + modernize the Sphinx documentation
Jul 20, 2026
5dbf871
Unify activation set + extend gradient optimizers/losses
Jul 20, 2026
ef6379d
Plan: refresh roadmap status (through Phase B) + open tasks
Jul 20, 2026
84ba210
Merge remote-tracking branch 'origin/dev' into backend-redesign
Jul 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
466 changes: 466 additions & 0 deletions BACKEND_REDESIGN_PLAN.md

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
docutils<0.18
sphinx-rtd-theme==0.5.1
sphinx-copybutton==0.3.1
sphinx>=7
sphinx-rtd-theme>=2.0
sphinx-copybutton>=0.5
pyrcn
scikit-learn
9 changes: 8 additions & 1 deletion docs/source/api/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,14 @@ PyRCN API

pyrcn
pyrcn.base
pyrcn.nn
pyrcn.echo_state_network
pyrcn.extreme_learning_machine
pyrcn.linear_model
pyrcn.cluster
pyrcn.model_selection
pyrcn.metrics
pyrcn.datasets
pyrcn.projection
pyrcn.preprocessing
pyrcn.postprocessing
pyrcn.util
9 changes: 9 additions & 0 deletions docs/source/api/pyrcn.metrics.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. _`pyrcn.metrics`:

pyrcn.metrics
=============

.. automodule:: pyrcn.metrics
:members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/source/api/pyrcn.model_selection.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. _`pyrcn.model_selection`:

pyrcn.model_selection
=====================

.. automodule:: pyrcn.model_selection
:members:
:undoc-members:
:show-inheritance:
64 changes: 64 additions & 0 deletions docs/source/api/pyrcn.nn.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
.. _`pyrcn.nn`:

pyrcn.nn
========

.. automodule:: pyrcn.nn

Reservoir layers
----------------

.. autoclass:: pyrcn.nn.Reservoir
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyrcn.nn.EulerReservoir
:members:
:undoc-members:
:show-inheritance:

Reservoir cells
---------------

.. autoclass:: pyrcn.nn.LeakyESNCell
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyrcn.nn.EulerESNCell
:members:
:undoc-members:
:show-inheritance:

Feature map and readout
------------------------

.. autoclass:: pyrcn.nn.InputFeatureMap
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyrcn.nn.IncrementalRidge
:members:
:undoc-members:
:show-inheritance:

.. autoclass:: pyrcn.nn.LinearReadout
:members:
:undoc-members:
:show-inheritance:

Gradient training
-----------------

.. autofunction:: pyrcn.nn.train_readout

.. autofunction:: pyrcn.nn.torch_generator

Weight initializers
-------------------

.. automodule:: pyrcn.nn.init
:members:
:undoc-members:
9 changes: 9 additions & 0 deletions docs/source/api/pyrcn.postprocessing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. _`pyrcn.postprocessing`:

pyrcn.postprocessing
====================

.. automodule:: pyrcn.postprocessing
:members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/source/api/pyrcn.preprocessing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. _`pyrcn.preprocessing`:

pyrcn.preprocessing
===================

.. automodule:: pyrcn.preprocessing
:members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/source/api/pyrcn.projection.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. _`pyrcn.projection`:

pyrcn.projection
================

.. automodule:: pyrcn.projection
:members:
:undoc-members:
:show-inheritance:
9 changes: 9 additions & 0 deletions docs/source/api/pyrcn.util.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.. _`pyrcn.util`:

pyrcn.util
==========

.. automodule:: pyrcn.util
:members:
:undoc-members:
:show-inheritance:
16 changes: 16 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,22 @@
]
master_doc = 'index'

# Render numpydoc "Attributes" sections as info fields rather than separate
# object descriptions (avoids duplicate-object-description warnings for
# dataclass attributes and properties).
napoleon_use_ivar = True

# Do not document scikit-learn's dynamically added metadata-routing methods
# (set_*_request); their docstrings reference sklearn-only glossary terms and
# labels that do not resolve in this documentation.
autodoc_default_options = {
'exclude-members': (
'set_fit_request,set_predict_request,set_partial_fit_request,'
'set_score_request,set_transform_request,'
'set_predict_proba_request,set_inverse_transform_request'
),
}

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

Expand Down
13 changes: 8 additions & 5 deletions docs/source/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Building your first Reservoir Computing Network
>>> from inspect import getmembers, isclass
>>> getmembers(blocks, isclass)
[('BatchIntrinsicPlasticity', <class 'pyrcn.base.blocks._input_to_node.BatchIntrinsicPlasticity'>),
('EulerNodeToNode', <class 'pyrcn.base.blocks._node_to_node.EulerNodeToNode'>),
('HebbianNodeToNode', <class 'pyrcn.base.blocks._node_to_node.HebbianNodeToNode'>),
('InputToNode', <class 'pyrcn.base.blocks._input_to_node.InputToNode'>),
('NodeToNode', <class 'pyrcn.base.blocks._node_to_node.NodeToNode'>),
Expand Down Expand Up @@ -94,10 +95,12 @@ Training a RCN

To train the ESN, only three steps are required:

1. Randomly distribute the time-series to each reservoir neuron (**Input-to-Node**).
2. Compute the state of each neuron based on the current input and the previous
state.
2. Compute a linear regression between the reservoir states and the target output.
1. Randomly distribute the time-series to each reservoir neuron
(**Input-to-Node**).
2. Compute the state of each neuron based on the current input and the
previous state.
3. Compute a linear regression between the reservoir states and the
target output.

These steps are handled via :py:func:`pyrcn.echo_state_network.ESNRegressor.fit`,
which is the most important function to train the ESN model:
Expand All @@ -120,6 +123,6 @@ Testing and predict using the ESN

.. doctest::

>>> y_pred = esn.predict(X[:4000])
>>> y_pred = esn.predict(X[:4000].reshape(-1, 1))

.. image:: _static/img/getting_started_mackey_glass_predicted.svg
4 changes: 2 additions & 2 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ It is actively developed to be extended into several directions:

* Interaction with `sktime <https://sktime.org/>`_
* Interaction with `hmmlearn <https://hmmlearn.readthedocs.io/en/stable/>`_
* More towards future work: Related architectures, such as Liquid State Machines (LSMs)
and Perturbative Neural Networks (PNNs)
* More towards future work: Related architectures, such as Liquid State
Machines (LSMs) and Perturbative Neural Networks (PNNs)

PyRCN has successfully been used for several tasks:

Expand Down
10 changes: 6 additions & 4 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _installation guide:

==================
Installation guide
==================
Expand All @@ -13,10 +15,10 @@ command in a PowerShell or CommandLine in Windows, or in a shell in Linux/MacOS:

python --version

As any package, **PyRCN** has several dependencies as listed in the `requirements.txt
<https://github.com/PlasmaControl/PyRCN/blob/main/requirements.txt>`_. To avoid any
unexpected interaction with the basic system as installed on your computer, we highly
recommend using a virtual environment
As any package, **PyRCN** has several dependencies, declared in
`pyproject.toml <https://github.com/PlasmaControl/PyRCN/blob/main/pyproject.toml>`_.
To avoid any unexpected interaction with the basic system as installed on your
computer, we highly recommend using a virtual environment

You can find more information about virtual environments, by checking the `Python
documentation on virtual environments and packages
Expand Down
2 changes: 2 additions & 0 deletions docs/source/introduction.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _whats rc:

Definition of Reservoir Computing
=================================

Expand Down
2 changes: 1 addition & 1 deletion examples/MNIST_regressors.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"from sklearn.linear_model import Ridge\n",
"from sklearn.preprocessing import MinMaxScaler\n",
"from sklearn.model_selection import RandomizedSearchCV, GridSearchCV, StratifiedKFold, ParameterGrid, cross_validate\n",
"from sklearn.utils.fixes import loguniform\n",
"from scipy.stats import loguniform\n",
"from sklearn.metrics import accuracy_score\n",
"\n",
"from pyrcn.model_selection import SequentialSearchCV\n",
Expand Down
14 changes: 7 additions & 7 deletions examples/Video_Door_state_Classification.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
"import pandas as pd\n",
"from tqdm import tqdm\n",
"import matplotlib.pyplot as plt\n",
"from IPython.display import set_matplotlib_formats\n",
"from matplotlib_inline.backend_inline import set_matplotlib_formats\n",
"set_matplotlib_formats('png', 'pdf')\n",
"from mpl_toolkits.axes_grid1 import make_axes_locatable\n",
"from matplotlib import ticker\n",
Expand All @@ -55,7 +55,7 @@
"from sklearn.metrics import make_scorer, ConfusionMatrixDisplay\n",
"from sklearn.model_selection import RandomizedSearchCV, GridSearchCV, ParameterGrid\n",
"from sklearn.cluster import MiniBatchKMeans\n",
"from sklearn.utils.fixes import loguniform\n",
"from scipy.stats import loguniform\n",
"from scipy.stats import uniform\n",
"from joblib import dump, load\n",
"\n",
Expand Down Expand Up @@ -259,10 +259,10 @@
"step3_esn_params = {'bias_scaling': np.linspace(0.0, 1.0, 11)}\n",
"step4_esn_params = {'alpha': loguniform(1e-5, 1e1)}\n",
"\n",
"kwargs_step1 = {'n_iter': 200, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, needs_proba=True)}\n",
"kwargs_step2 = {'n_iter': 50, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, needs_proba=True)}\n",
"kwargs_step3 = {'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, needs_proba=True)}\n",
"kwargs_step4 = {'n_iter': 50, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, needs_proba=True)}\n",
"kwargs_step1 = {'n_iter': 200, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, response_method='predict_proba')}\n",
"kwargs_step2 = {'n_iter': 50, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, response_method='predict_proba')}\n",
"kwargs_step3 = {'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, response_method='predict_proba')}\n",
"kwargs_step4 = {'n_iter': 50, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, response_method='predict_proba')}\n",
"\n",
"# The searches are defined similarly to the steps of a sklearn.pipeline.Pipeline:\n",
"searches = [('step1', RandomizedSearchCV, step1_esn_params, kwargs_step1),\n",
Expand Down Expand Up @@ -342,7 +342,7 @@
" 'leakage': loguniform(1e-5, 1e0),\n",
" 'bias_scaling': uniform(loc=0, scale=2)}\n",
"\n",
"kwargs_step0 = {'n_iter': 1000, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, needs_proba=True)}\n",
"kwargs_step0 = {'n_iter': 1000, 'random_state': 42, 'verbose': 1, 'n_jobs': -1, 'scoring': make_scorer(mean_squared_error, greater_is_better=False, response_method='predict_proba')}\n",
"\n",
"base_esn = ESNClassifier(**initially_fixed_params)\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion examples/Video_Door_state_Classification.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
RandomizedSearchCV, GridSearchCV,
ParameterGrid)
from sklearn.cluster import MiniBatchKMeans
from sklearn.utils.fixes import loguniform
from scipy.stats import loguniform
from scipy.stats import uniform
from joblib import dump, load

Expand Down
5 changes: 2 additions & 3 deletions examples/Video_Door_state_Classification_randomized_search.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@

from sklearn.metrics import make_scorer
from sklearn.model_selection import RandomizedSearchCV
from sklearn.utils.fixes import loguniform
from scipy.stats import uniform
from scipy.stats import loguniform, uniform
from joblib import dump, load


Expand Down Expand Up @@ -102,7 +101,7 @@ def read_file(fname, Nfr=-1):
kwargs_step0 = {'n_iter': 1000, 'random_state': 42, 'verbose': 1, 'n_jobs': 1,
'scoring': make_scorer(mean_squared_error,
greater_is_better=False,
needs_proba=True)}
response_method='predict_proba')}

base_esn = ESNClassifier(**initially_fixed_params)

Expand Down
4 changes: 2 additions & 2 deletions examples/digits-kmeans.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans

from src.pyrcn.util import get_mnist, tud_colors
from pyrcn.util import get_mnist


matplotlib.rc('image', cmap='binary')
Expand Down Expand Up @@ -92,7 +92,7 @@ def main():
labelbottom=False)

ax_barchart.bar(list(map(int, values)), cos_similarity[:, i],
tick_label=values, color=tud_colors['lightblue'])
tick_label=values, color='tab:blue')
# ax_barchart.set_xlim([0, 9])
ax_barchart.grid(which='both', axis='y')
ax_barchart.set_yticks([-1., 0., 1.], minor=False)
Expand Down
50 changes: 0 additions & 50 deletions examples/digits.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -520,56 +520,6 @@
" print(f\"{esn_cv}\\t{t_fit}\\t{t_inference}\\t{acc_score}\\t{mem_size}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, we can also use a PyTorch implementation of the ESN model"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"from pyrcn.nn import ESN\n",
"import torch"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"base_esn = ESN(input_size=8, hidden_size=50, num_layers=1, nonlinearity='tanh',\n",
" bias=True, input_scaling=0.016952130531190705,\n",
" spectral_radius=1.0214946051551315, bias_scaling=1.500499867548985,\n",
" bias_shift=0., input_sparsity=0.1, recurrent_sparsity=0.1,\n",
" bidirectional=False)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 50])\n"
]
}
],
"source": [
"for x, y in zip(X_train, y_train):\n",
" x = torch.Tensor(x).float()\n",
" print(base_esn(x)[0].shape)\n",
" break"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down
Loading
Loading