Backend redesign#63
Merged
Merged
Conversation
The nn/ PyTorch subpackage was an early draft and is out of scope for the scikit-learn compatibility modernization. Remove it along with the torch, torchvision and torchaudio dependencies so that `import pyrcn` no longer requires PyTorch. - Delete src/pyrcn/nn/ - Drop nn from the top-level package imports and __all__ - Strip torch from util.seed_everything (keep random/numpy seeding) - Remove torch/torchvision/torchaudio from pyproject dependencies Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prepare for scikit-learn >= 1.6 by removing reliance on private or deprecated symbols. No behavioral change. - base/_activations.py: define the inplace activation table locally instead of importing and mutating the private sklearn.neural_network._base.ACTIVATIONS global. - Drop the deprecated @_deprecate_positional_args decorator throughout; __init__/metric signatures are already keyword-only via `*`. - echo_state_network, extreme_learning_machine: use public sklearn.base.RegressorMixin in type hints instead of the private sklearn.linear_model._base.LinearModel. - node_to_node: import csr_matrix from scipy.sparse, not the removed scipy.sparse.csr submodule. - Replace the typing_extensions / sys.version_info shims with plain typing (Python >= 3.10) and drop now-dead `import sys`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Custom estimators relied on APIs removed or deprecated in scikit-learn 1.6: - BaseEstimator._validate_data / _check_n_features were removed. Replace them with the public sklearn.utils.validation.validate_data function. - The estimator type is now derived from tags and _estimator_type is deprecated. Per the official developer guide, mixins must precede BaseEstimator in the MRO so __sklearn_tags__ resolves the estimator type correctly; reorder the bases of every estimator accordingly. This restores is_regressor / is_classifier detection. - Coates: replace the getattr(clusterer, "_estimator_type") check with the public sklearn.base.is_clusterer helper. No behavioral change intended. The suite goes from 50 to 21 failing; the remaining failures are all in the sequence-aware metrics wrappers and are addressed separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The sequence-aware metric wrappers called scikit-learn's private _check_targets / _check_reg_targets helpers, whose signatures changed in 1.6 (sample_weight became positional), causing unpacking and validation errors. Replace those calls: flatten the per-sequence arrays with the public check_consistent_length + np.concatenate and derive the target type via the public sklearn.utils.multiclass.type_of_target. The public metric performs its own target validation, so the private helpers are no longer needed. Also route mean_squared_error(squared=False) to the public root_mean_squared_error, since the `squared` parameter was removed from sklearn.metrics.mean_squared_error in 1.6. PyRCN's own `squared` argument is preserved. No behavioral change intended. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The repository moved from TUD-STKS/PyRCN to PlasmaControl/PyRCN. Update all GitHub and mybinder links in the README, docs and example notebook. Links to other TUD-STKS repositories (Automatic-Music-Transcription, gci_estimation) are left unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Bump requires-python to >=3.10 (the floor for scikit-learn >= 1.6) and add a scikit-learn>=1.6 dependency floor; list numpy/scipy/joblib/pandas explicitly. - Fix the license classifier (was MIT; the LICENSE file and all source headers are BSD-3-Clause). - Single source of truth for the version: pyproject reads it dynamically from pyrcn._version, and _version.py is set to 0.0.18 (was 0.0.17post1). - Add test and examples optional-dependency extras. - Remove the redundant setup.cfg (pyproject is now canonical). - Fix requirements.txt: the malformed `requests[...]` lines were meant to be matplotlib/seaborn/ipywidgets/ipympl/tqdm; drop obsolete typing-extensions. - mypy: ignore_missing_imports for third-party packages without type stubs. - Read the Docs: build with Python 3.12 (>=3.10 is now required). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- CI matrix: Python 3.10-3.13 (drop 3.9); bump actions/checkout to v4 and actions/setup-python to v5 in both workflows. - Install the package via the `test` extra instead of an ad-hoc plugin list plus requirements.txt; run flake8 (blocking) and mypy (informational, the remaining strict-typing findings are deferred to the code-unification pass). - Remove pytest.ini, whose addopts required cov/mypy plugins just to collect tests; pyproject's [tool.pytest.ini_options] keeps a bare `pytest` working, and coverage is requested explicitly in CI. - Fix a flake8 E231 in tests so linting can cover the test suite too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply modern Python 3.10 idioms (the new requires-python floor) consistently: - Type hints: PEP 604 unions (Union[A, B] -> A | B, Optional[A] -> A | None) and PEP 585 builtin generics (Dict/List/Tuple -> dict/list/tuple); move Callable/Iterable to collections.abc. Applied with pyupgrade --py310-plus. - Replace str.format(...) calls with f-strings. - Sort and group imports consistently (isort) and add `from __future__ import annotations` to every module. - Reflow the multi-line signatures whose alignment shifted. Purely mechanical; no behavioral change. flake8 clean, suite unchanged (85 passed / 2 skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Remove the dead, commented-out NonlinearVectorAutoregression class that had been left as a module-level string literal in blocks/_input_to_node.py. - Replace `raise BaseException(...)` with `raise TypeError(...)` in the ESN and ELM partial_fit guards; BaseException should never be raised directly. - Fix a stray `%s` left in a NodeToNode hidden_layer_size error message. No behavioral change (aside from the more specific exception type on an invalid-regressor error path). flake8 clean, suite unchanged (85 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_make_sparse takes a random_state argument but drew the retained-weight indices from the global numpy RNG (np.random.choice), so the sparsity mask was not reproducible from a fixed seed. Use the passed random_state. This makes the sparse weight masks deterministic w.r.t. random_state (the exact mask for a given seed changes); the test suite is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Coates no longer calls check_random_state in __init__; it stores the raw random_state and resolves it in _extract_random_patches, so get_params / clone round-trip correctly. Numerically equivalent for a single fit. - Replace the mutable default arguments Coates(clusterer=KMeans()) and lorenz(x_0=[1.0, 1.0, 1.0]) with None sentinels, constructing the default inside the method/function to avoid a shared mutable instance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the remaining strict-mypy findings so `mypy src/pyrcn` is clean, and drop the informational `continue-on-error` from the CI type-check step. - util.batched: annotate the generator return as Iterator[tuple]. - util.value_to_tuple: correct the signature (value may be a tuple; size is an int) and always return. - BatchIntrinsicPlasticity: initialize self._m/_c as floats (they are assigned float distribution parameters). - Coates: type the pooling axis as `int | None`, and assert self.clusterer is not None at the post-fit use sites (it is resolved to KMeans() in fit). No behavioral change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply the same modernization used for the package to tests/: PEP 604 / PEP 585 type syntax and f-strings (pyupgrade --py310-plus), consistent import sorting with `from __future__ import annotations` (isort), and removal of unused imports (autoflake). No test logic changed; suite unchanged (85 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Living design document for replacing the compute backend with PyTorch while keeping the scikit-learn-compatible frontend. Captures the audit, the design decisions (data model, engine, numerics, dependency, FE/BE boundary, state semantics, params, training modes, block<->module companion pattern, staged public torch API) and a phased implementation roadmap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-add torch as a core dependency (D4) and create the private pyrcn.backend package that will hold the PyTorch compute layer (reservoir engine, input feature map, readout) behind the unchanged scikit-learn frontend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a TODO on the P1 (go-public) roadmap step to remove the "private / implementation-detail" disclaimers from the backend docstrings once the PyTorch surface is promoted to the public pyrcn.nn API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce pyrcn.util.check_sequences + SequenceBatch, the single validated entry point that normalizes the accepted public input forms into the canonical backend representation: - accepts a 2-D single sequence, a list/object-array of 2-D sequences, or a 3-D equal-length batch; - returns a zero-padded (n_sequences, max_length, n_features) array plus a per-sequence lengths vector; - a 2-D (T, F) input normalizes to a single sequence (N=1, single_sequence flag) to preserve the current 2-D behavior (INV-1); - resolves the task (sequence-to-sequence vs sequence-to-value) from target shapes, with an explicit task= override for the ambiguous case. Test-driven (tests/test_sequences.py). Not wired into the estimators yet, so the existing suite is unaffected (93 passed / 2 skipped). Will replace concatenate_sequences + the _check_if_sequence* heuristics at integration (A4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add pyrcn.backend.Reservoir, a batched leaky-integrator reservoir as a PyTorch
nn.Module implementing the same recurrence as the legacy NumPy NodeToNode:
h_t = (1 - leakage) * h_{t-1}
+ leakage * f(x_t + spectral_radius * (h_{t-1} @ W_hh))
Recurrent weights are a dense Parameter (requires_grad=False) assigned via
set_recurrent_weights; supports initial_state -> final_state and batched
(n_sequences, length, hidden_size) input.
Parity is proven by initializing the recurrent weights with PyRCN's own
NodeToNode (dense and sparse k_rec variants, spectral-radius-normalized) and
injecting the identical matrix into the torch module: single-step difference is
exactly 0.0 and full float64 trajectories agree to ~1e-16 across all
activations. Suite: 114 passed / 2 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Record how the torch modules are built: subclass nn.Linear (InputToNode) and nn.RNNCell (NodeToNode), reusing torch's fused single-step op and adding only leaky integration + the extra activations; the two-activation handling; and the deferred fused ESNCell / whole-sequence fast path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the hand-written recurrence with LeakyESNCell(nn.RNNCell): the tanh/relu path reuses torch's fused single-step cell op via super().forward and adds only leaky integration; logistic/identity/bounded_relu use affine + activation. weight_ih is a frozen identity (the reservoir input is added directly) and spectral_radius is folded into weight_hh. Reservoir is now a thin layer that loops the cell (lengths-agnostic, initial_state -> final_state). Behaviorally identical: the NodeToNode parity tests (weights injected from a real NodeToNode, float64) still hold to ~1e-16; suite 114 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add bidirectional=True to Reservoir: run the same cell forward and over the time-reversed input (shared weights, flip back, concatenate on the feature axis), matching NodeToNode.transform's fw+bw hstack. Parity-tested against a real bidirectional NodeToNode (float64, ~1e-16). initial_state carry is not combined with bidirectional (raises). Suite 115 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add pyrcn.backend init functions for the recurrent matrix: - normal_recurrent_weights: normal (optionally sparse via fan_in), normalized to unit spectral radius (spectral_normalize uses torch.linalg.eigvals); - simple_cycle_weights / delay_line_weights / delay_line_feedback_weights: the Rodan minimum-complexity topologies (deterministic structured matrices). Verified numerically: unit spectral radius for random/sparse, exact fan_in per column, exact structure + resulting spectral radius for SCR/DLR/DLRB, and reproducibility via a torch Generator. Parity with the legacy path stays via weight injection; these are for real initialization. Suite 121 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- EulerESNCell / EulerReservoir: the EuSN update h' = h + epsilon * f(x + h @ (recurrent_scaling*W + gamma*I)), reusing the fused cell op; parity-tested against EulerNodeToNode (weights injected, float64, ~1e-16). Add antisymmetric_recurrent_weights for its torch-native init (verified antisymmetric + reproducible). - Factor a shared _ReservoirCell base (identity weight_ih, param freeze, activation dispatch) and a shared _iterate loop, so the leaky and Euler cells do not duplicate machinery. - Hebbian: it only *learns* the recurrent weights in fit; transform is the standard recurrence, so its torch form is the existing Reservoir with the NumPy-learned weights injected. Parity-tested for all four learning rules. Suite 133 passed / 2 skipped; flake8 + mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add pyrcn.backend.InputFeatureMap, the torch companion of InputToNode: an nn.Linear subclass applying PyRCN's input_scaling/input_shift/bias_scaling/ bias_shift (folded into weight/bias) and the input activation. Parity-tested against InputToNode.transform (weights injected, float64, ~1e-16) across all activations and scaling/shift settings. Add torch-native input init: uniform_input_weights (optionally sparse, fan_in per hidden node), uniform_bias_weights, and bernoulli_input_weights (signed constants for minimum-complexity ESNs); property-tested. Factor the shared activation table into pyrcn.backend._activations and reuse it from the reservoir cells (no duplicated activation dicts). Suite 147 passed. Note: InputFeatureMap owns the input weights; the reservoir cell's weight_ih stays a frozen identity, so chaining them does not weight the input twice. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
InputToNode.fit derived the sparse fan-in from round(hidden_layer_size * sparsity), a formula borrowed from NodeToNode (where the input dimension equals hidden_layer_size). For InputToNode the input dimension is n_features, so with k_in set the fan-in became round(hidden_layer_size * k_in / n_features) instead of k_in: it crashed when that exceeded n_features (e.g. k_in=3, n_features=6, hidden=20 -> fan_in=10 > 6) and was silently wrong otherwise (e.g. digits k_in=5 -> fan_in=4). Set fan_in = k_in when k_in is given (k_in inputs per hidden node), matching the semantics the torch backend already implements. The default (sparsity-based, no k_in) path is unchanged. Extend the sparse test to assert each hidden node receives exactly k_in inputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add pyrcn.backend.IncrementalRidge, the torch companion of IncrementalRegression: accumulate K = sum(Z^T Z) and xTy = sum(Z^T y) over batches, then solve (K + alpha I) W = xTy once. This reproduces the sequence-fit flow, where each sequence contributes to the statistics (postpone_inverse=True) and the final call triggers the single solve. fit_intercept folds a bias by appending a ones column (as the legacy readout does); statistics are additive, so disjoint readouts merge with "+" (map-reduce). State is on-device; weights carry no gradient in the closed-form path. Parity vs the legacy IncrementalRegression by feeding identical Z and y: single-batch fit (1-D and multi-target, with/without intercept), postpone-then-solve sequence flow, and merge == concatenated fit, all in float64. Suite 155 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add pyrcn.backend._bridge: given a fitted InputToNode / NodeToNode block or an IncrementalRegression config, build the equivalent torch module (InputFeatureMap / Reservoir / EulerReservoir / IncrementalRidge) with the block's weights injected, reproducing the block's NumPy transform within precision. Plus fast-path membership predicates (input/node/regressor_is_backable) decided by exact type, so blocks with bespoke transforms (BatchIntrinsicPlasticity, FeatureUnion, external Ridge, normalize=True) correctly take the NumPy fallback. This is the shared foundation for wiring ESN*/ELM* onto the torch backend (fast path) while keeping arbitrary sub-estimators on the legacy path. Tests reproduce transform for leaky/bidirectional/Euler reservoirs, the input feature map, and the ridge readout, and check classification. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lback) ELMRegressor.fit/predict route through the torch InputFeatureMap + IncrementalRidge (via the config->backend bridge) when the input stage and regressor are the torch-backable pyrcn defaults; arbitrary sub-estimators (FeatureUnion input, external Ridge, normalize=True) and partial_fit keep the legacy numpy path. Behavior-preserving (float64; only precision differs). ELMClassifier inherits the fast path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lback) ESNRegressor.fit/predict (and ESNClassifier via inheritance) route the input feature map, reservoir recurrence, and closed-form readout through the torch backend when all components are the torch-backable pyrcn defaults; sequence mode accumulates the readout per sequence with a fresh zero state, matching the legacy loop. Arbitrary sub-estimators and partial_fit keep the numpy path. Behavior-preserving (float64). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a training-only washout (int, default 0) that drops the first `washout` reservoir states and matching targets per sequence before fitting the readout, discarding the start transient. Add predict(initial_state=None, return_state=False): seed the reservoir from a given state and optionally return the final state(s), exposing the backend's state-carry primitive (splitting a sequence and carrying the final state reproduces the whole-sequence pass). ESN-only (ELM has no reservoir state); requires the torch backend, raising NotImplementedError on the numpy fallback. washout survives clone via get_params(deep=False) and is validated in _validate_hyperparameters. Defaults are behavior-preserving. Suite 183 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the Parallel/delayed map-reduce branches from ESN _sequence_fit and ELM chunked fit: the torch fast path batches the whole computation, and the numpy fallback now accumulates sequences/chunks serially. n_jobs is kept in the fit signatures for API compatibility but ignored (no test exercises the fallback with n_jobs > 1). Drop the now-unused joblib and clone imports. __add__/__radd__ (regressor stat merge) are retained as public methods. Behavior-preserving; suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_backend_parity_estimators: prove the torch fast path equals the numpy fallback for identical configs across all four estimators and both sequence/non-sequence modes, using a native vs forced-fallback twin (the fallback wraps the input in a single-transformer FeatureUnion). Max observed difference ~3e-9 (float64 round-off) everywhere. test_model_selection_esn: GridSearchCV / RandomizedSearchCV / SequentialSearchCV / component-swap grids over bare-name params all fit and keep best_estimator_._use_torch True through clone, confirming model selection is unchanged and preserves the fast path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Scope note: the numpy reservoir engine (NodeToNode.transform / _pass_through_recurrent_weights), concatenate_sequences, and the numpy weight init cannot be retired under the fast-path + fallback design - the fast path still uses concatenate_sequences and numpy-init weights (injected into torch), and the fallback path + standalone block tests + parity tests still exercise the numpy reservoir loop. So A6 is a finalize pass, not a legacy removal; true retirement waits until the companion pattern (P1+) makes the blocks torch-native. - Remove ESN*/ELM* __add__/__radd__: only the dropped joblib sum(reg) used them, and they read regressor._K which does not exist on a torch fast-path estimator (dead + broken). Also drop two dead commented lines. - Add tests/test_backend_device.py: the backend modules run on CPU and honour device=; the CUDA path is skipped when no GPU is available. - Drop joblib from explicit dependencies (no longer imported directly; scikit-learn still pulls it transitively). Full suite (as CI runs it, no deselect) 196 passed / 3 skipped; flake8 src/pyrcn tests + mypy src/pyrcn clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
InputToNode/NodeToNode raised a ValueError for an unsupported activation
that dumped the ACTIVATIONS dict with function objects and memory
addresses (non-deterministic, unreadable) and was missing a space
("supported.Supported"). List the sorted supported activation names
instead. Validation stays in _validate_hyperparameters (fit-time, per
sklearn conventions), not __init__.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename pyrcn.backend -> pyrcn.nn and expose it as a documented public API (D13), built from the Phase-A companion modules (not the old removed draft). Weight initializers move to a pyrcn.nn.init submodule (mirroring torch.nn.init); the reservoir layers/cells, InputFeatureMap and IncrementalRidge are the top-level public surface. The frontend glue (_bridge) and the cell base (_ReservoirCell) stay private. Remove the "private / implementation detail / not public yet" disclaimer from the package docstring and add a pure-PyTorch usage example (verified via doctest). Register pyrcn.nn in the top-level package. Estimator and test imports updated backend -> nn / nn.init. No behavior change. Full suite 195 passed / 3 skipped; flake8 src/pyrcn tests + mypy src/pyrcn clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add docs/source/api/pyrcn.nn.rst (reservoir layers/cells, feature map, readout, and the pyrcn.nn.init initializers) following the existing autodoc pattern, and add it to the API toctree. Not build-verified locally (sphinx not installed); renders on Read the Docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the gradient-mode readout components to pyrcn.nn: - LinearReadout: a trainable nn.Linear-based readout (predict returns no-grad outputs), the counterpart of the closed-form IncrementalRidge. - train_readout: a minimal optimizer loop (optimizer/loss selected by name, weight_decay = ridge strength, optional mini-batching). Both are public. Consistency check (D9): with fixed, whitened features a gradient-trained LinearReadout converges to the closed-form ridge solution (max abs ~1e-6). Plus loss-decrease, shape/no-grad, and unknown optimizer/loss validation tests. 5 tests, ~4s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ELMRegressor/ELMClassifier gain solver="gradient": when all components are torch-backable, train a LinearReadout with an optimizer loop (optimizer / learning_rate / epochs / batch_size params) instead of the closed-form ridge solve; the regressor supplies the readout config (fit_intercept, alpha -> weight_decay). Defaults (solver="closed_form") are behavior-preserving; the new params are validated and flow through get_params(deep=False) for clone/GridSearch. Gradient mode requires native components (raises NotImplementedError otherwise). predict squeezes a trailing singleton so 1-D targets match the closed-form output shape. Tight gradient-vs-closed-form equivalence is covered on well-conditioned features in test_backend_gradient; here we assert the ELM gradient path learns + the structural contracts. Full suite 205 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ESNRegressor/ESNClassifier gain solver="gradient": with a fixed reservoir, compute states once (dropping washout per sequence) and train a LinearReadout with an optimizer loop instead of the closed-form ridge solve. Sequence mode concatenates all per-sequence states/targets to train the shared readout. Defaults (solver="closed_form") behavior-preserving; new params validated + in get_params for clone/GridSearch. Gradient mode requires native components. predict squeezes a trailing singleton so 1-D targets match the closed-form shape. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Seed the readout's weight init and the mini-batch shuffle from the estimator's random_state so gradient fits are deterministic: - LinearReadout gains a `generator` arg (reproducible weight init; the nn.Linear default draws from torch's global RNG). - train_readout gains a `generator` arg (seeds torch.randperm). - new pyrcn.nn.torch_generator(random_state) builds a seeded torch.Generator from an int / RandomState / None (public helper). - ESN*/ELM* gradient fits derive the generator from the input block's random_state and thread it through both. Same random_state now yields identical gradient predictions (new reproducibility tests); no global torch seeding needed. Suite 212 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a test demonstrating that a gradient-trained readout reaches the closed-form ridge solution even through the raw, ill-conditioned reservoir map: minimizing the same ridge objective (SSE + alpha||params||^2) with a second-order optimizer (L-BFGS) matches IncrementalRidge to ~1e-4. This makes concrete that gradient==closed-form is a matter of matching the objective and the optimizer/conditioning, not a fundamental barrier. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ESN* gain trainable_reservoir (bool, default False): with solver="gradient" the reservoir's recurrent weights are unfrozen and optimized jointly with the readout by backpropagating through the recurrence. The RC init is the starting point (weights injected then set requires_grad=True via the new Reservoir/EulerReservoir.set_recurrent_trainable). Each epoch recomputes the reservoir states (the fixed input feature map is applied once and detached); full-batch updates. Invalid trainable+closed_form is rejected in _validate_hyperparameters. Reproducible via random_state. predict now runs under torch.no_grad() (a trainable reservoir tracks grad otherwise). ESN-only (ELM has no reservoir). Sequence + non-sequence + classifier. Full suite 218 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the examples in line with the current codebase (they predated the sklearn-1.6 modernization). High-level API was preserved, so most only needed small fixes; no torch / pyrcn.nn is introduced. - tud_colors (removed from pyrcn.util) -> standard matplotlib colors (input-to-node, digits-kmeans; a local matplotlib tab10 palette in experiments / preprocessing-mnist). - from src.pyrcn... -> from pyrcn... (digits-kmeans). - loguniform moved sklearn.utils.fixes -> scipy.stats (mnist_regressors, musical_note_prediction, f0_extraction, sklearn_autoencoder, +notebooks). - make_scorer(needs_proba=True) -> response_method='predict_proba' (Video_Door_*, musical_note_prediction) for sklearn 1.9. - hidden_layer_state is now a method: esn.hidden_layer_state(X)[...] (mackey-glass-t17). - Removed SeqToLabelESNClassifier -> ESNClassifier; wash_out -> washout; dropped obsolete 'continuation' (spoken_digit_recognition*). - digits.ipynb: deleted the old removed pyrcn.nn/torch draft cells. - esn_impulse_responses.ipynb: dropped dead imports of removed local modules; setup_local.ipynb: fixed the ELMRegressor->ELMClassifier typo (GitHub #61); fixed a pre-existing syntax error in spoken_digit_recognition_auto_encoder.ipynb. All .py import cleanly (except optional third-party libs: librosa/tqdm/ madmom/mir_eval/IPython) and all notebooks remain valid JSON. Data-heavy examples were API-updated but not executed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
set_matplotlib_formats moved from IPython.display to matplotlib_inline.backend_inline in modern IPython/Jupyter. Update the four notebooks that import it so they run under a current Jupyter stack. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- trainable_input (bool, default False): with solver="gradient", train the input feature-map weights too (via InputFeatureMap.set_input_trainable), alone or together with trainable_reservoir -> a fully trainable RNN (input + reservoir + readout). Requires solver="gradient" (rejected with closed-form, alongside trainable_reservoir). A fixed feature map is still precomputed+detached; a trainable one is recomputed each epoch so grads reach its weights. - BPTT mini-batching: _torch_trainable_fit now batches over sequences (batch_size sequences/step, shuffled via the seeded generator; None = full-batch). Non-sequence stays full-BPTT over the single sequence. - Both are reproducible via random_state; new params flow through get_params(deep=False) and the classifier __init__. Full suite 222 passed / 3 skipped; flake8 + mypy clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the docs up to date with the upgraded package and make them build cleanly (0 Sphinx warnings/errors, down from 157). API reference - Add pyrcn.nn API page covering the new torch classes: Reservoir, EulerReservoir, LeakyESNCell, EulerESNCell, InputFeatureMap, IncrementalRidge, LinearReadout, train_readout, torch_generator, and the pyrcn.nn.init initializers. - Complete the API toctree: drop the dangling pyrcn.cluster entry, add the previously-missing pages (model_selection, metrics, datasets, projection, preprocessing, postprocessing, util). - Document the new estimator params on ESN*/ELM* (solver, optimizer, learning_rate, epochs, batch_size, trainable_reservoir, trainable_input, washout). conf.py - napoleon_use_ivar (removes duplicate-object-description warnings for Attributes) and exclude sklearn's dynamic set_*_request methods (removes meta-estimator / metadata_routing cross-ref noise). Docstring RST cleanup (docstrings only, no code changes) - Fix bullet/definition-list/indentation issues and neutralize dangling scikit-learn :ref:/:term:/:func: cross-references and unreferenced footnotes across base.blocks, datasets, nn, metrics, model_selection, util. Narrative docs - getting_started: refresh the doctests (add EulerNodeToNode, reshape the predict input, renumber the training steps); define the previously undefined :ref: labels (installation guide, whats rc); fix index and installation RST; point installation at pyproject.toml. - No em dashes in any docstrings. docs/requirements.txt modernized so the docs build on the current Sphinx stack. Suite 222 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#54 follow-up: drop the NumPy-only softmax/softplus from base.ACTIVATIONS (and their isolated tests). They were not supported by the torch backend, not invertible for BatchIntrinsicPlasticity, and never used as a block or estimator activation (so they would break on the torch fast path). The supported activation set is now the same five in both backends (tanh/identity/logistic/relu/bounded_relu), which the validation error message and the Literal type hints already reflect. Gradient extensibility: expand pyrcn.nn.OPTIMIZERS to {adam, adamw, sgd, rmsprop, adagrad} and LOSSES to {mse, mae, huber}, and add a user-facing `loss` param to ESN*/ELM* (validated, in get_params(deep=False), threaded through the gradient readout fit and the BPTT loop, and documented). optimizer validation now checks the full set. Full suite 220 passed / 3 skipped; flake8 + mypy clean; docs build clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # docs/source/installation.rst # pyproject.toml # src/pyrcn/__init__.py # src/pyrcn/base/_activations.py # src/pyrcn/base/blocks/_input_to_node.py # src/pyrcn/base/blocks/_node_to_node.py # src/pyrcn/echo_state_network/_esn.py # src/pyrcn/extreme_learning_machine/_elm.py # src/pyrcn/metrics/_classification.py # src/pyrcn/metrics/_regression.py # src/pyrcn/nn/__init__.py # src/pyrcn/nn/_activations.py # src/pyrcn/nn/init.py # src/pyrcn/util/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a PyTorch backend to PyRCN. The existing API is reused, but the PyTorch backend makes it easier to use PyRCN for forecasting, addressing issue #56 with a new API.