Backend redesign (#63) * Remove torch dependency and nn subpackage The nn/ PyTorch subpackage was an early draft and is out of scope for the scikit-learn compatibility modernization. Remove it along with the torch, torchvision and torchaudio dependencies so that `import pyrcn` no longer requires PyTorch. - Delete src/pyrcn/nn/ - Drop nn from the top-level package imports and __all__ - Strip torch from util.seed_everything (keep random/numpy seeding) - Remove torch/torchvision/torchaudio from pyproject dependencies Co-Authored-By: Claude Opus 4.8 (1M context) * Replace private/deprecated sklearn and scipy imports with public APIs Prepare for scikit-learn >= 1.6 by removing reliance on private or deprecated symbols. No behavioral change. - base/_activations.py: define the inplace activation table locally instead of importing and mutating the private sklearn.neural_network._base.ACTIVATIONS global. - Drop the deprecated @_deprecate_positional_args decorator throughout; __init__/metric signatures are already keyword-only via `*`. - echo_state_network, extreme_learning_machine: use public sklearn.base.RegressorMixin in type hints instead of the private sklearn.linear_model._base.LinearModel. - node_to_node: import csr_matrix from scipy.sparse, not the removed scipy.sparse.csr submodule. - Replace the typing_extensions / sys.version_info shims with plain typing (Python >= 3.10) and drop now-dead `import sys`. Co-Authored-By: Claude Opus 4.8 (1M context) * Migrate estimators to the scikit-learn 1.6+ developer API Custom estimators relied on APIs removed or deprecated in scikit-learn 1.6: - BaseEstimator._validate_data / _check_n_features were removed. Replace them with the public sklearn.utils.validation.validate_data function. - The estimator type is now derived from tags and _estimator_type is deprecated. Per the official developer guide, mixins must precede BaseEstimator in the MRO so __sklearn_tags__ resolves the estimator type correctly; reorder the bases of every estimator accordingly. This restores is_regressor / is_classifier detection. - Coates: replace the getattr(clusterer, "_estimator_type") check with the public sklearn.base.is_clusterer helper. No behavioral change intended. The suite goes from 50 to 21 failing; the remaining failures are all in the sequence-aware metrics wrappers and are addressed separately. Co-Authored-By: Claude Opus 4.8 (1M context) * Refactor metrics to use the public scikit-learn API The sequence-aware metric wrappers called scikit-learn's private _check_targets / _check_reg_targets helpers, whose signatures changed in 1.6 (sample_weight became positional), causing unpacking and validation errors. Replace those calls: flatten the per-sequence arrays with the public check_consistent_length + np.concatenate and derive the target type via the public sklearn.utils.multiclass.type_of_target. The public metric performs its own target validation, so the private helpers are no longer needed. Also route mean_squared_error(squared=False) to the public root_mean_squared_error, since the `squared` parameter was removed from sklearn.metrics.mean_squared_error in 1.6. PyRCN's own `squared` argument is preserved. No behavioral change intended. Co-Authored-By: Claude Opus 4.8 (1M context) * Update repository URLs to the PlasmaControl organization The repository moved from TUD-STKS/PyRCN to PlasmaControl/PyRCN. Update all GitHub and mybinder links in the README, docs and example notebook. Links to other TUD-STKS repositories (Automatic-Music-Transcription, gci_estimation) are left unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) * Modernize packaging metadata for the scikit-learn 1.6+ upgrade - Bump requires-python to >=3.10 (the floor for scikit-learn >= 1.6) and add a scikit-learn>=1.6 dependency floor; list numpy/scipy/joblib/pandas explicitly. - Fix the license classifier (was MIT; the LICENSE file and all source headers are BSD-3-Clause). - Single source of truth for the version: pyproject reads it dynamically from pyrcn._version, and _version.py is set to 0.0.18 (was 0.0.17post1). - Add test and examples optional-dependency extras. - Remove the redundant setup.cfg (pyproject is now canonical). - Fix requirements.txt: the malformed `requests[...]` lines were meant to be matplotlib/seaborn/ipywidgets/ipympl/tqdm; drop obsolete typing-extensions. - mypy: ignore_missing_imports for third-party packages without type stubs. - Read the Docs: build with Python 3.12 (>=3.10 is now required). Co-Authored-By: Claude Opus 4.8 (1M context) * Modernize CI and test configuration - CI matrix: Python 3.10-3.13 (drop 3.9); bump actions/checkout to v4 and actions/setup-python to v5 in both workflows. - Install the package via the `test` extra instead of an ad-hoc plugin list plus requirements.txt; run flake8 (blocking) and mypy (informational, the remaining strict-typing findings are deferred to the code-unification pass). - Remove pytest.ini, whose addopts required cov/mypy plugins just to collect tests; pyproject's [tool.pytest.ini_options] keeps a bare `pytest` working, and coverage is requested explicitly in CI. - Fix a flake8 E231 in tests so linting can cover the test suite too. Co-Authored-By: Claude Opus 4.8 (1M context) * Modernize type hints and unify imports across the package Apply modern Python 3.10 idioms (the new requires-python floor) consistently: - Type hints: PEP 604 unions (Union[A, B] -> A | B, Optional[A] -> A | None) and PEP 585 builtin generics (Dict/List/Tuple -> dict/list/tuple); move Callable/Iterable to collections.abc. Applied with pyupgrade --py310-plus. - Replace str.format(...) calls with f-strings. - Sort and group imports consistently (isort) and add `from __future__ import annotations` to every module. - Reflow the multi-line signatures whose alignment shifted. Purely mechanical; no behavioral change. flake8 clean, suite unchanged (85 passed / 2 skipped). Co-Authored-By: Claude Opus 4.8 (1M context) * Clean up outdated coding practices - Remove the dead, commented-out NonlinearVectorAutoregression class that had been left as a module-level string literal in blocks/_input_to_node.py. - Replace `raise BaseException(...)` with `raise TypeError(...)` in the ESN and ELM partial_fit guards; BaseException should never be raised directly. - Fix a stray `%s` left in a NodeToNode hidden_layer_size error message. No behavioral change (aside from the more specific exception type on an invalid-regressor error path). flake8 clean, suite unchanged (85 passed). Co-Authored-By: Claude Opus 4.8 (1M context) * Fix non-reproducible sparse weight initialization _make_sparse takes a random_state argument but drew the retained-weight indices from the global numpy RNG (np.random.choice), so the sparsity mask was not reproducible from a fixed seed. Use the passed random_state. This makes the sparse weight masks deterministic w.r.t. random_state (the exact mask for a given seed changes); the test suite is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) * Follow scikit-learn conventions for constructor parameters - Coates no longer calls check_random_state in __init__; it stores the raw random_state and resolves it in _extract_random_patches, so get_params / clone round-trip correctly. Numerically equivalent for a single fit. - Replace the mutable default arguments Coates(clusterer=KMeans()) and lorenz(x_0=[1.0, 1.0, 1.0]) with None sentinels, constructing the default inside the method/function to avoid a shared mutable instance. Co-Authored-By: Claude Opus 4.8 (1M context) * Make the package pass strict mypy and enforce it in CI Resolve the remaining strict-mypy findings so `mypy src/pyrcn` is clean, and drop the informational `continue-on-error` from the CI type-check step. - util.batched: annotate the generator return as Iterator[tuple]. - util.value_to_tuple: correct the signature (value may be a tuple; size is an int) and always return. - BatchIntrinsicPlasticity: initialize self._m/_c as floats (they are assigned float distribution parameters). - Coates: type the pooling axis as `int | None`, and assert self.clusterer is not None at the post-fit use sites (it is resolved to KMeans() in fit). No behavioral change. Co-Authored-By: Claude Opus 4.8 (1M context) * Modernize the test suite to match the package Apply the same modernization used for the package to tests/: PEP 604 / PEP 585 type syntax and f-strings (pyupgrade --py310-plus), consistent import sorting with `from __future__ import annotations` (isort), and removal of unused imports (autoflake). No test logic changed; suite unchanged (85 passed). Co-Authored-By: Claude Opus 4.8 (1M context) * Add backend redesign plan Living design document for replacing the compute backend with PyTorch while keeping the scikit-learn-compatible frontend. Captures the audit, the design decisions (data model, engine, numerics, dependency, FE/BE boundary, state semantics, params, training modes, block<->module companion pattern, staged public torch API) and a phased implementation roadmap. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A0: declare torch dependency and add backend package scaffolding Re-add torch as a core dependency (D4) and create the private pyrcn.backend package that will hold the PyTorch compute layer (reservoir engine, input feature map, readout) behind the unchanged scikit-learn frontend. Co-Authored-By: Claude Opus 4.8 (1M context) * Plan: note to drop the private-API disclaimer when pyrcn.nn goes public Add a TODO on the P1 (go-public) roadmap step to remove the "private / implementation-detail" disclaimers from the backend docstrings once the PyTorch surface is promoted to the public pyrcn.nn API. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A1: add check_sequences normalizer (TDD) Introduce pyrcn.util.check_sequences + SequenceBatch, the single validated entry point that normalizes the accepted public input forms into the canonical backend representation: - accepts a 2-D single sequence, a list/object-array of 2-D sequences, or a 3-D equal-length batch; - returns a zero-padded (n_sequences, max_length, n_features) array plus a per-sequence lengths vector; - a 2-D (T, F) input normalizes to a single sequence (N=1, single_sequence flag) to preserve the current 2-D behavior (INV-1); - resolves the task (sequence-to-sequence vs sequence-to-value) from target shapes, with an explicit task= override for the ambiguous case. Test-driven (tests/test_sequences.py). Not wired into the estimators yet, so the existing suite is unaffected (93 passed / 2 skipped). Will replace concatenate_sequences + the _check_if_sequence* heuristics at integration (A4). Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A2: torch reservoir module with NodeToNode parity (TDD) Add pyrcn.backend.Reservoir, a batched leaky-integrator reservoir as a PyTorch nn.Module implementing the same recurrence as the legacy NumPy NodeToNode: h_t = (1 - leakage) * h_{t-1} + leakage * f(x_t + spectral_radius * (h_{t-1} @ W_hh)) Recurrent weights are a dense Parameter (requires_grad=False) assigned via set_recurrent_weights; supports initial_state -> final_state and batched (n_sequences, length, hidden_size) input. Parity is proven by initializing the recurrent weights with PyRCN's own NodeToNode (dense and sparse k_rec variants, spectral-radius-normalized) and injecting the identical matrix into the torch module: single-step difference is exactly 0.0 and full float64 trajectories agree to ~1e-16 across all activations. Suite: 114 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) * Plan: document torch-module reuse strategy (D14) Record how the torch modules are built: subclass nn.Linear (InputToNode) and nn.RNNCell (NodeToNode), reusing torch's fused single-step op and adding only leaky integration + the extra activations; the two-activation handling; and the deferred fused ESNCell / whole-sequence fast path. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A2: rebuild the reservoir on nn.RNNCell (D14 reuse) Replace the hand-written recurrence with LeakyESNCell(nn.RNNCell): the tanh/relu path reuses torch's fused single-step cell op via super().forward and adds only leaky integration; logistic/identity/bounded_relu use affine + activation. weight_ih is a frozen identity (the reservoir input is added directly) and spectral_radius is folded into weight_hh. Reservoir is now a thin layer that loops the cell (lengths-agnostic, initial_state -> final_state). Behaviorally identical: the NodeToNode parity tests (weights injected from a real NodeToNode, float64) still hold to ~1e-16; suite 114 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A2: bidirectional reservoir with NodeToNode parity Add bidirectional=True to Reservoir: run the same cell forward and over the time-reversed input (shared weights, flip back, concatenate on the feature axis), matching NodeToNode.transform's fw+bw hstack. Parity-tested against a real bidirectional NodeToNode (float64, ~1e-16). initial_state carry is not combined with bidirectional (raises). Suite 115 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A2: torch-native reservoir weight init (TDD) Add pyrcn.backend init functions for the recurrent matrix: - normal_recurrent_weights: normal (optionally sparse via fan_in), normalized to unit spectral radius (spectral_normalize uses torch.linalg.eigvals); - simple_cycle_weights / delay_line_weights / delay_line_feedback_weights: the Rodan minimum-complexity topologies (deterministic structured matrices). Verified numerically: unit spectral radius for random/sparse, exact fan_in per column, exact structure + resulting spectral radius for SCR/DLR/DLRB, and reproducibility via a torch Generator. Parity with the legacy path stays via weight injection; these are for real initialization. Suite 121 passed. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A2: Euler + Hebbian reservoir variants (TDD) - EulerESNCell / EulerReservoir: the EuSN update h' = h + epsilon * f(x + h @ (recurrent_scaling*W + gamma*I)), reusing the fused cell op; parity-tested against EulerNodeToNode (weights injected, float64, ~1e-16). Add antisymmetric_recurrent_weights for its torch-native init (verified antisymmetric + reproducible). - Factor a shared _ReservoirCell base (identity weight_ih, param freeze, activation dispatch) and a shared _iterate loop, so the leaky and Euler cells do not duplicate machinery. - Hebbian: it only *learns* the recurrent weights in fit; transform is the standard recurrence, so its torch form is the existing Reservoir with the NumPy-learned weights injected. Parity-tested for all four learning rules. Suite 133 passed / 2 skipped; flake8 + mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) * Phase A2: torch InputFeatureMap (nn.Linear) + input init (TDD) Add pyrcn.backend.InputFeatureMap, the torch companion of InputToNode: an nn.Linear subclass applying PyRCN's input_scaling/input_shift/bias_scaling/ bias_shift (folded into weight/bias) and the input activation. Parity-tested against InputToNode.transform (weights injected, float64, ~1e-16) across all activations and scaling/shift settings. Add torch-native input init: uniform_input_weights (optionally sparse, fan_in per hidden node), uniform_bias_weights, and bernoulli_input_weights (signed constants for minimum-complexity ESNs); property-tested. Factor the shared activation table into pyrcn.backend._activations and reuse it from the reservoir cells (no duplicated activation dicts). Suite 147 passed. Note: InputFeatureMap owns the input weights; the reservoir cell's weight_ih stays a frozen identity, so chaining them does not weight the input twice. Co-Authored-By: Claude Opus 4.8 (1M context) * Fix InputToNode k_in: connect k_in inputs per hidden node InputToNode.fit derived the sparse fan-in from round(hidden_layer_size * sparsity), a formula borrowed from NodeToNode (where the input dimension equals hidden_layer_size). For InputToNode the input dimension is n_features, so with k_in set the fan-in became round(hidden_layer_size * k_in / n_features) instead of k_in: it crashed when that exceeded n_features (e.g. k_in=3, n_features=6, hidden=20 -> fan_in=10 > 6) and was silently wrong otherwise (e.g. digits k_in=5 -> fan_in=4). Set fan_in = k_in when k_in is given (k_in inputs per hidden node), matching the semantics the torch backend already implements. The default (sparsity-based, no k_in) path is unchanged. Extend the sparse test to assert each hidden node receives exactly k_in inputs. Co-Authored-By: Claude Opus 4.8 * Phase A3: torch-native closed-form ridge readout (TDD) Add pyrcn.backend.IncrementalRidge, the torch companion of IncrementalRegression: accumulate K = sum(Z^T Z) and xTy = sum(Z^T y) over batches, then solve (K + alpha I) W = xTy once. This reproduces the sequence-fit flow, where each sequence contributes to the statistics (postpone_inverse=True) and the final call triggers the single solve. fit_intercept folds a bias by appending a ones column (as the legacy readout does); statistics are additive, so disjoint readouts merge with "+" (map-reduce). State is on-device; weights carry no gradient in the closed-form path. Parity vs the legacy IncrementalRegression by feeding identical Z and y: single-batch fit (1-D and multi-target, with/without intercept), postpone-then-solve sequence flow, and merge == concatenated fit, all in float64. Suite 155 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 * Plan: mark Phase A0-A3 workstreams done Co-Authored-By: Claude Opus 4.8 * Phase A4: config-block -> torch-backend bridge (TDD) Add pyrcn.backend._bridge: given a fitted InputToNode / NodeToNode block or an IncrementalRegression config, build the equivalent torch module (InputFeatureMap / Reservoir / EulerReservoir / IncrementalRidge) with the block's weights injected, reproducing the block's NumPy transform within precision. Plus fast-path membership predicates (input/node/regressor_is_backable) decided by exact type, so blocks with bespoke transforms (BatchIntrinsicPlasticity, FeatureUnion, external Ridge, normalize=True) correctly take the NumPy fallback. This is the shared foundation for wiring ESN*/ELM* onto the torch backend (fast path) while keeping arbitrary sub-estimators on the legacy path. Tests reproduce transform for leaky/bidirectional/Euler reservoirs, the input feature map, and the ridge readout, and check classification. Co-Authored-By: Claude Opus 4.8 * Phase A4: wire ELM estimators onto the torch backend (fast path + fallback) ELMRegressor.fit/predict route through the torch InputFeatureMap + IncrementalRidge (via the config->backend bridge) when the input stage and regressor are the torch-backable pyrcn defaults; arbitrary sub-estimators (FeatureUnion input, external Ridge, normalize=True) and partial_fit keep the legacy numpy path. Behavior-preserving (float64; only precision differs). ELMClassifier inherits the fast path. Co-Authored-By: Claude Opus 4.8 * Phase A4: wire ESN estimators onto the torch backend (fast path + fallback) ESNRegressor.fit/predict (and ESNClassifier via inheritance) route the input feature map, reservoir recurrence, and closed-form readout through the torch backend when all components are the torch-backable pyrcn defaults; sequence mode accumulates the readout per sequence with a fresh zero state, matching the legacy loop. Arbitrary sub-estimators and partial_fit keep the numpy path. Behavior-preserving (float64). Co-Authored-By: Claude Opus 4.8 * Plan: mark A4 core (estimator torch integration) done Co-Authored-By: Claude Opus 4.8 * Phase A4b: ESN washout + predict initial_state / return_state Add a training-only washout (int, default 0) that drops the first `washout` reservoir states and matching targets per sequence before fitting the readout, discarding the start transient. Add predict(initial_state=None, return_state=False): seed the reservoir from a given state and optionally return the final state(s), exposing the backend's state-carry primitive (splitting a sequence and carrying the final state reproduces the whole-sequence pass). ESN-only (ELM has no reservoir state); requires the torch backend, raising NotImplementedError on the numpy fallback. washout survives clone via get_params(deep=False) and is validated in _validate_hyperparameters. Defaults are behavior-preserving. Suite 183 passed / 2 skipped. Co-Authored-By: Claude Opus 4.8 * Plan: mark A4b (washout + state carry) done Co-Authored-By: Claude Opus 4.8 * Phase A5: drop the joblib/loky sequence-parallel path Remove the Parallel/delayed map-reduce branches from ESN _sequence_fit and ELM chunked fit: the torch fast path batches the whole computation, and the numpy fallback now accumulates sequences/chunks serially. n_jobs is kept in the fit signatures for API compatibility but ignored (no test exercises the fallback with n_jobs > 1). Drop the now-unused joblib and clone imports. __add__/__radd__ (regressor stat merge) are retained as public methods. Behavior-preserving; suite green. Co-Authored-By: Claude Opus 4.8 * Phase A5: parity harness + model-selection tests on the torch backend test_backend_parity_estimators: prove the torch fast path equals the numpy fallback for identical configs across all four estimators and both sequence/non-sequence modes, using a native vs forced-fallback twin (the fallback wraps the input in a single-transformer FeatureUnion). Max observed difference ~3e-9 (float64 round-off) everywhere. test_model_selection_esn: GridSearchCV / RandomizedSearchCV / SequentialSearchCV / component-swap grids over bare-name params all fit and keep best_estimator_._use_torch True through clone, confirming model selection is unchanged and preserves the fast path. Co-Authored-By: Claude Opus 4.8 * Plan: mark A5 (parity + model-selection tests, drop joblib) done Co-Authored-By: Claude Opus 4.8 * Phase A6: finalize - remove dead merge operators, device test, deps Scope note: the numpy reservoir engine (NodeToNode.transform / _pass_through_recurrent_weights), concatenate_sequences, and the numpy weight init cannot be retired under the fast-path + fallback design - the fast path still uses concatenate_sequences and numpy-init weights (injected into torch), and the fallback path + standalone block tests + parity tests still exercise the numpy reservoir loop. So A6 is a finalize pass, not a legacy removal; true retirement waits until the companion pattern (P1+) makes the blocks torch-native. - Remove ESN*/ELM* __add__/__radd__: only the dropped joblib sum(reg) used them, and they read regressor._K which does not exist on a torch fast-path estimator (dead + broken). Also drop two dead commented lines. - Add tests/test_backend_device.py: the backend modules run on CPU and honour device=; the CUDA path is skipped when no GPU is available. - Drop joblib from explicit dependencies (no longer imported directly; scikit-learn still pulls it transitively). Full suite (as CI runs it, no deselect) 196 passed / 3 skipped; flake8 src/pyrcn tests + mypy src/pyrcn clean. Co-Authored-By: Claude Opus 4.8 * Plan: mark A6 (finalize) done; Phase A complete Co-Authored-By: Claude Opus 4.8 * Improve activation validation error message (#54) InputToNode/NodeToNode raised a ValueError for an unsupported activation that dumped the ACTIVATIONS dict with function objects and memory addresses (non-deterministic, unreadable) and was missing a space ("supported.Supported"). List the sorted supported activation names instead. Validation stays in _validate_hyperparameters (fit-time, per sklearn conventions), not __init__. Co-Authored-By: Claude Opus 4.8 * P1: promote the torch backend to a public pyrcn.nn package Rename pyrcn.backend -> pyrcn.nn and expose it as a documented public API (D13), built from the Phase-A companion modules (not the old removed draft). Weight initializers move to a pyrcn.nn.init submodule (mirroring torch.nn.init); the reservoir layers/cells, InputFeatureMap and IncrementalRidge are the top-level public surface. The frontend glue (_bridge) and the cell base (_ReservoirCell) stay private. Remove the "private / implementation detail / not public yet" disclaimer from the package docstring and add a pure-PyTorch usage example (verified via doctest). Register pyrcn.nn in the top-level package. Estimator and test imports updated backend -> nn / nn.init. No behavior change. Full suite 195 passed / 3 skipped; flake8 src/pyrcn tests + mypy src/pyrcn clean. Co-Authored-By: Claude Opus 4.8 * P1: document the public pyrcn.nn API (Sphinx page) Add docs/source/api/pyrcn.nn.rst (reservoir layers/cells, feature map, readout, and the pyrcn.nn.init initializers) following the existing autodoc pattern, and add it to the API toctree. Not build-verified locally (sphinx not installed); renders on Read the Docs. Co-Authored-By: Claude Opus 4.8 * Plan: mark P1 (public pyrcn.nn) done Co-Authored-By: Claude Opus 4.8 * Phase B1 core: trainable LinearReadout + train_readout helper Add the gradient-mode readout components to pyrcn.nn: - LinearReadout: a trainable nn.Linear-based readout (predict returns no-grad outputs), the counterpart of the closed-form IncrementalRidge. - train_readout: a minimal optimizer loop (optimizer/loss selected by name, weight_decay = ridge strength, optional mini-batching). Both are public. Consistency check (D9): with fixed, whitened features a gradient-trained LinearReadout converges to the closed-form ridge solution (max abs ~1e-6). Plus loss-decrease, shape/no-grad, and unknown optimizer/loss validation tests. 5 tests, ~4s. Co-Authored-By: Claude Opus 4.8 * Phase B1: gradient solver for the ELM estimators ELMRegressor/ELMClassifier gain solver="gradient": when all components are torch-backable, train a LinearReadout with an optimizer loop (optimizer / learning_rate / epochs / batch_size params) instead of the closed-form ridge solve; the regressor supplies the readout config (fit_intercept, alpha -> weight_decay). Defaults (solver="closed_form") are behavior-preserving; the new params are validated and flow through get_params(deep=False) for clone/GridSearch. Gradient mode requires native components (raises NotImplementedError otherwise). predict squeezes a trailing singleton so 1-D targets match the closed-form output shape. Tight gradient-vs-closed-form equivalence is covered on well-conditioned features in test_backend_gradient; here we assert the ELM gradient path learns + the structural contracts. Full suite 205 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 * Phase B1: gradient solver for the ESN estimators ESNRegressor/ESNClassifier gain solver="gradient": with a fixed reservoir, compute states once (dropping washout per sequence) and train a LinearReadout with an optimizer loop instead of the closed-form ridge solve. Sequence mode concatenates all per-sequence states/targets to train the shared readout. Defaults (solver="closed_form") behavior-preserving; new params validated + in get_params for clone/GridSearch. Gradient mode requires native components. predict squeezes a trailing singleton so 1-D targets match the closed-form shape. Co-Authored-By: Claude Opus 4.8 * Plan: mark Phase B1 (gradient readout solver) done Co-Authored-By: Claude Opus 4.8 * Phase B1: make gradient training reproducible via random_state Seed the readout's weight init and the mini-batch shuffle from the estimator's random_state so gradient fits are deterministic: - LinearReadout gains a `generator` arg (reproducible weight init; the nn.Linear default draws from torch's global RNG). - train_readout gains a `generator` arg (seeds torch.randperm). - new pyrcn.nn.torch_generator(random_state) builds a seeded torch.Generator from an int / RandomState / None (public helper). - ESN*/ELM* gradient fits derive the generator from the input block's random_state and thread it through both. Same random_state now yields identical gradient predictions (new reproducibility tests); no global torch seeding needed. Suite 212 passed. Co-Authored-By: Claude Opus 4.8 * Plan: gradient training reproducible via random_state Co-Authored-By: Claude Opus 4.8 * Phase B1: tight gradient<->closed-form consistency demo (real reservoir) Add a test demonstrating that a gradient-trained readout reaches the closed-form ridge solution even through the raw, ill-conditioned reservoir map: minimizing the same ridge objective (SSE + alpha||params||^2) with a second-order optimizer (L-BFGS) matches IncrementalRidge to ~1e-4. This makes concrete that gradient==closed-form is a matter of matching the objective and the optimizer/conditioning, not a fundamental barrier. Co-Authored-By: Claude Opus 4.8 * Phase B2: trainable reservoir (end-to-end gradient training / BPTT) ESN* gain trainable_reservoir (bool, default False): with solver="gradient" the reservoir's recurrent weights are unfrozen and optimized jointly with the readout by backpropagating through the recurrence. The RC init is the starting point (weights injected then set requires_grad=True via the new Reservoir/EulerReservoir.set_recurrent_trainable). Each epoch recomputes the reservoir states (the fixed input feature map is applied once and detached); full-batch updates. Invalid trainable+closed_form is rejected in _validate_hyperparameters. Reproducible via random_state. predict now runs under torch.no_grad() (a trainable reservoir tracks grad otherwise). ESN-only (ELM has no reservoir). Sequence + non-sequence + classifier. Full suite 218 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 * Plan: mark Phase B2 (trainable reservoir) done Co-Authored-By: Claude Opus 4.8 * Update examples for the modernized API (no torch usage) Bring the examples in line with the current codebase (they predated the sklearn-1.6 modernization). High-level API was preserved, so most only needed small fixes; no torch / pyrcn.nn is introduced. - tud_colors (removed from pyrcn.util) -> standard matplotlib colors (input-to-node, digits-kmeans; a local matplotlib tab10 palette in experiments / preprocessing-mnist). - from src.pyrcn... -> from pyrcn... (digits-kmeans). - loguniform moved sklearn.utils.fixes -> scipy.stats (mnist_regressors, musical_note_prediction, f0_extraction, sklearn_autoencoder, +notebooks). - make_scorer(needs_proba=True) -> response_method='predict_proba' (Video_Door_*, musical_note_prediction) for sklearn 1.9. - hidden_layer_state is now a method: esn.hidden_layer_state(X)[...] (mackey-glass-t17). - Removed SeqToLabelESNClassifier -> ESNClassifier; wash_out -> washout; dropped obsolete 'continuation' (spoken_digit_recognition*). - digits.ipynb: deleted the old removed pyrcn.nn/torch draft cells. - esn_impulse_responses.ipynb: dropped dead imports of removed local modules; setup_local.ipynb: fixed the ELMRegressor->ELMClassifier typo (GitHub #61); fixed a pre-existing syntax error in spoken_digit_recognition_auto_encoder.ipynb. All .py import cleanly (except optional third-party libs: librosa/tqdm/ madmom/mir_eval/IPython) and all notebooks remain valid JSON. Data-heavy examples were API-updated but not executed here. Co-Authored-By: Claude Opus 4.8 * Examples: use matplotlib_inline for set_matplotlib_formats set_matplotlib_formats moved from IPython.display to matplotlib_inline.backend_inline in modern IPython/Jupyter. Update the four notebooks that import it so they run under a current Jupyter stack. Co-Authored-By: Claude Opus 4.8 * Phase B2+: trainable_input flag + BPTT mini-batching (ESN) - trainable_input (bool, default False): with solver="gradient", train the input feature-map weights too (via InputFeatureMap.set_input_trainable), alone or together with trainable_reservoir -> a fully trainable RNN (input + reservoir + readout). Requires solver="gradient" (rejected with closed-form, alongside trainable_reservoir). A fixed feature map is still precomputed+detached; a trainable one is recomputed each epoch so grads reach its weights. - BPTT mini-batching: _torch_trainable_fit now batches over sequences (batch_size sequences/step, shuffled via the seeded generator; None = full-batch). Non-sequence stays full-BPTT over the single sequence. - Both are reproducible via random_state; new params flow through get_params(deep=False) and the classifier __init__. Full suite 222 passed / 3 skipped; flake8 + mypy clean. Co-Authored-By: Claude Opus 4.8 * Plan: mark B2+ (trainable_input + BPTT mini-batching) done Co-Authored-By: Claude Opus 4.8 * Docs: complete + modernize the Sphinx documentation Bring the docs up to date with the upgraded package and make them build cleanly (0 Sphinx warnings/errors, down from 157). API reference - Add pyrcn.nn API page covering the new torch classes: Reservoir, EulerReservoir, LeakyESNCell, EulerESNCell, InputFeatureMap, IncrementalRidge, LinearReadout, train_readout, torch_generator, and the pyrcn.nn.init initializers. - Complete the API toctree: drop the dangling pyrcn.cluster entry, add the previously-missing pages (model_selection, metrics, datasets, projection, preprocessing, postprocessing, util). - Document the new estimator params on ESN*/ELM* (solver, optimizer, learning_rate, epochs, batch_size, trainable_reservoir, trainable_input, washout). conf.py - napoleon_use_ivar (removes duplicate-object-description warnings for Attributes) and exclude sklearn's dynamic set_*_request methods (removes meta-estimator / metadata_routing cross-ref noise). Docstring RST cleanup (docstrings only, no code changes) - Fix bullet/definition-list/indentation issues and neutralize dangling scikit-learn :ref:/:term:/:func: cross-references and unreferenced footnotes across base.blocks, datasets, nn, metrics, model_selection, util. Narrative docs - getting_started: refresh the doctests (add EulerNodeToNode, reshape the predict input, renumber the training steps); define the previously undefined :ref: labels (installation guide, whats rc); fix index and installation RST; point installation at pyproject.toml. - No em dashes in any docstrings. docs/requirements.txt modernized so the docs build on the current Sphinx stack. Suite 222 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 * Unify activation set + extend gradient optimizers/losses #54 follow-up: drop the NumPy-only softmax/softplus from base.ACTIVATIONS (and their isolated tests). They were not supported by the torch backend, not invertible for BatchIntrinsicPlasticity, and never used as a block or estimator activation (so they would break on the torch fast path). The supported activation set is now the same five in both backends (tanh/identity/logistic/relu/bounded_relu), which the validation error message and the Literal type hints already reflect. Gradient extensibility: expand pyrcn.nn.OPTIMIZERS to {adam, adamw, sgd, rmsprop, adagrad} and LOSSES to {mse, mae, huber}, and add a user-facing `loss` param to ESN*/ELM* (validated, in get_params(deep=False), threaded through the gradient readout fit and the BPTT loop, and documented). optimizer validation now checks the full set. Full suite 220 passed / 3 skipped; flake8 + mypy clean; docs build clean. Co-Authored-By: Claude Opus 4.8 * Plan: refresh roadmap status (through Phase B) + open tasks Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: renierts Co-authored-by: Claude Opus 4.8 (1M context)

Uh oh!

Input Sanitazing for activation function #54

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions