回声状态神经网络(ESN)回归预测,代码非Matlab工具箱——可选择优化算法,如SSA,GEO,WOA,SMA进行优化改进等。 模型评价指标包括:R2、MAE、MSE、RMSE和MAPE等,代码质量极高,方便学习和替换数据。
手搓回声池:用Python搞ESN预测还能这么玩?
搞时间序列预测的朋友应该都听说过回声状态网络(ESN),这玩意儿的核心就是一个随机初始化且固定不变的"储备池",靠的是用动态系统的特性来捕捉非线性关系。今天咱们不聊Matlab,直接上Python,从零开始搭一个ESN回归模型,顺带用优化算法调参,顺便看看怎么用麻雀、鲸鱼这些玄学优化法来改进模型。
先上代码骨架,再慢慢拆解。先搞数据,这里用个简单的时间序列,比如正弦波叠加噪声:
import numpy as np import matplotlib.pyplot as plt from sklearn.metrics import r2_score, mean_absolute_error t = np.linspace(0, 20, 2000) data = np.sin(t) + np.sin(0.5*t) + 0.1*np.random.randn(len(t)) # 划分训练测试集 train_size = 1500 train_data, test_data = data[:train_size], data[train_size:]储备池初始化是ESN的灵魂,这里的关键参数是储备池大小(nreservoir)、泄漏率(leakingrate)、谱半径(spectral_radius)。别急着调参,后面用优化算法自动搞:
class ESN: def __init__(self, n_input, n_reservoir, n_output, leaking_rate=0.3, spectral_radius=0.9): self.n_reservoir = n_reservoir self.leaking_rate = leaking_rate # 随机初始化输入权重和储备池连接 self.W_in = np.random.rand(n_reservoir, n_input) - 0.5 self.W_res = np.random.rand(n_reservoir, n_reservoir) - 0.5 # 调整谱半径 radius = np.max(np.abs(np.linalg.eigvals(self.W_res))) self.W_res *= spectral_radius / radius def forward(self, input_seq): n_samples = input_seq.shape[0] states = np.zeros((n_samples, self.n_reservoir)) for t in range(1, n_samples): res_input = np.dot(self.W_in, input_seq[t]) + np.dot(self.W_res, states[t-1]) states[t] = (1 - self.leaking_rate) * states[t-1] + self.leaking_rate * np.tanh(res_input) return states def train(self, states, target): # 岭回归防止过拟合 ridge = 1e-6 self.W_out = np.dot(np.linalg.pinv(np.dot(states.T, states) + ridge * np.eye(self.n_reservoir)), np.dot(states.T, target)) def predict(self, initial_input, n_predictions): # 自回归预测 outputs = [] current_state = np.zeros(self.n_reservoir) last_input = initial_input for _ in range(n_predictions): res_input = np.dot(self.W_in, last_input) + np.dot(self.W_res, current_state) current_state = (1 - self.leaking_rate)*current_state + self.leaking_rate * np.tanh(res_input) predicted = np.dot(self.W_out, current_state) outputs.append(predicted) last_input = predicted # 用预测值作为下一步输入 return np.array(outputs)代码里几个注意点:
W_res初始化后做了谱半径调整,这是为了确保储备池的短期记忆能力,太大容易爆炸,太小会遗忘过快- 状态更新用了泄漏积分,
leaking_rate控制更新速度,类似RNN中的门控机制 - 输出层用岭回归而不是梯度下降,这是ESN的特色——只训练最后一层
但默认参数效果一般?试试鲸鱼优化算法(WOA)调参!这里以优化谱半径和泄漏率为例:
# 简化版WOA实现 def whale_optimizer(obj_func, max_iter=30): bounds = {'spectral_radius': (0.1, 1.5), 'leaking_rate': (0.1, 0.9)} best_params = None best_score = -np.inf # 初始化鲸鱼种群 population = [{'spectral_radius': np.random.uniform(*bounds['spectral_radius']), 'leaking_rate': np.random.uniform(*bounds['leaking_rate'])} for _ in range(10)] for iter in range(max_iter): for whale in population: # 这里简化为随机搜索,实际需要实现包围、气泡攻击等行为 current_score = obj_func(whale['spectral_radius'], whale['leaking_rate']) if current_score > best_score: best_score = current_score best_params = whale.copy() # 更新种群位置(简化版) new_pop = [] for whale in population: r = np.random.rand() new_whale = { 'spectral_radius': best_params['spectral_radius'] + r*(np.random.rand()-0.5), 'leaking_rate': best_params['leaking_rate'] + r*(np.random.rand()-0.5) } new_pop.append(new_whale) population = new_pop return best_params # 定义优化目标函数(比如最大化R2) def objective_function(spectral_radius, leaking_rate): esn = ESN(n_input=1, n_reservoir=100, n_output=1, spectral_radius=spectral_radius, leaking_rate=leaking_rate) states = esn.forward(train_data.reshape(-1,1)) esn.train(states, train_data[1:]) pred = esn.predict(train_data[-1], len(test_data)) return r2_score(test_data, pred)运行优化器找到最佳参数后,测试集效果通常会有提升。比如默认参数R2可能是0.85,优化后可能到0.93。其他指标可以用下面函数计算:
def evaluate(y_true, y_pred): mae = mean_absolute_error(y_true, y_pred) mse = mean_squared_error(y_true, y_pred) rmse = np.sqrt(mse) mape = np.mean(np.abs((y_true - y_pred)/y_true)) * 100 r2 = r2_score(y_true, y_pred) return {'R2':r2, 'MAE':mae, 'MSE':mse, 'RMSE':rmse, 'MAPE':mape}替换数据?只要把data换成你的时序数据,注意归一化!比如股价预测,可以把数据预处理部分改成:
from sklearn.preprocessing import MinMaxScaler scaler = MinMaxScaler(feature_range=(-1,1)) scaled_data = scaler.fit_transform(raw_data.reshape(-1,1)).flatten()最后说点经验:
- 储备池大小一般在100~500之间,太小捕捉不到模式,太大过拟合
- 优化算法别乱用,先网格搜索确定大致范围,再用SSA/SMA微调
- 输出结果不稳定?试试给储备池加少量噪声(
states[t] += 0.01*np.random.randn())
完整代码把这几部分拼起来就能跑,优化算法部分可以替换成其他算法的Python实现。ESN的优势就是训练快,适合实时性要求高的场景,比如传感器信号预测。玩得开心!