GJjT"ddlZddlmZmZddlmZddlmZddl m Z Gdde Z Gdde Z Gd d e Z y) N)CallableOptional)tanh)Modulec NeZdZdZ d dedededeeffd ZdZ d dZ xZ S) RNNaAn Elman recurrent layer. The input is a sequence of shape ``NLD`` or ``LD`` where: * ``N`` is the optional batch dimension * ``L`` is the sequence length * ``D`` is the input's feature dimension Concretely, for each element along the sequence length axis, this layer applies the function: .. math:: h_{t + 1} = \text{tanh} (W_{ih}x_t + W_{hh}h_t + b) The hidden state :math:`h` has shape ``NH`` or ``H``, depending on whether the input is batched or not. Returns the hidden state at each time step, of shape ``NLH`` or ``LH``. Args: input_size (int): Dimension of the input, ``D``. hidden_size (int): Dimension of the hidden state, ``H``. bias (bool, optional): Whether to use a bias. Default: ``True``. nonlinearity (callable, optional): Non-linearity to use. If ``None``, then func:`tanh` is used. Default: ``None``. input_size hidden_sizebias nonlinearityct||xst|_t |jst d|ddt j|z }||_tjj| |||f|_ tjj| |||f|_ |r*tjj| ||f|_yd|_y)Nz.Nonlinearity must be callable. Current value: .?lowhighshape)super__init__rr callable ValueErrormathsqrtr mxrandomuniformWxhWhhr )selfr r r r scale __class__s a/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/mlx/nn/layers/recurrent.pyrz RNN.__init__'s (0D))*@aP dii ,,&99$$U; *C% 99$$U; *D%   II  5&u[N  K  c d|jjdd|jd|jd|jduS)N input_dims=, hidden_size=z, nonlinearity=, bias=)rrr r r rs r" _extra_reprzRNN._extra_reprDsT$((..+,-++,- --.gdiit6K5L N r#c|j6tj|j||jj}n||jjz}g}t |j dD]d}|4tj|d|ddf||jj}n |d|ddf}|j|}|j|ftj|dS)N.axis) r raddmmrTrangerrr appendstack)rxhidden all_hiddenidxs r"__call__z RNN.__call__Ks 99 Atxxzz2ADHHJJA %C!!CaK.&$((**E3Q;&&v.F   f % &xx ,,r#)TNN) __name__ __module__ __qualname____doc__intboolrrrr*r8 __classcell__r!s@r"rr sJ>+/      x(  : -r#rcBeZdZdZ ddededeffd ZdZd dZxZ S) GRUaA gated recurrent unit (GRU) RNN layer. The input has shape ``NLD`` or ``LD`` where: * ``N`` is the optional batch dimension * ``L`` is the sequence length * ``D`` is the input's feature dimension Concretely, for each element of the sequence, this layer computes: .. math:: \begin{aligned} r_t &= \sigma (W_{xr}x_t + W_{hr}h_t + b_{r}) \\ z_t &= \sigma (W_{xz}x_t + W_{hz}h_t + b_{z}) \\ n_t &= \text{tanh}(W_{xn}x_t + b_{n} + r_t \odot (W_{hn}h_t + b_{hn})) \\ h_{t + 1} &= (1 - z_t) \odot n_t + z_t \odot h_t \end{aligned} The hidden state :math:`h` has shape ``NH`` or ``H`` depending on whether the input is batched or not. Returns the hidden state at each time step of shape ``NLH`` or ``LH``. Args: input_size (int): Dimension of the input, ``D``. hidden_size (int): Dimension of the hidden state, ``H``. bias (bool): Whether to use biases or not. Default: ``True``. r r r ct|||_dtj|z }t j j| |d|z|f|_t j j| |d|z|f|_ |r't j j| |d|zfnd|_ |r*t j j| ||f|_ yd|_ y)Nrr) rrr rrrrrWxWhbbhnrr r r r r!s r"rz GRU.__init__{s &dii ,,))##U1{?J*G$ ))##U1{?K*H$   II  5&uQ_xx ,,r#Tr9 r:r;r<r=r>r?rr*r8r@rAs@r"rCrC]s9B      6 *-r#rCcBeZdZdZ ddededeffd ZdZd dZxZ S) LSTMauAn LSTM recurrent layer. The input has shape ``NLD`` or ``LD`` where: * ``N`` is the optional batch dimension * ``L`` is the sequence length * ``D`` is the input's feature dimension Concretely, for each element of the sequence, this layer computes: .. math:: \begin{aligned} i_t &= \sigma (W_{xi}x_t + W_{hi}h_t + b_{i}) \\ f_t &= \sigma (W_{xf}x_t + W_{hf}h_t + b_{f}) \\ g_t &= \text{tanh} (W_{xg}x_t + W_{hg}h_t + b_{g}) \\ o_t &= \sigma (W_{xo}x_t + W_{ho}h_t + b_{o}) \\ c_{t + 1} &= f_t \odot c_t + i_t \odot g_t \\ h_{t + 1} &= o_t \text{tanh}(c_{t + 1}) \end{aligned} The hidden state :math:`h` and cell state :math:`c` have shape ``NH`` or ``H``, depending on whether the input is batched or not. The layer returns two arrays, the hidden state and the cell state at each time step, both of shape ``NLH`` or ``LH``. Args: input_size (int): Dimension of the input, ``D``. hidden_size (int): Dimension of the hidden state, ``H``. bias (bool): Whether to use biases or not. Default: ``True``. r r r ct|||_dtj|z }t j j| |d|z|f|_t j j| |d|z|f|_ |r-t j j| |d|zf|_ yd|_ y)Nrr) rrr rrrrrrFrGr rJs r"rz LSTM.__init__s &dii ,,))##U1{?J*G$ ))##U1{?K*H$   II  5&uQ_D!xxfdggii8$3JAq!Q 1 A 1 A A 1 A4x!a%'1u&F OOD !   f %%&(xx ,bhhxb.IIIr#r[)NNr\rAs@r"r^r^s:H      , Jr#r^)rtypingrrmlx.corecorermlx.nn.layers.activationsrmlx.nn.layers.baserrrCr^r#r"rpsD %*%O-&O-di-&i-XZJ6ZJr#