Suppose you have a dataset by the shape=(b, w, d):

dataset =
[
   [
      [e111 e112 ... e11d]
      [e121 e122 ... e12d]
      ...
      [e1w1 e1w2 ... e1wd]
   ]
   [
      [e211 e212 ... e21d]
      [e221 e222 ... e22d]
      ...
      [e2w1 e2w2 ... e2wd]
   ]
   ...
   [
      [eb11 eb12 ... eb1d]
      [eb21 eb22 ... eb2d]
      ...
      [ebw1 ebw2 ... ebwd]
   ]
]

Based on the above dataset, define X_i as follows:

X_i =
[
   [e1i1 e1i2 ... e1id]
   [e2i1 e2i2 ... e2id]
   ...
   [ebi1 ebi2 ... ebid]
]

So X_i.shape = (b, d) and if

S_0.shape = (b, n)
W_input.shape = (d, n)
W_output.shape = (n, n)

then below is the schema of a RNN: