Return to Colloquia & Seminar listing
Reservoir Computing with high non-linear separation and long- term memory for time-series data analysis
Special Events| Speaker: | John Butcher, Keele University |
| Location: | 3106 MSB |
| Start time: | Thu, Jul 5 2012, 12:10PM |
Description
Reservoir
Computing
(RC)
is
a
recent
addition
to
the
field
of
recurrent
neural
networks,
with
the
added
advantage
of
a
simple
and
fast
training
procedure.
This
talk
presents
the
use
of
reservoirs
when
applied
to
several
time-‐series
datasets
including
real-‐world
data
collected
from
an
engineering
application.
Through
this
analysis
an
antagonistic
trade-‐off
between
a
reservoir's
non-‐linear
mapping
and
ability
to
recall
inputs
from
the
past
was
observed
when
data
appeared
to
require
high
amounts
of
non-‐linearity
and
input
recall,
which
hindered
performance.
To
overcome
this
trade-‐off,
a
reservoir
was
combined
with
two
feedforward
layers
of
neurons
to
give
Reservoirs
with
Random
Static
Projections
(R2SP).
These
two
layers
were
borrowed
from
the
field
of
extreme
learning
machines
(ELMs),
where
it
was
conjectured
that
these
would
allow
the
reservoir
to
be
tuned
towards
maximising
its
memory
capacity
while
the
non-‐linear
transformation
of
the
input
was
taken
care
of
by
these
new
layers.
The
R2SP
architecture,
along
with
a
standard
reservoir
and
traditional
recurrent
neural
networks
were
applied
to
several
datasets,
where
the
R2SP
outperformed
the
standard
reservoir
approach.
The
properties
of
the
standard
reservoir
and
R2SP
were
analysed
where
it
was
found
that
the
reservoir
of
the
R2SP
was
tuned
towards
input
recall,
as
less
non-‐linear
transformation
of
the
input
was
required
from
it.
The
advantages
of
using
RC
approaches
was
apparent
not
only
from
the
improvement
in
performance
they
offered,
but
also
the
drastic
reduction
in
the
complexity
of
their
training
procedures.