Home
Browse All
DRL Collection Development Policy
Links to non-OHSU Collections
OHSU Campus Building Names
Digital Collections Blog
Log in
|
Help
Search
Advanced Search
Find results with:
error div
Add another field
Search by date
from
after
before
on
to
Searching collections:
Student Scholar Archive
Add or remove collections
Home
Student Scholar Archive
Dynamics and algorithms for stochastic search
Reference URL
To link to this object, paste this link in email, IM or document
To embed this object, paste this HTML in website
Dynamics and algorithms for stochastic search
View Description
Download
small (250x250 max)
medium (500x500 max)
large ( > 500x500)
Full Resolution
9532023642007_199510.orr.genevieve.pdf
Description
Rights
http://www.ohsu.edu/xd/education/library/services/theses-dissertations/rights-statement.cfm
Title
Dynamics
and
algorithms
for
stochastic
search
Creator.PersonalName
Orr
,
Genevieve
Beth
Thesis.Degree
Ph.D.
Thesis.Major
Computer Science and Engineering
Thesis.DateDegreeAwarded
October
1995
Institution
Oregon Graduate Institute of Science & Technology
Department
Dept. of Computer Science and Engineering
Thesis.Advisor/Mentor
Leen, Todd K.
Thesis.Committee
Otto, Steve W.
Barnard, Etienne
Fraser, Andrew M.
Subject.LCSH
Machine learning
Neural networks (Computer science)
Computer algorithms
Call Number
Q183.5.OGI O77 1995
Description.Abstract
In this
thesis
we
develop
a
mathematical
formulation
for the
learning
dynamics
of
stochastic
or
on-line
learning
algorithms
in
neural
networks
.
We
use
this
formulation
to
1)
model
the
time
evolution
of the
weight
space
densities
during
learning
,
2)
predict
convergence
regimes
with and
without
momentum
, and
3)
develop
a
new
efficient
algorithm
with
few
adjustable
parameters
which
we
call
adaptive
momentum
. In
stochastic
learning
, the
weights
are
updated
at
each
iteration
based
on a
single
exemplar
randomly
chosen
from the
training
set
.
Treating
the
learning
dynamics
a
Markov
process
,
we
show
that the
weight
space
probability
density
P(w
,
t
)
can
be
cast
as a
Kramers-Moyal
series
[equation
:
δP(w,t)
divided
by
δt
=
LKM
P(w,t)]
where
LKM
is
an
infinite-order
linear
differential
operator
, the
terms
of
which
involve
powers
of the
learning
rate
µ
.
We
present
several
approaches
for
truncating
this
series
so
that
approximate
solutions
can
be
obtained
.
One
approach
is
the
small
noise
expansion
where
the
weights
are
modeled
as a
sum
of a
deterministic
and
noise
component
.
However
, in
order
to
provide
more
accurate
solutions
,
we
also
develop
a
perturbation
expansion
in
µ
.
We
demonstrate
the
technique
on
equilibrium
weight-space
densities
.
Unlike
batch
learning
,
stochastic
updates
are
noisy
but
fast
to
compute
. The
speedup
can
be
dramatic
if
training
sets
are
highly
redundant
, and the
noise
can
decrease
the
likelihood
of
becoming
trapped
in
poor
local
minima
.
However
,
acceleration
techniques
based
on
estimating
the
local
curvature
of the
cost
surface
can
not be
implemented
stochastically
because
the
estimates
of
second
order
effects
are
much
too
noisy
.
Disregarding
such
effects
can
greatly
hinder
learning
in
problems
where
the
condition
number
of the
hessian
is
large
. A
matrix
of
learning
rates
(the
inverse
hessian)
that
scales
the
stepsize
according
to the
curvature
along
the
different
eigendirections
of the
hessian
is
needed
.
We
propose
adaptive
momentum
as a
solution
.
It
results
in an
effective
learning
rate
matrix
that
approximates
the
inverse
hessian
.
No
explicit
calculation
of the
hessian
or its
inverse
is
required
. This
algorithm
is
only
O(n)
in
both
space
and
time
,
where
n
is
the
dimension
of the
weight
vector
.
Language
eng
Type
Text
Format.Use
Needs Adobe Acrobat Reader to view.
Format.FileType
pdf
Format.FileSize
4360.215 KB
OCLC number
36305093
you wish to report:
Your comment:
Your Name:
Submit
Cancel
...
Back to top
Select the collections to add or remove from your search
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
Select All Collections
B
Beauty of the PNW Collection
C
Campus Collection
Classic Article Collection
Clinical Outcomes Research Initiative (CORI)
CSETech
F
FDA Drug Approval Documents
N
Naturopathic Medicine Historical Collection
O
OHSU Historical Collections & Archives
OHSU Oral History Collection
ONPRC Collection
S
Student Scholar Archive
500
You have selected:
1
OK
Cancel
Student Scholar Archive
Dynamics and algorithms for stochastic search 