Initializers¶
Initializers provide init values for network parameter blobs. In Caffe, they are called Fillers.
- class NullInitializer¶
An initializer that does nothing.
- class ConstantInitializer¶
Set everything to a constant.
- value¶
The value used to initialize a parameter blob. Typically this is set to 0.
- class XavierInitializer¶
An initializer based on [BengioGlorot2010], but does not use the fan-out value. It fills the parameter blob by randomly sampling uniform data from \([-S,S]\) where the scale \(S=\sqrt{3 / F_{\text{in}}}\). Here \(F_{\text{in}}\) is the fan-in: the number of input nodes.
Heuristics are used to determine the fan-in: For a 4D tensor parameter blob with the shape \((W,H,C,N)\), if \(C=N=1\), then this is considered as a parameter blob for an InnerProductLayer, and fan-in = \(W\). Otherwise, fan-in is \(W\times H\times C\).
[BengioGlorot2010] Y. Bengio and X. Glorot, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of AISTATS 2010, pp. 249-256.