store grain.autograd.BackProp object in returned variables from forward function
type-erased version of backward function used in grain.autograd.BackProp object
import numir; Embedding!float embed; auto w = [[1.0f, 2.0f], [3.0f, 4.0f]].nparray.variable; auto x = [0, 1, 0].variable; auto y = embed.forward(w, x); assert(y.sliced == [[1,2],[3,4],[1,2]]); auto gy = [[1f, 2f], [-1f, -2f], [1f, 0f]].nparray.variable; auto gw = embed.backward(gy)[0]; assert(gw.sliced == [[2f, 2f], [-1f, -2f]]); version (grain_cuda) { Embedding!float dembed; auto dy = dembed.forward(w.to!DeviceStorage, x.to!DeviceStorage); assert(dy.to!HostStorage.sliced == y.sliced); auto dgw = dembed.backward(gy.to!DeviceStorage)[0]; assert(dgw.to!HostStorage.sliced == gw.sliced); }
Emebedding ID into vector. TODO: support N-dim input. support sparse weight matrix