how to convert parameters dtype from float32 to float64 in gluon?

[zhaoningning]

I need to do this because model produce very small values, and float32 is not suitable in
this case, Does anyone who has any ideas about convert float32 parameter to float64 in gluon?

[zhreshold]

Cast data to float64, and the corresponding weights in model will be initialized to float64 accordingly once you call net(x)
Or there's per parameter function cast: https://mxnet.incubator.apache.org/api/python/gluon/gluon.html?highlight=parameterd#mxnet.gluon.Parameter.cast

[zhaoningning]

@zhreshold thx for your replies! I succeed using cast float32 paramster to float64. however, after cast parameters to float64, there is a error :
include/mxnet/././tensor_blob.h:217: Check failed: mshadow::DataType::kFlag == type_flag_ TBlob.get_with_shape: data type do not match specified type.Expected: 0 v.s. given 1, which happend when barckward() is called, and float32 parameters does not give this error; I use mxnet 1.0.0, and the loss layer is writed by myself, do you have any ideas about this error?

[zhaoningning]

@piiswrong @ry @pluskid

[reminisce]

Similar to #8799

[haojin2]

@zhaoningning Hello, do you still experience this issue? If so, would you please provide some example scripts to reproduce your issue? Thanks!

[apeforest]

I could reproduce this error using the following example from #8799. I will work on this issue.

import mxnet as mx
import numpy as np
from mxnet import autograd

def crit_func(a):
    q, l = mx.nd.linalg.gelqf(a)
    # Could as well write _, l = mx.nd.linalg.gelqf(a)
    # This circumvents the issue, but is highly obscure:
    #bogus = mx.nd.BlockGrad(q) * 0.0
    #return mx.nd.sum(l) + bogus
    return mx.nd.sum(l)

ctx = mx.cpu()

dtype = np.float32
a32 = mx.nd.random.normal(shape=(2, 3), ctx=ctx, dtype=dtype)
a32.attach_grad()
with autograd.record():
    crit32 = crit_func(a32)
    print ('crit32 = %f' % crit32.asscalar())
    crit32.backward()
    print ('grad32\n', a32.grad.asnumpy())

dtype = np.float64
a64 = mx.nd.Cast(a32, dtype=dtype)
a64.attach_grad()
with autograd.record():
    crit64 = crit_func(a64)
    print ('crit64 = %f' % crit64.asscalar())
    crit64.backward()
    print ('grad64\n', a64.grad.asnumpy())

[apeforest]

@zhaoningning Could you provide a small script of your issue. I would like to verify my fix since your description of the issue is not exactly the same as #8799. Thanks!

[lupesko]

@zhaoningning are you able to provide a script reproducing the issue?

[zhaoningning]

@apeforest @lupesko sorry,I can not reproduce this issue after such a long time...