Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

Python list as a gluon Dataset #12087

Description

@asitstands
data = mx.gluon.data.DataLoader([([1,2], 0), ([3, 4], 1)], batch_size=1, num_workers=2) 
for d, l in data:
    pass

This causes an error in the recent master.

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/site-packages/mxnet-1.3.0-py3.6.egg/mxnet/gluon/data/dataloader.py", line 154, in worker_loop
    dataset._fork()
AttributeError: 'list' object has no attribute '_fork'

The code has worked in some days ago. I guess #11370 brokes it. According to the API doc, any types with __getitem__ and __len__ can be used as a Dataset. #11370 is temporal but I think it would be a good idea to avoid breaking existing code. @zhreshold

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions