I see lots people implementing their own loading algorithms for ML projects.
Some don’t have threading or multiprocessing, others don’t check for memory protection of threads. End of the day it’s a mess and significant changes have to be made to others code to run it on other computers.
`ThreadedFileLoader` is python module capable of loading all file formats.
Installation:
pip install ThreadedFileLoader
Loading Images:
Here is an example that loads all jpg images from a folder.
Loading Datasets:
Loading Text Files:
Loading Other File Formats:
Here is the cool part, you can load any other file formats that aren’t even define in the module without having to deal with threading or multiprocessing.
Here the object_loader method was overloaded so that it can load text files.
