PR #21 introduced ability to download files (archived and remote) from cloud object storage. This was based around a manifest file concept (original generated for use with wget or some CLI tool) - and manifests txt are currently stored in the package (to determine files to download).
There are likely better ways to achieve this in a python environment (download links could be composed dynamically). Or additionally, something like boto3 (with public API/creds? ..noting data is public) could be used to list/check (or even download) files in object store.
PR #21 introduced ability to download files (archived and remote) from cloud object storage. This was based around a manifest file concept (original generated for use with
wgetor some CLI tool) - and manifests txt are currently stored in the package (to determine files to download).There are likely better ways to achieve this in a python environment (download links could be composed dynamically). Or additionally, something like
boto3(with public API/creds? ..noting data is public) could be used to list/check (or even download) files in object store.