Support generating metadata and downloading files from remote file server

[jdduprey]

Another issue related to functionality OOI might need when building out pbp pipelines:
The ability to generate metadata and download files from a remote file system accessible over http. For example the OOI raw data server.

Are there any other groups this would be useful for?

[carueda]

Thanks for the suggestion @jdduprey !

HTTP access should not be complicated to add and would be a nice (logical) addition to the current access mechanisms (local files, s3:, gs:). For future testing could you provide specific audio file URLs that are relevant to you? (I get a Not Found when clicking the URL above).

[jdduprey]

Hi Carlos,
This is the updated example link.
You wouldn't be able to test anything yet because the current archive format is mseed. However, OOI is planning to make all hydrophone data available in FLAC soon(ish). I'll ping this thread when that happens.

I've gotten a pbp pipeline working currently by pulling down the mseed to a cloud machine, converting to FLAC and using the currently supported local file access. (But my current solution isn't ideal in terms of compute resources)