How should we store 100 GB + data sets? Is git-lfs still the answer?

Data Management
2

This may not be applicable in this context, but https://google.github.io/flatbuffers/ looks interesting from an efficient data storage and access standpoint.

Serialization mechanisms