#crate

/

      • cratebot
        [itamar steinberg, CrateDB Community] Hello, we are trying to load a ~600G data from json files, we split the data to files - each ~3.2 G , when looking on disk space during the load after 4 files it takes 24G disk space that means 6G for each file (almost twice the size of the original file size) . that means ~1200G of disk space eventually .
      • so we will end up with a lot more disk space usage vs size of raw data.
      • we expected that the data will be compressed and to be less then file size
      • is there a parameter or something we miss regarding the disk size crate uses ? thanks
      • [Mohan Krishnamurthy, CrateDB Community] JDBC Connector
      • [Claus Matzinger, CrateDB Community] @itamars no, this will happen over time. The actual space CrateDB uses mostly depends on how easy to compress the data is (i.e. variability, etc)