I have 10K files to write in S3 every hour, how can I do this in parallel?
Use multiple threads or try an event mashin with an S3 driver. You need a throughput of approximately 3 files per second, which should not be difficult depending on the file size. To not reconnect between each file, be sure to use HTTP.
Comments
Post a Comment