Hello CSHARP, I'm looking for advice on how to handle very large binary file uploads with WebApi. It needs to scale well, anything from 1byte to 5TB.
My API has a client that will expose the upload as a method that takes a Stream. I'd like the user to be able to specify a Stream and for the API to be able to (without requiring 5TB of ram..) cache to to disk as the stream comes in.
If at all possible it would be ideal to be able to start processing the cached file on the server as soon as it starts to come in. I'll be forwarding it to Amazon S3.
I've looked at their SDK and they support a multipart random ordered concurrent upload for large files. Wondering if I could creating something similar to this.
Any advice would be appreciated.
by habathcx via /r/csharp