This post shows two ways you can upload a file to a Spring web application. I do not show the full code but parts of it.
Method 1 – using multipart/form-data
server code:
@PostMapping("/fileUpload")
public ResponseEntity<String> handleMultipartFileUpload(@RequestPart("file") MultipartFile file) {
if (file.isEmpty()) {
return ResponseEntity.badRequest().body("File is empty");
}
try {
long t0 = System.currentTimeMillis();
// Process the file here
byte[] bytes = file.getBytes();
String checksum = Utils.calculateMD5(bytes);
logger.info("received {} bytes with checksum {} in {} ms", bytes.length, checksum, System.currentTimeMillis() - t0);
return ResponseEntity.ok("File uploaded successfully");
} catch (Exception e) {
e.printStackTrace();
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body("Error uploading file");
}
}
client code:
with open('sample_file.txt', 'rb') as f:
files = {'file': f}
response = requests.post(file_upload_url, files=files)
# Printing the response
print(response.status_code)
print(response.text)
this method corresponds to following OpenAPI spec:
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
properties:
file:
type: string
format: binary
Method 2 – using application/octet-stream
server code:
@PostMapping(value = "/binaryUpload",
consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<String> handleBinaryUpload(HttpServletRequest httpServletRequest) {
try {
long t0 = System.currentTimeMillis();
var inputStream = httpServletRequest.getInputStream();
Checksum checksum = Utils.calculateMD5(inputStream);
logger.info("received {} bytes with checksum {} in {} ms", checksum.length(), checksum.checksum(), System.currentTimeMillis() - t0);
return ResponseEntity.ok("File uploaded successfully");
} catch (Exception e) {
e.printStackTrace();
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body("Error uploading file");
}
}
client code:
with open('sample_file.txt', 'rb') as f:
response = requests.post(binary_upload_url, data=f, headers={'Content-Type': 'application/octet-stream'})
# Printing the response
print(response.status_code)
print(response.text)
this method corresponds to following OpenAPI spec:
requestBody:
required: true
content:
application/octet-stream:
schema:
type: string
format: binary
Comparison
Method 1 gave me following error when I used a large file:
org.springframework.web.multipart.MaxUploadSizeExceededException: Maximum upload size exceeded
It can be fixed by adding following to application.properties [1]:
spring.servlet.multipart.max-file-size=10MB
spring.servlet.multipart.max-request-size=10MB
server.tomcat.max-swallow-size=100MB
With this I was able to upload my test file:
MainController : received 5646730 bytes with checksum 715a149541e7cf0995b341a2b18d3bfe in 12 ms
However how big is too big? where should the cap be?
Method 2 works as well:
MainController : received 5646730 bytes with checksum 715a149541e7cf0995b341a2b18d3bfe in 11 ms
in addition it does not require tweaking maximum file upload limit. It seems superior and better choice for sending very large blobs to a server.
On the other hand sending a file as multipart/form-data will allow you to access metadata about the file on the server [1].
Also checkout this question on SO.
This answer re: multipart/form-data on SO says:
It does not split huge files into parts.
With Method 2 I was able to upload 4GB file to the server without errors:
MainController : received 3848008288 bytes with checksum 8c22ba24008740228b73731969546db8 in 6529 ms
I think this method will also lend itself to the case when the client is NOT uploading a file from disk but rather connecting to a BLOB storage like AWS S3 and wants to ingest a blob from there (i.e., upload the blob to your server without having to save it to local disk). TODO: test it.