S3 Uploader for Directly Uploading Data to AWS S3 from Web and Mobile Devices

Our client was in need of a comprehensive solution for uploading large files to an S3 bucket. They also required a robust user management system that could assign specific bucket access to users, as well as a dashboard and reporting system for tracking user activity and file upload sizes.

Technology Used

Key Challenges

  • One of the major challenges was the size of the files to be uploaded.
  • The client needed to upload files as large as 40GB, and they wanted the functionality to resume an upload in case of browser closure or network issues.
  • For instance, if a user had already uploaded 10GB of a 20GB file and then closed the browser, they should be able to resume uploading the same file at the 10GB mark.

Solution Implementation

  • We developed a user management system that included access control list (ACL) functionality, using a module-based architecture with each module having its own access level based on user roles.
  • The roles were dynamic, enabling the admin to create custom roles and assign access as needed.
  • For the file upload functionality, we used AWS Amplify with an Async mode, and added the ability to pause and resume uploads.
  • If a user closed the browser or experienced network failure, the system would automatically consider it a pause event.
  • We also developed a desktop application with similar functionality, using the same microservice developed for the web app.