[On-Prem / Docker] How can i improve uploading lfs-objects speed to massive repository?

The process below is uploading a very large capacity (approximately 800GB) of lfs-objects. In the beginning, the speed was more than 10MB, but from the time when 60% passed, the speed has been around 200KB. This is a very low number.

The above process is steadily showing an upload speed of around 20MB. Over time, however, this process also slows down and falls below 10MB.

I want to know why uploading large lfs-ojbects is not getting this fast and I want to find a solution.

In fact, both processes are showing upload speeds that fall short of my expectations. Both Space and Repository were expecting very high transmission speeds because they were on the same PC, but this speed is less than that of a git server operated by Synology NAS.

Clearly, this speed is simply unacceptable because it uses a huge amount of computer resources. How can I increase the upload speed? I once shut down the process and restarted the computer and docker, then I tried restarting the lfs-objects upload but it was still showing low speeds then.

0
2 comments

JackToStudios Official, thanks for sharing this info! It looks suspicious that Minio uses so many CPU resources. May I ask if there's any correlation between the increased CPU usage and the decreased upload speed?

Apart from this, just to make sure we're on the same page: are all components of the Space installation running on the same host? To be more precise, are Minio and DB volumes located on the same host as containers?

0

1. there's any correlation between the increased CPU usage and the decreased upload speed?

No. CPU usage is always 100% (it's mean docker use cpu resource over 2000%, using all logical processor)  in uploading progress. And most of the usage is used by minio.

 

2. Are all components of the Space installation running on the same host?

Yes, all container has same host. I run docker on windows desktop

0

Please sign in to leave a comment.