Qubes OS Main repo/iso Mirroring Hints

Original forum link: https://forum.qubes-os.org/t/26127
Original poster: Ali MJ
Created at: 2024-04-29 10:41:21
Posts count: 5
Likes count: 7
Tags: updates

Background

I did some research on the Qubes main repository/iso download mirror (ftp.qubes-os.org). There is a brief short page on Qubes OS site with little information on hosting a Qubes OS mirror. Since I used to host an unofficial Qubes mirror in the past, I decided to post this guideline for people who might be interested on hosting a Qubes OS mirror.

Brief Technical Details

Mirroring of Distro mirrors is usually perfored via rsync(1). Qubes source mirror does not provide rsync protocol to the public (or I am not aware of it). But some of its mirrors provide it which makes it perfect for analysis or Tier-2/Personal mirroring. Recommended syncing frequency is not mentioned. This is usually 3 or 4 times a day for release based Distros and is scheduled via a systemd timer/service pair (by comparison, rolling Distros might do it every few minutes via some special trigger!).

Mirror size

Total hard drive space required to host a Qubes OS mirror at the moment is 822.75GB (822,748,334,768 bytes). This info can be obtained prior to the main sync via:

rsync -rtlHnvh --stats rsync://mirrors.kernel.org/qubes

For personal/corporate usage, user are most probably interested only on parts of the mirror. The mirror could be broken to: - 150.79G for ISO files of R4.0 to R4.2.1 including beta and release candidates. Considering R4.0 is EOL, maybe its ISO should be axed from main mirror (there might be one for keeping historic releases). - 671.94G repo directory could be broken to: - 145.71M Archlinux repository - 8.98G Debian repository also used by Whonix templates. (Maybe R2.0, 3.1, 3.2, 4.0 related ones should be axed) - 265MB debu directory (possible Debian Unstable Builds?) - 657.82G yum directory contains repositories for both Fedora templates as well as dom0. Also the template images. It could be broken to: - R4.0: 91.78G - Again, maybe this should be axed from the main mirror. - R4.1: 343.49G - R4.2: 219.04G. If anyone wants only the stable repositories: 21.46GB and stable templates: 29.29GB. - R4.3: 3.52G

Debian and Fedora mirrors of each release might contain packages for EOL templates. But axing them is most probably not a good idea.

Required Bandwidth

Required bandwidth for source mirrors (not personal) is not documented. This is usually at least 1Gb/s for most Distros.