This blog post is part of a CTF challenge.

I’ve tried to search for this answer publicly but I can’t find whether is it possible to install a big number of packages at once from the requirements.txt.

To give you more context on this weird condition, I started working at TeamCTF21ProdSec and I found that they are storing their python packages in their private repository. I’m working on a project that analyse all these packages so I’ve managed to retrieve all the packages names in the requirements.txt (literally thousands of private packages). Well retrieving the packages names was not really difficult since all the packages names follows the atlasctf_21_prod_[0-2]?[0-9] regular expression (all the regex combinations are almost existing) and similar ones.

I was asking about whether there is any limit because the number of packages is huge and I don’t want the analysis to abort especially that the analysis is going to be quick (we’re using a scaling infrastructure which means that the installation of all these packages should be done in less than 10 minutes. The issue is since the installation run every 10 minutes to make sure that the stored private packages are analysed as soon as possible, we are disabling the installation output. That’s why I don’t believe that there is a way to check if there was an issue with the requirements.txt length or number of lines.

I don’t know whether what is passed from the first part of the following command to the second part can be broken (too much for the pipe |). Basically, we’re using this command: 

cat requirement.txt | pip3 install -I 

And with the following configuration of /etc/pip.conf

[global]
extra-index-url = http://<private_IP_and_port>/simple/
trusted-host = <private_IP>

With this, the packages are installed from the private repository.

We avoided installing the packages using the -r requirements.txt parameter to avoid any issue coming from pip3.

For anyone that is curious what if a project was decommissioned, well before the packages start their installation, the old installed packages are removed.

Thanks