webysther/packagist-mirror

Build mirror of packagist

1.1.1 2019-11-11 03:35 UTC

This package is auto-updated.

Last update: 2020-01-15 15:22:48 UTC


README

Build Status docker Status docker pulls Minimum PHP Version Packagist Codecov Quality Score Mentioned in Awesome composer

This repository make possible to you create a metadata mirror and keep in sync with packagist.org.

Mirror creation

❤️ Recommended by packagist.org ❤️

If you're using PHP Composer, commands like create-project, require, update, remove are often used. When those commands are executed, Composer will download information from the packages that are needed also from dependent packages. The number of json files downloaded depends on the complexity of the packages which are going to be used. The further you are from the location of the packagist.org server, the more time is needed to download json files. By using mirror, it will help save the time for downloading because the server location is closer.

⚙️ How it works?

This project aims to create a local mirror with ease, allowing greater availability for companies/countries that want to use the composer but do not want to depend on the infrastructure of third parties. It is also possible to create a public mirror to reduce the load on the main repository and allow a better distribution of requests around the world, help us creating a public mirror!

When creating your local copy the child mirror (you) adds a list of other mirrors to use when creating your own mirror, if any mirror fails to deliver the metadata file the client automatically fetches the original file from the main mirror, could be packagist.org or even another. If you lose your connection to the server or any other problem that prevents you from continuing is okay, it can return from where it stopped running. After the mirror is created, the next runs will only look for the main mirror delta optimally and efficiently.

Mirror creation

🌎 Packagist public metadata mirrors observatory around the world

🛫 Amazing data mirrors used to download repositories metadata built using this recommended repository or another:

Theses lists are ordered by country and sync time.

Location Mirror Maintainer Github Sync Since
Brazil packagist.com.br Webysther main Continuously Q3'17
China php.cnpkg.org Eagle Wu fork Every minute Q3'18
China packagist.mirrors.sjtug.sjtu.edu.cn Shanghai Jiao Tong University fork Every hour Q2'19
Czech Republic packagist.hostuj.to HOSTUJ TO fork Every 5 minutes 🆕Q1'20
India packagist.in Varun Sridharan fork Every minute Q2'19
India packagist.vrkansagara.in Vallabh Kansagara fork Every 5 minutes Q4'19
Indonesia packagist.phpindonesia.id Indra Gunawan fork Every 30 seconds Q3'18
Indonesia packagist.ianmustafa.com Ian Mustafa fork Every 30 seconds Q3'19
Indonesia packagist.telkomuniversity.ac.id Telkom University fork Every 5 minutes 🆕Q1'20
Japan packagist.dev.studio-umi.jp Studio Umi fork Every minute 🆕Q1'20
South Africa packagist.co.za SolidWorx fork Every 5 minutes Q3'18
South Korea packagist.kr PackagistKR fork Every minute Q3'18
Thailand packagist.mycools.in.th Jarak Kritkiattisak fork Every 5 minutes Q4'19
USA packagist-mirror.wmflabs.org Wikimedia fork Every 5 minutes Q3'18

⚠️ Not based on this source code:

Location Mirror Maintainer Github Sync Since
China mirrors.aliyun.com Aliyun Every 5 minutes
China mirrors.cloud.tencent.com Tecent Cloud Every day
Japan packagist.jp Hiraku forked Every 2 minutes Q4'14
Japan packagist.kawax.biz Kawax another Every hour Q4'18

🛑 Not working as a mirror of packagist.org (checked at Q1'20):

Location Mirror Maintainer Github Reason At least
China mirrors.huaweicloud.com Huawei Cloud Outdated Q3'19
China packagist.phpcomposer.com Outdated Q4'19

If you know any new mirror based or not on this one, please create a issue or a pull request with the new data.

Check status page for health mirror's.

World Map

The colors represent the topology drawn below, only show servers in better condition at country level.

🚀 Create your own mirror

Topology

With docker and nginx:

The mirror creation save all data as .gz to save disk space and CPU, you need to enable reverse gz decode when a client ask for the decompressed version, normally used only for legacy composer clients.

Change you nginx configuration of gzip_static and gunzip as is:

Create a website on a default nginx instalattion:

sudo vim /etc/nginx/sites-available/packagist.com.br.conf
server {
    index.html;

    server_name packagist.com.br www.packagist.com.br;

    location / {
        try_files $uri $uri/ =404;
        gzip_static on;
        gunzip on;
    }
}

💡Tip: use a machine with 2GB at least of memory, with that all metadata keep to the memory helping the nginx and disk to not be consumed at all.

After install nginx edit /etc/crontab:

* * * * * root docker run --name mirror --rm -v /var/www:/public \
-e MAINTAINER_REPO='packagist.com.br' \
-e APP_COUNTRY_NAME='Brazil' \
-e APP_COUNTRY_NAME='br' \
-e MAINTAINER_MIRROR='Webysther' \
-e MAINTAINER_PROFILE='https://github.com/Webysther' \
-e MAINTAINER_REPO='https://github.com/Webysther/packagist-mirror' \
-e URL='packagist.com.br' \
webysther/packagist-mirror

to more options about image go to docker repository.

Put inside your ~/.*rc (~/.bashrc/~/.zshrc/~/.config/fish/config.fish):

alias logs='watch -n 0.5 docker logs --tail 10 -t mirror'

Update your env vars and see monitoring packagist mirror creation:

source ~/.*rc
logs

👷 Install

Using with docker repository or composer local:

$ git clone https://github.com/Webysther/packagist-mirror.git
$ cd packagist-mirror && composer install
$ cp .env.example .env

Schedule the command to create and update the mirror:

$ php bin/mirror create -vvv

🐧 Development & Contributing

Please see CONTRIBUTING and CONDUCT for details.

🛣️ Roadmap

2020

  • Support for Drupal metadata
  • Translate readme.md and index of mirror.
  • Fully IaC with terraform.
  • More recipes to AWS/Azure/GCP/DigitalOcean and another cloud providers.
  • Support for Heroku.
  • Support for kubernetes.

2021

  • Support gz disabled for limited configuration access to Apache/Nginx.
  • Support full mirror mode: Github/Gitlab.
  • Integration with twity.
  • Integration with composer-registry-manager.

📋 Requirements

The following versions of PHP are supported by this version.

  • PHP >=7.2

🧪 Testing

$ vendor/bin/phpunit

🥂 Credits

💙 Other correlated projects

☮️ License

MIT License. Please see License File for more information.

FOSSA Status