Reference:#
- https://docs.astral.sh/uv/configuration/indexes/#pinning-a-package-to-an-index
- Reference Repository: test-uv-index
In the last session, we mentioned how to configure separate CPU or CUDA versions of torch and torchaudio for Pytorch using tool.uv.index..
One user asked if it is possible to configure the Python mirror source for uv. I happened to have this need as well, because sometimes I find that uv cannot use my proxy; I can download directly with pip at a speed of 6MB/s, but downloading a 20MB Numpy with uv takes 7 minutes.
At that time, I discovered that we specified an index-url for torch, and in fact, when we change the source for pip, we also change the index-url.
For example:
# Tsinghua Source
https://pypi.tuna.tsinghua.edu.cn/simple
# Aliyun Source
https://mirrors.aliyun.com/pypi/simple/
# Tencent Source
http://mirrors.cloud.tencent.com/pypi/simple
# Douban Source
http://pypi.douban.com/simple/
These are also all index-url. Thus, we refer to the previous configuration of torch's index-url:
dependencies = [
"funasr==1.2.4",
"pyaml==25.1.0",
"torch==2.1.0",
"torchaudio==2.1.0",
]
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu" },
]
torchaudio = [
{ index = "pytorch-cpu" },
]
It can be easily concluded that we can configure the mirror source like this:
dependencies = [
"numpy==1.26.4",
"matplotlib==3.10.0"
]
[[tool.uv.index]]
name = "tsinghua"
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
explicit = true
[[tool.uv.index]]
name = "aliyun"
url = "https://mirrors.aliyun.com/pypi/simple/"
explicit = true
[tool.uv.sources]
numpy = [
{ index = "tsinghua"},
]
matplotlib = [
{ index = "aliyun"}
]
This means we use the Aliyun source to download matplotlib and the Tsinghua source to download numpy.
Then we write a simple script to test it.
However, I found that in uv.lock, the packages we specified indeed pointed to tsinghua and aliyun, but all the dependency packages were not resolved to use the mirror sources and still used pypi.org.
[[package]]
name = "matplotlib"
version = "3.10.0"
source = { registry = "https://mirrors.aliyun.com/pypi/simple/" }
dependencies = [
{ name = "contourpy" },
{ name = "cycler" },
{ name = "fonttools" },
{ name = "kiwisolver" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "pillow" },
{ name = "pyparsing" },
{ name = "python-dateutil" },
]
sdist = { url = "https://mirrors.aliyun.com/pypi/packages/68/dd/fa2e1a45fce2d09f4aea3cee169760e672c8262325aa5796c49d543dc7e6/matplotlib-3.10.0.tar.gz", hash = "sha256:b886d02a581b96704c9d1ffe55709e49b4d2d52709ccebc4be42db856e511278" }
wheels = [
{ url = "https://mirrors.aliyun.com/pypi/packages/09/ec/3cdff7b5239adaaacefcc4f77c316dfbbdf853c4ed2beec467e0fec31b9f/matplotlib-3.10.0-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:2c5829a5a1dd5a71f0e31e6e8bb449bc0ee9dbfb05ad28fc0c6b55101b3a4be6" },
...
[[package]]
name = "numpy"
version = "1.26.4"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/65/6e/09db70a523a96d25e115e71cc56a6f9031e7b8cd166c1ac8438307c14058/numpy-1.26.4.tar.gz", hash = "sha256:2a02aba9ed12e4ac4eb3ea9421c420301a0c6460d9830d74a9df87efa4912010", size = 15786129 }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a7/94/ace0fdea5241a27d13543ee117cbc65868e82213fb31a8eb7fe9ff23f313/numpy-1.26.4-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9ff0f4f29c51e2803569d7a51c2304de5554655a60c5d776e35b4a41413830d0", size = 20631468 },
...
[[package]]
name = "packaging"
version = "24.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d0/63/68dbb6eb2de9cb10ee4c9c14a0148804425e13c4fb20d61cce69f53106da/packaging-24.2.tar.gz", hash = "sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f", size = 163950 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", hash = "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", size = 65451 },
]
Why is this happening? I found that it was because my English was not good.
explicit = true means exclusion.
An index can be marked as explicit = true to prevent packages from being installed from that index unless explicitly pinned to it. For example, to ensure that torch is installed from the pytorch index, but all other packages are installed from PyPI, add the following to your pyproject.toml:
An index can be marked as explicit = true to prevent packages from being installed from that index unless explicitly pinned to it. For example, to ensure that torch is installed from the pytorch index, but all other packages are installed from PyPI, add the following to your pyproject.toml:
So, if we only want to specify it occasionally, we use the explicit = true parameter. Then normally, it still uses our pypi.
Additionally, it also provides a default parameter.
By default, uv includes the Python Package Index (PyPI) as the "default" index, i.e., the index used when a package is not found on any other index. To exclude PyPI from the list of indexes, set default = true on another index entry (or use the --default-index command-line option):
By default, uv includes the Python Package Index (PyPI) as the "default" index, i.e., the index used when a package is not found on any other index. To exclude PyPI from the list of indexes, set default = true on another index entry (or use the --default-index command-line option):
The default parameter will directly replace the pypi source and exclude it. It will no longer use the pypi source, even if the package is not found in the mirror sources, it will not search pypi.
If only the default is set, there are actually some issues. Because sometimes some packages are not timely maintained or updated, or the mirror site occasionally has issues, which can cause you to be unable to work normally.
[tool.uv.index] also has an option without parameters. That is to write it like this:
[[tool.uv.index]]
# Optional name for the index.
name = "pytorch"
# Required URL for the index.
url = "https://download.pytorch.org/whl/cpu"
Indexes are prioritized in the order in which they’re defined, such that the first index listed in the configuration file is the first index consulted when resolving dependencies, with indexes provided via the command line taking precedence over those in the configuration file.
Indexes are prioritized in the order in which they’re defined, such that the first index listed in the configuration file is the first index consulted when resolving dependencies, with indexes provided via the command line taking precedence over those in the configuration file.
By default, uv includes the Python Package Index (PyPI) as the "default" index, i.e., the index used when a package is not found on any other index. To exclude PyPI from the list of indexes, set default = true on another index entry (or use the --default-index command-line option):
By default, uv includes the Python Package Index (PyPI) as the "default" index, i.e., the index used when a package is not found on any other index. To exclude PyPI from the list of indexes, set default = true on another index entry (or use the --default-index command-line option):
That is, without the `default` and `explicit` parameters. This is similar to `conda channels`:<br>
```shell
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
It will use the order you defined as its priority order, with the topmost being the highest priority, and if nothing is found, it will use default, which is pypi, and can be set by yourself. You can also use explicit to specify the index for specific packages, but you need to specify the source, like this:
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu" },
]
torchaudio = [
{ index = "pytorch-cpu" },
]
A quite perfect and concise design.
Thus, our final configuration is as follows:
[[tool.uv.index]]
# Optional name for the index. This can be omitted
name = "tsinghua"
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
default = true
[[tool.uv.index]]
name = "aliyun"
url = "https://mirrors.aliyun.com/pypi/simple/"
We use the Tsinghua source to replace pypi and exclude pypi, then give Aliyun as a backup. When our Tsinghua source (which is defined above Aliyun, so it has a higher priority) cannot find anything, it will try to search in the Aliyun source.
You can see https://github.com/MrXnneHang/test-uv-index/blob/master/uv.lock
This time, Tsinghua source is the main character in the uv.lock. =-=